Contributing author:
John D. Seeger, PharmD, DrPH
Vice President, Epidemiology
RTI Health Solutions
Assessing Feasibility in Research
Embarking on an observational study requires that a thorough feasibility assessment be conducted. This evaluation should consider the research question, data sources, study measures (exposure, outcome, and covariates), sample size, operational plans, and the study's motivation (e.g., regulatory requirements). Feasibility depends on these elements being combined in a way to produce results that meet scientific standards. Even if initially deemed feasible, the journey of scientific discovery may reveal challenges that call into question the study's continuation. Researchers should recognize warning signs that indicate when to discontinue a study or address its deficiencies.
How to Recognize Futility in Research
Not every research study follows a linear path to success. Researchers should be able to recognize when a study has reached a point of futility—when continuing is unlikely to yield meaningful results or when the costs outweigh the potential benefits. Signs indicating when to pivot or abandon a project are critical but often underrecognized.
Warning Signs of a Potentially Futile Research Study
- Failure to Address the Research Question
- Scenarios: Inadequate initial research question, changes in the research question, or changes in the research landscape.
- Potential Remedies: Modify the study question to match new circumstances.
- Insufficient Sample Size
- Scenarios: Inadequate number of exposed individuals or study outcomes.
- Potential Remedies: Extend the study duration or bring in additional data sources to increase size.
- Limited Data Quality
- Scenarios: Data source lacks important variables or has substantial gaps.
- Potential Remedies: Acquire additional data or use imputation to address missingness.
- Unforeseen External Variables
- Scenarios: Changes in research priorities.
- Potential Remedies: Seek alternative funding or scale back the project.
- Unresolved Ethical Dilemmas
- Scenarios: Medical intervention being studied appears to cause injury.
- Potential Remedies: Modify the study to address ethical issues or discontinue the study.
- Exhaustion of Resources
- Scenarios: Costs exceed the value of obtaining the answer, or accrual is slower than expected.
- Potential Remedies: Seek alternative funding or scale back the project.
- Inconclusive or Overwhelmingly Negative Results
- Scenarios: Interim results prompt a reappraisal of feasibility.
- Potential Remedies: Revise power estimation or consider conditional power calculations.
Interim Analyses and Conditional Power
Interim analyses allow researchers to evaluate data at predetermined points. If the analysis indicates low power or a lack of effect, continuing the study may be deemed futile. Conditional power reflects the likelihood of achieving a positive outcome if the trial continues. Low conditional power suggests that the research may not be worth continuing.
Ethical Research Considerations and Resource Management
Researchers should weigh the costs of continuing against potential benefits. Sometimes, it is better to cut losses and invest resources in a new direction that holds greater potential for public health benefits. Continuing a study with low power or diminishing returns raises ethical concerns about resource allocation. Researchers have a duty to recommend stopping studies unlikely to provide valuable insights, protecting participants and preventing unnecessary expenditure of resources. Recognizing futility in research helps maintain scientific integrity and promotes transparent reporting.
Discontinuing a study is never easy but is sometimes necessary. Recommending to discontinue a study may lead to a sense that time and resources have been wasted. However, the ethical concerns, unexpected risks, failure to meet objectives, insufficient enrollment, changes in scientific understanding, and resource limitations that inform the decision to discontinue will become even greater if the study continues and does not deliver as expected. Recognizing futility ensures efficient resource allocation and ethical, scientifically sound research efforts. By conducting interim analyses and monitoring accumulating results, researchers can and should reveal futility and provide counsel on whether to develop remedies or stop the study.