OBJECTIVE: To characterise the study design features and methods of external control arm (ECA) studies in recently submitted marketing authorisation (MA) applications to the European Medicines Agency (EMA), stratified according to the EMA’s opinion of the evidence.
METHOD: EMA assessment documents from 01-01-2022 to 31-07-2024 were screened for ECA design. We double-extracted data and classified ECA evidence as unsupportive, supportive, or acceptable for the final MA recommendation according to the EMA’s feedback.
RESULTS: Of 236 medicines, 22 (9%) included evidence based on an ECA. Of these, 18 were designated as orphan drugs and 20 had a positive recommendation in their MA application. The motivation for using ECA evidence was to provide a control arm for clinical trials, either single-arm (14) or randomised (4), or to compare long-term outcomes in open-label extension phases (4). The main types of data used for the ECAs were health records from observational studies (14), clinical trial data (4), or a combination of observational, clinical trial, and literature review data (4).
For applications with unsupportive ECA evidence (8/22), EMA’s concerns included insufficient methodological details, post hoc data-driven decisions for analysis and covariates, differences between the trial and external cohorts in terms of their baseline characteristics, calendar time, disease stage, and endpoint characterisation, as well as the risk of residual confounding and selection bias.
In supportive cases (11/22), when presented alongside primary evidence from clinical trials, the EMA considered ECA evidence as supportive, confirmatory, or useful for contextualising results. The choice of ECAs was deemed justifiable in the absence of placebo arms, alternative data, or existing treatment options. ECA evidence was considered more valid in scenarios with large treatment effects, objective endpoints, and predictable disease courses, but still insufficient to replace controlled trial results due to “inherent biases”.
In acceptable cases (3/22), ECA evidence formed the basis for the final recommendation. EMA’s feedback on these ECAs emphasised: a strong justification for the ECA due to extremely rare diseases, unethical placebo arms due to severe, rapid-progressing diseases without treatment options, well described baseline comparability of populations with good regional and temporal overlap, and sensitivity analyses with consistent results.
CONCLUSIONS: The increasing availability of observational data and the recently published guidelines on ECAs present an opportunity for the use of ECAs in regulatory applications of treatments for rare diseases. In the past 2.5 years, several MA applications to the EMA have included ECA evidence, mainly for orphan drugs and when primary evidence is from a single-arm trial. Based on the reviewed EMA feedback, ECAs can help support the primary study results, but they are generally insufficient to serve as the main source of evidence.
Many of the concerns highlighted by the EMA regarding ECA evidence can be addressed through robust methodologies. Study design features and methods that can strengthen the robustness of ECA studies include clearly defining the causal question, specifying the study design features and the analysis plan before data analysis, assessing the populations’ exchangeability in terms of baseline characteristics, geography and calendar time distribution, disease stage, and exposure and outcome characterisation methods, and conducting relevant sensitivity analyses and net bias assessments. Other limitations, such as residual confounding, are inherent to observational studies, but making the required assumptions explicit can improve the interpretability of results.
While ECA evidence can meaningfully contribute to regulatory decisions, especially in rare diseases with limited patient numbers, it must be backed by a compelling rationale and be methodologically robust. A strong justification for the need of an ECA, a detailed description of the populations’ comparability, pre-specification of analyses, and sensitivity analyses are valued by the EMA when assessing the reliability of the ECA evidence. These findings can inform the design of future ECA studies to improve their likelihood for acceptance in regulatory decisions.
