Why Most Acute Stroke Studies Are Positive in Animals but Not in Patients: A Systematic Comparison of Preclinical, Early Phase, and Phase 3 Clinical Trials of Neuroprotective Agents

Objective To analyze why numerous acute stroke treatments were successful in the laboratory but failed in large clinical trials. Methods We searched all phase 3 trials of medical treatments for acute ischemic stroke and corresponding early clinical and experimental studies. We compared the overall efficacy and assessed the impact of publication bias and study design on the efficacy. Furthermore, we estimated power and true report probability of experimental studies. Results We identified 50 phase 3 trials with 46,008 subjects, 75 early clinical trials with 12,391 subjects, and 209 experimental studies with >7,141 subjects. Three (6%) phase 3, 24 (32%) early clinical, and 143 (69.08%) experimental studies were positive. The mean treatment effect was 0.76 (95% confidence interval [CI] = 0.70–0.83) in experimental studies, 0.87 (95% CI = 0.71–1.06) in early clinical trials, and 1.00 (95% CI = 0.95–1.06) in phase 3 trials. Funnel plot asymmetry and trim-and-fill revealed a clear publication bias in experimental studies and early clinical trials. Study design and adherence to quality criteria had a considerable impact on estimated effect sizes. The mean power of experimental studies was 17%. Assuming a bias of 30% and pre-study odds of 0.5 to 0.7, this leads to a true report probability of <50%. Interpretation Pivotal study design differences between experimental studies and clinical trials, including different primary end points and time to treatment, publication bias, neglected quality criteria and low power, contribute to the stepwise efficacy decline of stroke treatments from experimental studies to phase 3 clinical trials. Even under conservative estimates, less than half of published positive experimental stroke studies are truly positive. ANN NEUROL 2020;87:40–51