Multigroup, Adaptively Randomized Trials Are Advantageous for Comparing Coronavirus Disease 2019 (COVID-19) Interventions
Authors: Amalia S. Magaret, Shevin T. Jacob, M. Elizabeth Halloran, Katherine A. Guthrie, Craig A. Magaret, Christine Johnston, Noah R. Simon, Anna Wald
Journal: Annals of Internal Medicine (2020)

Ideas and Opinions6 October 2020Multigroup, Adaptively Randomized Trials Are Advantageous for Comparing Coronavirus Disease 2019 (COVID-19) InterventionsFREEAmalia S. Magaret, PhD, Shevin T. Jacob, MD, MPH, M. Elizabeth Halloran, MD, DSc, Katherine A. Guthrie, PhD, Craig A. Magaret, MS, Christine Johnston, MD, MPH, Noah R. Simon, PhD, and Anna Wald, MD, MPHAmalia S. Magaret, PhDUniversity of Washington and Fred Hutchinson Cancer Research Center, Seattle, Washington (A.S.M., M.E.H., C.J., A.W.)Search for more papers by this author, Shevin T. Jacob, MD, MPHLiverpool School of Tropical Medicine, Liverpool, United Kingdom, and University of Washington, Seattle, Washington (S.T.J.)Search for more papers by this author, M. Elizabeth Halloran, MD, DScUniversity of Washington and Fred Hutchinson Cancer Research Center, Seattle, Washington (A.S.M., M.E.H., C.J., A.W.)Search for more papers by this author, Katherine A. Guthrie, PhDFred Hutchinson Cancer Research Center, Seattle, Washington (K.A.G., C.A.M.)Search for more papers by this author, Craig A. Magaret, MSFred Hutchinson Cancer Research Center, Seattle, Washington (K.A.G., C.A.M.)Search for more papers by this author, Christine Johnston, MD, MPHUniversity of Washington and Fred Hutchinson Cancer Research Center, Seattle, Washington (A.S.M., M.E.H., C.J., A.W.)Search for more papers by this author, Noah R. Simon, PhDUniversity of Washington, Seattle, Washington (N.R.S.)Search for more papers by this author, and Anna Wald, MD, MPHUniversity of Washington and Fred Hutchinson Cancer Research Center, Seattle, Washington (A.S.M., M.E.H., C.J., A.W.)Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/M20-2933 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail We propose platform trials with outcome-adaptive randomization to efficiently select the most effective coronavirus disease 2019 (COVID-19) treatments. The global spread of severe acute respiratory syndrome coronavirus 2 infection is alarming in its geographic scope and in the number of associated deaths. There are currently no treatments proven to decrease mortality from COVID-19 further than what can be achieved through supportive care. Thus far, the choice of therapeutics has been limited to existing, repurposed medications. Given that some of the medications are perceived to have low toxicity, many have been embraced without evidence. Although remdesivir was recently found to shorten time to symptom resolution, evidence for survival benefit is inconclusive (1).Well-designed, placebo-controlled trials have begun. On 1 June 2020, we searched ClinicalTrials.gov and found 308 phase 2 and 3 intervention studies aimed at COVID-19 that were open to enrollment; an additional 287 trials were posted but not yet recruiting (2). Those already recruiting are largely treatment trials testing known antiviral medications (remdesivir, favipiravir, and oseltamivir), antimalarials (hydroxychloroquine), immunosuppressive drugs known to be effective in the treatment of inflammatory or autoimmune disorders (sarilumab, tocilizumab, baricitinib, and others), or antiretrovirals for treatment of HIV infection (lopinavir, ritonavir, darunavir, and cobicistat).Several large trials are already engaged in simultaneous testing of multiple treatment strategies in separate groups, with plans to discontinue any group that is definitively inferior at planned interim analyses, a format known as a platform trial. For example, the DisCoVeRy (Trial of Treatments for COVID-19 in Hospitalized Adults) trial in France is testing standard of care; remdesivir; lopinavir and ritonavir; and lopinavir, ritonavir, and interferon β-1a. The World Health Organization's Solidarity trial includes hydroxychloroquine in addition to those interventions listed above. The United Kingdom's RECOVERY (Randomised Evaluation of COVID-19 Therapy) trial is simultaneously testing 5 treatments as well, and REMAP-CAP (Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia) has amended its international treatment trial for community-acquired pneumonia (3–6). Other multigroup trials are taking place in Belgium, Norway, Spain, and the United States. In a short time, many trials have been initiated to test potentially beneficial interventions, some with adaptive design incorporated. Although cost, availability, and regulatory issues may limit the appropriateness or feasibility of testing certain treatments in some settings, these large trials demonstrate the feasibility of testing several interventions simultaneously to facilitate direct comparison.A further innovation in the design of the clinical trials would be a broader use of outcome-adaptive randomization, a specific adaptive design feature that potentially reduces the number of deaths or other adverse outcomes incurred during a trial. To favor groups with more advantageous outcomes, outcome-adaptive randomization updates the allocation proportions on the basis of observed outcomes from cumulatively enrolled persons to date. For example, consider a study with a favorable or desirable binary outcome. After early group-specific event rates are observed during an interim analysis, outcome-adaptive randomization permits reallocation of twice as many subsequent participants to a group with a high event rate than to a group with half that event rate. These later-enrolled participants stand to benefit from the experience gained earlier in the trial. This approach is suitable for COVID-19 disease outcomes, which are known rapidly: Patients either recover or die within a few weeks. As such, the follow-up period for antiviral studies using remdesivir ranges from 14 to 29 days (1, 3). Although platform trials can drop ineffective groups at interim analyses, outcome-adaptive randomization further concentrates allocation among the current, better-performing groups. A simulation study done in 2016 showed that, relative to platform trials allowing early dropping of ineffective groups, a platform trial additionally incorporating outcome-adaptive randomization (or response-adaptive randomization) can lead to an 18% decrease in the number of poor outcomes (7). The advantages of updating randomization allocation are thus also measured in adverse outcomes averted.A few issues related to adaptive randomization require consideration. The first is drift, which occurs when participants randomly assigned at later stages of the trial have a different pretreatment outcome risk relative to those enrolled earlier. Drift could occur in this setting if the stage of illness at the time of presentation for medical care changes as diagnostics become more available, or if the virulence of the infection changes through viral mutation or repeated person-to-person transmissions. For example, if widespread diagnostic testing is made available, persons enrolling later may present at an earlier stage of infection. In this scenario, genuine treatment benefit leading to higher allocation in some groups may not be distinguishable from benefit due to earlier presentation. However, this problem can be mitigated through stratification on the stages of allocation or through the use of previously developed rerandomization tests (8).The second issue to consider when updating randomization proportions during a trial is that early study results can be highly variable. When few participants are enrolled, some groups can appear to do better than others on the basis of chance alone. The problems of bias and inflated significance induced by decisions made early in a sequentially designed trial can be addressed by validated post hoc corrections that account for potentially exaggerated treatment benefit estimates (9).A third concern is the need for clinicians providing care to patients to remain agnostic about the relative potential benefit of the treatments they provide, even as randomization allocations change. Should care providers' equipoise falter before the end of the study, they may be strongly tempted to ignore subsequent treatment assignments. Approaches that mitigate this concern include masking investigators, when possible, and separating the roles of clinicians who are providing treatment from those assessing outcomes (10).In summary, an adaptively designed, multigroup trial with outcome-adaptive randomization is especially appropriate for this large-scale severe acute respiratory syndrome coronavirus 2 outbreak. If interventions are tested separately over the next few months, additional time will be required to conduct direct comparison of the most effective treatments. A collaborative effort will help us to widely implement the most effective treatments as quickly as possible, and with potentially more persons receiving the most effective treatments. These recommendations complement those of the World Health Organization's R&D Blueprint group, which recently encouraged core protocols that are maintained until a definitive answer about efficacy is reached, perhaps spanning multiple infectious disease outbreaks (11).References1. National Institutes of Health. NIH clinical trial shows remdesivir accelerates recovery from advanced COVID-19. Accessed at www.nih.gov/news-events/news-releases/nih-clinical-trial-shows-remdesivir-accelerates-recovery-advanced-covid-19 on 10 April 2020. Google Scholar2. U.S. National Library of Medicine. ClinicalTrials.gov database. Accessed at www.clinicaltrials.gov on 10 April 2020. Google Scholar3. Institut National de la Santé Et de la Recherche Médicale. Trial of treatments for COVID-19 in hospitalized adults (DisCoVeRy). Accessed at https://clinicaltrials.gov/ct2/show/NCT04315948 on 10 April 2020. Google Scholar4. World Health Organization. “Solidarity” clinical trial for COVID-19 treatments. Accessed at www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments on 10 April 2020. Google Scholar5. University of Oxford. Randomised evaluation of COVID-19 therapy (RECOVERY). Accessed at https://clinicaltrials.gov/ct2/show/NCT04381936 on 10 April 2020. Google Scholar6. The Australian and New Zealand Intensive Care Research Centre. REMAP-CAP response to the novel COVID-19 pandemic. Accessed at www.remapcap.org/coronavirus on 10 April 2020. Google Scholar7. Saville BR, Berry SM. Efficiencies of platform clinical trials: a vision of the future. Clin Trials. 2016;13:358-66. [PMID: 26908536] doi:10.1177/1740774515626362 CrossrefMedlineGoogle Scholar8. Simon R, Simon NR. Using randomization tests to preserve type I error with response-adaptive and covariate-adaptive randomization. Stat Probab Lett. 2011;81:767-772. [PMID: 21769160] CrossrefMedlineGoogle Scholar9. Wittes J. Stopping a trial early—and then what? Clin Trials. 2012;9:714-20. [PMID: 22879573] doi:10.1177/1740774512454600 CrossrefMedlineGoogle Scholar10. U.S. Food and Drug Administration. Adaptive design clinical trials for drugs and biologics guidance for industry. Accessed at www.fda.gov/regulatory-information/search-fda-guidance-documents/adaptive-design-clinical-trials-drugs-and-biologics-guidance-industry on 10 April 2020. Google Scholar11. Dean NE, Gsell PS, Brookmeyer R, et al. Creating a framework for conducting randomized clinical trials during disease outbreaks. N Engl J Med. 2020;382:1366-1369. [PMID: 32242365] doi:10.1056/NEJMsb1905390 CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAuthors: Amalia S. Magaret, PhD; Shevin T. Jacob, MD, MPH; M. Elizabeth Halloran, MD, DSc; Katherine A. Guthrie, PhD; Craig A. Magaret, MS; Christine Johnston, MD, MPH; Noah R. Simon, PhD; Anna Wald, MD, MPHAffiliations: University of Washington and Fred Hutchinson Cancer Research Center, Seattle, Washington (A.S.M., M.E.H., C.J., A.W.)Liverpool School of Tropical Medicine, Liverpool, United Kingdom, and University of Washington, Seattle, Washington (S.T.J.)Fred Hutchinson Cancer Research Center, Seattle, Washington (K.A.G., C.A.M.)University of Washington, Seattle, Washington (N.R.S.)Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M20-2933.Corresponding Author: Amalia S. Magaret, PhD, University of Washington, Box 359300, Seattle, WA 98195-9300; e-mail, amag@uw.edu.Current Author Addresses: Dr. Magaret: University of Washington, Box 359300, Seattle, WA 98195-9300.Dr. Jacob: Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, United Kingdom.Drs. Halloran and Guthrie and Mr. Magaret: Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, PO Box 19024, Seattle, WA 98109-1024.Drs. Johnston and Wald: Harborview Medical Center, University of Washington, 325 9th Avenue, Box 359928, Seattle, WA 98104.Dr. Simon: University of Washington, Box 357232, Seattle, WA 98195-7232.Author Contributions: Conception and design: A.S. Magaret, M.E. Halloran, A. Wald.Drafting of the article: A.S. Magaret, C.A. Magaret, A. Wald.Critical revision of the article for important intellectual content: A.S. Magaret, S.T. Jacob, M.E. Halloran, K.A. Guthrie, A. Wald.Final approval of the article: A.S. Magaret, S.T. Jacob, M.E. Halloran, K.A. Guthrie, C.A. Magaret, C. Johnston, N.R. Simon, A. Wald.Statistical expertise: A.S. Magaret, M.E. Halloran, K.A. Guthrie, C.A. Magaret, N.R. Simon.Administrative, technical, or logistic support: C.A. Magaret, A. Wald.This article was published at Annals.org on 11 June 2020. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetails Metrics Cited byResponse-Adaptive Randomization in Clinical Trials: From Myths to Practical ConsiderationsUses of equipoise in discussions of the ethics of randomized controlled trials of COVID-19 therapies 6 October 2020Volume 173, Issue 7Page: 576-577 ePublished: 11 June 2020 Issue Published: 6 October 2020 Copyright & PermissionsCopyright © 2020 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...