Misinterpretation of Clinical Research Findings and COVID-19 Mortality
Authors: Arturo Casadevall, Liise-anne Pirofski
Journal: Annals of Internal Medicine (2023)

Ideas and OpinionsJuly 2023Misinterpretation of Clinical Research Findings and COVID-19 MortalityFREEArturo Casadevall, MD, PhD, and Liise-anne Pirofski, MDArturo Casadevall, MD, PhDDepartment of Molecular Microbiology and Immunology, Johns Hopkins School of Public Health, Baltimore, Maryland (A.C.), and Liise-anne Pirofski, MDDivision of Infectious Diseases, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, New York (L.P.).Author, Article, and Disclosure Informationhttps://doi.org/10.7326/M23-0737 Annals Author Insight Video - Arturo Casadevall, MD, PhD, and Liise-anne Pirofski, MD In this video, Arturo Casadevall, MD, PhD, and Liise-anne Pirofski, MD, offer additional insight into the article, "Misinterpretation of Clinical Research Findings and COVID-19 Mortality." (Duration 3:58) SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail The first 3 years of the COVID-19 pandemic witnessed an unprecedented pace of research that delivered vaccines, antiviral agents, and anti-inflammatory therapies that dramatically lessened the morbidity and mortality of COVID-19. However, some research findings led to clinical practice recommendations that were later associated with excess mortality.Early in the pandemic, when no proven therapies were available, there were reports that the antimalarial drug hydroxychloroquine had antiviral effects. This prompted a flurry of research to determine whether it could be repurposed for COVID-19 and led to the U.S. Food and Drug Administration (FDA) authorizing its use (1). These studies had many flaws, including not providing sufficient information to evaluate efficacy. One was later retracted (1). Nonetheless, hydroxychloroquine continued to be used despite many well-done studies showing that it was not only ineffective but associated with increased mortality (2) and despite its authorization being revoked by the FDA.Although corticosteroids reduced mortality in critically ill patients with COVID-19 in a large randomized controlled trial (RCT) (3), rather than being broadly beneficial, their effect was limited to patients who required oxygen. Importantly, this limitation was biologically plausible because corticosteroids reduce inflammation and severe COVID-19 is caused by immune-mediated damage. However, the drug was then used in many hospitalized patients who were in an earlier phase of the disease, in whom a benefit had not been established. This led to massive overuse and worse outcomes, including increased mortality in patients who were not critically ill (4). Such use in non–critically ill patients was associated with a detrimental outcome in a meta-analysis (5) and ran counter to recommendations.Failure to incorporate known principles of antibody therapy into the design of COVID-19 therapy trials also had a detrimental effect (6). The FDA authorized COVID-19 convalescent plasma (CCP) for hospitalized patients with COVID-19 in August 2020, based on both biological plausibility and evidence from a large observational study (7). By the fall of 2020, an estimated 30% to 40% of all hospitalized patients in the United States had received CCP. An epidemiologic analysis found that CCP use and mortality were inversely correlated and estimated that CCP may have saved about 100 000 lives (7). However, in early 2021, RCTs from the United Kingdom, Italy, and Canada that mainly evaluated CCP in severely ill hypoxic patients who were hospitalized reported that CCP did not reduce mortality (6). Despite FDA authorization for certain patients, this prompted guideline committees to recommend against CCP, and its use decreased. A temporal trend analysis found that reduced CCP use in the winter of 2020–2021 was associated with 30 000 excess deaths (7). Two years later, an analysis of aggregated RCT evidence from more than 3 dozen trials suggested that CCP significantly reduced mortality in hospitalized patients who were treated with high-titer plasma early in the disease course (8).Today, hydroxychloroquine is no longer used, corticosteroids are primarily used in critically ill patients, and CCP has found a niche for treatment of immunocompromised patients (9). Although different factors resulted in the misuse of each agent for COVID-19, reconciling biological plausibility with the results of clinical studies might have prevented errors that led to detrimental outcomes. For hydroxychloroquine, the assumption that an antimalarial drug could be an effective antiviral was initially based on its antiviral activity in vitro. However, observational and randomized trials in patients with COVID-19 did not demonstrate efficacy (1), highlighting that in vitro biological plausibility must be validated in vivo. For corticosteroids, their efficacy in critically ill patients in a study that also included non–critically ill patients in whom they were not effective led to widespread use despite the knowledge that immunosuppressive therapy impairs antiviral immunity. The immunosuppressive effect of corticosteroids is a basic tenet of their mechanism of action.For CCP, recommendations against its use were based on negative results from trials that, like studies of corticosteroids, were largely conducted in hospitalized patients in whom it was unlikely to be effective, such as those who required oxygen supplementation. This is consistent with prior use of antibody therapies that were generally only effective soon after disease onset—a premise that was validated in a meta-analysis of outpatient COVID-19 RCTs (10). With regard to biological plausibility, CCP efficacy in early COVID-19 is consistent with its antiviral activity, and its inefficacy in the later inflammatory phase is consistent with the likelihood that specific antibody cannot reverse established pulmonary inflammation. Discarding CCP despite its established antiviral properties, a century of successful use of antibody therapies for viral diseases, and incontrovertible evidence from vaccine trials that specific antibody prevents disease progression resulted in the failure to take advantage of the window during which it may have been life-saving.One common thread in the foibles associated with the use of hydroxychloroquine, corticosteroids, and CCP was the failure to consider biological plausibility in clinical decision making. Although it could be argued that the pathogenesis of COVID-19 was not fully understood in the first year of the pandemic, it was known that a related virus, SARS-CoV-1, rapidly transitioned from a viral to an inflammatory phase of disease and that most patients died due to the latter. For hydroxychloroquine, biological plausibility was clung to because of in vitro data, although efficacy was disproved in vivo; for corticosteroids, biological plausibility was not sufficiently considered based on their known mechanism of action and their use in non–critically ill patients, in whom trial data showed they were not effective; and for CCP, biological plausibility was not considered in the design of RCTs of hospitalized patients based on its known mechanism of antiviral action—namely, antiviral activity and historical evidence of clinical efficacy.Clinical research amid the pandemic underscores the need for clinicians conducting trials to consider biological plausibility in study design and interpretation. Even during a pandemic, physicians should consider the complex interplay between the scientific underpinnings of the relevant disease and clinical outcomes. This could refine clinical practice and avoid missteps in the future, which would be a positive legacy from the COVID-19 pandemic.References1. Manivannan E, Karthikeyan C, Moorthy NSHN, et al. The rise and fall of chloroquine/hydroxychloroquine as compassionate therapy of COVID-19. Front Pharmacol. 2021;12:584940. [PMID: 34025393] doi:10.3389/fphar.2021.584940 CrossrefMedlineGoogle Scholar2. Axfors C, Schmitt AM, Janiaud P, et al. Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nat Commun. 2021;12:2349. [PMID: 33859192] doi:10.1038/s41467-021-22446-z CrossrefMedlineGoogle Scholar3. Horby P, Lim WS, Emberson JR, et al; RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384:693-704. [PMID: 32678530] doi:10.1056/NEJMoa2021436 CrossrefMedlineGoogle Scholar4. Sahu AK, Mathew R, Bhat R, et al. Steroids use in non-oxygen requiring COVID-19 patients: a systematic review and meta-analysis. QJM. 2021;114:455-463. [PMID: 34347106] doi:10.1093/qjmed/hcab212 CrossrefMedlineGoogle Scholar5. Patel C, Parmar K, Patel D, et al. Effect of corticosteroid therapy on mortality in COVID-19 patients—a systematic review and meta-analysis. Rev Med Virol. 2022;32:e2386. [PMID: 35971278] doi:10.1002/rmv.2386 CrossrefMedlineGoogle Scholar6. Focosi D, Franchini M, Pirofski LA, et al. COVID-19 convalescent plasma and clinical trials: understanding conflicting outcomes. Clin Microbiol Rev. 2022;35:e0020021. [PMID: 35262370] doi:10.1128/cmr.00200-21 CrossrefMedlineGoogle Scholar7. Casadevall A, Dragotakes Q, Johnson PW, et al. Convalescent plasma use in the USA was inversely correlated with COVID-19 mortality. Elife. 2021;10. [PMID: 34085928] doi:10.7554/eLife.69866 CrossrefMedlineGoogle Scholar8. Senefeld JW, Gorman EK, Johnson PW, et al. Mortality rates among hospitalized patients with COVID-19 treated with convalescent plasma: a systematic review and meta-analysis. medRxiv. Preprint posted online 12 January 2023. doi:10.1101/2023.01.11.23284347 CrossrefGoogle Scholar9. Bloch EM, Focosi D, Shoham S, et al. Guidance on the use of convalescent plasma to treat immunocompromised patients with COVID-19. Clin Infect Dis. 2023. [PMID: 36740590] doi:10.1093/cid/ciad066 CrossrefMedlineGoogle Scholar10. Levine AC, Fukuta Y, Huaman MA, et al. COVID-19 convalescent plasma outpatient therapy to prevent outpatient hospitalization: a meta-analysis of individual participant data from five randomized trials. Clin Infect Dis. 2023. [PMID: 36809473] doi:10.1093/cid/ciad088 CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment John A. BudnyPharmaCal, Ltd.22 May 2023 Hydroxychloroquine Use for SARS-CoV-2 is Determined by Pathodynamics of the Infection Casadevall and Pirofski assert that hydroxychloroquine (HCQ) is not an effective treatment for COVID-19 [sic] and HCQ was associated with increased mortality. The basis of their assertion was a published meta-analysis of randomized trials during the COVID-19 pandemic, where HCQ was used to treat SARS-CoV-2 infections (1). Unfortunately, Casadevvall and Pirofski did not recognize that the randomized trials that were used in the meta-analysis did not contain studies with longitudinal data on HCQ treatment effects for the various stages of SARS-CoV-2 infections. There is nothing wrong with the data from the selected studies used in the meta-analysis, provided that when a treatment is administered, it does not affect the subsequent pathology caused by the disease. However, that is not the case with HCQ therapy for SARS-CoV-2 infections. Casadervall and Pirofski used the meta-analysis data to arrive at their two erroneous conclusions that 1) HCQ was ineffective in treating SARS-CoV-2 infections, and 2) The use of HCQ was associated with increased mortality from SARS-CoV-2 infections (2). In fact, the second conclusion is the exact opposite of Casadevall and Pirofski (2): NOT using or restricting the use of HCQ at a strategic time point in the SARS-CoV-2 infection timeline will result in more deaths. The critical and all-important issue in the therapeutic value of HCQ for infections caused by the SARA-CoV-2 virus is the pathodynamic nature of the infection (3). It has been shown, albeit not understood or rejected, that there is a critical window of the SARS-CoV-2 infection progression: early treatment in the infection lifecycle is necessary to avoid the adverse consequence of the SARS-CoV-2 infection: if the virus is prohibited from entering the host's cell, the disease is unable to progress (3, 4, 5). The lack of appreciation of the pathodynamic nature of SARS-CoV-2 infections by Casadevall and Pirofski not only results in incorrect information but also puts infected individuals at greater risk by not appropriately using HCQ as an early administered preventive therapeutic. Finally, it is puzzling, if not troubling, that two published studies (4, 5) that validate the early use of HCQ in SARS-CoV-2 infections were not mentioned by Casadervall and Pirofski. The value of science is to seek and arrive at truth, not attempt to validate a preconceived conclusion. References Axfors, C, Schmitt, AM, Janiaud, P, et al. Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nat Commun. 2021; 12:2349[PMID:33859192] doi:10.1038/s41467-021-22446-z Budny, JA. COVID-19. Hydroxychloroquine and the importance of disease progression. Toxicology Research. 2021, 1-5. Doi: 10.1093/toxres/tfab015 Arshad, S, Kilgore, P, Chaudhry, ZS, et al. 2020; Treatment with hydroxychloroquine, azithromycin, and in combination in patients hospitalized with COVID-19. Intl J Infect Dis 2020; doi.org/10.1016/j.ijid.2020.06.099. Million, M., Lagier, J.-C., Gautret, P, Raoult, D. et al.. Full-length title: Early treatment of COVID-19 patients with hydroxychloroquine and azithromycin: A retrospective analysis of 1061 cases in Marseille, France. Travel Medicine and Infectious Disease 2020; .doi.org/10.1016/j.tmaid.2020.101738 Avidan, MS, Dehbi, H-M and Delany-Moretlwe, S. Hydroxychloroquine as Post Exposure Prophylaxis for Covid-19. N Engl Med 2020; 383(11): 1087- 1088 Arturo Casadevall (1), Liise-anne Pirofski (2)(1)Department of Molecular Microbiology and Immunology, Johns Hopkins School of Public Health, (2) Division of Infectious Diseases, Albert Einstein College of Medicine and Montefiore Medical Center21 June 2023 Author Response to Budny Dr. Budny argues that the therapeutic value of hydroxychloroquine has been underestimated by use at the wrong time during COVID-19. We agree that antiviral agents need to be used early in the course of COVID-19, as has been amply demonstrated for small molecule antiviral drugs, monoclonal antibodies, and convalescent plasma. However, in contrast to these therapies, hydroxychloroquine was found to be ineffective as pre-exposure prophylaxis (1) and a meta-analyzes of individual participants in 11 randomized controlled trials revealed no evidence of efficacy when it was used early in disease (2). Furthermore, studies of hydroxychloroquine dosing and pharmacokinetics revealed that its antiviral effect in vitro was unlikely to occur in vivo with the dosages that were used clinically (3), reducing biological plausibility that it could be effective for COVID-19. REFERENCES Rajasingham R, Bangdiwala AS, Nicol MR, Skipper CP, Pastick KA, Axelrod ML, et al. Hydroxychloroquine as Pre-exposure Prophylaxis for Coronavirus Disease 2019 (COVID-19) in Healthcare Workers: A Randomized Trial. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2021;72(11):e835-e43. Mitjà O, Reis G, Boulware DR, Spivak AM, Sarwar A, Johnston C, et al. Hydroxychloroquine for treatment of non-hospitalized adults with COVID-19: A meta-analysis of individual participant data of randomized trials. Clinical and translational science. 2023;16(3):524-35. Fan J, Zhang X, Liu J, Yang Y, Zheng N, Liu Q, et al. Connecting Hydroxychloroquine In Vitro Antiviral Activity to In Vivo Concentration for Prediction of Antiviral Effect: A Critical Step in Treating Patients With Coronavirus Disease 2019. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2020;71(12):3232-6. Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-0737. Author, Article, and Disclosure InformationAuthors: Arturo Casadevall, MD, PhD; Liise-anne Pirofski, MDAffiliations: Department of Molecular Microbiology and Immunology, Johns Hopkins School of Public Health, Baltimore, Maryland (A.C.)Division of Infectious Diseases, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, New York (L.P.).Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-0737.Corresponding Author: Arturo Casadevall, MD, PhD, Johns Hopkins School of Public Health, 615 North Wolfe Street, Baltimore, MD 21205; e-mail, acasade1@jhu.edu.Author Contributions: Conception and design: A. Casadevall, L. Pirofski.Analysis and interpretation of the data: L. Pirofski.Drafting of the article: A. Casadevall, L. Pirofski.Critical revision for important intellectual content: A. Casadevall, L. Pirofski.Final approval of the article: A. Casadevall, L. Pirofski.Administrative, technical, or logistic support: A. Casadevall.This article was published at Annals.org on 16 May 2023. PreviousarticleNextarticle Advertisement Annals Author Insight Video - Arturo Casadevall, MD, PhD, and Liise-anne Pirofski, MD In this video, Arturo Casadevall, MD, PhD, and Liise-anne Pirofski, MD, offer additional insight into the article, "Misinterpretation of Clinical Research Findings and COVID-19 Mortality." (Duration 3:58) FiguresReferencesRelatedDetails Metrics Current IssueJuly 2023Volume 176, Issue 7Page: 993-994KeywordsAntibody therapyAntiviral therapyBlood plasmaClinical trialsCOVID-19Food and Drug AdministrationHospitalizationsMortality ePublished: 16 May 2023 Issue Published: July 2023 Copyright & PermissionsCopyright © 2023 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...