Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials
Authors: Frank W Rockhold
Javed Akram
Ravi K Amaravadi
Cathrine Axfors
Andreas M Schmitt
Stephen R Walsh
Janneke Van&#x27;t Hooft
Felipe Jurado-Camacho
Susanna Naggie
Lisa N Cowan
Patricia Meza-Meneses
Benno Kreuels
Shehnoor Azhar
Rossana Rosa
Yaseen M Arabi
Chien-Yu Cheng
Rogelio Perez-Padilla
Wu-Pu Lin
Bruno Igau
Anissa Elfakir
John P A Ioannidis
Steve A Webb
Thomas Hills
Ireri Thirion-Romero
Peter G Kremsner
Yi-Wen Huang
Shu-Hsing Cheng
Anthony C Gordon
Lewis A Novack
Bryan J McVerry
Perrine Janiaud
Thuy Le
Robert Rolfe
Thomas Benfield
Vincent Dubee
Olav Dalgard
Andy I M Hoepelman
Sebastian Rodriguez-Llamazares
Muhammad Shahzad
Susan C Morpeth
Ahmad Mourad
Yi-Chun Lin
Leila Belkhir
Shanti Narayanasamy
Todd B Seto
Shaimaa Soliman
Cheng-Yu Kuo
David Moher
Steven N Goodman
Wuelton M Monteiro
Fernando F de Almeida E Val
Nicholas A Turner
Arantxa Remigio-Luna
Marcus V G de Lacerda
Magnus Nakrem Lyngbakken
Sean M O&#x27;Brien
Fernando G Zampieri
Nwora Lance Okeke
Marvin A H Berrevoets
Helge Røsjø
Yehuda Z Cohen
Srinivas Murthy
Benjamin S Abella
Wu Zhong
Andrea B Troxel
Carmen M Hernandez-Cardenas
Derek C Angus
Lindsey R Baden
Jesper M Weehuizen
Khalid S Khan
Lars G Hemkens
Ehab F Abdo
Ronghua Jin
Hon-Lai Wong
Vanderson S Sampaio
Ting-Yu Tseng
Robert J Ulrich
Colin McArthur
Mark J Mulligan
Léna Perez
Norma E Rivera-Martinez
Wei-Sheng Chung
Laurent Perrin
Jason E Stout
Tse-Hung Lin
Alistair Nichol
Sherief Abd-Elsalam
Cheng-Pin Chen
Gisely C de Melo
Arthur W Baker
Lennie Derde
Rebekah Wrenn
Tsung-Chia Chen
Maria Velinova
Journal: Nature Communications (2021)
Discipline: Multidisciplinary
Keywords: COVID-19; hydroxychloroquine; chloroquine; randomized clinical trials; meta-analysis
Affiliations: Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA., Department for Women's and Children's Health, Uppsala University, Uppsala, Sweden., Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland., Department of Medical Oncology, University of Basel, Basel, Switzerland., Amsterdam University Medical Center, Amsterdam University, Amsterdam, the Netherlands., Tropical Medicine and Infectious Diseases Department, Faculty of Medicine, Tanta University, Tanta, Egypt., Tropical Medicine and Gastroenterology Department, Faculty of Medicine, Assiut University, Assiut, Egypt., Department of Emergency Medicine, University of Pennsylvania, Philadelphia, PA, USA., Department of Internal Medicine, Vice Chancellor, University of Health Sciences, Lahore, Punjab, Pakistan., Abramson Cancer Center and Department of Medicine, University of Pennsylvania, Philadelphia, PA, USA., Department of Critical Care Medicine, The Clinical Research Investigation and Systems Modeling of Acute Illness (CRISMA) Center, University of Pittsburgh, Pittsburgh, PA, USA., the UPMC Health System Office of Healthcare Innovation, University of Pittsburgh Medical Centre, Pittsburgh, PA, USA., Intensive Care Department, King Saud Bin Abdulaziz University for Health Sciences and King Abdullah International Medical Research Center, Riyadh, Saudi Arabia., Department of Public Health, University of Health Sciences, Lahore, Punjab, Pakistan., Division of Infectious Diseases, Brigham and Women's Hospital, Boston, MA, USA., Department of Medicine, Division of Infectious Diseases and International Health, Duke University Medical Center, Durham, NC, USA., Infectious Diseases Department, Cliniques universitaires Saint-Luc, Université Catholique de Louvain, Brussels, Belgium., Center of Research & Disruption of Infectious Diseases, Department of Infectious Diseases, Copenhagen University Hospital, Amager and Hvidovre, Hvidovre, Denmark., Department of Internal Medicine, Elisabeth-Tweesteden hospital, Tilburg, Netherlands., Department of Infectious Diseases, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan., Department of Internal Medicine, Taichung Hospital, Ministry of Health and Welfare, Taichung, Taiwan., Sanofi, Bridgewater, NJ, USA., Department of Infectious Diseases, Division of Medicine, Akershus University Hospital, Lørenskog, Norway., Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway., Fundação de Medicina Tropical Dr. Heitor Vieira Dourado, Manaus, AM, Brazil., Instituto Leonidas e Maria Deane - ILMD, FIOCRUZ-AM, Manaus, AM, Brazil., Universidade do Estado do Amazonas, Manaus, AM, Brazil., Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands., Intensive Care Centre, University Medical Center Utrecht, Utrecht, Netherlands., Infectious and Tropical Diseases Department, Angers University Hospital, Angers, France., Ividata Life Sciences, Levallois-Perret, France., Department of Surgery and Cancer, Anaesthetics, Pain Medicine, and Intensive Care Medicine, Imperial College London and Imperial College Healthcare NHS Trust, London, UK., Critical Care Department, Instituto Nacional de Enfermedades Respiratorias Ismael Cosío Villegas, Ciudad de México, Mexico., Medical Research Institute of New Zealand, Wellington, New Zealand., Auckland City Hospital, Auckland, New Zealand., Department of Infectious Diseases, University Medical Center Utrecht, Utrecht, Netherlands., Department of Internal Medicine, Chang Hua Hospital, Ministry of Health and Welfare, Changhua, Taiwan., Beijing Youan Hospital, Capital Medical University, Beijing, People's Republic of China., Department of Preventive Medicine & Public Health, University of Granada, Hospital Real, Avenida del Hospicio, Granada, Granada, Spain., Institute of Tropical Medicine, University of Tübingen, Tübingen, Germany., Centre de Recherches Médicales de Lambaréné, Lambaréné, Gabon., German Center for Infection Research, Partner Site Tübingen, Tübingen, Germany., Department of Medicine, Division of Tropical Medicine and Division of Infectious Diseases, University Medical Center Hamburg-Eppendorf, Hamburg, Germany., Department of Tropical Medicine, Bernhard Nocht Institute for Tropical Medicine, Hamburg, Germany., Department of Internal Medicine, Pingtung Hospital, Ministry of Health and Welfare, Pingtung, Taiwan., Department of Internal Medicine, Taipei Hospital, Ministry of Health and Welfare, New Taipei City, Taiwan., Division of Medicine, Akershus University Hospital, Lørenskog, Norway., School of Epidemiology and Preventive Medicine, Australian and New Zealand Intensive Care Research Centre, Monash University, Melbourne, VIC, Australia., Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA., Hospital Regional de Alta especialidad de Ixtapaluca, Ixtapaluca, Mexico., Middlemore Hospital, Auckland, New Zealand., Department of Medicine, Duke University Medical Center, Durham, NC, 27710, USA., Department of Microbiology, NYU Grossman School of Medicine, New York, NY, USA., Department of Internal Medicine, Division of Infectious Diseases and Immunology, NYU Grossman School of Medicine, New York, NY, USA., University of British Columbia School of Medicine, Vancouver, BC, Canada., Department of Intensive Care, Alfred Health, Melbourne, VIC, Australia., Department of Anesthesia and Intensive Care, St Vincent's University Hospital, Dublin, Ireland., School of Medicine and Medical Sciences, University College Dublin, Dublin, Ireland., Division of Infectious Diseases, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA., Department of Biostatistics and Bioinformatics, Duke University Medical Center and Duke Clinical Research Institute, Durham, NC, USA., Excelya, Montpellier, France., Department of Smoking and COPD, Instituto Nacional de Enfermedades Respiratorias Ismael Cosío Villegas, Ciudad de México, Mexico., Sanofi, Montpellier, France., Hospital Regional de Alta especialidad de Oaxaca, Oaxaca, Mexico., UnityPoint Health, Des Moines, IA, USA., Division of Research and Innovation, Akershus University Hospital, Lørenskog, Norway., Fundação de Vigilância em Saúde do Amazonas, Manaus, AM, Brazil., University of Hawaii John A. Burns School of Medicine, Honolulu, HI, USA., The Queen's Medical Center, Honolulu, HI, USA., Department of Pharmacology, University of Health Sciences, Lahore, Punjab, Pakistan., Public Health and Community Medicine, Menoufia University, Menoufia, Egypt., Division of Biostatistics, Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA., Department of Medicine, Division of Infectious Diseases and Immunology, NYU Grossman School of Medicine, New York, NY, USA., St. John of God Hospital, Subiaco, WA, Australia., PRA Health Science, Groningen, Netherlands., Department of Internal Medicine, Keelung Hospital, Ministry of Health and Welfare, Keelung, Taiwan., Research Institute, HCor-Hospital do Coração, São Paulo, Brazil., Research Institute, BRICNet - Brazilian Research in Intensive Care Network, São Paulo, Brazil., IDor Research Institute, São Paulo, Brazil., National Engineering Research Center for the Emergency Drug, Beijing Institute of Pharmacology and Toxicology, Beijing, People's Republic of China., Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada., Stanford University School of Medicine, Stanford, CA, USA., Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA., Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, CA, USA., Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany., Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. ., Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland. ., Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany. .

Abstract:  Substantial COVID-19 research investment has been allocated to randomized clinical trials (RCTs) on hydroxychloroquine/chloroquine, which currently face recruitment challenges or early discontinuation. We aim to estimate the effects of hydroxychloroquine and chloroquine on survival in COVID-19 from all currently available RCT evidence, published and unpublished. We present a rapid meta-analysis of ongoing, completed, or discontinued RCTs on hydroxychloroquine or chloroquine treatment for any COVID-19 patients (protocol: https://osf.io/QESV4/ ). We systematically identified unpublished RCTs (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, Cochrane COVID-registry up to June 11, 2020), and published RCTs (PubMed, medRxiv and bioRxiv up to October 16, 2020). All-cause mortality has been extracted (publications/preprints) or requested from investigators and combined in random-effects meta-analyses, calculating odds ratios (ORs) with 95% confidence intervals (CIs), separately for hydroxychloroquine and chloroquine. Prespecified subgroup analyses include patient setting, diagnostic confirmation, control type, and publication status. Sixty-three trials were potentially eligible. We included 14 unpublished trials (1308 patients) and 14 publications/preprints (9011 patients). Results for hydroxychloroquine are dominated by RECOVERY and WHO SOLIDARITY, two highly pragmatic trials, which employed relatively high doses and included 4716 and 1853 patients, respectively (67% of the total sample size). The combined OR on all-cause mortality for hydroxychloroquine is 1.11 (95% CI: 1.02, 1.20; I² = 0%; 26 trials; 10,012 patients) and for chloroquine 1.77 (95%CI: 0.15, 21.13, I² = 0%; 4 trials; 307 patients). We identified no subgroup effects. We found that treatment with hydroxychloroquine is associated with increased mortality in COVID-19 patients, and there is no benefit of chloroquine. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities. 

Retracted: No

PubPeer comments:
 There is concern regarding possible selective reporting in this meta-analysis. The authors did not update their meta-analysis with the new data available since October. For instance, the numbers they used for the HYDRA trial are quite different than what was subsequently reported in the preprint. This does not appear to follow a criteria prespecified in the protocol. Updating the meta-analysis may impact the significance of the overall result. More importantly, it would bring relevant additional information for the  low-dose subgroup. The authors choose an unconventional estimator for the pooled point estimate and confidence interval. Although prespecified, It is quite peculiar that it happens to be the only one that yields a statistically significant increase in mortality. In the SOLIDARITY trial, some tablets included 155mg of chloroquine base rather than hydroxychloroquine. High-dose chloroquine was shown to be harmful in a Brazilian trial. The authors did not calculate separately each treatment effect estimate from the raw data. , 
 The authors present a meta-analysis of published and unpublished trials on chloroquine and hydroxychloroquine in the treatment of COVID-19. The main result of the analysis is a significant increase in mortality associated with hydroxychloroquine use, derived mostly from inpatient studies. There are however some issues with this meta-analysis: * The authors did not include the data that was reported after the SOLIDARITY trial (i.e. about six months ago). For instance, the results for the HYDRA trial (the one with largest weight in the  low-dose subgroup) included in the meta-analysis are quite different than what was subsequently reported in the preprint. This does not appear to follow a prespecified criteria and raises concern about possible selective reporting. The authors have indeed stopped including data when the increase in mortality was marginally significant. *The authors used a rather unconventional estimator to calculate the pooled point estimate and confidence intervals. Although prespecified, it is quite peculiar that it happens to be the only one that yields a (marginally) significant increase in mortality. The fact that other estimators find no significant increase, together with the previous comment, suggest that the authors have overinterpreted their result and that the average effect of HCQ in hospitalized patients was probably mostly neutral. * One of the trials included in the chloroquine meta-analysis (NCT04323527) does not have a proper comparator, since it compares two dosing regimen of the same drug. The authors incorrectly classified it as a placebo-controlled trial. That study, which carries 96.7% of the weight in the meta-analysis, does not meet the inclusion criteria and the authors need publish an erratum. *In the SOLIDARITY trial, some tablets included 155mg of chloroquine base, rather than hydroxychloroquine. Axfors and al. did not calculate the effect estimate separately from the raw data, not respecting their protocol. , 
 I wish to elaborate further on the previous comment. The trial mistakenly included in the chloroquine meta-analysis, which is published in the JAMA, was comparing  low-dose (in fact, already pretty high dose) with  high-dose (in fact, a dose that was predicted to be toxic) chloroquine. It was stopped for toxicity. The Hartung-Knapp method used by the authors is well known to give anticonservative confidence intervals (i.e. too narrow). Since the test for overall effect is only borderline significant (p=0.02), and since other methods that do not have that problem did not yield a significant difference, I believe that the authors are somehow overselling their result. Based on this result, one of the authors claimed in the media that their analysis shows that HCQ use in COVID-19 has caused 100 000 deaths. Moreover, HKSJ is known to underperform when the standard error of the studies vary a lot. Here, there were more events in RECOVERY trial than in all the other trials combined, so it is precisely a situation where this method is should be used with caution. The meta-analysis presented by the authors is seriously lagging and does not include more recent data made publicly available in the past six months. Some of this data was available since the fall of 2020 already, so there would have been plenty of time to update. One can only wonder why the authors chose not to, so it is interesting to see what happens when the publicly reported data from the following trials is taken into account: -ORCHID trial -DISCOVERY trial -updated HYDRA trial -NCT04391127 -NCT04358081 Now, even suboptimal HKSJ method no longer yields a significant difference for the test of overall effect (p=0.08). Interestingly, there is weak evidence for a dose-response effect: the test for subgroup difference ( low dose against  high dose ) gives p=0.04, compared to p=0.29 reported in the paper. To summarize, I could say that the strong conclusions only rely on the analysis not being up-to-date and using a suboptimal (in that context) method to estimate the confidence interval. The incorrect inclusion of the Brazilian trial in the chloroquine meta-analyis also suggests lack of rigor from both authors and Nature s reviewers. , 
 Precision to the previous comment: HKSJ gives anticonservative CIs **when the heterogeneity is small**, as is the case here. The reported tau^2 is 0. In that case using HK SJ without the ad hoc modification is questionable since it gives CIs that are even narrower than fixed-effect model. , 
 In light of the above, I would like to ask the authors to please justify clarify the following points: 1) Including a study in the CQ meta-analysis that does not meet the inclusion criteria. Is there any scientific reason not to publish an erratum to correct the error? 2) What were the stopping rules for updating the meta-analysis? The paper was submitted late september, updated in October with results from the solidarity trial, but not thereafter. Why not e.g. update the analysis with the orchid trial published early november? 3) How to justify scientifically the use of the Hartung-Knapp method without modification in a situation with tau^2=0? This is the most sensible point, as it seems the maths behind the principal result of the study (statistical association with mortality) are unsound. ,