Seeing the Positive in Negative Studies
Authors: Eliseo Guallar, Steven N. Goodman, A. Russell Localio, Alisa J. Stephens-Shields, Christine Laine
Journal: Annals of Internal Medicine (2023)

Understanding Clinical ResearchApril 2023Seeing the Positive in Negative StudiesEliseo Guallar, MD, DrPH, Steven N. Goodman, MD, PhD, A. Russell Localio, PhD, Alisa J. Stephens-Shields, PhD, and Christine Laine, MD, MPHEliseo Guallar, MD, DrPH Annals of Internal Medicine, Philadelphia, Pennsylvania, and Departments of Epidemiology and Medicine, and Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (E.G.), Steven N. Goodman, MD, PhD Annals of Internal Medicine, Philadelphia, Pennsylvania, and Department of Epidemiology and Population Health and Department of Medicine, Stanford University School of Medicine, Stanford, California (S.N.G.), A. Russell Localio, PhD Annals of Internal Medicine and Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania (A.R.L., A.J.S.), Alisa J. Stephens-Shields, PhD Annals of Internal Medicine and Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania (A.R.L., A.J.S.), and Christine Laine, MD, MPH Annals of Internal Medicine, Philadelphia, Pennsylvania (C.L.).Author, Article, and Disclosure Informationhttps://doi.org/10.7326/M23-0576 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Identifying ineffective health interventions can be as important as identifying effective ones (1). Consequently, Annals is interested in publishing "negative" studies. We base our editorial decisions not on the "positivity" of findings (that is, their statistical significance) but on the importance of the research question and methodological quality.Unfortunately, few negative studies provide strong evidence for treatment ineffectiveness (2). To do so, they must have results statistically compatible only with a narrow range of small effects. If we can exclude sources of bias and agree that clinically important effects lie outside this range, we can conclude that the intervention is ...References1. Borenstein M, Hedges LV, Higgins JPT, et al. Publication bias. In: Borenstein M, Hedges LV, Higgins JPT, et al, eds. Introduction to Meta-Analysis. J Wiley; 2009:277-92. Google Scholar2. Liao JM, Stack CB, Goodman S. Annals Understanding Clinical Research: interpreting results with large P values. Ann Intern Med. 2018;169:485-486. [PMID: 30208404] doi:10.7326/M18-2003 LinkGoogle Scholar3. Wang TY, Wahed AS, Morris A, et al. Effect of thromboprophylaxis on clinical outcomes after COVID-19 hospitalization. Ann Intern Med. 2023;176:515-523. doi:10.7326/M22-3350 LinkGoogle Scholar4. Spyropoulos AC, Ageno W, Albers GW, et al. Rivaroxaban for thromboprophylaxis after hospitalization for medical illness. N Engl J Med. 2018;379:1118-1127. [PMID: 30145946] doi:10.1056/NEJMoa1805090 CrossrefMedlineGoogle Scholar5. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed). 1986;292:746-750. [PMID: 3082422] doi:10.1136/bmj.292.6522.746 CrossrefMedlineGoogle Scholar6. Hayes RJ, Moulton LH. Cluster Randomised Trials. 2nd ed. CRC Pr; 2017. Google Scholar7. Freiman JA, Chalmers TC, Smith H, et al. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial—survey of 71 "negative" trials. N Engl J Med. 1978;299:690-694. [PMID: 355881] doi:10.1056/NEJM197809282991304 CrossrefMedlineGoogle Scholar8. Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311:485. [PMID: 7647644] doi:10.1136/bmj.311.7003.485 CrossrefMedlineGoogle Scholar9. Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. The American Statistician. 2016;70:129-133. CrossrefGoogle Scholar10. Petitti DB, Teutsch SM, Barton MB, et al. Update on the methods of the U.S. Preventive Services Task Force: insufficient evidence. Ann Intern Med. 2009;150:199-205. [PMID: 19189910] doi:10.7326/0003-4819-150-3-200902030-00010 LinkGoogle Scholar Author, Article, and Disclosure InformationAuthors: Eliseo Guallar, MD, DrPH; Steven N. Goodman, MD, PhD; A. Russell Localio, PhD; Alisa J. Stephens-Shields, PhD; Christine Laine, MD, MPHAffiliations: Annals of Internal Medicine, Philadelphia, Pennsylvania, and Departments of Epidemiology and Medicine, and Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (E.G.) Annals of Internal Medicine, Philadelphia, Pennsylvania, and Department of Epidemiology and Population Health and Department of Medicine, Stanford University School of Medicine, Stanford, California (S.N.G.) Annals of Internal Medicine and Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania (A.R.L., A.J.S.) Annals of Internal Medicine, Philadelphia, Pennsylvania (C.L.).Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-0576.Corresponding Author: Eliseo Guallar, MD, DrPH, Johns Hopkins Bloomberg School of Public, Welch Center for Prevention, Epidemiology, and Clinical Research, Room 2-645, Baltimore, MD 21205; e-mail, [email protected]edu.Author Contributions: Conception and design: E. Guallar.Drafting of the article: E. Guallar.Critical revision of the article for important intellectual content: S.N. Goodman, E. Guallar, C. Laine, A.R. Localio, A.J. Stephens-Shields.Final approval of the article: S.N. Goodman, E. Guallar, C. Laine, A.R. Localio, A.J. Stephens-Shields.Statistical expertise: S.N. Goodman, E. Guallar, A.R. Localio, A.J. Stephens-Shields.This article was published at Annals.org on 21 March 2023. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetails Metrics April 2023Volume 176, Issue 4Page: 561-562KeywordsClinical epidemiologyClinical trial reportingConfidence intervalsResearch reporting guidelines ePublished: 21 March 2023 Issue Published: April 2023 Copyright & PermissionsCopyright © 2023 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...