Designed Learning: Missing Data in Clinical Research
Authors: Catharine B. Stack, Trevor Butterworth, Rebecca Goldin
Journal: Annals of Internal Medicine (2018)

Editorials15 May 2018Designed Learning: Missing Data in Clinical ResearchFREECatharine B. Stack, PhD, Trevor Butterworth, BA, MPhil, MS, and Rebecca Goldin, PhDCatharine B. Stack, PhDDeputy Editor, Statistics Annals of Internal Medicine (C.B.S.), Trevor Butterworth, BA, MPhil, MSSense About Science USA, Alexandria, Virginia (T.B.), and Rebecca Goldin, PhDGeorge Mason University, Fairfax, Virginia (R.G.)Author, Article, and Disclosure Informationhttps://doi.org/10.7326/M18-0534 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Countless articles and textbooks have been written on the topic of missing data. In 2010, the National Research Council published recommendations for the prevention and treatment of missing data in clinical trials, which were developed by a panel of experts assembled by the U.S. Food and Drug Administration (1). However, reviews on the topic provide evidence that learning is slow and the effect of these written resources on published articles is hardly noticeable (2, 3). Many factors contribute to this problem. One solution could involve creating new mechanisms to deliver the information directly to the communities of scientists at the front lines. In our continued effort to explain analytic methods to clinical researchers in new ways (4, 5), we joined forces with Sense About Science USA to communicate concepts of missing data through graphic design. Sense About Science USA enlisted the data visualization expertise of Accurat. The product, "Missing Data" (http://labs.annals.org/missingdata), is an early result of our collaboration."Missing Data" combines illustrations of key concepts (provided in the left pane of the screen) with explanatory text (in the right pane). Material updates as users move down the page, and users can also jump to specific sections by clicking on the outline in the center of the page. White vertical lines denote observed data items, and red forward slashes represent missing data items. This interactive Web site transitions from instructive scenarios where researchers commonly encounter missing data to descriptions of statistical concepts, such as missing completely at random, missing at random, and missing not at random, in order to set a foundation for understanding proper analytic methods. Two case studies based on published Annals articles allow users to see how this knowledge is applied in practice (6, 7). The module explores how researchers might go wrong and explains why a simple fix will not work.Missing data are ubiquitous in clinical research, and extracting valid conclusions from incomplete data sets requires thought and knowledge. We hope this interactive guide will help clinical researchers to better understand their missing data so that they can minimize lost data when possible, handle it more optimally within analyses, carry out meaningful sensitivity analysis, and involve statisticians when needed. We also hope this module stimulates new approaches to communicating statistical concepts to researchers.References1. National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Washington, DC: National Academies Pr; 2010. doi:10.17226/12955 CrossrefGoogle Scholar2. Fielding S, Ogbuagu A, Sivasubramaniam S, MacLennan G, Ramsay CR. Reporting and dealing with missing quality of life data in RCTs: has the picture changed in the last decade? Qual Life Res. 2016;25:2977-83. [PMID: 27650288] CrossrefMedlineGoogle Scholar3. Bell ML, Fiero M, Horton NJ, Hsu CH. Handling missing data in RCTs; a review of the top medical journals. BMC Med Res Methodol. 2014;14:118. [PMID: 25407057] doi:10.1186/1471-2288-14-118 CrossrefMedlineGoogle Scholar4. Liao JM, Stack CB. Annals Understanding Clinical Research: implications of missing data due to dropout. Ann Intern Med. 2017;166:596-8. [PMID: 28241264]. doi:10.7326/M17-0195 LinkGoogle Scholar5. Talluri R, Ranadive R, Rao JK, Laine C, Griswold M. Bringing data to life: interactive visualizations of complex data. Ann Intern Med. 2017;167:820-1. [PMID: 29132157]. doi:10.7326/M17-2903 LinkGoogle Scholar6. Bronfort G, Hondras MA, Schulz CA, Evans RL, Long CR, Grimm R. Spinal manipulation and home exercise with advice for subacute and chronic back-related leg pain: a trial with adaptive allocation. Ann Intern Med. 2014;161:381-91. [PMID: 25222385]. doi:10.7326/M14-0006 LinkGoogle Scholar7. Tomasson G, Peloquin C, Mohammad A, Love TJ, Zhang Y, Choi HK, et al. Risk for cardiovascular disease early and late after a diagnosis of giant-cell arteritis: a cohort study. Ann Intern Med. 2014;160:73-80. [PMID: 24592492] LinkGoogle Scholar Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAffiliations: Deputy Editor, Statistics Annals of Internal Medicine (C.B.S.)Sense About Science USA, Alexandria, Virginia (T.B.)George Mason University, Fairfax, Virginia (R.G.)Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M18-0534.Corresponding Author: Catharine B. Stack, PhD, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106-1572; e-mail, [email protected]acponline.org.Current Author Addresses: Dr. Stack: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106-1572.Mr. Butterworth: Sense About Science USA, 732 North Washington Street, Alexandria, VA 22314.Dr. Goldin: Department of Mathematical Sciences, George Mason University, 4400 University Drive, Fairfax, VA 22030.This article was published at Annals.org on 10 April 2018. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetails Metrics Cited bySLE clinical trials: impact of missing data on estimating treatment effects 15 May 2018Volume 168, Issue 10Page: 744KeywordsClinical trialsDisclosureFood and Drug AdministrationScientistsStatistical data ePublished: 10 April 2018 Issue Published: 15 May 2018 Copyright & PermissionsCopyright © 2018 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...