Introduction: Improvement and Measurement in the Era of Electronic Health Records
Authors: Andrew Auerbach, David W. Bates
Journal: Annals of Internal Medicine (2020)

Supplement2 June 2020Introduction: Improvement and Measurement in the Era of Electronic Health RecordsFREEAndrew Auerbach, MD, MPH and David W. Bates, MD, MScAndrew Auerbach, MD, MPHUniversity of California, San Francisco, San Francisco, California (A.A.)Search for more papers by this author and David W. Bates, MD, MScBrigham and Women's Hospital, Boston, Massachusetts (D.W.B.)Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/M19-0870 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Since the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act (1), health care has undergone a sea change with the widespread implementation of electronic health records (EHRs). In the decade since HITECH, EHRs are now nearly universal throughout the United States, producing innumerable implications for care delivery. Changes in care delivery are also producing dramatic effects on the ways in which quality improvement is carried out and measured. However, how improvement and measurement efforts in the EHR era are carried out may not be keeping pace with improvements in how local improvement programs can create generalizable knowledge—a step that requires consideration of not only the rigor of the implementation, but also the rigor of with which the improvement is reported.Some EHR-enabled approaches to improving health care, such as machine learning and artificial intelligence, offer great promise yet do not necessarily work well with traditional methodologies of evaluation and reporting. Other improvement efforts, such as those targeting costs and value, may benefit from the availability of new implementation tools that have otherwise been difficult to incorporate into paper-based implementation strategies. For nearly all kinds of research in the EHR era, the role of vendor-developed EHR tools—both their opportunities and constraints—underlie how care improvements are framed and executed.With this supplement, we hope to help guide those carrying out improvement and innovation efforts in the EHR era toward more robust approaches that will in turn provide more robust results. We hope to aid those same investigators and innovators by providing guidance as to how to describe their findings in ways that are rigorous and understandable. And we hope that the combination of these first 2 goals leads to the third: the framing of a more useful and implementable body of literature about how to implement change in the digital era.We have solicited a group of papers that cover key issues in translating research into the digital era, focusing on cross-cutting issues in the new era (Table). With each paper, we hope to answer such questions as: What are the factors that define high-quality research of this type? What should researchers be attentive to when designing, conducting, and reporting their work? What should clinicians, administrators, patients, and policymakers be looking for as they consider the application of the results of such research into their own endeavors?Table. Topics Addressed in This SupplementTopics Covered in the SupplementContext and ApproachIn a paper on context and approach in reporting evaluations of EHR-based implementation by Haynes and colleagues (2), the authors make the point that understanding the local, nonelectronic environment for research can be as important as the intervention itself. This article also covers key issues in EHR-oriented evaluation more broadly.Observational StudiesObservational studies are a cornerstone of learning about patients, epidemiology of diseases, and patterns of care delivery; EHRs bring new opportunities for this type of research. Callahan and associates (3) argue that EHRs enable gathering of broad and complicated data sets that provide the opportunity to develop ongoing cohorts, many of which were not previously possible, as well as to carry out effectiveness studies in the “real world.” The authors point out parallels to what is known about observational data from non-EHR eras, as well as key new considerations—such as clear definitions of a “computable phenotype” of patient cohorts.Randomized TrialsRandomized trials are the gold-standard approach for determining whether one intervention works better than another, but they have traditionally been expensive, time-consuming, and far removed from care delivery. However, EHRs can address these considerations—with them, trials can often be done more quickly and at a fraction of the cost traditionally required. Pletcher and colleagues (4) offer some examples of this phenomenon and examine randomized trials of EHR-based interventions and how they should be reported. Many of the basic precepts are the same as for randomized controlled trials in general, but there are also new wrinkles.Safety and Quality ImprovementSingh and Sittig (5) offer a framework for studies focusing on safety and quality improvement by using EHRs. They provide an approach to designing and reporting studies that focus on quality and safety as their target outcomes, and they emphasize the importance of individual sociotechnical issues in developing and reporting EHR-based quality improvement and safety studies.Clinical Decision SupportClinical decision support (CDS) is arguably the most important intervention type enabled by EHRs, though its effectiveness has been highly variable. Kawamoto and McDonald (6) suggest that a substantial proportion of EHR-based interventions will involve forms of CDS to influence care and discuss some of the ways CDS can have more impact. This paper also provides a CDS publication checklist, which will help frame the design of studies using CDS as part of their research plan.Standardization Versus CustomizationOne of the key tensions in informatics and EHRs is standardization versus customization. If you standardize, it is much easier to spread widely, but if you customize, local uptake and performance may be better. If we are to get benefit from EHRs, it will be critical to become better at managing this and then reporting it in research evaluations. Wright and coworkers (7) review this issue and suggest that translation of interventions from one site to another often requires adapting and customizing them at the translation step. The authors also discuss how papers might provide information about opportunities to achieve such translation effectively.Workflow and WorkaroundsA pivotal issue with EHRs has been utilizing them effectively; if they are hard to use, require substantial extra time, and do not support improvement, they can both create provider burnout and fail to improve quality, safety, or value. Studying workflow and workarounds in EHR-supported work thus represents a critical frontier, which Zheng and colleagues (8) consider. Ironically, although EHRs collect streams of data that enable detailed analysis of workflow, these data are often unavailable to researchers. Unraveling these issues represent a key to realizing system improvement.Patient EngagementThere is broad agreement that patients should become more involved with their care, and EHRs offer powerful tools for improving engagement, including personal health records and capture of patient-reported outcomes measures among others, yet realizing of these approaches is still nascent. Lyles and coworkers (9) discuss issues around understanding patient engagement in health using EHRs, and how studies that address this should be reported.ValueThe burning platform in U.S. health care today is its poor value: Our costs are twice as high as those of the rest of the world. But EHRs offer tremendous insights into both the value of what is being done, and how it could be improved. Rudin and colleagues (10) discuss how EHRs can be leveraged to improve value and cover key issues around research and value using EHRs.Artificial Intelligence and Machine LearningOne of the most exciting technologies around care improvement is artificial intelligence, and medicine has lagged other industries in adopting it. However, that is changing quickly, and EHRs will undoubtedly enable this transition. Bates and associates (11) go through some of the challenges that artificial intelligence creates—such as models that are not static but evolve in real time—and how interventions involving machine learning and artificial intelligence should be reported.Broadly, EHRs offer great opportunity to improve safety, quality, and value, but many of these benefits are not yet being realized. Some of the possibilities and precepts are like those in the pre-EHR world, but many have changed—specifically, cohort identification, doing large numbers of practical randomized trials, CDS, opportunities for patient engagement, and leveraging such technologies as artificial intelligence are fundamentally different. We believe that addressing the issues raised in this supplement and performing the necessary research to get to better care is critical.Key Research and Reporting NeedsAs a group, these papers touch on issues that are common in research and care improvement, but 2 critical research and reporting needs stand out. The first is a need for transparency in reporting how electronic tools have been constructed. Suggested approaches include screenshots and visual representations of the tools themselves, as well as sharing of code and data models (particularly when a digital tool is an algorithm or machine-learning model). Transparency, or open-source initiatives, could be considered a potential hazard to innovators or companies whose intellectual property might be disclosed or intuited based on the information available, but this risk might be mitigated by the benefits of improved recognition of their products, and additional market access. Greater transparency and ease of adoption from a technical standpoint may provide substantial advantages. Many EHR vendors have resisted sharing screenshots, even when a specific issue has caused safety concerns, but federal regulations in the 21st Century Cures Act (12) forbid gag clauses that have been present in some vendor contracts and to this point have prohibited discussion and demonstration of problems in vendor systems.The second key issue is that investigators and innovators need to provide enough contextual information in their publications to understand whether the digital research methods might be tested or adopted at another site. In addition to traditional guidelines, such as SQUIRE (Standards for QUality Improvement Reporting Excellence) 2.0 (13), in the digital era this level of context should include not only technical features but also factors critical to adding any new digital tool to clinical care, such as where the tool fits in existing workflows, how introduction of the tool was governed and managed, and the social and technical milieu of the health care setting. Context, user-engagement approaches, and how the tools might or should be tailored to individual systems, will be critical almost regardless of whether screenshots or source code is shared freely. Understanding the interaction among context, effective use, and outcomes will be more practical to study in the digital era (especially as many human activities are already tracked by electronic systems).Addressing these 2 key themes will catalyze knowledge generation by using methods that are greatly enabled in the digital era, such as the use of data for observational research studies or randomized trials. Research involving and encouraging patients in the use of digital health tools are strongly dependent on issues that lie outside the digital tools themselves. Use of artificial intelligence and machine learning, and studies of how work is carried out in electronic work spaces, are also dependent on providing adequate technical and contextual information for the studies to be interpreted and adopted elsewhere. The need for improved interoperability of health data is critical (14), but understanding and utilizing newly interoperable data will require attention to many of the factors discussed in the specific areas examined by this supplement.The widespread availability of digital health tools in the United States provides a generational opportunity for groundbreaking innovation in health care delivery. The digital transformation of the delivery system also provides an opportunity to affect how we carry out research and discovery. The practicality of randomizing interventions by using EHRs also decreases the costs of data collection by making them a byproduct of routine care. Increasing the generalizability of research methods and results will be further catalyzed by improvements in a health information technology exchange, particularly if the context of the users and sites are accounted for in study design and interpretation. These papers provide an overview of the opportunities available and help map the way forward.References 1. Department of Health and Human Services, HITECH Act. Accessed at https://www.hhs.gov/sites/default/files/ocr/privacy/hipaa/understanding/coveredentities/hitechact.pdf on 10 March 2019. Google Scholar2. Haynes RB, del Fiol G, Michelson M, et al. Context and approach in reporting evaluations of electronic health record–based implementation projects. Ann Intern Med. 2020;172:S73-S78. doi:10.7326/M19-0874 LinkGoogle Scholar3. Callahan A, Shah NH, Chen JH. Research and reporting considerations for observational studies using electronic health record data. Ann Intern Med. 2020;172:S79-S84. doi:10.7326/M19-0873 LinkGoogle Scholar4. Pletcher MJ, Flaherman V, Najafi N, et al. Randomized controlled trials of electronic health record interventions: design, conduct, and reporting considerations. Ann Intern Med. 2020;172:S85-S91. doi:10.7326/M19-0877 LinkGoogle Scholar5. Singh H, Sittig DF. A sociotechnical framework for safety-related electronic health record research reporting: the SAFER reporting framework. Ann Intern Med. 2020;172:S92-S100. doi:10.7326/M19-0879 LinkGoogle Scholar6. Kawamoto K, McDonald CJ. Designing, conducting, and reporting clinical decision support studies: recommendations and call to action. Ann Intern Med. 2020;172:S101-S109. doi:10.7326/M19-0875 LinkGoogle Scholar7. Wright A, McCoy AB, Choudhry NK. Recommendations for the conduct and reporting of research involving flexible electronic health record–based interventions. Ann Intern Med. 2020;172:S110-S115. doi:10.7326/M19-0880 LinkGoogle Scholar8. Zheng K, Ratwani RM, Adler-Milstein J. Studying workflow and workarounds in electronic health record–supported work to improve health system performance. Ann Intern Med. 2020;172:S116-S122. doi:10.7326/M19-0871 LinkGoogle Scholar9. Lyles CR, Nelson EC, Frampton S, et al. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med. 2020;172:S123-S129. doi:10.7326/M19-0876 LinkGoogle Scholar10. Rudin RS, Friedberg MW, Shekelle P, et al. Getting value from electronic health records: research needed to improve practice. Ann Intern Med. 2020;172:S130-S136. doi:10.7326/M19-0878 LinkGoogle Scholar11. Bates DW, Auerbach A, Schulam P, et al. Reporting and implementing interventions involving machine learning and artificial intelligence. Ann Intern Med. 2020;172:S137-S144. doi:10.7326/M19-0872 LinkGoogle Scholar12. 21st Century Cures Act. HealthIT.gov. Accessed at www.healthit.gov/topic/laws-regulation-and-policy/notice-proposed-rulemaking-improve-interoperability-health on 10 March 2019. Google Scholar13. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25:986-92. [PMID: 26369893] doi:10.1136/bmjqs-2015-004411 CrossrefMedlineGoogle Scholar14. Washington V, DeSalvo K, Mostashari F, et al. The HITECH era and the path forward. N Engl J Med. 2017;377:904-6. [PMID: 28877013] doi:10.1056/NEJMp1703370 CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAffiliations: University of California, San Francisco, San Francisco, California (A.A.)Brigham and Women's Hospital, Boston, Massachusetts (D.W.B.)Acknowledgment: The authors thank Sharon Donovan for her expert and tireless work in managing this supplement from conception to publication.Financial Support: By the Gordon and Betty Moore Foundation, Palo Alto, California.Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M19-0870.Corresponding Author: Andrew Auerbach, MD, MPH, University of California, San Francisco, 533 Parnassus Avenue, San Francisco, CA 94117.Current Author Addresses: Dr. Auerbach: University of California, San Francisco, 533 Parnassus Avenue, San Francisco, CA 94117.Dr. Bates: Division of General Internal Medicine, Brigham and Women's Hospital, 1620 Tremont Street, Boston, MA 02115.This article is part of the Annals supplement “Implementing, Studying, and Reporting Health System Improvement in the Era of Electronic Health Records.” The Moore Foundation (contract number 7107) provided funding for publication of this supplement. Andrew D. Auerbach, MD, MPH (University of California, San Francisco); David W. Bates, MD, MSc (Brigham and Women's Hospital, Harvard Medical School, and Harvard School of Public Health); Jaya K. Rao, MD, MHS (Annals Deputy Editor); and Christine Laine, MD, MPH (Annals Editor in Chief), served as editors for this supplement. Nextarticle Advertisement FiguresReferencesRelatedDetails Metrics Cited byFactors associated with smoking cessation attempts in a public, safety-net primary care systemSharing Data 2 June 2020Volume 172, Issue 11_SupplementPage: S69-S72KeywordsArtificial intelligenceElectronic medical recordsHealth careHealth information technologyMachine learningQuality improvementResearch reporting guidelinesSafetySafety studies ePublished: 2 June 2020 Issue Published: 2 June 2020 Copyright & PermissionsCopyright © 2020 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...