Toward Reuse of Clinical Data for Research and Quality Improvement: The End of the Beginning?
Authors: Mark G. Weiner
Journal: Annals of Internal Medicine (2009)

Editorials1 September 2009Toward Reuse of Clinical Data for Research and Quality Improvement: The End of the Beginning?FREEMark G. Weiner, MD and Peter J. Embi, MD, MSMark G. Weiner, MDFrom University of Pennsylvania, Philadelphia, PA 19104, and University of Cincinnati, Cincinnati, OH 45267. and Peter J. Embi, MD, MSFrom University of Pennsylvania, Philadelphia, PA 19104, and University of Cincinnati, Cincinnati, OH 45267.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-151-5-200909010-00141 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Today's health information technology (HIT) landscape provides an unprecedented convergence of comprehensive electronic health records, robust computational processing power, data-sharing capabilities, and emerging financial incentives that favor the widespread adoption and meaningful use of such systems. Even as the primary value proposition for HIT adoption by clinical practices remains a matter of debate (1), 2 articles in this issue (2, 3) help to illustrate the potential for secondary benefits by discussing the value of distributed health data networks for improving research and health care quality.The multiple goals of population health, pharmaceutical surveillance, comparative effectiveness research, and other major initiatives being advanced to address the needs of our health care and research enterprises have created a growing need for access to high-quality, patient-level health information. One approach to enabling such access is the creation of centralized repositories to which data can be transferred and then readily accessed to answer questions, an approach that certainly has its merits. Alternatively, the distributed health data networks described in this issue can be designed with appropriate methods, policies, and systems to enable access and use of data housed in their original, disparate locations. As Maro and colleagues point out (2), a distributed approach has potential advantages—given today's regulatory, economic, and cultural realities—and the technological capabilities to enable such an approach exist today. Why, then, do we not see more widespread use of such systems?The history of HIT successes and failures indicates that the technological challenges involved often pale in comparison with the many socio-organizational issues that must be understood and addressed to enable HIT advances (4). Any visitor to a local technology superstore knows how easy and inexpensive it is to collect and store vast amounts of data—the equivalent of thousands of patient records—in a device that fits as easily in a shirt pocket today as a 1.44-megabyte floppy disk did in 1990. However, the ease of information dissemination this capability enables appropriately raises personal privacy and intellectual property concerns. Adequate funding is another important factor to the success of HIT initiatives; however, funding alone does not guarantee success. One need only look at the many recent examples of failed regional health information organizations—and their successful counterparts—to appreciate the importance of effective governance structures, regulatory policies, and properly aligned organizational incentives in establishing and sustaining an effective distributed health data network (5).In addition to these prerequisites, developers and those who attempt to leverage HIT resources have recognized even more fundamental issues inherent to reusing clinical data. Although systems to enable clinical research from large health information collections have been around for some time (6–9), they have often been criticized because the quality and comprehensiveness of the clinical data were not up to research standards or the analytical methods used to overcome these limitations were inadequate to overcome systematic biases inherent to data collected primarily for clinical care (10–12).Has data quality improved since the time of these earlier systems? In recent years, increased data capture in electronic systems, improvements in the speed and standardization of data transfers between systems, and the ability to leverage data from multiple clinical sources (such as objective test results or therapeutic information) have reduced the oft-criticized reliance on administrative and billing data sources. However, no purely technical solution can overcome the capture of inaccurate information by the user of a clinical information system. As such, nontechnical innovations that help improve the accuracy of recorded information and incentivize consistently accurate data collection are critical to the success of research initiatives that rely on the presence of such data.Is data capture more comprehensive now than in the earlier systems? Comprehensive data capture all relevant exposures and outcomes, features that are essential to meaningful comparative effectiveness assessments. Within a single health system, improved technology has facilitated the automated capture of more comprehensive data in electronic form, which minimizes the need for laborious manual data extraction from paper charts. However, many patients receive care across several health systems and alternative health care settings, such that a single, comprehensive, longitudinal record rarely exists for any given patient. Unfortunately, the goal of widespread information system interoperability—to enable integrated health information access that spans different health systems and vendor products—remains elusive. Achieving interoperability and ensuring adherence to common standards will be critical to the success of secondary-use initiatives (13). Without it, distributed data access could have limited value—or lead to misinterpretation of the prevalence of and relationships between exposures and outcomes.Finally, have the methods for analyzing routinely collected clinical data improved? The process of clinical care introduces treatment bias, in which the statistical association between therapy and outcome is confounded by measured and unmeasured factors that influence both the choice of treatment and the likelihood of the outcome. Instrumental variable and propensity score analyses, which have been applied for years in the social sciences, and new methods, such as prior-event rate ratios (14), are increasingly being applied in the medical domain to overcome treatment biases, with variable degrees of success and acceptance. Understanding the clinical circumstances and types of research questions for which these methods may yield valid results—perhaps even consistent with those of traditional clinical studies—requires simulations, sensitivity analyses, and validation against the findings of randomized trials.The promises of our current HIT environment are clearly great and growing. The efforts of biomedical informaticians, health services researchers, biostatisticians, and others have significantly advanced our knowledge of how to collect, organize, retrieve, analyze, and apply health data to improve individual patient care, as well as for such additional purposes as population health and biomedical research. However, significant collaborative effort by many of the stakeholders involved, including health care institutions and clinicians, HIT vendors, researchers, informaticians, regulators and policymakers, payers, and patients, is required to realize the full promise of these resources. If history is any indication, fostering and nurturing this collaboration will be challenging and take some time. Nevertheless, as the 2 articles in this issue help illustrate, current technology and existing models of success have put us in a better position today than we have been in before to realize the promises of HIT to advance research and create a safer and more efficient health care enterprise. To quote Sir Winston Churchill, "This is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning."Mark G. Weiner, MDUniversity of PennsylvaniaPhiladelphia, PA 19104Peter J. Embi, MD, MSUniversity of CincinnatiCincinnati, OH 45267References1. Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med. 2009;360:1477-9. [PMID: 19321856] CrossrefMedlineGoogle Scholar2. Maro JC, Platt R, Holmes JH, Strom BL, Hennessy S, Lazarus R, et al. Design of a national distributed health data network. Ann Intern Med. 2009;151:341-4. LinkGoogle Scholar3. Pace WD, Cifuentes M, Valuck RJ, Staton EW, Brandt EC, West DR. An electronic practice-based network for observational comparative effectiveness research. Ann Intern Med. 2009;151:338-40. LinkGoogle Scholar4. Kaplan B, Harris-Salamone KD. Health IT success and failure: recommendations from literature and an AMIA workshop. J Am Med Inform Assoc. 2009;16:291-9. [PMID: 19261935] CrossrefMedlineGoogle Scholar5. Adler-Milstein J, McAfee AP, Bates DW, Jha AK. The state of regional health information organizations: current activities and financing. Health Aff (Millwood). 2008;27:w60-9. [PMID: 18073225] MedlineGoogle Scholar6. Johnson SB, Hripcsak G, Chen J, Clayton P. Accessing the Columbia Clinical Repository. Proc Annu Symp Comput Appl Med Care. 1994;:281-5. [PMID: 7949935] MedlineGoogle Scholar7. Steib SA, Reichley RM, McMullin ST, Marrs KA, Bailey TC, Dunagan WC, et al. Supporting ad-hoc queries in an integrated clinical database. Proc Annu Symp Comput Appl Med Care. 1995;:62-6. [PMID: 8563360] MedlineGoogle Scholar8. Tierney WM, McDonald CJ. Practice databases and their uses in clinical research. Stat Med. 1991;10:541-57. [PMID: 2057654] CrossrefMedlineGoogle Scholar9. Safran C, Porter D, Lightfoot J, Rury CD, Underhill LH, Bleich HL, et al. ClinQuery: a system for online searching of data in a teaching hospital. Ann Intern Med. 1989;111:751-6. [PMID: 2802433] LinkGoogle Scholar10. Hlatky MA. Using databases to evaluate therapy. Stat Med. 1991;10:647-52. [PMID: 2057662] CrossrefMedlineGoogle Scholar11. Byar DP. Problems with using observational databases to compare treatments. Stat Med. 1991;10:663-6. [PMID: 2057663] CrossrefMedlineGoogle Scholar12. McDonald CJ, Hui SL. The analysis of humongous databases: problems and promises. Stat Med. 1991;10:511-8. [PMID: 2057652] CrossrefMedlineGoogle Scholar13. Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC, et al. Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc. 2007;14:1-9. [PMID: 17077452] CrossrefMedlineGoogle Scholar14. Tannen RL, Weiner MG, Xie D. Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings. BMJ. 2009;338:b81. [PMID: 19174434] CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAffiliations: From University of Pennsylvania, Philadelphia, PA 19104, and University of Cincinnati, Cincinnati, OH 45267.Grant Support: Dr. Weiner's contributions were supported in part by the Agency for Healthcare Quality and Research (5-R18-HS-017099-02, "Crossing the Quality Assessment Chasm: Aligning Measured and True Quality of Care") and the National Center for Research Resources, National Institutes of Health (1-UL1-RR-024134-04, Institutional Clinical and Translational Science Award). Dr. Embi's contributions were supported in part by the National Center for Research Resources, National Institutes of Health (1-UL1-RR-026314-01, Institutional Clinical and Translational Science Award) and the National Library of Medicine, National Institutes of Health (1-R01-LM-009533-01A1, "Evaluating EHR-based, Point-of-Care Trial Recruitment Across Clinical Settings").Disclosures:Grants received: M.G. Weiner (National Heart, Lung, and Blood Institute).Corresponding Author: Mark G. Weiner, MD, University of Pennsylvania, 423 Guardian Drive, Room 1116, Philadelphia, PA 19104; e-mail, [email protected]med.upenn.edu.Current Author Addresses: Dr. Weiner: University of Pennsylvania, 423 Guardian Drive, Room 1116, Philadelphia, PA 19104.Dr. Embi: University of Cincinnati, 231 Albert Sabin Way, Box 670840, Cincinnati, OH 45267. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoAn Electronic Practice-Based Network for Observational Comparative Effectiveness Research Wilson D. Pace , Maribel Cifuentes , Robert J. Valuck , Elizabeth W. Staton , Elias C. Brandt , and David R. West Design of a National Distributed Health Data Network Judith C. Maro , Richard Platt , John H. Holmes , Brian L. Strom , Sean Hennessy , Ross Lazarus , and Jeffrey S. Brown Metrics Cited byTemporal relationship of computed and structured diagnoses in electronic health record dataMulti-Center Healthcare Data Quality Measurement Model and Assessment Using OMOP CDMResearch Integrated Network of Systems (RINS): a virtual data warehouse for the acceleration of translational researchThe Indiana Learning Health System Initiative: Early experience developing a collaborative, regional learning health systemAssessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experienceUse of electronic health records to support a public health response to the COVID-19 pandemic in the United States: a perspective from 15 academic medical centersTheme Identification for Linked Medical DataLeveraging Real-World Data for the Selection of Relevant Eligibility Criteria for the Implementation of Electronic Recruitment Support in Clinical TrialsFactors Affecting the Quality of Person-Generated Wearable Device Data and Associated Challenges: Rapid Systematic ReviewUsing Electronic Medical Record Data for Research in a Healthcare Information and Management Systems Society (HIMSS) Analytics Electronic Medical Record Adoption Model (EMRAM) Stage 7 Hospital in Beijing: Cross-sectional StudySerious Mental Illness and Health-Related Factors Associated with Regional Emergency Department UtilizationComparison of physical examination and laboratory data between a clinical study and electronic health recordsComputable Phenotype Implementation for a National, Multicenter Pragmatic Clinical TrialIdentifying Ectopic Pregnancy in a Large Integrated Health Care Delivery System: Algorithm ValidationEvaluating the demographic and clinical minimum data sets of Iranian National Electronic Health RecordAutomating the Capture of Structured Pathology Data for Prostate Cancer Clinical Care and ResearchElectronic health records in IS research: Quality issues, essential thresholds and remedial actionsA tale of three subspecialties: Diagnosis recording patterns are internally consistent but Specialty-DependentThe seven key challenges for the future of computer-aided diagnosis in medicineBarriers and facilitators to data quality of electronic health records used for clinical research in China: a qualitative studyChallenges and Opportunities for Using Big Health Care Data to Advance Medical Science and Public HealthImplementing a Novel Quality Improvement-Based Approach to Data Quality Monitoring and Enhancement in a Multipurpose Clinical RegistryDataGauge: A Practical Process for Systematically Designing and Implementing Quality Assessments of Repurposed Clinical DataDeveloping a Standardization Algorithm for Categorical Laboratory Tests for Clinical Big Data Research: Retrospective StudyUnlocking the Power of Artificial Intelligence and Big Data in MedicineMultidisciplinary teams and ICT: a qualitative study exploring the use of technology and its impact on multidisciplinary team meetingsResearch IT maturity models for academic health centers: Early development and initial evaluationIntelligent and effective informatic deconvolution of "Big Data" and its future impact on the quantitative nature of neurodegenerative disease therapyDatabase Quality and Access Issues Relevant to Research Using Anesthesia Information Management System DataAssessing privacy risks in population health publications using a checklist-based approachCommunity Health Worker Impact on Chronic Disease Outcomes Within Primary Care Examined Using Electronic Health RecordsA Shared Opportunity for Improving Electronic Medical Record DataAnonymization for outputs of population health and health services research conducted via an online data centerThe Learning Healthcare System and Cardiovascular Care: A Scientific Statement From the American Heart AssociationFeasibility of automatic evaluation of clinical rules in general practiceRoutinedaten – das ungenutzte Potenzial in der VersorgungsforschungAutonomous Decentralized Privacy-Enabled Data Preparation Architecture for Multicenter Clinical Observational ResearchA Preparation Framework for EHR Data to Construct CBR Case-BaseDevelopment of a Web-Based Clinical Decision Support System for Drug Prescription: Non-Interventional Naturalistic Description of the Antipsychotic Prescription Patterns in 4345 Outpatients and Future ApplicationsUsing primary care electronic health record data for comparative effectiveness research: experience of data quality assessment and preprocessing in The NetherlandsIUD services among primary care practices in New York CityHealth Information Exchange in Emergency MedicineData Governance and Strategies for Data IntegrationConstruction of quality-assured infant feeding process of care data repositories: definition and design (Part 1)Computer-supported feedback message tailoring: theory-informed adaptation of clinical audit and feedback for learning and behavior changeIndividual privacy versus public good: protecting confidentiality in health researchDesign and Implementation of a Privacy Aware Framework for Sharing Electronic Health RecordsAn integrated, ontology-driven approach to constructing observational databases for researchBuilding bridges across electronic health record systems through inferred phenotypic topicsBenefits and Risks in Secondary Use of Digitized Clinical Data: Views of Community Members Living in a Predominantly Ethnic Minority Urban NeighborhoodOntology-Based Data Integration between Clinical and Research SystemsFuture Directions for Translational InformaticsUsing patient lists to add value to integrated data repositoriesTraits and types of health data repositoriesExtension of the primary care research object model (PCROM) as clinical research information model (CRIM) for the learning healthcare systemThe linked medical data access control frameworkThe Use and Misuse of Biomedical Data: Is Bigger Really Better?Improving Case-Based Reasoning Systems by Combining K-Nearest Neighbour Algorithm with Logistic Regression in the Prediction of Patients' Registration on the Renal Transplant Waiting ListMeasure, Report, Improve: The Quest for Best Practices for High-Quality Care in Critical Care TransportCaveats for the Use of Operational Electronic Health Record Data in Comparative Effectiveness ResearchComputerized provider documentation: findings and implications of a multisite study of clinicians and administrators: Table 1Enabling Genomic-Phenomic Association Discovery without Sacrificing AnonymityMethods and dimensions of electronic health record data quality assessment: enabling reuse for clinical researchUsing EHRs to integrate research with patient care: promises and challenges: Table 1Practices and perspectives on building integrated data repositories: results from a 2010 CTSA surveyInformatics Support of Outcome MeasuresResearch Patient Data WarehousingA Database De-identification Framework to Enable Direct Queries on Medical Data for Secondary UseElectronic Medical Records (EMRs), Epidemiology, and Epistemology: Reflections on EMRs and Future Pediatric Clinical ResearchMethods for monitoring and prognosis of clinical status of patients in acute phase of myocardial infarction for computer network based clinical decision support systemA Case-Based Retrieval System Using Natural Language Processing and Population-Based VisualizationABJS Carl T. Brighton Workshop on Health Informatics in Orthopaedic Surgery: Editorial CommentEvaluating re-identification risks with respect to the HIPAA privacy rule 1 September 2009Volume 151, Issue 5Page: 359-360KeywordsClinical trialsElectronic medical recordsHealth careHealth care quality assessmentHealth informaticsHealth information technologyInformation storage and retrievalInformation technologyPatientsResearch quality assessment ePublished: 1 September 2009 Issue Published: 1 September 2009 Copyright & PermissionsCopyright © 2009 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...