Artificial intelligence approaches using natural language processing to advance EHR-based clinical research

The wide adoption of electronic health record systems in health care generates big real-world data that open new venues to conduct clinical research. As a large amount of valuable clinical information is locked in clinical narratives, natural language processing techniques as an artificial intelligence approach have been leveraged to extract information from clinical narratives in electronic health records. This capability of natural language processing potentially enables automated chart review for identifying patients with distinctive clinical characteristics in clinical care and reduces methodological heterogeneity in defining phenotype, obscuring biological heterogeneity in research concerning allergy, asthma, and immunology. This brief review discusses the current literature on the secondary use of electronic health record data for clinical research concerning allergy, asthma, and immunology and highlights the potential, challenges, and implications of natural language processing techniques. The wide adoption of electronic health record systems in health care generates big real-world data that open new venues to conduct clinical research. As a large amount of valuable clinical information is locked in clinical narratives, natural language processing techniques as an artificial intelligence approach have been leveraged to extract information from clinical narratives in electronic health records. This capability of natural language processing potentially enables automated chart review for identifying patients with distinctive clinical characteristics in clinical care and reduces methodological heterogeneity in defining phenotype, obscuring biological heterogeneity in research concerning allergy, asthma, and immunology. This brief review discusses the current literature on the secondary use of electronic health record data for clinical research concerning allergy, asthma, and immunology and highlights the potential, challenges, and implications of natural language processing techniques. Over the past decade, electronic health record (EHR) systems have been increasingly implemented at US hospitals and clinics. Thus, the role of artificial intelligence (AI) (computer-handled human task) as a tool for enhancing clinical care and research is becoming important in a broad range of clinical practices. For example, a few recent AI works in the field of respiratory disease and other clinical areas highlight feasibility of interpretation of pulmonary function test results and their associated diagnosis, early identification of asymptomatic left ventricular dysfunction from 12-lead electrocardiogram, and early detection of atypia or carcinoma in situ from breast tissue biopsy.1Topalovic M. Das N. Burgel P.R. Daenen M. Derom E. Haenebalcke C. et al.Pulmonary Function Study Investigators; Pulmonary Function Study Investigators:. Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests.Eur Respir J. 2019; 53Crossref Scopus (69) Google Scholar, 2Attia Z.I. Kapa S. Lopez-Jimenez F. McKie P.M. Ladewig D.J. Satam G. et al.Screening for cardiac contractile dysfunction using an artificial intelligence–enabled electrocardiogram.Nat Med. 2019; 25: 70-74Crossref PubMed Scopus (394) Google Scholar, 3Mercan E. Mehta S. Bartlett J. Shapiro L.G. Weaver D.L. Elmore J.G. Assessment of machine learning of breast pathology structures for automated differentiation of breast cancer and high-risk proliferative lesions.JAMA Netw Open. 2019; 2: e198777Crossref PubMed Scopus (37) Google Scholar These works may reflect only a small aspect of opportunities that EHR-based informatics research can offer in the future. Specifically, large amounts of detailed longitudinal patient information, including clinical history, laboratory tests, medications, interventions, and prognoses, are accumulated, updated, and available electronically. These large clinical databases are valuable data sources for clinical and translational research. Major initiatives have been established to exploit this crucial resource, including the Clinical and Translational Science Awards Program,4National Institutes of Health. Clinical and Translational Science Awards (CTSA): Foundation for Accelerated Discovery and Efficient Translation. 2011. Bethesda, Maryland. Available at: https://ncats.nih.gov/ctsa_2011/index.html. Accessed January 13, 2020.Google Scholar the Electronic Medical Records and Genomics Network, which links DNA biorepositories with EHRs,5Lemke A.A. Wu J.T. Waudby C. Pulley J. Somkin C.P. Trinidad S.B. Community engagement in biobanking: experiences from the eMERGE Network.Genomics Soc Policy. 2010; 6: 35-52Crossref Google Scholar the Clinical Data Research Networks Program supported by the Patient-Centered Outcomes Research Institute for comparative effectiveness research,6Fleurence R.L. Curtis L.H. Califf R.M. Platt R. Selby J.V. Brown J.S. Launching PCORnet, a national patient-centered clinical research network.J Am Med Inform Assoc. 2014; 21: 578-582Crossref PubMed Scopus (336) Google Scholar and the Observational Health Data Sciences and Informatics initiative (http://ohdsi.org/), which now contains a data network with more than 660 million subjects, all of which are normalized to the Observational Medical Outcomes Partnership Common Data Model. The great potential of these large initiatives highly depends on efficient and effective data mining approaches leading to knowledge discovery and enhancing clinical practice,7Gardner R.L. Cooper E. Haskell J. Harris D.A. Poplau S. Kroth P.J. et al.Physician stress and burnout: the impact of health information technology.J Am Med Inform Assoc. 2019; 26: 106-114Crossref PubMed Scopus (228) Google Scholar,8MedicalEconomicsEHRs should be a tool not a task. 2019.https://www.medicaleconomics.com/article/ehrs-should-be-tool-not-taskGoogle Scholar as a large amount of clinical information is locked in clinical narratives. For example, clinicians spend a significant amount of their time documenting clinical notes such as history of present illness, clinical history, narrative description on physical examination findings, pulmonary function test reports, and radiology or operative reports. Now, an online patient portal is an important platform for communication between clinicians and patients largely in free text and a valuable source for obtaining patient-reported outcomes. Experts from International Data Corporation have estimated that unstructured data accounts for more than 80% of currently available health care data.9Martin-Sanchez F. Verspoor K. Big data in medicine is driving big changes.Yearb Med Inform. 2014; 9: 14-20Crossref PubMed Scopus (109) Google Scholar Regarding this, natural language processing (NLP) techniques have been leveraged to extract information from clinical narratives for clinical research. Now, this capability of NLP systems enables AI-assisted cohort selection and/or matching from EHRs, which may make clinical trials or studies virtual or, at least, streamlined.10Department of Biomedical Informatics, Harvard Medical SchoolNational NLP Clinical Challenges (n2c2). 2019.https://n2c2.dbmi.hms.harvard.edu/Date accessed: September 23, 2019Google Scholar, 11Xiong Y. Shi X. Chen S. Jiang D. Tang B. Wang X. et al.Cohort selection for clinical trials using hierarchical neural network.J Am Med Inform Assoc. 2019; 26: 1203-1208Crossref PubMed Scopus (12) Google Scholar, 12Fang G. Annis I.E. Elston-Lafata J. Cykert S. Applying machine learning to predict real-world individual treatment effects: insights from a virtual patient cohort.J Am Med Inform Assoc. 2019; 26: 977-988Crossref PubMed Scopus (14) Google Scholar, 13Chen L. Gu Y. Ji X. Lou C. Sun Z. Li H. et al.Clinical trial cohort selection based on multi-level rule-based natural language processing system.J Am Med Inform Assoc. 2019; 26: 1218-1226Crossref PubMed Scopus (23) Google Scholar For example, the 2018 National NLP Clinical Challenges (n2c2) Shared-Task and Workshop on Cohort Selection for Clinical Trials was organized for competition. One group developed a rule-based NLP system for 13 selection criteria for a clinical trial using 202 training cohort and 86 testing cohort and achieved an F-measure of 0.90 (weighted mean of a test’s precision [positive predictive value (PPV)] and recall [sensitivity]).13Chen L. Gu Y. Ji X. Lou C. Sun Z. Li H. et al.Clinical trial cohort selection based on multi-level rule-based natural language processing system.J Am Med Inform Assoc. 2019; 26: 1218-1226Crossref PubMed Scopus (23) Google Scholar Given the labor-intensive nature of selecting a proper clinical trial cohort meeting the enrollment and exclusion criteria, this capability of an NLP system may potentially enable a virtual clinical trial and observational studies apart from enhancing our abilities for clinical trials. Also, because EHR-based research as a source of real-world evidence is complementary to traditional randomized controlled trial–based evidence as recently recognized by the Food and Drug Administration,14Jarow J.P. LaVange L. Woodcock J. Multidimensional evidence generation and FDA regulatory decision making: defining and using “real-world” data.JAMA. 2017; 318: 703-704Crossref PubMed Scopus (132) Google Scholar the utility of NLP as an efficient and effective approach leveraging EHRs for real-world evidence is likely to be increasingly recognized. In this review, first we will focus on discussing the current literature on NLP-based research in allergy, asthma, and immunology on the basis of our systematic literature review. We refer readers to the recent review on broad information extraction approaches (beyond allergy, asthma, and immunology) from EHRs for further understanding.15Wang Y. Wang L. Rastegar-Mojarad M. Moon S. Shen F. Afzal N. et al.Clinical information extraction applications: a literature review.J Biomed Inform. 2018; 77: 34-49Crossref PubMed Scopus (338) Google Scholar,16Shivade C. Raghavan P. Fosler-Lussier E. Embi P.J. Elhadad N. Johnson S.B. et al.A review of approaches to identifying patient phenotype cohorts using electronic health records.J Am Med Inform Assoc. 2014; 21: 221-230Crossref PubMed Scopus (276) Google Scholar In the second part, we will introduce NLP to clinicians and highlight the role of NLP in data mining and knowledge discovery from EHRs using specific case studies. We will conclude this review with a discussion of the implications of NLP-based clinical research and care in allergy, asthma, and immunology. To assess the current state of EHR-based research using NLP in the field of asthma, allergy, and immunology, a comprehensive literature search of several databases from January 1, 2000, to August 13, 2019, English language, was conducted (see Table E1 in this article’s Online Repository at www.jacionline.org for the detailed method for our systematic literature search and the summary of results of each study). Only 21 articles were included in this systematic review by excluding abstracts, review articles, research protocols, unavailable full articles, non–EHR- or non–NLP-based studies, and nonmatching topics. Overall, NLP-based research in the field of asthma, allergy, and immunology is limited in the literature. Herein, we summarize the current state of NLP-based research leveraging EHRs for allergic disorders and discuss the key findings on the limitations of NLP-based research from the literature review. NLP is most commonly applied for determining asthma status or outcomes as shown in Table E1. There are a total of 13 publications that used NLP logic (computer follows the preset rules in the program) or NLP machine learning (ML) (computer learns latent rules through human guidance called annotation [supervised] or no human guidance [unsupervised]) to determine asthma status or outcomes. Apart from 9 reports from our research group,17Wu S.T. Juhn Y.J. Sohn S. Liu H. Patient-level temporal aggregation for text-based asthma status ascertainment.J Am Med Inform Assoc. 2014; 21: 876-884Crossref PubMed Scopus (15) Google Scholar, 18Wu S.T. Sohn S. Ravikumar K.E. Wagholikar K. Jonnalagadda S.R. Liu H. et al.Automated chart review for asthma cohort identification using natural language processing: an exploratory study.Ann Allergy Asthma Immunol. 2013; 111: 364-369Abstract Full Text Full Text PDF PubMed Scopus (48) Google Scholar, 19Wu S. Liu S. Sohn S. Moon S. Wi C-i Juhn Y. et al.Modeling asynchronous event sequences with RNNs.J Biomed Inform. 2018; 83: 167-177Crossref PubMed Scopus (26) Google Scholar, 20Wi C.I. Sohn S. Ali M. Krusemark E. Ryu E. Liu H. et al.Natural language processing for asthma ascertainment in different practice settings.J Allergy Clin Immunol Pract. 2018; 6: 126-131Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar, 21Sohn S. Wi C.-I. Wu S.T. Liu H. Ryu E. Krusemark E. et al.Ascertainment of asthma prognosis using natural language processing from electronic medical records.J Allergy Clin Immunol. 2018; 141: 2292-2294.e3Abstract Full Text Full Text PDF PubMed Scopus (13) Google Scholar, 22Kaur H. Sohn S. Wi C.I. Ryu E. Park M.A. Bachman K. et al.Automated chart review utilizing natural language processing algorithm for asthma predictive index.BMC Pulm Med. 2018; 18: 34Crossref PubMed Scopus (33) Google Scholar, 23Wi C.I. Sohn S. Rolfes M.C. Seabright A. Ryu E. Voge G. et al.Application of a natural language processing algorithm to asthma ascertainment: an automated chart review.Am J Respir Crit Care Med. 2017; 196: 430-437Crossref PubMed Scopus (49) Google Scholar, 24Sohn S. Wi C.I. Juhn Y.J. Liu H. Analysis of clinical variations in asthma care documented in electronic health records between staff and resident physicians.Stud Health Technol Inform. 2017; 245: 1170-1174PubMed Google Scholar, 25Sohn S, Wang Y, Wi C-I, Krusemark EA, Ryu E, Ali MH, et al. Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions [published online ahead of print November 30, 2017]. J Am Med Inform Assoc. https://doi.org/10.1093/jamia/ocx138.Google Scholar there are only 4 publications that used NLP algorithms in asthma status or outcomes.26Afzal Z. Engelkes M. Verhamme K.M. Janssens H.M. Sturkenboom M.C. Kors J.A. et al.Automatic generation of case-detection algorithms to identify children with asthma from large electronic health record databases.Pharmacoepidemiol Drug Saf. 2013; 22: 826-833Crossref PubMed Scopus (28) Google Scholar, 27Himes B.E. Kohane I.S. Ramoni M.F. Weiss S.T. Characterization of patients who suffer asthma exacerbations using data extracted from electronic medical records.AMIA Annu Symp Proc. 2008; : 308-312PubMed Google Scholar, 28Liang H. Tsui B.Y. Ni H. Valentim C.C.S. Baxter S.L. Liu G. et al.Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence.Nat Med. 2019; 25: 433-438Crossref PubMed Scopus (281) Google Scholar As shown in Table E1, overall, the performance of NLP systems for asthma status and outcomes appears to vary depending on the nature of asthma concepts, outcomes, and NLP methods. Our previous work primarily applied NLP (logic or ML) to detect asthma status or prognosis on the basis of predetermined criteria instead of searching free-text terms as discussed in the following section. Only 1 article used NLP to define atopic dermatitis on the basis of a predetermined criteria.29Gustafson E. Pacheco J. Wehbe F. Silverberg J. Thompson W. A machine learning algorithm for identifying atopic dermatitis in adults from electronic health records.IEEE Int Conf Healthc Inform. 2017; 2017: 83-90PubMed Google Scholar In this study, the authors developed an ML algorithm for identifying atopic dermatitis in adults from EHRs using Hanifin-Rajka and UK Working Party criteria. The study used 3 nonoverlapping data subsets stratified by class label: training (60%), validation (20%), testing (20%) of 562 adult patients. They reported precision (69%-84%), recall (51%-75%), and F (0.62-0.79) depending on structured code only, NLP only, and structured code-NLP and type of atopic dermatitis criteria. Another area to which NLP has been applied is identification of allergy entry in EHRs. Even if standard terminologies for allergy entry could be searched and retrieved, such terminologies exist in an inconsistent or nonstandardized format (ie, lexical variation), posing challenges for identifying specific allergy status from EHRs For example, Systematized Nomenclature of Medicine—Clinical Terms (standardized clinical terminology system developed and maintained by SNOMED International, a nonprofit international organization, for the purpose of information entry into EHRs [International Classification of Diseases (ICD) is a classification system for information retrieval]) was used to match terms referring to allergy entry in EHRs but matching tends to be low (eg, 1 study reported 82% for SNOWMED-CT) because 6% of all allergies and 48% of food allergies were in a free-text format.30Plasek J.M. Goss F.R. Lai K.H. Lau J.J. Seger D.L. Blumenthal K.G. et al.Food entries in a large allergy data repository.J Am Med Inform Assoc. 2016; 23: e79-e87Crossref PubMed Scopus (18) Google Scholar Because of this reason, NLP had to be used for addressing this challenge (second most common use cases for NLP after asthma), detection of specific allergy status from EHRs. There are 4 NLP-based studies (3 studies based on NLP logic for food allergy,30Plasek J.M. Goss F.R. Lai K.H. Lau J.J. Seger D.L. Blumenthal K.G. et al.Food entries in a large allergy data repository.J Am Med Inform Assoc. 2016; 23: e79-e87Crossref PubMed Scopus (18) Google Scholar any allergy entries in emergency department setting,31Goss F.R. Plasek J.M. Lau J.J. Seger D.L. Chang F.Y. Zhou L. An evaluation of a natural language processing tool for identifying and encoding allergy information in emergency department clinical notes.AMIA Annu Symp Proc. 2014; 2014: 580-588PubMed Google Scholar and food/drug allergy32Epstein R.H. St Jacques P. Stockin M. Rothman B. Ehrenfeld J.M. Denny J.C. Automated identification of drug and food allergies entered using non-standard terminology.J Am Med Inform Assoc. 2013; 20: 962-968Crossref PubMed Scopus (19) Google Scholar and 1 study based on supervised NLP-ML for any allergy misspelling33Lai K.H. Topaz M. Goss F.R. Zhou L. Automated misspelling detection and correction in clinical free-text records.J Biomed Inform. 2015; 55: 188-195Crossref PubMed Scopus (71) Google Scholar). Their overall performance of NLP algorithms was excellent (F-measures [mean of PPV and sensitivity] of 94%-99%30Plasek J.M. Goss F.R. Lai K.H. Lau J.J. Seger D.L. Blumenthal K.G. et al.Food entries in a large allergy data repository.J Am Med Inform Assoc. 2016; 23: e79-e87Crossref PubMed Scopus (18) Google Scholar,32Epstein R.H. St Jacques P. Stockin M. Rothman B. Ehrenfeld J.M. Denny J.C. Automated identification of drug and food allergies entered using non-standard terminology.J Am Med Inform Assoc. 2013; 20: 962-968Crossref PubMed Scopus (19) Google Scholar,33Lai K.H. Topaz M. Goss F.R. Zhou L. Automated misspelling detection and correction in clinical free-text records.J Biomed Inform. 2015; 55: 188-195Crossref PubMed Scopus (71) Google Scholar or slightly lower F-measures of 80%-92% depending on allergies31Goss F.R. Plasek J.M. Lau J.J. Seger D.L. Chang F.Y. Zhou L. An evaluation of a natural language processing tool for identifying and encoding allergy information in emergency department clinical notes.AMIA Annu Symp Proc. 2014; 2014: 580-588PubMed Google Scholar). These are promising results for application to clinical practice. There is 1 article for detecting smoking status related to asthma exacerbation in EHRs on the basis of NLP logic, with PPV of 79% and specificity of 90%.34Meystre S.M. Deshmukh V.G. Mitchell J. A clinical use case to evaluate the i2b2 Hive: predicting asthma exacerbations.AMIA Annu Symp Proc. 2009; 2009: 442-446PubMed Google Scholar One article used NLP and multiple ML classifiers to extract and identify anaphylaxis incidence from unstructured data (F-measure, 048-0.95).35Segura-Bedmar I. Colon-Ruiz C. Tejedor-Alonso M.A. Moro-Moro M. Predicting of anaphylaxis in big data EMR by exploring machine learning approaches.J Biomed Inform. 2018; 87: 50-59Crossref PubMed Scopus (22) Google Scholar Another article used NLP to extract and determine the incidence of hypersensitivity reaction to nonsteroidal anti-inflammatory drug from EHRs.36Blumenthal K.G. Lai K.H. Huang M. Wallace Z.S. Wickner P.G. Zhou L. Adverse and hypersensitivity reactions to prescription nonsteroidal anti-inflammatory agents in a large health care system.J Allergy Clin Immunol Pract. 2017; 5: 737-743.e3Abstract Full Text Full Text PDF PubMed Scopus (48) Google Scholar There are several key findings for the current state of NLP-based research in the field of asthma, allergy, and immunology. First, overall, NLP is underused in the field of asthma, allergy, and immunology. Although there is some NLP-based research in allergy and asthma, little research based on NLP approaches has been done in atopic dermatitis or allergic rhinitis. Apart from NLP algorithms for asthma criteria and prognosis reported by our group discussed later,20Wi C.I. Sohn S. Ali M. Krusemark E. Ryu E. Liu H. et al.Natural language processing for asthma ascertainment in different practice settings.J Allergy Clin Immunol Pract. 2018; 6: 126-131Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar, 21Sohn S. Wi C.-I. Wu S.T. Liu H. Ryu E. Krusemark E. et al.Ascertainment of asthma prognosis using natural language processing from electronic medical records.J Allergy Clin Immunol. 2018; 141: 2292-2294.e3Abstract Full Text Full Text PDF PubMed Scopus (13) Google Scholar, 22Kaur H. Sohn S. Wi C.I. Ryu E. Park M.A. Bachman K. et al.Automated chart review utilizing natural language processing algorithm for asthma predictive index.BMC Pulm Med. 2018; 18: 34Crossref PubMed Scopus (33) Google Scholar, 23Wi C.I. Sohn S. Rolfes M.C. Seabright A. Ryu E. Voge G. et al.Application of a natural language processing algorithm to asthma ascertainment: an automated chart review.Am J Respir Crit Care Med. 2017; 196: 430-437Crossref PubMed Scopus (49) Google Scholar,37Seol H.Y. Sohn S. Liu H. Wi C. Ryu E. Park M.A. et al.Early identification of childhood asthma: the role of informatics in an era of electronic health records.Front Pediatr. 2019; 7: 113Crossref PubMed Scopus (4) Google Scholar NLP algorithms for complex concepts such as predetermined criteria for allergic disorders are significantly limited, recognizing that valuable information required for complex predetermined criteria for allergic disorders exists in a free-text format of EHRs. NLP is a useful tool for identifying a cohort with distinctive clinical characteristics in clinical conditions with significant heterogeneity such as asthma. For example, because NLP has a technical feature that researchers may modify the existing NLP algorithms for a condition (eg, asthma) to identify a subgroup of patients with distinctive clinical characteristics (eg, atopic asthma or asthma with impaired lung function), which may not be feasible with ICD codes for asthma. Second, the current literature on EHR-based research using NLP has a tendency to not capture and use a temporal component for outcome events of interest. Because NLP leverages free texts of clinicians’ notes in EHRs, it is more accurate than structured code for detecting the index date of allergic disorders. Because the status of allergic disorders in large-scale clinical studies is frequently determined by a survey question about a physician diagnosis of asthma (eg, “have you ever been told that you (or your child) have a diagnosis of asthma?”) without knowing the index date, NLP is useful for detecting the index date (when one fulfilled predetermined criteria as shown in our research work discussed later) of asthma or other outcomes such as remission on a large scale. For clinical care and research, it is crucially important to determine the index or incidence date of outcome events (eg, asthma onset and asthma remission or relapse) because temporality is an essential aspect to discern the relationship between predictors and outcomes. In this respect, NLP systems have the capability for capturing temporal components of outcome events over time because NLP programs detect temporal components for each note for a specific concept. Third, as one of the major challenges in informatics research, portability and external validation of AI algorithms are often difficult to assess because of patient confidentiality and technology transfer, only a few studies have addressed this challenge. Clinical practice and workflow vary across institutions, which results in different practice settings and reporting schemes for generating EHRs. Also, it has been demonstrated that clinical language is not homogeneous, but instead consists of heterogeneous characteristics (eg, syntactic variation or semantic variation).25Sohn S, Wang Y, Wi C-I, Krusemark EA, Ryu E, Ali MH, et al. Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions [published online ahead of print November 30, 2017]. J Am Med Inform Assoc. https://doi.org/10.1093/jamia/ocx138.Google Scholar All these factors pose challenges for applying an NLP algorithm developed from one institution to another and need to be considered in portability of NLP algorithms. This is an active research area for addressing these challenges. Finally, not least, apart from its accuracy of EHR-based AI tools, another important aspect for assessing properness or utility of such tools is acceptability, user interface, usability, or workflow for the entire care process to ensure EHR-based AI tools are helpful rather than a task or burden to clinicians or researchers. Dr Homer Warner, a pioneer in medical informatics, said “Medical informatics = 10% technology + 10% medicine + 80% sociology,” highlighting the crucial importance of designing and implementing informatics tools including NLP considering patient-centered, clinician-empowering, and team-based workflow. In this respect, there is no widely accepted standard metrics assessing or reporting these important domains. We further discuss this important issue and challenge in the later section.