A b s t r a c t Objective: The aim of this study was to develop a method based on natural language processing (NLP) that automatically maps an entire clinical document to codes with modifiers and to quantitatively evaluate the method. Methods: An existing NLP system, MedLEE, was adapted to automatically generate codes. The method involves matching of structured output generated by MedLEE consisting of findings and modifiers to obtain the most specific code. Recall and precision applied to Unified Medical Language System (UMLS) coding were evaluated in two separate studies. Recall was measured using a test set of 150 randomly selected sentences, which were processed using MedLEE. Results were compared with a reference standard determined manually by seven experts. Precision was measured using a second test set of 150 randomly selected sentences from which UMLS codes were automatically generated by the method and then validated by experts. Results: Recall of the system for UMLS coding of all terms was .77 (95% CI .72-.81), and for coding terms that had corresponding UMLS codes recall was .83 (.79-.87). Recall of the system for extracting all terms was .84 (.81-.88). Recall of the experts ranged from .69 to .91 for extracting terms. The precision of the system was .89 (.87-.91), and precision of the experts ranged from .61 to .91. Extraction of relevant clinical information and UMLS coding were accomplished using a method based on NLP. The method appeared to be comparable to or better than six experts. The advantage of the method is that it maps text to codes along with other related information, rendering the coded output suitable for effective retrieval. The electronic patient record contains a rich source of valuable clinical information, which could be used for a wide range of automated applications aimed at improving the health care process, such as alerting for potential medical errors, generating a patient problem list, and assessing the severity of a condition. However, generally, such applications are not yet feasible because much of the information is in textual form, 1 which is not suitable for use by other automated applications. In addition, for the information to be effectively utilized by a broad range of clinical applications, it must be interoperable among different clinical applications, different clinical domains, and different institutions. The Clinical Document Architecture (CDA) effort, 2 which is a key effort that addresses interoperability, has established the importance of a common well-delineated representational model for each document in a document class (e.g., radiology reports) and also for the concepts in the documents. An effective method that automatically maps relevant text in clinical documents into standardized concepts would help further the CDA effort. Many reports have been published 3-10 on methods that automatically map clinical text to concepts within a standardized coding system, such as the Unified Medical Language System (UMLS). These approaches, which primarily have been applied to indexing applications, use methods based on string matching, statistical processing, or linguistic processing, utilizing part of speech tagging (i.e., classifying words as nouns, verbs, etc.) followed by identification of noun phrases. However, for accurate retrieval of clinical information, coding concepts in the documents in isolation of other information, such as modifiers, is not enough because the underlying meaning of the concepts is significantly affected by other co-occurring concepts. For example, in a patient document, concepts may be modified by negation (i.e., without significant tremors), by temporal information (i.e., previously admitted for pneumonia), by family history (i.e., family history of heart ), or by modifiers denoting that the event may not have actually occurred (i.e., exposed to tuberculosis, admitted for syncope workup). For many automated applications, such as alerting applications, to perform effectively, modifiers of the concepts also have to be accounted for. Some work concerning identification of negated concepts has been reported, 11, 12 which is critical for many automated applications. However, other types of modifiers were not detected. In this report we describe a coding method that uses advanced NLP techniques to generate structured encoded output consisting of findings and corresponding modifiers. The method attempts to find the most granular corresponding code by matching the structured output, which was generated as a result of parsing the sentences, with structures in a coding table in which the structures have been associated with codes. A code is obtained by successfully matching a finding along with modifiers, based on an assumption that a match consisting of a finding with the most modifiers is preferable to a match consisting of the same finding with fewer modifiers because it is the most specific. For example, for the sentence, Status post myocardial infarction in 1995, the UMLS code C0856742 (corresponding to the UMLS concept post mi) would be obtained along with a date modifier with the value 1995. Note that the code C0027051 (corresponding to the UMLS concept myocardial infarction) would also be appropriate, but is not as specific as the one corresponding to post mi, and therefore is not chosen. In addition, we also describe an evaluation of the method and the results that were obtained. The Background section discusses related work, the Methods section describes the techniques and the evaluation design, the Results section presents performance measures, the Discussion section discusses the strengths and weaknesses of our method and issues associated with encoding and with the evaluation.
REF extract structured output from notes in the form of Unified Medical Language System (UMLS) codes.
Automated Encoding of Clinical Documents Based on Natural Language Processing
Journal of the American Medical Informatics Association : JAMIA
Journal of the American Medical Informatics Association : JAMIA
Computer Science Medicine