Context-aware Medication Event Extraction from Unstructured Text (2023)

Abstract: Accurately capturing medication history is crucial in delivering high-quality medical care. The extraction of medication events from unstructured clinical notes, however, is challenging because the information is presented in complex narratives. We address this challenge by leveraging the newly released Contextualized Medication Event Dataset (CMED) as part of our participation in the 2022 National NLP Clinical Challenges (n2c2) shared task. Our study evaluates the performance of various pretrained language models in this task. Further, we find that data augmentation coupled with domain-specific training provides notable improvements. With experiments, we also underscore the importance of careful data preprocessing in medical event detection.

## introduction ensuring the accuracy of a patient's treatment history is essential for delivering high-quality medical care. this allows healthcare professionals to assess the effectiveness of existing treatments, detect possible medication-related problems, and suggest appropriate future treatment options @xcite . various forms of treatment changes, however, are often absent from structured electronic data sources, being recorded only in clinical narratives @xcite . an accurate extraction of medication event information from unstructured data in patients' medical records is thus crucial for a complete understanding of their treatments. when extracting medication changes from clinical text, it is necessary to take into account various forms of contextual information, due to the narrative and longitudinal nature of clinical documentation. clinical text often documents events over a patient's medical history, and providers may also record the reasoning behind their medical decisions. these factors result in complex events that cannot be properly captured by extracting medi-cation changes alone, without considering the surrounding clinical context. this is especially true when developing a medication change extraction system to support real-world applications, such as medication timeline generation @xcite or medication reconciliation @xcite . indeed, as @xcite have argued, the use of sophisticated natural language processing (nlp) information extraction (ie) become a necessity when the automatic extraction of relevant medical information is required by large-scale or real-time applications further downstream, such as clinical research and decision support. this study investigates how to extract information about changes to patient medications from clinical text using the contextualized medication event dataset (cmed) developed by @xcite and subsequently released to the community as a shared task in 2022 national nlp clinical challenges (n2c2) foot_0 . this consists of three tasks: (i) medication extraction, to extract all medication mentions in clinical notes, (ii) event classification, to identify whether a medication change is discussed in an event,and (iii) context classification, to classify the contextual information of a medication change event along five orthogonal dimensions, with each dimension further classified into multiple attributes of the event. for the first medical named entity recognition task, we note that @xcite demonstrate significant improvements with the use of biobert, a domain-specific model initialized with bert and then pretrained on pubmed abstracts and pubmed central full text articles. thus, we proceed to use biobert as well, providing comparisons against popular general purpose language models like bert @xcite . additionally, we also utilize bio+clinial bert @xcite , another domain-specific model initialized with biobert and further pretrained on notes from the mimic-iii dataset @xcite . we evaluate the performance of several pretrained language models for the second and third tasks. specifically, we examine three popular general purpose models -bert, roberta @xcite , and xlnet @xcite -and one domain-specific pretrained model, bio+clinical bert. while many downstream natural language understanding tasks are readily successful when a large pretrained language model is tuned for the task in hand, we observe that the accuracy of clinical event detection crucially depends on careful data preprocessing. in particular, identifying the proper linguistic context from surrounding text is of utmost importance. to that end, we develop and employ a sentence detection method tailored to this task, leading to a better performance by all models. we also find that augmenting the data with the ddi (drug-drug interaction) corpus @xcite leads to overall improvements in medication change detection and its context classification. ## related work the first task in contextualized medication event extraction is to extract the mention of medicationsclearly, a medical named entity recognition (ner) task. medical ner, in general, includes identifying other types of entities such as diseases, symptoms, proteins, or patient information (see @xcite for an overview). to identify medication names in particular, many approaches have been proposed. early methods relied explicitly on domain ontology or dictionaries @xcite , rules @xcite , and subsequently, contextual rules and automatically learned rules @xcite . a comprehensive survey of this literature has been conducted by @xcite . more recent approaches are hybrid, combining lstm and its variants with conditional random fields (crf) or other graphical models @xcite . even more recent, however, are techniques that utilize transformer models (e.g., bert). there is some work to further indicate that combining bert with bilstm-crf improves medical ner @xcite , while others demonstrate the improvements in using domain-specific pretraining with bert initialization @xcite . identifying medication change events and classifying their attributes, however, is a significantly less explored problem. this is due largely to the scarcity of annotated resources, but to a lesser extent, also to the complexity of the language used in clinical narratives to describe such events. initially, research heavily relied on annotated datasets like the 2009 i2b2 and the 2013 ddi datasets @xcite . some early work focused on very specific events of clinical relevance, such as @xcite , who inspect medication discontinuation, or @xcite , who focus on whether medication was started, stopped, increased, or decreased. in another approach, @xcite introduced temporal information into their labels. in spite of the success on individual datasets, these approaches employ rule-based decisions and classical supervised learning algorithms like support vector machines (svms) or maximum entropy modeling, which are unlikely to generalize across multiple datasets with linguistic variation without extensive supervision for each dataset. for a detailed understanding of treatments, such as extracting the dosage, frequency, or mode of drug administration, or in determining its relation to other phenomena like adverse drug effects, generalizable success in this task carries immense significance. it is thus worth noting that recent methods leveraging neural architectures and models have shown promise in medical event extraction and classification tasks @xcite . @xcite use a neural top-down transition based parser and achieve results comparable to bilstm models for medical entity and event detection. perhaps the closest to our study is the approach of @xcite , who tune bert on covid-19 data to identify various events of clinical significance, such as symptoms, severity, and assertion. this body of work is distinct from ours, however, since it does not delve into classification of event attributes involving complex temporal or conditional expressions. several studies @xcite @xcite @xcite have examined the detection of negated medical concepts in clinical text. however, none of them specifically focus on identifying medication change events. moreover, they have not looked at the combined identification of negation and the actor responsible for that negation. early work on negation detection in clinical texts was based on negation lexicons, and rule-based algorithms using them. most notable among them is negex @xcite . although these have been superceded by others who combine lexicons with dependency structures or other linguistic features @xcite , we use an implementation of negex built into the popular spacy 2 library, called negspacy. our results (sec. 6) show that in spite of its simplicity, this approach suffices. with the contextual medication event dataset, cmed, @xcite provide annotated data capturing five orthogonal dimensions of contextual information related to medication change events. further, they demonstrate the viability of svms and transformer-based models in detecting and classifying these events. very recently, @xcite have explored an avenue similar to ours, with domain-specific language models based on bert. in these, it has been noted that sentences that mention multiple drugs are particularly difficult to detect and classify. by contrast, our work investigates data augmentation and taskspecific preprocessing in conjunction with the user of domain-specific language models. in particular, we develop and use a custom sentence extraction module in our pipeline, which improves the accuracy of these models on the given tasks. ## the cmed dataset the cmed dataset comprises annotated clinical notes, where each medication mention is assigned 2 spacy.io/ one or more event labels from the three categories: (1) disposition, indicating the mention of a medication change, e.g., "prescribed albuterol for shortness of breath", (2) nodisposition, indicating that the mediction is mentioned with no indication of change, e.g., "patient continues to take aspirin", and (3) undetermined, indicating a lack of clarity or evidence regarding medication change, e.g., "plan: lasix". for each event identified as disposition, the clinical context is provided along five orthogonal dimensions, viz., action, actor, negation, certainty, and temporality. we describe these next. ## data preprocessing we observe that in cmed, most medication mentions are labeled with one event and a set of corresponding attributes. there is, however, a small fraction (< 90 instances in the training set), where the drug mention is labeled with two events and two separate sets of attributes, as noted by @xcite as well. further, we underscore the frequent presence of sentences containing multiple drug mentions (approx. 78%), with a substantial fraction (over 50%) of such sentences mentioning four or more drugs simultaneously. this aspect of the dataset significantly increases the complexity of detecting and classifying contextual information from clinical sentences. finally, we note that some events cannot be accurately labeled based solely on the sentence in which they appear, and additional context from neighboring sentences becomes necessary to determine the correct attributes. the first step in the medication information extraction task is to prepare the dataset by extracting the sentences containing medication information. however, due to the unstructured and lengthy nature of medical notes, accurately identifying the start and end of a sentence containing a medication mention is challenging. accordingly, relying solely on tools like, say, spacy, for their inbuilt sentence parser for this task does not produce satisfactory results. therefore, we develop a customized approach to accurately identify the sentences that contained medication names, which served as a crucial first step towards performing accurate medication event extraction. next, we describe the steps of this process. algorithm 1 get_conjunction(head) 1: acc ← [], list_heads ← [head] 2: while list_heads ̸ = [] do 3: new_heads ← [] 4: in our example, this approach leads to two such simpler expressions, "started lisinopril 10 mg p.o. daily" and "substituted for diltiazem". ## approach in this section, we explain our technical approach to the tasks of medication mention extraction, medical event identification, and medication event attribute classification. further, we devote a separate description of the steps we take to detect negation. ## evaluation to evaluate the accuracy of medication mention extraction systems, we employ two criteria: strict and lenient match. the strict criteria demands an exact match between the predicted and true medication mention spans. the lenient match criteria, on the other hand, considers a predicted medication mention to be correct if at least one token in the predicted mention overlaps with a token in the true mention. while strict criteria may provide a more conservative performance estimate, lenient criteria can identify more correct predictions, but at the expense of higher false positive rates. to evaluate the event and attribute classification systems, we employ precision, recall, and f1 scores, reporting both macro-and micro-averages. ## conclusion our analysis of cmed and its constituent tasks reveal three main characteristics. first, it is often necessary to consider additional context beyond the specific sentence containing the medication mention to accurately label medication references. this context could include information from previous or subsequent sentences, the patient's medical history, or other relevant information further away in a document (as often found in the assessment and plan sections) that could impact the interpretation of the medication mention. second, we observe the prevalence of multiple medication references within a single sentence, which poses a challenge for accurate extraction. finally, accurate identification of the start and end of a sentence containing a medication mention is also challenging, since standard sentence splitting and tokenization methods often fail in clinical notes, especially if task-specific or domain-specific preprocessing is not done. we especially underscore the importance of data preprocessing when training or fine-tuning models for the medical domain. in this work, for example, we perform abbreviation resolution, coreference resolution, syntactic dependency-based sentence extraction, and a custom sentence extraction with phrase chunking. similar to other recent findings, our study demonstrates that pretrained language models are extremely effective in complex clinical information extraction, when fine-tuned on carefully chosen domain data. overall, our study affirms the utility of transformer-based models, particularly biobert and bio+clinical bert, in medication information extraction from clinical notes. we also exhibit the additional advantage of training such models with augmented domain data.