Weakly Supervised Medication Regimen Extraction from Medical Conversations (2020)

Abstract: Automated Medication Regimen (MR) extraction from medical conversations can not only improve recall and help patients follow through with their care plan, but also reduce the documentation burden for doctors. In this paper, we focus on extracting spans for frequency, route and change, corresponding to medications discussed in the conversation. We first describe a unique dataset of annotated doctor-patient conversations and then present a weakly supervised model architecture that can perform span extraction using noisy classification data. The model utilizes an attention bottleneck inside a classification model to perform the extraction. We experiment with several variants of attention scoring and projection functions and propose a novel transformer-based attention scoring function (TAScore). The proposed combination of TAScore and Fusedmax projection achieves a 10 point increase in Longest Common Substring F1 compared to the baseline of additive scoring plus softmax projection.

## introduction patients forget 40-80% of the medical information provided by healthcare practitioners immediately @xcite and misconstrue 48% of what they think they remembered @xcite , and this adversely affects patient adherence. automatically extracting information from doctor-patient conversations can help patients correctly recall doctor's instructions and improve compliance with the care plan @xcite . on the other hand, clinicians spend up to 49.2% of their overall time on ehr and desk work, and only 27.0% of their total time on direct clinical face time with * work done as an intern at abridge ai inc. dr: limiting your alcohol consumption is important, so, and, um, so, you know, i would recommend vitamin d 1 to be taken 1 . have you had fosamax 2 before? pt: i think my mum did. dr: okay, fosamax 2 , you take 2 one pill 2 on monday and one on thursday 2 . dr: do you use much caffine? pt: no, none. dr: okay, this is 3 actonel 3 and it's one tablet 3 once a month 3 . dr: do you get a one month or a three months supply in your prescriptions? figure 1 : an example excerpt from a doctorpatient conversation transcript. here, there are three medications mentioned indicated by the superscript. the extracted attributes, change, route and frequency, for each medications are also shown. in this work, we focus on extracting medication regimen (mr) information @xcite from the doctor-patient conversations. specifically, we extract three attributes, i.e., frequency, route and change, corresponding to medications discussed in the conversation (figure 1 ). medication regimen information can help doctors with medication orders cum renewals, medication reconciliation, verification of reconciliations for errors, and other medicationcentered ehr documentation tasks. it can also improve patient engagement, transparency and better compliance with the care plan @xcite . mr attribute information present in a conversation can be obtained as spans in text (figure 1 ) or can be categorized into classification labels ( the main contributions of this work are as follows. we present a way of setting up an mr attribute extraction task from noisy classification data (section 2). we propose a weakly supervised model architecture which utilizes attention bottleneck inside a classification model to perform span extraction (section 3 & 4). in order to favor sparse and contiguous extractions, we experiment with two variants of attention projection functions (section 3.1.2), namely, softmax and fusedmax @xcite . further, we propose a novel transformer-based attention scoring function tascore (section 3.1.1). the combination of tascore and fusedmax achieves significant improvements in extraction performance over a phrase-based (22 lcsf1 points) and additive softmax attention (10 lcsf1 points) baselines. ## medication regimen (mr) using weak supervision medication regimen (mr) consists of information about a prescribed medication akin to attributes of an entity. in this work, we specifically focus on frequency, route of the medication and any change in the medication's dosage or frequency as shown in figure 1 . for example, given the conversation excerpt and the medication "fosamax" as shown in figure 1 , the model needs to extract the spans "one pill on monday and one on thursday", "pill" and "you take" for attributes frequency, route and change, respectively. the major challenge, however, is to perform the attribute span extraction using noisy classification labels with very few or no span-level labels. the rest of this section describes the dataset used for this task. ## background there have been several successful attempts to use neural attention @xcite to extract information from text in an unsupervised manner @xcite @xcite . attention scores provide a good proxy for importance of a particular token in a model. however, when there are multiple layers of attention, or if the encoder is too complex and trainable, the model no longer provides a way to produce reliable and faithful importance scores @xcite . we argue that, in order to bring in the faithfulness, we need to create an attention bottleneck in our classification + extraction model. the attention bottleneck is achieved by employing an attention function which generates a set of attention weights over the encoded input tokens. attention bottleneck forces the classifier to only see the portions of input that pass through it, thereby enabling us to trade the classification performance for extraction performance and getting span extraction with weak supervision from classification labels. in the rest of this section, we provide general background on neural attention and present its variants employed in this work. this is followed by the presentation of our complete model architecture in the subsequent sections. ## model our classification + extraction model uses mr attributes classification labels to extract mr attributes. the model can be divided into three phases: identify, classify and extract (figure 3 ). the identify phases encodes the input text and medication name and uses the attention bottleneck to produce attention over the text. classify phase computes the context vector using the attention from the identify phases and classifies the context vectors. finally, the extract phase uses the attention from the identify phase to extract spans corresponding to mr attributes. ## metrics token-wise f1 (tf1): each token in text is either part of the extracted span (positive class) for an attribute or not (negative class). token-wise f1 score is the f1 score of the positive class obtained by considering all the tokens in the dataset as separate binary classification data points. tf1 is calculated separately for each attribute. ## results and analysis ## related work existing literature directly related to our work can be bucketed into two categories -related methods and related tasks. methods: the recent work on generating rationales/explanations for deep neural network based classification models @xcite @xcite is closely related to ours in terms of the methods used. most of these works use binary latent variables to perform extraction as an intermediate step before classification. our work is closely related to @xcite , who use attention scores to generate rationales for classification models. these works, however, focus on generating faithful and plausible explanation for classification as opposed to extracting the spans for attributes of an entity, which is the focus of our work. moreover, our method can be generalized to any number of attributes while all these methods would require a separate model for each attribute. tasks: understanding doctor-patient conversations is starting to receive attention recently @xcite . @xcite performs mr extraction by framing the problem as a generative question answering task. this approach is not efficient at inference time -it requires one forward pass for each attribute. moreover, unlike a span extraction model, the generative model might produce hallucinated facts. @xcite obtain mr attributes as spans in text; however, they use a fully supervised approach which requires a large dataset with spanlevel labels. ## conclusion and future work we provide a framework to perform mr attribute extraction from medical conversations with weak supervision using noisy classification labels. this is done by creating an attention bottleneck in the classification model and performing extraction using the attention weights. after experimenting with several variants of attention scoring and projection functions, we show that the combination of our transformer-based attention scoring function (tascore) combined with fusedmax* achieves significantly higher extraction performance compared to the other attention variants and a phrase-based baseline. while our proposed method achieves good performance, there is still room for improvement, especially for text with multiple medications. data augmentation by swapping or masking medication names is worth exploring. an alternate direction of future work involves improving the naturalness of extracted spans. auxiliary supervision using a language modeling objective would be a promising approach for this.