A Multi-Task Learning Framework for Extracting Drugs and Their Interactions from Drug Labels

Abstract: Preventable adverse drug reactions as a result of medical errors present a growing concern in modern medicine. As drug-drug interactions (DDIs) may cause adverse reactions, being able to extracting DDIs from drug labels into machine-readable form is an important effort in effectively deploying drug safety information. The DDI track of TAC 2018 introduces two large hand-annotated test sets for the task of extracting DDIs from structured product labels with linkage to standard terminologies. Herein, we describe our approach to tackling tasks one and two of the DDI track, which corresponds to named entity recognition (NER) and sentence-level relation extraction respectively. Namely, our approach resembles a multi-task learning framework designed to jointly model various sub-tasks including NER and interaction type and outcome prediction. On NER, our system ranked second (among eight teams) at 33.00% and 38.25% F1 on Test Sets 1 and 2 respectively. On relation extraction, our system ranked second (among four teams) at 21.59% and 23.55% on Test Sets 1 and 2 respectively.

Preventable adverse drug reactions (ADRs) introduce a growing concern in the modern healthcare system as they represent a large fraction of hospital admissions and play a significant role in increased health care costs BIBREF0 . Based on a study examining hospital admission data, it is estimated that approximately three to four percent of hospital admissions are caused by adverse events BIBREF1 ; moreover, it is estimated that between 53% and 58% of these events were due to medical errors BIBREF2 (and are therefore considered preventable). Such preventable adverse events have been cited as the eighth leading cause of death in the U.S., with an estimated fatality rate of between 44,000 and 98,000 each year BIBREF3 . As drug-drug interactions (DDIs) may lead to preventable ADRs, being able to extract DDIs from structured product labeling (SPL) documents for prescription drugs is an important effort toward effective dissemination of drug safety information. The Text Analysis Conference (TAC) is a series of workshops aimed at encouraging research in natural language processing (NLP) and related applications by providing large test collections along with a standard evaluation procedure. The Drug-Drug Interaction Extraction from Drug Labels track of TAC 2018 BIBREF4 , organized by the U.S. Food and Drug Administration (FDA) and U.S. National Library of Medicine (NLM), is established with the goal of transforming the contents of SPLs into a machine-readable format with linkage to standard terminologies.
We focus on the first two tasks of the DDI track involving named entity recognition (NER) and relation extraction (RE). Task 1 is focused on identifying mentions in the text corresponding to precipitants, interaction triggers, and interaction effects. Precipitants are defined as substances, drugs, or a drug class involved in an interaction. Task 2 is focused on identifying sentence-level interactions; concretely, the goal is to identify the interacting precipitant, the type of the interaction, and outcome of the interaction. The interaction outcome depends on the interaction type as follows. Pharmacodynamic (PD) interactions are associated with a specified effect corresponding to a span within the text that describes the outcome of the interaction. Naturally, it is possible for a precipitant to be involved in multiple PD interactions. Pharmacokinetic (PK) interactions are associated with a label from a fixed vocabulary of National Cancer Institute (NCI) Thesaurus codes indicating various levels of increase/decrease in functional measurements. For example, consider the sentence: “There is evidence that treatment with phenytoin leads to to decrease intestinal absorption of furosemide, and consequently to lower peak serum furosemide concentrations.” Here, phenytoin is involved in a PK interaction with the label drug, furosemide, and the type of PK interaction is indicated by the NCI Thesaurus code C54615 which describes a decrease in the maximum serum concentration (C INLINEFORM0 ) of the label drug. Lastly, unspecified (UN) interactions are interactions with an outcome that is not explicitly stated in the text and usually indicated through cautionary statements. Figure FIGREF1 features a simple example of a PD interaction that is extracted from the drug label for Adenocard, where the precipitant is digitalis and the effect is “ventricular fibrillation.”
Herein, we describe the training and testing data involved in this task and the metrics used for evaluation. In Section SECREF5 , we describe our modeling approach, our deep learning architecture, and our training procedure.
Each drug label is a collection of sections (e.g., DOSAGE & ADMINISTRATION, CONTRAINDICATIONS, and WARNINGS) where each section contains one or more sentences. Each sentence is annotated with a list of zero or more mentions and interactions. The training data released for this task contains 22 drug labels, referred to as Training-22, with gold standard annotations. Two test sets of 57 and 66 drug labels, referred to as Test Set 1 and 2 respectively, with gold standard annotations are used to evaluate participating systems. As Training-22 is a relatively small dataset, we additionally utilize an external dataset with 180 annotated drug labels dubbed NLM-180 BIBREF5 (more later). We provide summary statistics about these datasets in Table TABREF3 . Test Set 1 closely resembles Training-22 with respect to the sections that are annotated. However, Test Set 1 is more sparse in the sense that there are more sentences per drug label (144 vs. 27), with a smaller proportion of those sentences having gold annotations (23% vs. 51%). Test Set 2 is unique in that it contains annotations from only two sections, namely DRUG INTERACTIONS and CLINICAL PHARMACOLOGY, the latter of which is not represented in Training-22 (nor Test Set 1). Lastly, Training-22, Test Set 1, and Test Set 2 all vary with respect to the distribution of interaction types, with Training-22, Test Set 1, and Test Set 2 containing a higher proportion of PD, UN, and PK interactions respectively.
We used the official evaluation metrics for NER and relation extraction based on the standard precision, recall, and F1 micro-averaged over exactly matched entity/relation annotations. For either task, there are two matching criteria: primary and relaxed. For entity recognition, relaxed matching considers only entity bounds while primary matching considers entity bounds as well as the type of the entity. For relation extraction, relaxed matching only considers precipitant drug (and their bounds) while primary matching comprehensively considers precipitant drugs and, for each, the corresponding interaction type and interaction outcome. As relation extraction evaluation takes into account the bounds of constituent entity predictions, relation extraction performance is heavily reliant on entity recognition performance. On the other hand, we note that while NER evaluation considers trigger mentions, triggers are ignored when evaluating relation extraction performance.
We propose a multi-task learning framework for extracting drug-drug interactions from drug labels. The framework involves branching paths for each training objective (corresponding to sub-tasks) such that parameters of earlier layers (i.e., the context encoder) are shared.
Since only drugs involved in an interaction (precipitants) are annotated in the ground truth, we model the task of precipitant recognition and interaction type prediction jointly. We accomplish this by reducing the problem to a sequence tagging problem via a novel NER tagging scheme. That is, for each precipitant drug, we additionally encode the associated interaction type. Hence, there are five possible tags: T for trigger, E for effects, and D, K, and U for precipitants with pharmacodynamic, pharmacokinetic, and unspecified interactions respectively. As a preprocesssing step, we identify the label drug in the sentence, if it is mentioned, and bind it to a generic entity token (e.g. “LABELDRUG”). We additionally account for label drug aliases, such as the generic version of a brand-name drug, and bind them to the same entity token. Table TABREF7 shows how the tagging scheme is applied to the simple example in Figure FIGREF1 . A drawback is that simplifying assumptions must be made that will hamper recall; e.g., we only consider non-overlapping mentions (more later).
Once we have identified the precipitant offsets (as well as of triggers/effects) and the interaction type for each precipitant, we subsequently predict the outcome or consequence of the interaction (if any). To that end, we consider all entity spans annotated with K tags and assign them a label from a static vocabulary of 20 NCI concept codes corresponding to PK consequence (i.e., multiclass classification) based on sentence-context. Likewise, we consider all entity spans annotated with D tags and link them to mention spans annotated with E tags; we accomplish this via binary classification of all pairwise combinations. For entity spans annotated with U tags, no outcome prediction is made.
Our proposed deep neural network is illustrated in Figure FIGREF8 . We utilize Bi-directional Long Short-Term Memory networks (Bi-LSTMs) and convolutional neural networks (CNNs) designed for natural language processing as building blocks for our architecture BIBREF6 , BIBREF7 . Entity recognition and outcome prediction share common parameters via a Bi-LSTM context encoder that composes a context representation at each timestep based on input words mapped to dense embeddings and character-CNN composed representations. We use the same character-CNN representation as described in a prior work BIBREF8 ; however, in this work, we omit the character type embedding. A Bi-LSTM component is used to annotate IOB tags for joint entity recognition and interaction type prediction (or, NER prediction) while a CNN with two separate dense output layers (one for PK and one for PD interactions) is used for outcome prediction. We consider NER prediction to be the main objective with outcome prediction playing a secondary role. When predicting outcome, the contextual input is arranged such that candidate entity (and effect) mentions are bound to generic tokens; the resulting representation is referred to as “entity-bound word embeddings” in Figure FIGREF8 .
We denote INLINEFORM0 as an abstract function, representing a standard bi-directional recurrent neural network with LSTM units, where INLINEFORM1 is the number of input vector representations (e.g., word embeddings) in the sequence and INLINEFORM2 and INLINEFORM3 are the dimensionality of the input and output representations respectively. We similarity denote INLINEFORM4 to represent a standard CNN that maps an INLINEFORM5 matrix to a vector representation of length INLINEFORM6 , where INLINEFORM7 is a list of window (or kernel) sizes that are used in the convolution.
Let the input be a sentence of length INLINEFORM0 represented as a matrix INLINEFORM1 , where each row corresponds to a word embedding of length INLINEFORM2 . Moreover, let INLINEFORM3 represent the word at position INLINEFORM4 of the sentence such that each of the INLINEFORM5 rows correspond to a character embedding of length INLINEFORM6 . The purpose of the context encoder is to encode each word of the input with surrounding linguistic features and long-distance dependency information. To that end, we employ the use of a Bi-LSTM network to encode S as a context matrix INLINEFORM7 where INLINEFORM8 is a hyper-parameter of the network. Concretely, DISPLAYFORM0 
where INLINEFORM0 denotes the INLINEFORM1 row of INLINEFORM2 and INLINEFORM3 is the vector concatenation operator. Essentially, for each word, we compose character representations using a CNN with a window size of three and concatenate them to pre-trained word embeddings; we stack the concatenated vectors as rows of a new matrix that is ultimately fed as input to the Bi-LSTM context encoder. The INLINEFORM4 row of INLINEFORM5 , denoted as INLINEFORM6 , represents the entire context centered at the INLINEFORM7 word. As an implementation detail, we chose INLINEFORM8 and INLINEFORM9 to be the maximum sentence and word length (according to the training data) respectively and pad shorter examples with zero vectors.
The network for the NER objective manifests as a stacked Bi-LSTM architecture when we consider both the context encoder and the entity recognition component. Borrowing from residual networks BIBREF9 , we re-inforce the input by concatenating word embeddings to the intermediate context vectors before feeding it to the second Bi-LSTM layer. Concretely, the final entity recognition matrix INLINEFORM0 is composed such that DISPLAYFORM0 
The output at each position INLINEFORM0 is INLINEFORM1 
where INLINEFORM0 is the INLINEFORM1 row of INLINEFORM2 and INLINEFORM3 and INLINEFORM4 are network parameters such that INLINEFORM5 denotes the number of possible IOB tags such as O, B-K, I-K and so on. In order to obtain a categorical distribution, we apply the SoftMax function to INLINEFORM6 such that INLINEFORM7 
where INLINEFORM0 is the vector of probability estimates serving as a categorical distribution over INLINEFORM1 tags for the word at position INLINEFORM2 . We optimize by computing the standard categorical cross-entropy loss for each of the INLINEFORM3 individual tag predictions. The final loss to be optimized is the mean over all INLINEFORM4 individually-computed losses.
A stacked Bi-LSTM architecture improves over a single Bi-LSTM architecture given its capacity to learn deep contextualized embeddings. While we showed that the stacked approach is better for this particular task in Section SECREF19 , it is not necessarily the case that a stacked approach is better in general. We offer an alternative explanation and motivation for using a stacked architecture for this particular problem based on our initial intuition as follows. First, we note that a standalone Bi-LSTM is not able to handle the inference aspect of NER, which entails learning IOB constraints. As an example, in the IOB encoding scheme, it is not possible for a I-D tag to immediately follow a B-E tag; in this way, the prediction of a tag is directly dependent on the prediction of neighboring tags. This inference aspect is typically handled by a linear-chain CRF. We believe that a stacked Bi-LSTM at least partially handles this aspect in the sense that the first Bi-LSTM (the context encoder) is given the opportunity to form independent preliminary decisions while the second Bi-LSTM is tasked with to making final decisions (based on preliminary ones) that are more globally consistent with respect to IOB constraints.
To predict outcome, we construct a secondary branch in the network path that involves convolving over the word and context embeddings made available in earlier layers. We first define a relation representation INLINEFORM0 that is produced by convolving with window sizes 3, 4, and 5 over the context vectors concatenated to entity-bound versions of the original input; concretely, INLINEFORM1 
where INLINEFORM0 is the entity-bound version of INLINEFORM1 . Based on this outcome representation, we compose two separate softmax outputs: one for PK interactions and one for PD interactions. Concretely, the output layers are INLINEFORM2 
and INLINEFORM0 
where INLINEFORM0 and INLINEFORM1 are probability estimates serving as a categorical distribution over the outcome label space for PD and PK respectively and INLINEFORM2 , INLINEFORM3 , INLINEFORM4 , and INLINEFORM5 are parameters of the network. For PK, INLINEFORM6 given there are 20 possible NCI Thesaurus codes corresponding to PK outcomes. For PD, INLINEFORM7 as it is a binary classification problem to assess whether the precipitant and effect pair encoded by INLINEFORM8 are linked. We optimize using the standard categorical cross-entropy loss on both objectives.
In NLM-180, there is no distinction between triggers and effects; moreover, PK effects are limited to coarse-grained (binary) labels corresponding to increase or decrease in function measurements. Hence, a direct mapping from NLM-180 to Training-22 is impossible. As a compromise, NLM-180 “triggers” were mapped to Training-22 triggers in the case of unspecified and PK interactions. For PD interactions, we instead mapped NLM-180 “triggers” to Training-22 effects, which we believe to be appropriate based on our manual analysis of the data. Since we do not have both trigger and effect for every PD interaction, we opted to ignore trigger mentions altogether in the case of PD interactions to avoid introducing mixed signals. While trigger recognition has no bearing on relation extraction performance, this policy has the effect of reducing the recall upperbound on NER by about 25% (more later on upperbound). To overcome the lack of fine-grained annotations for PK outcome in NLM-180, we deploy the well-known bootstrapping approach BIBREF10 to incrementally annotate NLM-180 PK outcomes using Training-22 annotations as seed examples. To mitigate the problem of semantic drift, in each bootstrap cycle, we re-annotated by hand predictions that were not consistent with the original NLM-180 coarse annotations (i.e., active learning BIBREF11 ).
We train the three objective losses (NER, PK outcome, and PD outcome) in an interleaved fashion at the minibatch BIBREF12 level. We use word embeddings of size 200 pre-trained on the PubMed corpus BIBREF13 as input to the network; these are further modified during back-propagation. For the character-level CNN, we set the character embedding size to 24 with 50 filters over a window size of 3; the final character-CNN composition is therefore of length 50. For each Bi-LSTM, the hidden size is set to 100 such that context vectors are 200 in length. For outcome prediction, we used window sizes of 3, 4, and 5 with 50 filters per window size; the final vector representation for outcome prediction is therefore 150 in length.
A held-out development set of 4 drug labels is used for tuning and validation. The models are trained for 30 epochs with check-pointing; only the check-point with the best performance on the development set is kept for testing. We dynamically set the mini-batch size INLINEFORM0 as a function of the number of examples INLINEFORM1 such that the number of training iterations is roughly 300 per epoch (and also constant regardless of training data size); concretely, INLINEFORM2 . As a form of regularization, we apply dropout BIBREF14 at a rate of 50% on the hidden representations immediately after a Bi-LSTM or CNN composition. The outcome objectives are trained such that the gradients of the context encoder weights are downscaled by an order of magnitude (i.e., one tenth) to encourage learning at the later layers. When learning on the NER objective – the main branch of the network – the gradients are not downscaled in the same manner. Moreover, when training on the NER objective, we upweight the loss penalty on “relation” tags (non-O tags) by a factor of 10, which forces the model to prioritize differentiation between different types of interactions over span segmentation. We additionally upweight the loss penalty by a factor of 3 on Training-22 examples compared to NLM-180 examples. We optimize using the Adam BIBREF15 optimization method. These hyper-parameters were tuned during initial experiments.
In this section, we present and discuss the results of our cross-validation experiments. We then describe the “runs” that were submitted as challenge entries and present our official challenge results. We discuss these results in Section SECREF28 .
We present the results of our initial experiments in Table TABREF20 . Evaluations were produced as as result of 11-fold cross-validation over Training-22 with two drug labels per fold. Instead of macro-averaging over folds, and thereby weighting each fold equally, we evaluate on the union of all 11 test-fold predictions.
The upperbound in Table TABREF20 is produced by reducing Training-22 (with gold labels) to our sequence-tagging format and then reverting it back to the original official XML format. Lowered recall is mostly due to simplifying assumptions; e.g., we only consider non-overlapping mentions. For coordinated disjoint cases such as “X and Y inducers”, we only considered “Y inducers” in our simplifying assumption. Imperfect precision is due to discrepancies between the tokenization scheme used by our method and that used to produce gold annotations; this leads to the occasional mismatch in entity offsets during evaluation.
Using a stacked Bi-LSTM trained on the original 22 training examples (Table TABREF20 ; row 1) as our baseline, we make the following observations. Incorporating NLM-180 resulted in a significant boost of more than 20 F1-points in relation extraction performance and more than 10 F1-points in NER performance (Table TABREF20 ; row 2), despite the lowered upperbound on NER recall as mentioned in Section SECREF5 . Adding character-CNN based word representations improved performance marginally, more so for NER than relation extraction (Table TABREF20 ; row 3). We also implemented several tweaks to the pre-processing and post-processing aspects of the model based on preliminary error analysis including (1) using drug class mentions (e.g., “diuretics”) as proxies if the drug label is not mentioned directly; (2) removing modifiers such as moderate, strong, and potent so that output conforms to official annotation guidelines; and (3) purging predicted mentions with only stopwords or generic terms such as “drugs” or “agents.” These tweaks improved performance by more than two F1-points across both metrics (Table TABREF20 ; row 4).
Based on early experiments with simpler models tuned on relaxed matching (not shown in Table TABREF20 and not directly comparable to results displayed in Table TABREF20 ), we found that a stacked Bi-LSTM architecture improves over a single Bi-LSTM by approximately four F1-points on relation extraction (55.59% vs. 51.55% F1 tuned on the relaxed matching criteria). We moreover found that omitting word embeddings as input at the second Bi-LSTM results in worse performance at 52.91% F1.
We also experimented with using Temporal Convolution Networks (TCNs) BIBREF16 as a “drop-in” replacement for Bi-LSTMs. Our attempts involved replacing only the second Bi-LSTM with a TCN (Table TABREF20 ; row 4) as well as replacing both Bi-LSTMs with TCNs (Table TABREF20 ; row 5). The results of these early experiments were not promising and further fine-tuning may be necessary for better performance.
Our final system submission is based on a stacked Bi-LSTM network with character-CNNs trained on both Training-22 and NLM-180 (corresponding to row 4 of Table TABREF20 ). We submitted the following three runs based on this architecture:
A single model.
An ensemble over ten models each trained with randomly initialized weights and a random development split. Intuitively, models collectively “vote” on predicted annotations that are kept and annotations that are discarded. A unique annotation (entity or relation) has one vote for each time it appears in one of the ten model prediction sets. In terms of implementation, unique annotations are incrementally added (to the final prediction set) in order of descending vote count; subsequent annotations that conflict (i.e., overlap based on character offsets) with existing annotations are discarded. Hence, we loosely refer to this approach as “voting-based” ensembling.
A single model with pre/post-processing rules to handle modifier coordinations; for example, “X and Y inducers” would be correctly identified as two distinct entities corresponding to “X inducers” and “Y inducers.” Here, we essentially encoded “X and Y inducers” as a single entity when training the NER objective; during test time, we use simple rules based on pattern matching to split the joint “entity” into its constituents.
Eight teams participated in task 1 while four teams participated in task 2. We record the relative performance of our system (among others in the top 5) on the two official test sets in Table TABREF24 . For each team, we only display the performance of the best run for a particular test set. Methods are grouped by the data used for training and ranked in ascending order of primary relation extraction performance followed by entity recognition performance. We also included a single model trained solely on Training-22, that was not submitted, for comparison. Our voting-based ensemble performed best among the three systems submitted by our team on both NER and relation extraction. In the official challenge, this model placed second overall on both NER and relation extraction.
Tang et al. BIBREF20 boasts the top performing system on both tasks. In addition to Training-22 and NLM-180, the team trained and validated their models on a set of 1148 sentences sampled from DailyMed labels that were manually annotated according to official annotation guidelines. Hence, strictly speaking, their method is not directly comparable to ours given the significant difference in available training data.
While precision was similar between the three systems (with exceptions), we observed that our ensemble-based system benefited mostly from improved recall. This aligns with our initial expectation (based on prior experience with deep learning models) that an ensemble-based approach would improve stability and accuracy with deep neural models. Although including NLM-180 as training data resulted in significant performance gains during 11-fold cross validation, we find that the same improvements were not as dramatic on either test sets despite the 800% gain in training data. As such, we offer the following analysis. First, we suspect that there may be a semantic or annotation drift between these datasets as annotation guidelines evolve over time and as annotators become more experienced. To our knowledge, the datasets were annotated in the following order: NLM-180, Training-22, and finally Test Sets 1 and 2; moreover, Test Sets 1 and 2 were annotated by separate groups of annotators. Second, having few but higher quality examples may be more advantageous than having many but lower quality examples, at least for this particular task where evaluation is based on matching exact character offsets. Finally, we note that the top performing system exhibits superior performance on Test Set 1 compared to Test Set 2; interestingly, we observe an inverse of the scenario in our own system. This may be an indicator that our system struggles with data that is more “sparse” (as previously defined in Section SECREF2 ).
We presented a method for jointly extracting precipitants and their interaction types as part of a multi-task framework that additionally detects interaction outcome. Among three “runs”, a ten model voting-ensemble was our best performer. In future efforts, we will experiment with Graph Convolution Networks BIBREF21 over dependency trees as a “drop-in” replace for Bi-LSTMs to assess its suitability for this task.
This research was conducted during TT's participation in the Lister Hill National Center for Biomedical Communications (LHNCBC) Research Program in Medical Informatics for Graduate students at the U.S. National Library of Medicine, National Institutes of Health. HK is supported by the intramural research program at the U.S. National Library of Medicine, National Institutes of Health. RK and TT are also supported by the U.S. National Library of Medicine through grant R21LM012274.

Figure and table captions:
Figure 1: An example illustrating the DDI task
Table 1: Characteristics of datasets
Table 2: Example of the tagging scheme
Figure 2: The multi-task neural network for DDI extraction
Table 3: Preliminary results based on 11-fold cross validation over Training-22 with two held-out drug labels per fold. When NLM-180 is incorporated, the training data used for each fold consists of 20 non-held out drug labels from Training-22 and all 180 drug labels from NLM-180.
Table 4: Comparison of our method with that of other teams in the top 5. Only the best performing method of each team is shown; methods are grouped by available training data and ranked in ascending order by relation extraction (primary) performance followed by entity recognition performance. 1This model was not submitted and is shown for reference only 2HS refers to a private dataset of 1148 sentences manually-annotated by Tang et al. [21] according to official guidelines

Questions about this paper:
What were the sizes of the test sets?

Answers:
Test set 1 contained 57 drug labels and 8208 sentences and test set 2 contained 66 drug labels and 4224 sentences