A Dual-Attention Network for Joint Named Entity Recognition and Sentence Classification of Adverse Drug Events (2020)

Abstract: An adverse drug event (ADE) is an injury resulting from medical intervention related to a drug. Automatic ADE detection from text is either fine-grained (ADE entity recognition) or coarse-grained (ADE assertive sentence classification), with limited efforts leveraging inter-dependencies among the two granularities. We instead propose a multi-grained joint deep network to concurrently learn the ADE entity recognition and ADE sentence classification tasks. Our joint approach takes advantage of their symbiotic relationship, with a transfer of knowledge between the two levels of granularity. Our dual-attention mechanism constructs multiple distinct representations of a sentence that capture both task-specific and semantic information in the sentence, providing stronger emphasis on the key elements essential for sentence classification. Our model improves state-of- art F1-score for both tasks: (i) entity recognition of ADE words (12.5% increase) and (ii) ADE sentence classification (13.6% increase) on MADE 1.0 benchmark of EHR notes.

## introduction background. adverse drug events (ades), injuries resulting from medical intervention, are a leading cause of death in the united states and cost around @xmath0130 billion every year @xcite . early detection of ade incidents aids in the timely assessment, mitigation and prevention of future occurrences of ades. natural language processing techniques have been recognized as instrumental in identifying ades and related information from unstructured text fields of spontaneous reports and electronic health records (ehrs) and thus in improving drug safety monitoring and pharmacovigilance @xcite . fine-grained ade detection identifies named ade entities at the word-level, while coarsegrained ade detection (also ade assertive text classification) identifies complete sentences describing drug-related adverse effects. @xcite 's system for identification of ade assertive sentences in medical case reports targets the important application of detecting underreported and under-documented adverse drug effects. lastly, multi-grained ade detection identifies ade information at multiple levels of granularity, namely, both entity and sentence level. as example, figure 1 displays ade and non-ade sentences. the first is an ade sentence where the mentions of drugname and ade entities have the appropriate relationship with each other. second and third sentences show that the mention of an ade entity by itself is not sufficient to assert a drug-related adverse side effect. recently, deep learning-based sequence approaches have shown some promise in extracting fine-grained ades and related named entities from text @xcite . however, the prevalence of entity-type ambiguity remains a major hurdle, such as, distinguishing between indication entities as the reason for taking a drug versus ade entities as unintended outcomes of taking a drug. coarsegrained sentence-level detection performs well in identifying ade descriptive sentences, but is not equipped to detect fine-grained information such as words associated with ade related named entities. unfortunately, when the interaction between these two extraction tasks is ignored, we miss the opportunity of the transfer of knowledge between the ade entity and sentence prediction tasks. attention-based neural network models have been shown to be effective for text classification tasks @xcite from alignment attention in translation @xcite to supervising attention in binary text classification @xcite . previous approaches typically apply only a single round of attention focusing on simple semantic information in our ade detection task, instead, key elements of the sentence can be linked to multiple categories of task-specific semantic information of the named entities (ade, drug, indication, severity, dose etc.). thus, single attention is insufficient in exploring this multi-aspect information and consequently risks losing important cues. proposed approach. in our work, we tackle the above shortcomings by designing a dual-attention based neural network model for multi-grained joint learning, called mgade, that jointly identifies both ade entities and ade assertive sentences. the design of mgade is inspired by multi-task recurrent neural network architectures for jointly learning to label tokens and sentences in a binary classification setting @xcite . in addition, our model makes use of a supervised selfattention mechanism based on entity-level predictions to guide the attention function -aiding it in tackling the above entity-type ambiguity problem. we also introduce novel strategies of constructing multiple complementary sentence-level representations to enhance the performance of sentence classification. our key contributions include: 1. joint model. we jointly model ade entity recognition as a multi-class sequence tagging problem and ade assertive text classification as binary classification. our model leverages the mutually beneficial relationships between these two tasks, e.g. ## conclusion we propose a dual-attention network for multigrained ade detection to jointly identify ade entities and ade assertive sentences from medical narratives. our model effectively supports knowledge sharing between the two levels of granularity, i.e., words and sentences, improving the overall quality of prediction on both tasks. our solution features significant performance improvements over stateof-the-art models on both tasks. our mgade architecture is pluggable, in that other sequential learning models including bert @xcite or other models for sequence labelling and text classification could be substituted in place of the bi-lstm sequential representation learning model. we leave this enhancement of our model and its study to future work.