DDI - M u G : Multi-aspect Graphs for Drug-Drug Interaction Extraction (2022)

Abstract: Drug-drug interaction (DDI) may leads to adverse reactions in patients, thus it is important to extract such knowledge from biomedical texts. However, previously proposed approaches typically focus on capturing sentence-aspect information while ignoring valuable knowledge concerning the whole corpus. In this paper, we propose a Multi-aspect Graph-based DDI extraction model, named DDI-MuG. We first employ a bio-specific pre-trained language model to obtain the token contextualized representations. Then we use two graphs to get syntactic information from input instance and word co-occurrence information within the entire corpus, respectively. Finally, we combine the representations of drug entities and verb tokens for the final classification. It is encouraging to see that the proposed model outperforms all baseline models on two benchmark datasets. To the best of our knowledge, this is the first model that explores multi-aspect graphs to the DDI extraction task, and we hope it can establish a foundation for more robust multi-aspect works in the future.

## introduction according to statistics from the u.s. centers of disease control and prevention, from 2015 to 2018, 48.6 % of americans used at least one prescription drug in 30 days 1 . more seriously, 20% of the elderly took more than 10 drugs simultaneously @xcite . however, drug-drug interaction (ddi) may occur when patients take multiple drugs, resulting in reduced drug effectiveness or even, possibly, adverse drug reactions (adrs) @xcite . therefore, the study of ddi extraction can be considerably important to patients' healthcare, as well as clinical research. currently, a number of drug databases, such as dailymed @xcite , twosides @xcite and @xcite can be used for retrieving ddi knowledge directly. however, with the exponential growth in biomedical literature, huge amounts of the most current and valuable knowledge remain hidden in biomedical literature @xcite . thus, the development of an automatic tool to extract ddi is an urgent need. during the past few years, various deep learningbased approaches, such as @xcite @xcite @xcite @xcite have been proposed to extract ddi knowledge. it is worth noting that compared with convolutional neural networks (cnns) and long short-term memory (lstm), which are sequential-based architectures, graph neural networks (gnns) can better deal with complex structural knowledge. based on this, @xcite combined a bio-specific bert @xcite and graph convolutional network (gcn) @xcite to capture contextualized representation together with syntactic knowledge. @xcite adopted the graph attention network (gat) @xcite on an enhanced dependency graph to obtain higher-level drug representations for ddi extraction. however, as examples in to alleviate the issues mentioned above, in this work, we propose a multi-aspect graphs-based ddi extraction model, ddi-mug, which can make use of the information in both sentence and corpus aspects. first, we use pubmedbert to obtain sentence semantic representation. we then apply a gcn with an average pooling layer to capture syntactic features from the input instance, and another gcn with average pooling is employed to model the word co-occurrence in the corpus level simultaneously. after that, an attentive pooling is used to integrate and obtain the optimal feature from the output of pubmedbert and both sentence-aspect and corpus-aspect graphs. finally, we employ a fully connected neural network in the output layer for the classification. our proposed model is evaluated on two benchmark datasets: ddiextraction-2013 @xcite and tac 2018 corpora @xcite . experimental results show that our proposed model improves the performance of ddi extraction effectively. to recap, the main contributions of our work can be summarized as follows: the rest of the paper is organized as follows. first, we introduce the background in section 1. then, several related works are introduced in section 2. next, in section 3, we explain the framework in the proposed model in detail. we then describe the two benchmark datasets, evaluation metrics, and parameters setting in section 4. section 5 presents the experimental results and discussion, and finally, we conclude this work in section 6. knowledge in many applications is exceedingly complex for a single-aspect network to learn robust representations. multi-aspect networks have thus emerged naturally in different fields. khan and blumenstock (2019) developed a multi-aspect gcns model to consider different aspects of phone networks for poverty research. they employed subspace analysis and a manifold ranking procedure in order to merge multiple views and prune the graph, respectively. liu et al. (2020) first constructed semantic-based, syntactic-based, and sequential-based text graphs, and then utilized an inter-graph propagation to coordinate heterogeneous information among graphs. in order to exploit richer sources of graph edge information, gong and cheng (2019) resorted to multidimensional edge weights to encode edge directions. similarly, huang et al. (2020) used multidimensional edge weights to exploit multiple attributes, adapting the edge weights before entering into the next layer. ## methods the architecture of the proposed model is illustrated in figure 1 . first, we obtain the contextual semantic representation of the input instances by pubmedbert. then, a sentence-aspect graph is constructed to encode the syntactic feature from the dependency path, while a corpus-aspect graph is used to explore word co-occurrence within the entire corpus. based on the vocabulary and instances analysis, we find that the part-of-speech (pos) tag of words, especially words corresponding to verbs, might be helpful for the final representation. therefore, we subsequently feed the representations of verbs and drug entities from pub-medbert, together with the two graphs, into an attentive pooling layer, to distinguish important features from all representations. finally, a fully connected layer with softmax is employed to perform the classification. the process is described in the following subsections in detail. ## experiments in our experiments, two public ddi extraction corpora, i.e., ddiextraction-2013 and tac 2018, were used to evaluate the proposed model. this section introduces the two corpora in detail and then presents the evaluation metrics and parameters setting. ## conclusions in this paper, we propose ddi-mug, a novel multiaspect graphs framework for ddi extraction task. concretely, a bio-specific pre-trained language model, pubmedbert, is firstly employed to encode the context information of each word from the aspect of sentence semantic information. then, two graphs are utilized to explore sentence syntactic and corpus word co-occurrence information, respectively. after that, attentive pooling mechanism is employed to update the representations of drug entities and verbs. finally, by feeding the concatenated representation of the two drugs and verbs into a fully connected and softmax classifier, the interaction between two drugs is obtained. extensive comparison experiments with baseline models on two public datasets verify the effectiveness of utilizing multi-aspect graphs in the ddi extraction task. for the future work, there are at least two directions could be considered. firstly, the performance on categories with small training samples, like int in the ddiextraction-2013 corpora, is unsatisfactory. the solution of contrastive learning can be explored. secondly, drug knowledge from external databases could be integrated in the architecture for richer drug representations.