BERTC hem- DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information (2020)

Abstract: Traditional biomedical version of embeddings obtained from pre-trained language models have recently shown state-of-the-art results for relation extraction (RE) tasks in the medical domain. In this paper, we explore how to incorporate domain knowledge, available in the form of molecular structure of drugs, for predicting Drug-Drug Interaction from textual corpus. We propose a method, BERTChem-DDI, to efficiently combine drug embeddings obtained from the rich chemical structure of drugs (encoded in SMILES) along with off-the-shelf domain-specific BioBERT embedding-based RE architecture. Experiments conducted on the DDIExtraction 2013 corpus clearly indicate that this strategy improves other strong baselines architectures by 3.4% macro F1-score.

## introduction concurrent administration of two or more drugs to a patient to cure an ailment might lead to positive or negative reaction (side-effect). these kinds of interactions are termed as drug-drug interactions (ddis). predicting drug-drug interactions (ddi) is a complex task as it requires to understand the mechanism of action of two interacting drugs. a large number of efforts by the researchers have been witnessed in terms of automatic extraction of ddis from the textual corpus (sahu and anand, 2018), @xcite , @xcite , @xcite and predicting unknown ddi from the knowledge graph @xcite , @xcite . automatic extraction of ddi from texts aids in maintaining the databases with high coverage and help the medical experts in their diagnosis and novel experiments. in parallel to the progress of ddi extraction from the textual corpus, some efforts have been observed recently where the researchers came up with various strategies of augmenting chemical structure information of the drugs @xcite and textual description of the drugs @xcite to improve drug-drug interaction prediction performance from corpus and knowledge graphs. the ddi prediction from the textual corpus has been framed by the earlier researchers as relation classification problem. earlier methods (sahu and anand, 2018), @xcite , @xcite , @xcite for relation classification are based on cnn or rnn based neural networks. recently, with the massive success of the pretrained language models @xcite , @xcite in many nlp classification / sequence labeling tasks, we formulate the problem of ddi classification as a relation classification task by leveraging both the entities and sentence-level information. we propose a model that leverages both domain-specific contextual embeddings (bio-bert) @xcite from the target entities and also external chemical structure information of the target entities (drugs). in the recent years, representation learning has played a pivotal role in solving various machine learning tasks. in addition to information of drug entities from the text, we make use of the rich hidden representation obtained from the molecule generation using variational auto-encoder @xcite representation of the drugs to learn the chemical structure representation. during unsupervised learning of chemical structure information of the drugs using variational autoencoder @xcite , we make use of the canonical smiles representation (simplified molecular input line entry system) obtained from the drugbank @xcite . we illustrate the overview of the proposed method in figure 1 . experiments conducted on the ddiextraction 2013 corpus @xcite reveals that this method outperforms the existing baseline models and is in line with the new direction of research of fusing various information to boost ddi classification performance. in a nutshell, the major contributions of this work are summarized as follows: ## methodology given a sentence s with target drug entities d 1 and d 2 , the task is to classify the type of relation (y) the drugs hold between them, @xmath0, ...., y n ), where n denotes the number of relation types. ## results and discussion in this section, we provide a detailed analysis of the various results and findings that we have observed during experimentation. we have demonstrated ## conclusion in this paper, we develop an approach for ddi relation classification based on pre-trained language model and chemical structure representation of drugs. experiments on the benchmark ddi dataset proves the efficacy of our method. possible directions of further research might be to explore knowledge graph based drug representation combined with textual description and other relation specific embeddings obtained from various ontologies.