Extracting Drug-Drug and Protein-Protein Interactions from Text using a Continuous Update of Tree-Transformers (2023)

Abstract: Understanding biological mechanisms requires determining mutual protein-protein interactions (PPI). Obtaining drug-drug interactions (DDI) from scientific articles provides important information about drugs. Extracting such medical entity interactions from biomedical articles is challenging due to complex sentence structures. To address this issue, our proposed model utilizes tree-transformers to generate the sentence representation first, and then a sentence-to-word update step to fine-tune the word embeddings which are again used by the tree-transformers to generate enriched sentence representations. Using the tree-transformers helps the model preserve syntactical information and provide semantic information. The fine-tuning provided by the continuous update step adds improved semantics to the representation of each sentence. Our model outperforms other prominent models with a significant performance boost on the five standard PPI corpora and a performance boost on the one benchmark DDI corpus that are used in our experiments.

## introduction with the rapid expansion of scientific literature, most biological knowledge is now stored as text and can be accessed through scientific publications. the medline database has experienced a steady annual growth of over 4% for the last two decades, currently boasting a collection of over 29 million records from diverse sources. this is an increase of 3 million records compared to 2020 and over 8 million records compared to 2014, as cited in @xcite . the vast amount of textual data in biomedical research articles presents an invaluable opportunity for automated biomedical information retrieval to leverage this wealth of information. as biomedical data continues to expand exponentially and the inherent complexity of textual representations, automated methods for information retrieval plays a pivotal role in aiding biolo-gists in locating pertinent information, managing databases, and providing decision support to medical practitioners. numerous studies have been conducted to extract valuable information from these texts, encompassing various domains such as protein-protein interactions, chemical-disease relationships, clinical correlations, drug-drug interactions, and more. the internal biological processes within a cell, such as cellular organization, signal transduction, and immune response, are predominantly governed by interactions between different proteins @xcite . to comprehend the molecular mechanisms underlying these biological processes, knowledge of protein-protein interactions (ppi) is indispensable @xcite . these interactions have significant relevance in the biomedical domain, including drug target examination @xcite and signal proteins @xcite . consequently, the identification of protein-protein interactions (ppis) leads to a deeper understanding of the functions, regulation, and communication between various proteins @xcite . the primary objective of ppi recognition is to extract the relationships between protein entities mentioned in a document @xcite . a drug-drug interaction (ddi) refers to a modification in the effects of one drug due to the presence of another drug @xcite . while clinical trials for pre-market identification of interactions are challenging, obtaining ddi information from scientific articles is a faster, cost-effective, and reliable approach to reducing adverse effects. furthermore, in order to practice evidence-based medicine and mitigate drug-related accidents, comprehensive extraction of ddi knowledge from pharmaceutical literature is crucial @xcite . automatic ddi extraction can prove highly beneficial for the pharmaceutical industry, offering an efficient means of reducing the time spent by healthcare professionals reviewing the medical literature. biomedical literature contains a wealth of information about protein-protein interactions (ppis) and drug-drug interactions (ddis), but this information is often unstructured. manual extraction of these interactions from biomedical literature is a laborious, resource-intensive, and costly task, given the sheer volume of published studies @xcite . consequently, the automatic extraction of ppis and ddis from biomedical literature has emerged as a vital research area, garnering attention from numerous researchers. while the information may be scattered throughout the document, the current study focuses on detecting these interactions within individual sentences, similar to previous studies @xcite @xcite . an instance of a sentence that demonstrates protein-protein interactions can be found in the study by @xcite , where it states: "at 89.3 nmol/l, maximal migration of ccr1 and ccr8 transfected cells was prompted by lec and at 5.6 nmol/l, cell adhesion also occurred." this sentence highlights two protein-protein interactions involving lec and ccr1, as well as lec and ccr8. however, it is important to note that there is no correlation mentioned between proteins ccr1 and ccr8 in this context. an instance of a ddi-containing sentence is @xcite : "to determine whether probenecid has a direct effect on the distribution of cloxacillin, the elimination and distribution of cloxacillin was studied in six patients, five lacking kidney function and one with a partially impaired renal function, in the presence or absence of probenecid." this sentence mentions two drugs: probenecid, and cloxacillin. however, the interaction between them is negative, as no concrete interaction is stated. during the extraction of relationships between target proteins or drugs, we have addressed three key concerns. firstly, how to tackle the challenge of retrieving relations when the mentioned proteins or drugs are widely separated in the text. secondly, how to preserve better semantics by handling the phrasal structure of the text, allowing for the effective extraction of ppis or ddis and capturing relevant information. lastly, what is the impact of updating word and non-leaf node representations in the tree-structured networks based on the sentence at hand, as opposed to using fixed representations from pre-trained models, and how this influences the generated sentence representation for the task of ppi and ddi extractions. to address the above-mentioned three considerations we have proposed a model combining a constituency tree-transformer (for preserving phraselevel information in the text), and a dependency tree-transformer (to consider relations between long distant drugs or proteins in the text) where each of them generates sentence representations which are then combined. finally, a sentenceto-word update step is introduced following the concept from @xcite to update the word and non-leaf nodes of the tree-transformers to generate refined sentence representation. this approach serves the purpose of fine-tuning bertbased word embeddings for these tasks. but the advantage of this approach is that we do not need to fine-tune millions of parameters in the bertbased models. our study includes a thorough analysis of the performance of the proposed models on benchmark ppi and ddi datasets. the results demonstrate the superiority of our proposed model compared to previous prominent models in the field. the comprehensive analysis highlights the effectiveness and efficacy of our approach in accurately extracting protein-protein and drug-drug interactions from biomedical literature. foot_0 ## related work in the initial stages of biomedical entity relation extraction research, co-occurrence and pattern recognition techniques were commonly employed @xcite . however, with advancements in technology, machine learning techniques have gained prominence due to their superior performance. early approaches involved feature engineering and kernel methods to construct a feature set, followed by classification using support vector machines or other classifiers @xcite . in recent years, deep learning techniques, leveraging the widespread use of deep learning in natural language processing (nlp), have been successfully applied to ppi and ddi extraction in several re-search works @xcite @xcite . @xcite proposed a three-channel convolution neural network for extracting ppis from the text. recent work in ppi and ddi extraction often utilizes recurrent neural network (rnn) models that treat textual representations as sequences @xcite @xcite . however, these models may miss semantic compositions when biomedical entities lie at distant positions in the text, as they only consider word order and ignore linguistic structure @xcite . in contrast, recursive neural networks, also known as tree-structured neural network models @xcite , process sentences represented in a parsed tree form, capturing both syntax and semantics in a more effective manner. there have also been investigations into graph-based methods, where models operate on a fully connected graph composed of either word or phrase nodes @xcite . these approaches aim to leverage the structural information present in the data for improved performance in ppi and ddi extraction. @xcite utilized molecular structure and description of the drugs for retrieving ddis. @xcite fine-tuned pubmedbert to extract relations between drugs. following this, @xcite utilized a knowledge graph with pubmedbert for the ddi extraction task. ## proposed model in this section, we provide details of our model for the protein-protein and drug-drug interaction extraction tasks. our model contains three key modules: two tree-transformers, as described in @xcite , for preserving the semantic and syntactical information, and a sentence-to-word update step for updating the word and intermediate node representations in the tree-transformers to generate refined representations of the sentences. in this current work, we have added an update of the word embeddings after the sentence-to-word update step which enriches the input to the combination of the two tree-transformers and the heterogeneous graph attention network, which were first proposed for the ppi extraction task in singha roy and mercer (2023). in this section, we first discuss how each module functions individually, and then elaborate on how these modules are integrated into our proposed model with the expanded workflow. ## experimental setup and analysis of results in this section, the performance of the proposed model is evaluated using the f1-score. the ppi (protein-protein interaction) and ddi (drug-drug interaction) extraction tasks have been formulated as classification tasks. the section also includes a demographical overview of the five primary ppi corpora and the standard ddi corpus used in the evaluation, as well as a discussion of the preprocessing techniques employed on these corpora. the efficacy of the proposed model is compared to leading sequential, tree-structured, and graphbased architectures that have been previously propounded for these biomedical entity inter-relation extraction tasks. ## conclusions and future work from these results and discussions in the previous sections, we can conclude that our model performs significantly better than the other prominent models even without using any additional features. the tree-transformers enable the proposed model to capture better semantics along with syntactical information. additionally, the sentence-to-word update module provides more task-specific context-aware information, generating enriched word embeddings that further enhance the sentence representations for the ppi and ddi extraction tasks. although the model has achieved a significantly improved performance over the previous models, still there is scope for further improvement. including a knowledge-graph, like in @xcite , may improve the model performance with proper knowledge about the ddi extraction task. moreover, the current models find ppis and ddis that are given in a single sentence. using an additional layer of hierarchy that represents documentto-sentence relations over the sentence-to-word update module, this work can be extended to extract relations between biomedical entities lying in different sentences.