Amalgamation of protein sequence, structure and textual information for improving protein-protein interaction identification (2020)

Abstract: An in-depth exploration of protein-protein interactions (PPI) is essential to understand the metabolism in addition to the regulations of biological entities like proteins, carbohydrates, and many more. Most of the recent PPI tasks in BioNLP domain have been carried out solely using textual data. In this paper, we argue that incorporating multimodal cues can improve the automatic identification of PPI. As a first step towards enabling the development of multimodal approaches for PPI identification, we have developed two multi-modal datasets which are extensions and multi-modal versions of two popular benchmark PPI corpora (BioInfer and HRPD50). Besides, existing textual modalities, two new modalities, 3D protein structure and underlying genomic sequence, are also added to each instance. Further, a novel deep multi-modal architecture is also implemented to efficiently predict the protein interactions from the developed datasets. A detailed experimental analysis reveals the superiority of the multi-modal approach in comparison to the strong baselines including unimodal approaches and state-of the-art methods over both the generated multi-modal datasets. The developed multi-modal datasets are available for use at https://github.com/sduttap16/MM_PPI_NLP .

## introduction understanding protein-protein interactions (ppi) is indispensable to comprehend different biological processes such as translation, protein functions @xcite , gene functions @xcite , metabolic pathways, etc. the ppi information helps researchers to discover disease mechanisms and plays seminal role in designing the therapeutic drugs @xcite . over the years, a significant amount of protein-protein interaction information has been published in scientific articles in unstructured text formats. however, in recent years, there has been an exponential rise in the number of biomedical publications @xcite . therefore, it becomes imperative, urgent and of extreme interest to develop an intelligent information extraction system to assist biologists in curating and maintaining ppi databases. this pressing need has motivated biomedical natural language processing (bionlp) researchers to automatically extract ppi information by exploring various ai techniques. recent advancements in deep learning @xcite have opened up new avenues in solving different well-known problems ranging from computational biology @xcite , machine translations @xcite , image captioning @xcite . subsequently, there is a notable trend in using deep learning for solving different natural language processing (nlp) tasks in the biomedical and clinical domains @xcite including the identification of protein-protein interactions from biomedical corpora @xcite . multi-modal deep learning models, combining information from multiple sources/modalities, show promising results compared to the conventional single modal-based models while solving various nlp tasks like sentiment and emotion recognition @xcite , natural language generation, machine translation @xcite @xcite etc. there exist few popular multi-modal datasets which are extensively used in solving various problems in nlp like emotion recognition from conversations @xcite , image captioning @xcite , sentiment analysis @xcite , etc. compared to single modal-based approaches, multi-modal techniques provide a more comprehensive perspective of the dataset under consideration. despite the popularity of multi-modal approaches in solving traditional nlp tasks, there is a dearth of multi-modal datasets in bionlp domain especially for the ppi identification task. the available ppi benchmark datasets contain solely the textual knowledge of different protein pairs, which do not help in anticipating the molecular properties of the proteins. hence, along with the textual information, incorporation of molecular structure or underlying genomic sequence can aid in understanding the regulations of the protein interactions. the integration of multi-modal features can help in obtaining deeper insights but the concept of multimodal architecture, for textual and biological aspects, has not been cultivated much in the bionlp domain @xcite . ## related works there are few works @xcite @xcite which focus on rule-based ppi information extraction method such as co-occurrence rules (stapley and benoit, 1999) from the biomedical texts. in @xcite , relation is extracted from entire sentence by considering the shallow syntactic information. @xcite utilize semi-supervised learning and cosine similarity to find the shortest dependency path (sdp) between protein entities. some important kernel-based methods for ppi extraction task are graph kernel @xcite , bagof-word (bow) kernel @xcite , editdistance kernel @xcite and all-path kernel @xcite . @xcite presented an attention-based bidirectional long shortterm memory networks (bilstm) model that uses sdp between protein pairs, latent pos and position generated instances of our multi-modal dataset protein pairs gene pairs pdb id pairs ensembl id pairs interaction type protein1 protein2 gene1 gene2 pdb1 pdb2 ensembl1 ensembl2 megalin and cubilin: multifunctional endocytic receptors protein1 and protein2 are two structurally different endocytic receptors that interact to serve such functions megalin cubilin lrp2 cubn 2m0p 3kq4 ensg00000081479 ensg00000107611 true megalin and protein1: multifunctional endocytic receptors megalin and protein2 are two structurally different endocytic receptors that interact to serve such functions cubilin cubilin cubn cubn 3kq4 3kq4 ensg00000107611 ensg00000107611 false protein1 and cubilin: multifunctional endocytic receptors megalin and protein2 are two structurally different endocytic receptors that interact to serve such functions cubilin megalin cubn lrp2 3kq4 2m0p ensg00000107611 ensg00000081479 false megalin and protein1: multifunctional endocytic receptors protein2 and cubilin are two structurally different endocytic receptors that interact to serve such functions cubilin megalin cubn lrp2 3kq4 2m0p ensg00000107611 ensg00000081479 false protein1 and protein2: multifunctional endocytic receptors megalin and cubilin are two structurally different endocytic receptors that interact to serve such functions cubilin megalin cubn lrp2 3kq4 2m0p ensg00000107611 ensg00000081479 false protein1 and cubilin: multifunctional endocytic receptors protein2 and cubilin are two structurally different endocytic receptors that interact to serve such functions megalin megalin lrp2 lrp2 2m0p 2m0p ensg00000081479 ensg00000081479 false megalin and cubilin: multifunctional endocytic receptors megalin and cubilin are two structurally different endocytic receptors that interact to serve such functions an instance from hrpd50 obtained 3d structure of proteins from pdb id obtained fasta sequence of proteins from ensembl id generated multi-modal instances from an instance of hrpd50 biomeedical corpora. embeddings for ppi extraction. some of the popular deep learning based ppi extraction techniques are reported by @xcite @xcite . ## dataset formation and preprocessing in this study, we have extended, improved, and further developed two popular benchmark ppi corpora, namely bioinfer 3 and hrpd50 4 dataset for the multi-modal scenario. along with the textual information, these enhanced multi-modal datasets contain the biological counterparts of the interacting or non-interacting protein pairs. biological information comes from the underlying fasta sequence and the atomic structures of interacting protein pairs. ## problem formalization our goal is to develop a deep multi-modal architecture that can efficiently predict whether two proteins are interact with each other or not from the developed multi-modal datasets. formally, consider the multi-modal dataset @xmath0, m 2 , m 3 are three different deep learning based models for text, structure and sequence modality, respectively. the extracted features are fused by self attention mechanism (f sa ) which is finally fed to an activation function(f act ) for predicting protein interactions. ## proposed methodology the major steps of our proposed multi-modal architecture are shown in figure 3 . ## experimental results and analysis in this section, we have briefly described the details of the hyper-parameters and the comparative analysis of the proposed deep multi-modal architecture. to explore the role of developed multi-modal datasets along with the proposed multi-modal architecture for predicting the protein interactions, several experiments are conducted for evaluating each modality and also different combinations of the modalities. additionally, we have compared the performance of our multi-modal approach with various state-of-the-art methods. ## conclusion and future work in this work, we have generated some multi-modal protein-protein interaction databases by amalgamating protein structures and sequences with existing text information available in the biomedical literature. the process of generating multi-modal datasets from ppi corpora is illustrated with some examples. besides, we have proposed a novel deep multi-modal architecture for managing the multimodal scenario for ppis. for each modality (textual, protein sequence and protein atomic structure), we have developed different deep learning models for efficient feature extractions. a detailed comparative analysis proves that the proposed multi-modal architecture outperforms other strong baselines and existing models. future work aims at enhancing sequence feature extraction methods to improve the classification performance as those suffer from low accuracy. further there are plenty of options for improving the fusion technique to enhance the overall performance of the model.