# A Hybrid Deep Learning Model for Protein-Protein Interactions Extraction from Biomedical Literature
## 1 Introduction  
Protein-protein interactions (PPIs) play important roles in various biological processes and are of pivotal importance in the regulation of biological systems, and are consequently implicated in the development of disease states [1]. The exponentially increasing size of biomedical literature and the limited ability of manual curators to discover PPIs in text has led to delays in keeping PPI databases, such as BIND (The Biomolecular Interaction Network Database) [2], MINT (The Molecular INTeraction Database) [3], and The IntAct molecular interaction database [4], updated with the current findings. Consequently, this causes a bottleneck when leveraging the valuable information that is currently available in order to develop personalized health care solutions. Previous studies have explored different methods for PPI extraction. The dominant techniques generally fall under the following three broad categories: co-occurrence-based methods [5], rule-and-pattern-based methods [6, 7], and statistical machine learning (ML)-based methods [8, 9, 10, 11, 12]. Co-occurrence- based methods measure the correlation between each pair of entities by co-occurrence. A major weakness of these methods is their tendency for having a high recall, but a low precision. This is mainly because when entities do not appear in pairs in the training set, no co-occurrence with correlation can be recorded. Rule-and-pattern-based methods employ predefined patterns and rules to match the labelled sequence. Although traditional rule-and-pattern-based methods have achieved high accuracy, their sophistication in pattern design and attenuated recall performance make them unsuitable for practical usage. Compared with co-occurrence- and rule-pattern-based methods, ML-based methods show a much better performance and generalization. In particular, the recent surge of interest in deep learning (DL) methods is due to the fact that they have been shown to outperform previous state-of-the-art techniques for PPI extraction tasks. Generally, ML based approaches cast the problem of PPIs extraction into a classification problem. Suppose to extract the binary PPIs between entity \(e_{1}\) and entity \(e_{2}\) in a given sentence \(=w_{1}w_{2}\ldots e_{1}\ldots e_{2}\ldots w_{n}\), where \(w_{i}\) is a word in \(S\). The classification model is constructed to output 1 when \(e_{1}\) and \(e_{2}\) are related, otherwise it is 0. The inputs of the classification model are the features extracted from \(S\). A key difference between traditional ML and DL is in how features are extracted and represented. Traditional ML-based methods usually collect words around target entities as features, such as unigram, bigram, and some semantic and syntactic features, and then these features are put into a bag-of-words model and encoded into one-hot type representations. However, such representations are unable to capture semantic relations among words or phrases and fail in generalizing the long context dependency [13]. The former issue is rendered as \"vocabulary gap\" (e.g., the words \"depend\" and \"rely\" are different in one-hot representations, albeit their similar linguistic functions). The latter one is introduced because of the n-order Markov restriction that attempts to alleviate the issue of \"curse of dimensionality.\" Moreover, the inability to extract features automatically leads to laborious manual efforts in designing features. Different from one-hot encoding representation, word distribution representation is proposed to solve the \"vocabulary gap\" problem by mapping words to dense vectors of real numbers in a low-dimensional space [14; 15]. In addition to using words as features, some semantic and syntactic features, such as part-of-speech (POS), word position, and dependency path between two entities, can also be embedded in their distribution representations for feature learning. Recent studies have proposed several feature embedding methods combining DL models for PPI extraction.