The Integration of Multiple Feature Representations for Protein Protein Interaction Classification Task

Background In order to extract and retrieve protein protein interaction (PPI) information from text, automatic detecting protein interaction relevant articles for database curation is a crucial step. The vast majority of this research used the “bag-of-words” representation, where each feature corresponds to a single word. For the sake of capturing more information left out from this simple bag-of-word representation, we examined alternative ways to represent text based on advanced natural language techniques, i.e. protein named entities, and biological domain knowledge, i.e. trigger keywords. Results These feature representations are evaluated using SVM classifier on the BioCreAtIvE II benchmark corpus. On their own the new representations are not found to produce a significant performance improvement based on the statistical significance tests. On the other hand, the performance achieved by the integration of 70 trigger keywords and 4 protein named entities features is comparable with that achieved by using bagof-words alone. In addition, the only 4 protein named entities features (4PNE) obtained the best recall performance (98.13%). Conclusions In general, our work supports that more sophisticated natural language processing (NLP) techniques and more advanced usage of these techniques need to be developed before better text representations can be produced. The feature representations with simple NLP techniques would benefit the real-life detecting system implemented with great efficiency and speed without losing the classification performance and exhaustive curation system. Background With the rapid growth of biological and biomedical research, the volume of published biological and biomedical articles is expanding at an increasing rate. Characterizing protein protein interaction (PPI) from sentences is one of the most important biomedical problems, which is crucial to understanding not only the functional role of individual proteins but also the organization of the entire biological processes. In practice, before detecting protein interaction descriptions in sentences, it is necessary to pick out those articles which contain relevant information relative to protein interactions. However, due to the rapid growth of the biomedical literature and the increasing number of newly discovered proteins, it is becoming difficult for the interaction database curators to keep up with the literature by manually detecting and curating protein protein interaction information. In recent years, the work on biomedical literature classification has interested a lot of researchers in Biology, Computer Science and Linguistics. Although the previous attempts to evaluate text classification in the biomedical domain have been made in the context of the TREC Genomics track [1], the recent BioCreAtIvE II Challenge [2] proposed a specific Protein Interaction Article Sub-task 1 (IAS) focusing on the detection of protein protein interaction relevant articles from PubMed titles and abstracts. Most of the participated teams adopted traditional bag-of-words approach to represent text. No advanced NLP techniques or components but stemming and stop words list were adopted. Even though few teams used POS tagging, shallow parsing and sentence splitting, they achieved worse performance than those who used the simple bag-of-words approach. None of these teams adopted complicated advanced NLP techniques, such as NER. In this challenge, we (team 57) achieved the best F1 (0.78) performance using 900 words weighted by tf.rf scheme [3]. Even though the simple bag-of-words approach has been widely-used for text representation and performed well in practice, it was criticized for ignoring a great deal of useful information from the original document. Therefore, researchers have also adopted several different ways to represent text for biological text classification, for example, predefined entities and keywords [4], expert-defined rules [5], local patterns [6], etc. However, since these features have been examined on the gene expression information, we are interested to explore multiple features for the specific protein protein interaction information. For this purpose, in this paper, we investigated multiple feature representations to further improve the text classification performance. Besides the traditional domainindependent bag-of-words approach and term weighting methods, we explored other domain-dependent features, i.e. protein interaction trigger keywords, protein named entities. In addition, the integration of these multiple features for this specific PPI text classification task has been evaluated on the BioCreAtIvE II corpus. To the best of our knowledge, so far no such work has been explored in the PPI task. Methods In this BioCreAtIvE PPI IAS subtask, we examined different ways for text representation based on information extraction (e.g. named entities) and domain experts (trigger keywords). We also explored the performance of their integration and 1 This result is achieved after BioCreAtIvE refined the released test corpus by removing 37 relevant and 36 nonrelevant abstracts during the post-evaluation period. Since the published released results were evaluated on a smaller data set (677 total instances), they are a bit different with our results (750 total instances) in this paper since BioCreAtIvE II has not published which test abstracts were removed from the initial test collection. the baseline bag-of-words approach. In addition, to check whether the different feature representations are significantly different from each other, we did the statistical significance tests on these representations. Data Corpus The training corpus of BioCreAtIvE II challenge in year 2006 is a collection of abstracts which contains 3536 true positive documents (64.3%) which are relevant fors which contains 3536 true positive documents (64.3%) which are relevant for PPI curation and 1959 true negative documents (35.7%) which are not relevant for PPI curation from the two databases, i.e. IntAct and MINT. After the training period, participants received a test data of 750 unlabeled abstracts and had to classify them and submit the test results in one week. The Porter's stemming was performed to reduce words to their base forms. Stop words (513 stop words), punctuations and numbers were removed. The threshold of the minimal term length is 3 (since many biological keywords contain 3 letters, such as acronym). The resulting vocabulary has 24648 words (terms or features). By using the λ statistics ranking metric for feature selection, the top P= {200, 300, 400, 450, 500, 1000, 1500} features from positive and negative categories were selected from the training set. Since the best performance has been achieved using 900 features (bag-of-words) in our previous experiments based on a through evaluation [3], then we only reported this best result by using 900 features in this paper. The bag-of-words Approach Feature Representation The representation that dominates the text classification literature is known as the “bag-of-words”. For most bag-of-words representations, each feature corresponds to a single word found in the training corpus, usually with case information and punctuation removed. Often infrequent and frequent words are removed from the original text. Sometimes a list of stop words (functional or connective words that are assumed to have no information content) is also removed. Typically, there is some attempt to make the features more statistically independent, i.e. removing suffixes from words using a stemming algorithm. Stemming has the effect of mapping several morphological forms of words to a common feature (in most cases, the stemmed root may not be a complete word). Besides feature type, another important issue is term (i.e. feature) weighting. Different features have different importance in a text and thus an important indicator represents how much the feature contributes to the semantics of document. Term weighting methods assign appropriate weights to terms to improve the performance of text categorization. We have earlier proposed a new effective supervised term weighting method tf.rf (see [7] for more details), which has been confirmed to perform significantly better than other methods (including tf.idf and other supervised term weighting methods) on two widely-used newswire benchmark corpora, i.e. Reuters corpus and 20 Newsgroups corpus, cross different learning methods. Therefore, in this new domain, we examine the results of tf.rf method as well. Protein Named Entities (PNEs) A very basic observation about bag-of-words representation is that a great deal of the information from the original document is discarded. Paragraph, sentence and word order is disrupted, and syntactic structures are also broken. The end result is that the text is represented incoherent to humans in order to make it coherent to a machine learning algorithm. The goal of using protein named entities (PNEs) as features is to attempt to capture some of the information left out of the bag-of-words representation, especially for the specific PPI task. Recognizing named entities like gene, protein and virus, is quite important for biomedical information retrieval and information extraction. It is a challenging task because there is no standard naming conventions of named entities in the biomedical domain, being much more difficult than the one in the news domain. For example, many biomedical entity names are descriptive and have many words and numbers. One biomedical entity name may be with various spelling forms with capitalization or hyphen or even various irregular abbreviations. In this paper, we adopted an existing named entity recognition system named PowerBioNE [8], which is based on a Hidden Mixture Markov Model. In this recognition system, various evidential features are integrated through a HMM-based recognizer to deal with various complex naming conventions in