# Automated Confirmation of Protein Annotation Using NLP and the UniProtKB Database
## 2 Methods
### Training Workflow Phase  
#### 2.2.1 Training Corpus Construction and Pre-Processing  
To train our individual models, we needed to acquire both positive and negative training data. We used publications from the UniProtKB/Swiss-Prot literature database, which are associated with ECO (Evidence and Conclusion Ontology) codes to represent the evidence type. Initially, we considered the use of publication abstracts, but publication titles resulted in greater accuracy with faster training and classification. We believe this is because both negative and positive abstracts share many words in common, which introduce considerable noise, thereby making classification more difficult. In general, publication titles available in the UniProtKB/Swiss-Prot database, which have been manually curated, seem to best summarize the substance of the publications. They often include significant words denoting experimental validation of protein function. For example, in the title \"Identification, isolation, and cloning of a _Bacillus thuringiensis_ CryIAc toxin-binding protein from the midgut of the lepidopteran insect _Heliothis virescens_\", \"Identification\", \"isolation\", and \"cloning\" infer experimentation and \"toxin-binding\" infers function. Such word usage in titles often distinguishes positive examples (titles) from negative examples (titles). In contrast, although negative examples may contain words such as \"characterization\", the titles do not indicate experimental validation of protein function. Some examples are  
Figure 2: Overview of the classification workflow for protein annotation.  
\"Structure and transcription of the Drosophila melanogaster vermilion gene and several mutant alleles\" and \"Variation in fumanisin and ochratoxin production associated with differences in biosynthetic gene content in _Aspergillus niger_ and _A. welwitschia_ isolates from multiple crop and geographic origins\". For positive examples, PubMed IDs with the ECO code ECO:0000269 were used; this code is used for manually curated information for which published experimental evidence exists [25; 26]. For negative examples, PubMed IDs with the ECO code ECO:0000303 were used; this code is used for manually curated information with no experimental evidence of statements presented in scientific articles. Both the positive and negative publication titles were collected by means of mapping from PubMed IDs to publication titles using BioPython programming tools [27]. The data were obtained from the UniProtKB/Swiss-Prot database on or before 10 April 2019, and we processed 14,101 positive examples and 14,101 negative examples to use for our training data.  
Our pre-processing includes segmenting the text into component words and symbols, converting uppercase characters and words to lowercase, removing all the punctuation and non-alphabetic symbols, and stop-word filtering. Specifically, our filtering uses a dictionary of stop words from the Natural Language ToolKit [28]. Stop words such as \"the\" and \"an\" convey no experimental evidence; thus, we eliminate them to reduce the dimensionality of our term space, that is, the number of words we need to vectorize. We also added word stemming, which truncates words that are phonologically related to their root word. While this reduced the total number of words, performance was degraded because the pre-trained word embedding models are based on complete words. Hence, we dispensed with word stemming.