Evaluating Protein Transfer Learning with TAPE Abstract  Protein modeling is an increasingly popular area of machine learning research. Semisupervised learning has emerged as an important paradigm in protein modeling due to the high cost of acquiring supervised protein labels, but the current literature is fragmented when it comes to datasets and standardized evaluation techniques. To facilitate progress in this field, we introduce the Tasks Assessing Protein Embeddings (TAPE), a set of five biologically relevant semisupervised learning tasks spread across different domains of protein biology. We curate tasks into specific training, validation, and test splits to ensure that each task tests biologically relevant generalization that transfers to reallife scenarios. We benchmark a range of approaches to semisupervised protein representation learning, which span recent work as well as canonical sequence learning techniques. We find that selfsupervised pretraining is helpful for almost all models on all tasks, more than doubling performance in some cases. Despite this increase, in several cases features learned by selfsupervised pretraining still lag behind features extracted by stateoftheart nonneural techniques. This gap in performance suggests a huge opportunity for innovative architecture design and improved modeling paradigms that better capture the signal in biological sequences. TAPE will help the machine learning community focus effort on scientifically relevant problems. Toward this end, all data and code used to run these experiments are available at https:github.comsonglabcaltape.

Ontologybased transporter substrate annotation for benchmark datasets Construction of benchmark datasets for supervised learning requires a label or class to be assigned to each datapoint. This is done by the constructor of the dataset in those cases where the label is not directly taken from a reference source. In transporter substrate prediction, during the dataset construction step, a class is assigned to each protein that reflects the substrate transported across the biological membrane. This substrate class assignment is typically conducted through manual curation process in which details regarding the assignment are not explained. Biological databases are consistently growing and many entries are updated; therefore, automating the data collection stage is desirable. This work aims to automate the transporter substrate data collection process in a consistent and reproducible manner, and eliminate external dataset curator judgment. To achieve this, we propose an automated tool that assigns a substrate class by using available annotations and delegating the broader class assignment to previously established ontologies. Two case studies have been used to evaluate the automation tool and to analyze the available number of substrates in the current biological databases.