Combining graph and sequence information to learn protein representations
Authors: Hassan Kané
Mohamed Coulibali
Pelkins Ajanoh
Ali Abdalla
Venue:  (2020)
Keywords: NLP
Protein
Representation Learning

Abstract: Computational methods that infer the function of proteins are key to understanding life at the molecular level. In recent years, representation learning has emerged as a powerful paradigm to discover new patterns among entities as varied as images, words, speech, molecules. In typical representation learning, there is only one source of data or one level of abstraction at which the learned representation occurs. However, proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks. Given that protein function is an emergent property of all these levels of interactions in this work, we learn joint representations from both amino acid sequence and multilayer networks representing tissue-specific protein-protein interactions. Using these representations, we train machine learning models that outperform existing methods on the task of tissue-specific protein function prediction on 10 out of 13 tissues. Furthermore, we outperform existing methods by 19% on average.

Under review as a conference paper at ICLR 2020
AUGMENTING PROTEIN NETWORK WITH SEQUENCE INFORMATION
EMBEDDINGS
Anonymous authors Paper under double-blind review
ABSTRACT
Computational methods that infer the function of proteins are key to understand- ing life at the molecular level. In recent years, representation learning has emerged as a powerful paradigm to discover new patterns among entities as varied as im- ages, words, speech, molecules. In typical representation learning, there is only one source of data or one level of abstraction at which the learned representation occurs. However, proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks. Given that protein function is emergent property of all these levels of interac- tions in this work, we learn joint representations from both amino acid sequence and multilayer networks representing tissue-specifie protein-protein interactions. Using these hybrid representations, we show that simple machine learning mod- els trained using these hybrid representations outperform existing network-bass methods the task of tissue-specific protein function prediction 13 out of 13 tissues. Furthermore, these representations outperform existing ones by 14% on average.
INTRODUCTION
Proteins can be described by their primary, secondary, tertiary, and quaternary structure or even nodes in protein-protein interaction networks (Creighton) 1993). Some proteins with similar sequences play similar roles; others with high levels of sequence similarity can play different roles. To add further nuance, the same protein can play different roles depending the tissue it is in and the state of that tissue. Understanding the relationship between these different levels of structure and the role that a protein plays is one of the grand challenges of biology. Recent availability of high- throughput experimental data and machine- learning based computational methods can be useful for unveiling and understanding such patterns.
We frame the problem of understanding the relationship between these complementary data sources and tissue-specific protein function as one of developing protein embeddings on top of which simple machine learning models can be trained to map a given protein to its tissue-specific function.
In this work we constructed new protein representations combining different levels of abstraction. More specifically, we constructed a 128-dimensional vector for each protein where the first 64 di- mensions are derived from the amino acid sequence and the remaining dimensions are obtained from embedding the protein into lissue-specffic protein-protein interaction networks. Such repre- sentations are then used to train a simple linear classifier to predict tissuc-specific protein function. This approach outperforms network- approaches which usually only information from the protein-protein interaction network.
The main contribution of this paper include:
Approaching the problem of tissue-specific protein function prediction from the angle of representation learning using information ranging from amino acid sequence to multilayer networks including tissue-specific
interaction
Experimentally showing that such representations outperform network- methods 13 out of 13 tissues for which we perform the experiments. The best method outperforms current ones by 14% on average.

---

Under review as a conference paper at ICLR 2020
An ablation analysis that demonstrated that our state-of-the results are result of the joint embeddings
2 RELATED WORK
Computational methods to predict the function of proteins fall into several categories. An important step of the pipeline is developing representations for proteins. Most existing methods focus one level of biological abstraction and develop a representation specific to this level. For example, when looking the primary structure, the first attempt computationally predict the role of protein is through sequence homology. That is, using database of protein whose sequence and function known, methods using string similarity will find the closest proteins and use heuristics make prediction based on such similarity. These methods use dynamic programming and hierarchical clustering to align multiple sequence to perform homology and find the distance of given protein to multiple proteins stored in a database. (Feng & Doolittle 1987) (Corpet 1988) (Corpet 1988)
Edgar 2004)
Beyond sequence homology, local polypeptide chains are grouped under patterns called protein do- mains (Bateman et al., 2004). Protein domains evolve independently of the rest of the protein chain. They are often thought of as evolutionary advantageous building blocks which are conserved across species and proteins. The presence of such building blocks in protein is used as a proxy to infer function and protein family. Pfam is database of protein families that includes their annotations and multiple sequence alignments generated using hidden Markov models and has 17,929 families used characterize unknown on the basis of motif presence.
Recently, inspired by the methods used in natural language processing, researchers have developed character-level language models by training algorithms such as long short-term memory (LSTM) (Hochreiter & Schmidhuber 1997) networks to predict the next amino acid given the previous amino acids. Many recent works have gone into training and investigating the properties learned by such language models and found that they encode many biochemical properties and can be used to re- cover protein families. More specifically UniRep (Alley et al. 2019) uses multiplicative LSTM (Krause et al., 2016) trained to perform next amino acid prediction on 24 million UniRef50 (Suzek) jet al,, (007) amino acid sequences. The trained model is used to generate a single fixed-length vector representation of the input sequence by globally averaging intermediate mL.TTM numeri- cal summaries. Seq Vee (Heinzinger et al., 2019) works by training bi-directional language model ELMo (Peters al. 2018) on UniRef50. While such models are useful descriptors and encoders of biochemical properties, they lack the local context needed to infer protein function.
While all previously-citer methods develop representations of proteins with the basic molecular components, other methods treat proteins like social networks. Proteins rarely accomplish a function in isolation and need to bind with other proteins, in a specific tissue in given state accomplish a function. Using this insight, many methods describe proteins using such signals. That is, using a "guilt by association principle," they take the perspective that the role of a protein can be inferred from understanding which other proteins it interacts with (Letovsky & Kasif 2003) (Vazquez et al.) 2003] (Mostafavi al. 2008). Representation learning methods formalizing such principles usually take as input a protein-pre protein interaction network represented as graph and use methods such as matrix decomposition (Tang et al., 2011)) and node embeddings (Grover & Leskoved (016) to develop a vector representation grouping neighboring nodes into a similar position. However, these methods do not take into account the rich information that can learned by examining protein's primary sequence. We aim to synthesize the previous approaches, and also take more contextual information about the tissues in which proteins interact. We use OhmNet (Zitnik & Leskovec) 2017) to include the tissue hierarchy and develop tissue-specific node embeddings taking into account local neighborhoods among proteins as well as local neighborhoods among tissues.
3
METHODS
The main idea we present is to integrate information at different levels of the biological hierarchy into the leamned representation of each protein. We used information from two sources: the amino acid sequence and the tissuc-specific protein-proted interaction network. We combined these rep- resentations by concatenating them into a 128 dimensional vector and trained linear classifier to
2

---

Under review as a conference paper at ICLR 2020
predict tissue- speuue- protein functions in a one all fashion. That is, each classifier is binary classifier to predict a given protein plays given role in a specific tissue. We measure the area under the curve for each classifier and average it to have tissue-specific AUROC.
3.1 AMINO ACID SEQUENCE REPRESENTATION
To represent the amino acid sequence, we used recent works such as UniRep and SeqVec treat the amino acids as an alphabet and the amino acid sequence as a string in that discrete alphabet. They learn representations by leveraging the millions of protein sequences available to train machine learning model to predict the next amino acid given the previously seen amino acids. More specifi- cally UniRep uses multiplicative LSTTM train to perform next amino acid prediction on 24 million UniRe150 amino acid sequences. The trained model used to generate a single ixxedlength vector representation of the input sequence by globally averaging intermediate mLSTM numerical sum- maries. Seq Vec works by training bi-directional language model ELMo on UniRef50.
3.2 TISSUE-SPECIFIC PROTEIN NETWORK EMBEDDING
For the second source of representation, we used different methods: Ohmnet and Node2 Node2vec learns a mapping of nodes to a now-dimensional space of features that maximizes the likelihood of preserving network neighborhoods of nodes.
OhmNet encourages sharing of similar features among proteins with similar network neighborhoods and among proteins activated in similar tissues.
Given that the task of tissue-specific protein function prediction is introduced in OhmNet and uses 128 dimensional vector to compare it with other methods, all of our vectors are also constructed to produce 128 dimensional vectors.
3.3 DUMMY VECTORS
To perform controlled experiments that ablate various sources of information, we constructed dummy vectors that we concatenated with either the amino acid sequence representation the tissue-specific protein network embedding. These vectors are: Random64, a 64 dimensional ran- dom vector where each dimension is generated by sampling from uniform distribution in the [-1,1] interval. Random 128 is the corresponding 128 dimensional random vector. O-pad, which simply pads the remaining dimensions with Os.
EXPERIMENTS
4.1 EXPERIMENTAL. SETUP
The goal of each experiment is to solve multi-label binary classification problem. Each label is binary and represents a specific function (more precisely cellular function from the Gene Ontology) in specific tissue. On each tissue, we aim to match every active protein with zero, one more tissue-specific functions. Using a multi-output linear classifier model, we then, for each tissue, use separate linear classifier to predict every single protein functional activation.
We evaluate and compare the protein representations from the original Ohmnet versus the augmented versions introduced in this paper. In this experiment, we run a 10-fold cross-validation with each method over 13 complex tissues (those mapped with more than one function in the Gene Ontology). Prior to that random oversampling is present in almost all tissues. With each fold, the protein embeddings are split between training set (90%) and validation set (10%) in randomly stratified fashion. This training/test split ration is done to reproduce the OhmNet setting. The task at hand is to predict the unseen validation set after fitting the training set. The name of the representations includes the data sources used to generate the 128 dimensional vectors. More details including scores for specific tissues are available in the appendix.
the training data to make for the class imbalance
Out of the 13 tissues we've tried. Some highlight results include:
Node2Vec-Seq Vec outperforms Node2Vec 13/13 times
3

---

Under review as a conference paper at ICLR 2020
Table 1: Average AUROC for tissue-specifie protein function prediction (10-fold cross validation)
Ohmnet128
Node2Vec128
Ohmnet-SeqVen
Node2Vec-Seq Vec
Chmnet-Randon
Random 128
AUROC 0.52
0.53
0.6
0.58
0.51 0.5
Node2Vec-Seq Vec outperforms Veq 8/13 times
Node2Vec-SegVen Vec achieves best performance 6/13 times
Ohmnet-Seqvec achieves best result 2/13 times and is very close second 3/13 times.
Ohmnet-Seqvec scores consistently higher than both Ohmnet and Seqvec separately (re- spectively 11/13 times and 9/13 times)
Either hybrid representations (Node2Vec-Seq and Ohmnet-Seqvec) therefore achieve best performance 8/13 times
On average Ohmnet-Seq Vec has a 14% higher AUROC than pure OhmNet
On average Node2Vec-Seq has 11% higher AUROC than pure Node2Vec
Looking how Ohmnet-SeqVec and Node2Vec-Sed Vec performs (a similar trend is observed for UniRep) shows that both Unirep and Seqvee add significant and new information that's not captured by tissue hierarchy protein-protein protein interaction alone.
The average AUROC score from Random big higher than what could be expected from such representations thanks to the spikes (Placenta, Epidermis) which might also result from the huge functional class imbalance within those two tissues which, given the uniformity of the data, gets them more often than not the right side of the hyperplane. Another explanation might be the low amount of data (respectiveley 35 and 72 active proteins) available on those two tissues!
5 CONCLUSION
In this work, we have looked at how conceptually different representations of proteins could interact and complement each other for the task of predicting function. We have shown that by merging information from two task-independent representations of proteins, we make consistently better tissue-specific function predictions in 13 complex tissues. Our ablation analysis demonstrates that the improved results are consequence of integrating information from different levels of the bio- logical hierarchy.
6 DISCUSSION/FUTURE WORK
This work explores various ways of learning representations of proteins to understand protein func- tion in its given biological context. One key takeaway that combining representations from dif- ferent level of biological abstractions leads to improved representations as judged by their ability predict tissue-specific protein function. Recent work on developing representation from amino acid sequence enables us to take advantage of the vast amount of unlabeled sequences and work directly with proteins whether or not they have been aligned with existing sequences annotated using known families.
the current experimental setting, we only focused on 13 tissues which had more than functions and between 90 and 1400 active proteins. Further work can be done by looking at more compre- hensive set of tissues and functions. Additionally, we trained relatively simply classifiers in a one all manners; more powerful approaches using complex models should naturally be explored.
Recent work has also developed embeddings encoding 3D protein structure. These embeddings are currently missing in this work and could also be integrated in subsequent work help understand the relative importance sequence, structure and protein interaction network to predict tissue-speicife function.
We hope that our work spurs more research in representations that integrate information from mul- tiple levels of the biological hierarchy and provide insight into the function of proteins and cells.
4

---

Under review as a conference paper at ICLR 2020
REFERENCES
Ethan C Alley, Grigory Khimulya, Surojit Biswas, Mohammed AlQuraishi, and George M Church. Unified rational protein engineering with sequence-only deep representation learning. bioRxiv, pp. 589333, 2019.
Alex Bateman, Lachlan Coin, Richard Durbin, Robert D) Finn, Volker Hollich, Sam Griffiths-Iones, Ajay Khanna, Mhairi Marshall, Simon Moxon, Erik LL Sonnhammer, et al. The pfam protein families database. Nucleic acids research, 32(suppL 1133--111) 2004.
Florence Corpet. Multiple sequence alignment with hierarchical clustering. Nucleic acids research, (0822):10811-0888 1988.
Thomas E Creighton. Proteins: structures and molecular properties. Macmillan, 1993.
Robert C Edgar. Muscle: multiple sequence alignment with high accuracy and high throughput. Nucleic acids research, 32(5): 2004.
Da-Fei Feng and Russell F Doolittle. Progressive sequence alignment as a prerequisitetoo correct phylogenetic trees. Journal of molecular evolution, 25(4):351-360, 1987.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 855-864. ACM, 2016.
Michael Heinzinger, Ahmed Elnaggar, Yu Wang, Christian Dallago, Dmitrii Nachaev, Florian Matthes, and Burkhard Rost. Modeling the language of life-deep learning protein sequences. bioRxiv, pp. 614313, 2019.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Ben Krause, Liang Lu, lain Murray, and Steve Renals. Multiplicative 1stm for sequence modelling. arXiv preprint 1oXx::6609907999 2016.
Stanley Letovsky and Simon Kasif. Predicting protein function from protein/protein interaction
data: probabilistic approach. Bioinformatics, 19(supp_ 2003.
Sara Mostafavi, Debajyoti Ray, David Warde-Farley, Chris Grouios, and Quaid Morris. Genema- nia: real-time multiple association network integration algorithm for predicting gene function. Genome biology, 91):S4, 2008
Matthew E Peters, Mark Neumann, Mohit lyyer. Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations, arXiv preprint arXiv:1802.05360 2018.
Baris E Suzek, Hongzhan Huang, Peter McGarvey, Raja Mazumder, and Cathy H Wu. Uniref: comprehensive and non-redundant uniprot reference clusters. Bioinformatics, 128-01; 2007.
Lei Tang, Xufei Wang, and Huan Liu. Scalable learning of collective behavior. IEEE Transactions on Knowledge and Data Engineering, 1080-110,- 2011.
Alexei Vazquez. Alessandro Flammini, Amos Maritan, and Alessandro Vespignani. Global protein function prediction from protein-protedd interaction networks. Nature biotechnology, 21(6))697, 2003.
Marinka Zitnkk and Jure Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics, 33( 33(14):110--1198, 2017.
A APPENDICES
5

---

Under review as a conference paper at ICLR 2020
Ohmnet-unirep
0.62
0.58
0.58
0.54
0.64
0.55
0.56
0.62
0.57
0.57
0.56
0.24
0.56
0.24
0.6
Ohmnet-seqvee
Table 2: Average test scores per tissue for each representation after 10-fold cross validation run
0.62
0.6
0.67
0.58
0.49
0.55
0.6
0.61
0.65
0.61
0.6
0.6
nan
0.6
0.24
Ohmnet-rand
0. 1.49
0.5
0.45
0.52
0.45
0.57
0.49
0.44
0.5
0.48
0.49
0.5
0.78
0.51
0.23
N2vec64
0.64
0.54
0.52
0.59
man
0.51
0.51
0.59
0.54
0.6
0.58
0.54
0.39
0.55
0.23
0hmnet64
0.56
0.47
0.54
0.54
0.45
0.46
0.52
0.47
0.52
0.56
0.59
0.53
0.44
0.51
0.23
N2vec
0.64
0.55
0.53
0.57
0.48
0.52
0.53
0.58
0.58
0.52
0.34
0.53
0.23
0.5
0.6
Ohmnet128
0.55
0.51
0.54
0.53
0.41
0.53
0.44
0.55
0.56
0.58
0.53
0.49
0.52
0.24
0.6
Proteins
741
267
90
148
45
111
72
81
127
625
318
611
35
central central.nervous. www..neeeuuu...e hervoos-system system
nervous. system
Avg. AUROC
Std. AUROC
blood. vessel
lymphocyte
epidermis
leukocyte
cartilage
placenta
tissue
blood
bone
brain
kidney
eye

---

Under review as a conference paper at ICLR 2020
Random 128
Table 3: Average test scores per tissue for each representation after 10-fold cross- validation run (rest)
0 0.49
0.51
0.51
0.57
0.55
0.45
0.58
0.41
0.48
0.49
0.51
0.48
nan
0.5
0.23
Unirep
0.61
0.61
0.57
0.58
0.59
0.58
0.57
0.56
0.62
0.58
0.56
0.53
0.42
0.57
0.25
Seqvec
0.6
0.59
0.65
0.57
0.46
0.56
0.61
0.58
0.68
0.6
0.57
0.59
nan
0.59
0.24
N2vec-unirep
0.62
0.58
0.42
0.56
0.47
0.49
0.62
0.59
0.57
0.54
0.49
0.55
0.23
0.6
0.6
N2vec-seqvee
0.65
0.58
0.56
0.43
0.55
0.54
0.64
0.65
0.62
0.61
0.56
0.58
0.23
0.6
0.6
Proteins
741
267
90
148
45
111
72
81
127
625
318
611
35
central central www.nnerooussssyeooo system
nervous. system
Avg. AUROC
Std. AUROC
blood. vessel
lymphocyte
epidermis
leukocyte
cartilage
placenta
fissue
blood
bone
brain
kidney
eye


Reviews:
[{"review_id": "S1x_LgRTtr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a method to incorporate both sequence information and graph information to learn the protein representations. The idea is very straightforward. Basically, it used the embedding from OhmNet [Marinka et al, 2017] for the graph information and used the sequence information from UniRep [Ethan et al, 2019] or SeqVec [Michael et al, 2019]. It uses one experiment to show the performance of the combination of the two pieces of information.\n\nThis paper should be rejected for the following reasons: \n(1) The paper is obviously in the preliminary form without too much polish.\n (2) The simple combination of the results from two published articles is not that interesting\n(3) the presentation of the paper and idea is not in an acceptable form (the authors should at least draw a figure to show the big idea of the paper).\n(4) the experiment is not convincing (there is only one experiment and it is not compared with the other state-of-the-art methods; since an embedding of a protein can be of broad usage, the authors should give its performance on four tasks: protein function prediction (GO term) [Maxat et al, 2018], enzyme function prediction (EC number) [Yu et al, 2018], protein secondary structure prediction [Sheng et al, 2016], protein contact map prediction [Jinbo Xu, 2019]) \n(5) The learned embedding is not well discussed. The author should at least visualize the embeddings and check the physical and biological meaning of those embeddings, if possible. \n\nSince this manuscript would be for sure and have to be largely rewritten in the future, I would not give too many detailed suggestions but some high-level suggestions if the authors would like to refine this manuscript further and submit it somewhere else or ICLR next year:\n(1) Further improve the idea of combining different sources of information. Combining different pieces of information will definitely be helpful but the authors should figure out a way to use them in a more natural way.\n(2) Compared with other methods, which can combine different sources of information.\n(3) Run more experiments on various tasks instead of one: protein function prediction (GO term), enzyme function prediction (EC number), protein secondary structure prediction, protein contact map prediction\n(4) Refine the representation of the paper.\n\nReferences:\n[Marinka et al, 2017] Predicting multicellular function through multi-layer tissue networks, 2017, https://arxiv.org/abs/1707.04638\n[Ethan et al, 2019] Unified rational protein engineering with sequence-based deep representation learning, 2019, Nature Methods\n[Michael et al, 2019] Modeling the Language of Life – Deep Learning Protein Sequences, 2019, https://www.biorxiv.org/content/10.1101/614313v2\n[Maxat et al, 2018] DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier, 2018, Bioinformatics\n[Yu et al, 2018] DEEPre: sequence-based enzyme EC number prediction by deep learning, 2018, Bioinformatics\n[Sheng et al, 2016] Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields, 2016, Scientific Reports\n[Jinbo Xu, 2019] Distance-based protein folding powered by deep learning, 2019, PNAS", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "S1gcZeCCFB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this study, the authors develop a method to predict the function of proteins from their structure as well as the network of proteins with which they interact in a given tissue. The method consists in training a linear classifier on the output of two existing embedding methods, UniRep/SeqVec and OhmNet, respectively embedding the amino acid sequences and the tissue-specific protein-protein interaction networks. This method improves prediction of protein function by 19% compared to OhmNet alone.\n\nAlthough the topic is important and the article clearly written, I would tend to reject this article because there is no innovation in ML that would justify presentation at ICLR.\n\nStrengths:\n- the article is well-written and straight-forward. Prior art is well-described.\n- timely and important topic (prediction of protein function), where ML is likely to have an big impact.\n- positive scientific result (prediction is improved compared to prior art).\n\nWeakness:\n- the ML aspect of this work is entirely based on prior art, the main innovation consisting in fitting a linear classifier on concatenated features extracted by two existing embedding methods (UniRep/SeqVec and OhmNet).\n\n\nAdditional feedback:\n- In the ablation studies, why not include the condition SeqVec-Random and UniRep-random?\n-\"The average AUROC score from Random is a big higher than what could be expected from such representations thanks to the spikes (Placenta, Epidermis) which might also result from the huge functional class imbalance within those two tissues which, given the uniformity of the data, gets them more often than not on the right side of the hyperplane. \"\n=> unclear sentence.\n- \"is a big higher\" => typo\n- \"beta sheets .\" => typo\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HJl3xM52tr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work tries to predict the protein functional activation on a tissue by combining the information from amino acid sequence, and tissue-specific protein-protein interaction network. The authors claim that with this joint representation, their model outperforms current methods (Omhnet) on 10 out of 13 tissues by a larger margin(19% on average).\n\nNotations:\nThe notations in experiment is a little bit confusing. In Table 1, the authors refer to different representations with Ohmnet128, Ohmnet64, Ohmnet-Unirep, etc. However, these are not consistent to the ones introduced in Section 4.1: Ohmnet, Ohmnet64-Unirep64, etc. And \"0-pad\" is introduced in section 3.3 while they denote one method as \"Ohmnet64-0Padded\" in section 4.1. It would be difficult for the reader to infer the meaning of these abbreviations.\n\nMethod:\n\n--amino acid sequence representation:\nIt would be better to report the explained variance when using Principle Component Analysis (PCA) to project the 1024-dimensional output vector of SeqVec to 64 dimensional space.  And the authors can show us more results of different projected dimensions (with different explained variance of the PCA).\n\nExperiments:\n\n--model:\nMaybe the authors can provide us more information about the model they use. For classification, what exactly the linear model is? For learning representation, is there any modification of the structure and hyperparameter of UniRef, SeqVec and OhmNet? And is there any regularization? Showing training details like batch size, epochs would be helpful, too.\n\n--data:\nIt would be better to show the details of the data this paper uses, like what the data looks like, what is the size, the distribution, and the pre-processing. What's more, since validation set is used for tuning, it would be better to report the results on test set.\n\n--result:\nIn the second paragraph of Section 4.1, it would be more clear to use a table instead of words to show the results. What's more, what's exactly the 13 tissues this paper is using? Why they are chosen? Exactly what is the AUROC of each protein in each tissue?  What the learning curves look like?\n\nAnother big issue is, what \"current methods\" is this paper comparing its result with? It seems like the authors are comparing their implementation of Ohmnet-SeqVec + linear model with Ohmnet + linear model, and report that the former one is of 19% higher AUROC than the latter. But how about the results of other models/methods on the same task in the literature. Is there anyone using similar joint representation and what is their results? \n\n--conclusion:\nSince the proposed methods only achieve best results  in 10 out of 13 tissues, it is improper to claim \"… we make consistently better tissue-specific function predictions in 13 complex tissues …\".\n\nIn conclusion, I find this is an interesting paper, that the authors tries to combine amino acid sequence representation and tissue information to predict the activation of protein on specific tissue. However, the authors should perform more rigorous experiment, and show us more implementation details. What's more, comparing results with the start-of-art methods on the same task setting is important, too.\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a linear classifier based on a concatenation of two types of features for protein function prediction. The two features are constructed using methods from previous papers, based on peptide sequence and protein-protein interactions. 

All the reviewers agree that the problem is an important one, but the paper as it is presented does not provide any methodological advance, and weak empirical evidence of better protein function prediction. Therefore the paper would require a major revision before being suitable for ICLR.

Decision: Reject