Original: C OMBINING G RAPH AND S EQUENCE INFORMATION TO LEARN PROTEIN REPRESENTATION
Revised: C OMBINING G RAPH AND S EQUENCE INFORMATION TO LEARN PROTEIN REPRESENTATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Computational methods that infer the function of proteins are key to understanding life at the molecular level.
Revised: Computational methods that infer the function of proteins are key to understanding life at the molecular level.

Original: In recent years, representation learning has emerged as a powerful paradigm to discover new patterns among entities as varied as images, words, speech, molecules.
Revised: In recent years, representation learning has emerged as a powerful paradigm to discover new patterns among entities as varied as images, words, speech, molecules.

Original: In typical representation learning, there is only one source of data or one level of abstraction at which the learned representation occurs.
Revised: In typical representation learning, there is only one source of data or one level of abstraction at which the learned representation occurs.

Original: However, proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks.
Revised: However, proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks.

Original: Given that protein function is an emergent property of all these levels of interactions in this work, we learn joint representations from both amino acid sequence and multilayer networks representing tissue-speciﬁc protein-protein interactions.
Revised: Given that protein function is an emergent property of all these levels of interactions in this work, we learn joint representations from both amino acid sequence and multilayer networks representing tissue-speciﬁc protein-protein interactions.

Original: Using these  representations, we train machine learning models that outperform existing  methods on the task of tissue-speciﬁc protein function prediction on10 out of 13 tissues.
Revised: Using these hybrid representations, we show that simple machine learning models trained using these hybrid representations outperform existing network-based methods on the task of tissue-speciﬁc protein function prediction on 13 out oftissues.

Original: Furthermore, we outperform existing methods by 19% on average.
Revised: Furthermore, these representations outperform existing ones by 14% on average.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: With few exceptions, all cells in the human body have the same genetic information.

Original: Yet the human body has more than 200 types of cells which are combined into four broad tissue categories.

Original: Proteins differences play a fundamental role in the variation among cell type, as proteins carry out various activities such as catalyzing biochemical reactions, acting as messengers, and providing structure to tissues(Creighton, 1993).

Original: Understanding how the structure of proteins relates to their function can shed light on the inner workings of cells.

Original: Proteins are generally understood through four levels of structures (Creighton, 1993).The primary structure of a protein refers to its linear chain of amino acids.
Revised: Proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks (Creighton, 1993).

Original: The secondary structure refers to local folded structures that form within a polypeptide due to interactions between atoms of the backbone.

Original: The most common types of secondary structures are alpha helices and beta sheets .

Original: The tertiary structure is the three dimensional shape of the protein.

Original: The protein quaternary structure is the number and arrangement of multiple folded protein subunits in a multi-subunit complex.

Original: Understanding the relationship between these different levels of structure and the role that a protein plays is one of the grand challenges of biology.

Original: Some proteins with similar sequences play similar roles; others with high levels of sequence similarity can play different roles.
Revised: Some proteins with similar sequences play similar roles; others with high levels of sequence similarity can play different roles.

Original: To add further nuance, the same protein can play different roles depending on the tissue it is in and the state of that tissue.
Revised: To add further nuance, the same protein can play different roles depending on the tissue it is in and the state of that tissue.

Revised: Understanding the relationship between these different levels of structure and the role that a protein plays is one of the grand challenges of biology.

Original: Recent availability of high-throughput experimental data and machine-learning based computational methods can be useful for unveiling and understanding such patterns.
Revised: Recent availability of highthroughput experimental data and machine-learning based computational methods can be useful for unveiling and understanding such patterns.

Original: In this work, we approach this problem from the angle of representation learning. Representation learning enables us to frame the problem of understanding the relationship between protein structure and  function as one of embedding proteins in a vector space capturing several key properties and using such embeddings in the context of predictive tasks. From this angle, learning representations in the form ofvector embeddings for proteins whichare predictive of their tissue-speciﬁc function allows to train simple machine learning models mapping a given protein to its tissue-speciﬁc function.
Revised: We frame the problem of understanding the relationship between these complementary data sources and tissue-speciﬁc protein function as one of developing protein embeddings on top of which simple machine learning models can be trained to map a given protein to its tissue-speciﬁc function.

Original: In this work we constructed new protein representations combining different levels of abstraction.
Revised: In this work we constructed new protein representations combining different levels of abstraction.

Original: More speciﬁcally, we constructed a 128-dimensional vector for each protein where the ﬁrst 64 dimensions are derived from the amino acid sequence and the remaining 64 dimensions are obtained from embedding the protein into a tissue-speciﬁc protein-protein interaction networks.
Revised: More speciﬁcally, we constructed a 128-dimensional vector for each protein where the ﬁrst 64 dimensions are derived from the amino acid sequence and the remaining 64 dimensions are obtained from embedding the protein into a tissue-speciﬁc protein-protein interaction networks.

Original: Such representations are then used to train a simple linear classiﬁer to predict tissue-speciﬁc protein function.
Revised: Such representations are then used to train a simple linear classiﬁer to predict tissue-speciﬁc protein function.

Original: This approach outperformsexisting network-based approaches which usually only use information from the protein-protein interaction network.
Revised: This approach outperforms network-based approaches which usually only use information from the protein-protein interaction network.

Original: The main contribution of this paper include:
Revised: The main contribution of this paper include:

Original: • Approaching the problem of  protein function prediction from the angle of representation learning using information ranging from amino acid sequence to multilayer networks including tissue-speciﬁc protein-protein interaction • Experimentally show that such representations outperform existing methods on 10 out of 13 tissues for which we perform the experiments.
Revised: • Approaching the problem of tissue-speciﬁc protein function prediction from the angle of representation learning using information ranging from amino acid sequence to multilayer networks including tissue-speciﬁc protein-protein interaction • Experimentally showing that such representations outperform network-based methods on 13 out of 13 tissues for which we perform the experiments.

Original: The best method outperforms current ones by 19% on average.
Revised: The best method outperforms current ones by 14% on average.

Original:  An ablation analysis that demonstrated that our state-of-the-art results are a result of the joint embeddings
Revised: • An ablation analysis that demonstrated that our state-of-the-art results are a result of the joint embeddings

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Computational methods to predict the function of proteins fall into several categories.
Revised: Computational methods to predict the function of proteins fall into several categories.

Original: An important step of the pipeline is developing representations for proteins.
Revised: An important step of the pipeline is developing representations for proteins.

Original: Most existing methods focus on one level of biological abstraction and develop a representation speciﬁc to this level.
Revised: Most existing methods focus on one level of biological abstraction and develop a representation speciﬁc to this level.

Original: For example, when looking at the primary structure, the ﬁrst attempt to computationally predict the role of a protein is through sequence homology.
Revised: For example, when looking at the primary structure, the ﬁrst attempt to computationally predict the role of a protein is through sequence homology.

Original: That is, using a database of protein whose sequence and function is known, methods using string similarity will ﬁnd the closest proteins and use heuristics to make a prediction based on such similarity.
Revised: That is, using a database of protein whose sequence and function is known, methods using string similarity will ﬁnd the closest proteins and use heuristics to make a prediction based on such similarity.

Original: These methods use dynamic programming and hierarchical clustering to align multiple sequence to perform homology and ﬁnd the distance of a given protein to multiple proteins stored in a database.
Revised: These methods use dynamic programming and hierarchical clustering to align multiple sequence to perform homology and ﬁnd the distance of a given protein to multiple proteins stored in a database.

Original: (Feng & Doolittle, 1987)
Revised: (Feng & Doolittle, 1987)

Original: (Corpet, 1988) (Corpet, 1988)
Revised: (Corpet, 1988) (Corpet, 1988)

Original: (Edgar, 2004)
Revised: (Edgar, 2004)

Original: Beyond sequence homology, local polypeptide chains are grouped under patterns called protein domains (Bateman et al., 2004).
Revised: Beyond sequence homology, local polypeptide chains are grouped under patterns called protein domains (Bateman et al., 2004).

Original: Protein domains evolve independently of the rest of the protein chain.
Revised: Protein domains evolve independently of the rest of the protein chain.

Original: They are often thought of as evolutionary advantageous building blocks which are conserved across species and proteins.
Revised: They are often thought of as evolutionary advantageous building blocks which are conserved across species and proteins.

Original: The presence of such building blocks in protein is used as a proxy to infer function and protein family.
Revised: The presence of such building blocks in protein is used as a proxy to infer function and protein family.

Original: Pfam is a database of protein families that includes their annotations and multiple sequence alignments generated using hidden Markov models and has 17,929 families used to characterize unknown on the basis of motif presence.
Revised: Pfam is a database of protein families that includes their annotations and multiple sequence alignments generated using hidden Markov models and has 17,929 families used to characterize unknown on the basis of motif presence.

Original: Recently, inspired by the methods used in natural language processing, researchers have developed character-level language models by training algorithms such as long short-term memory (LSTM) (Hochreiter & Schmidhuber, 1997) networks to predict the next amino acid given the previous amino acids.
Revised: Recently, inspired by the methods used in natural language processing, researchers have developed character-level language models by training algorithms such as long short-term memory (LSTM) (Hochreiter & Schmidhuber, 1997) networks to predict the next amino acid given the previous amino acids.

Original: Many recent works have gone into training and investigating the properties learned by such language models and found that they encode many biochemical properties and can be used to recover protein families.
Revised: Many recent works have gone into training and investigating the properties learned by such language models and found that they encode many biochemical properties and can be used to recover protein families.

Original: More speciﬁcally UniRep (Alley et al., 2019) uses a multiplicative LSTM (Krause et al., 2016) trained to perform next amino acid prediction on 24 million UniRef50 (Suzek et al., 2007) amino acid sequences.
Revised: More speciﬁcally UniRep (Alley et al., 2019) uses a multiplicative LSTM (Krause et al., 2016) trained to perform next amino acid prediction on 24 million UniRef50 (Suzek et al., 2007) amino acid sequences.

Original: The trained model is used to generate a single ﬁxed-length vector representation of the input sequence by globally averaging intermediate mLSTM numerical summaries.
Revised: The trained model is used to generate a single ﬁxed-length vector representation of the input sequence by globally averaging intermediate mLSTM numerical summaries.

Original: SeqVec (Heinzinger et al., 2019) works by training bi-directional language model
Revised: SeqVec (Heinzinger et al., 2019) works by training bi-directional language model

Original: ELMo (Peters et al., 2018) on UniRef50.
Revised: ELMo (Peters et al., 2018) on UniRef50.

Original: While such models are useful descriptors and encoders of biochemical properties, they lack the local context needed to infer protein function.
Revised: While such models are useful descriptors and encoders of biochemical properties, they lack the local context needed to infer protein function.

Original: While all previously-cited methods develop representations of proteins with the basic molecular components, other methods treat proteins like social networks.
Revised: While all previously-cited methods develop representations of proteins with the basic molecular components, other methods treat proteins like social networks.

Original: Proteins rarely accomplish a function in isolation and need to bind with other proteins, in a speciﬁc tissue in a given state to accomplish a function.
Revised: Proteins rarely accomplish a function in isolation and need to bind with other proteins, in a speciﬁc tissue in a given state to accomplish a function.

Original: Using this insight, many methods describe proteins using such signals.
Revised: Using this insight, many methods describe proteins using such signals.

Original: That is, using a “guilt by association principle,” they take the perspective that the role of a protein can be inferred from understanding which other proteins it interacts with (Letovsky & Kasif, 2003)
Revised: That is, using a “guilt by association principle,” they take the perspective that the role of a protein can be inferred from understanding which other proteins it interacts with (Letovsky & Kasif, 2003)

Original: (Vazquez et al., 2003) (Mostafavi et al., 2008).
Revised: (Vazquez et al., 2003) (Mostafavi et al., 2008).

Original: Representation learning methods formalizing such principles usually take as input a protein-protein interaction network represented as a graph and use methods such as matrix decomposition (Tang et al., 2011) and node embeddings (Grover & Leskovec, 2016) to develop a vector representation grouping neighboring nodes into a similar position.
Revised: Representation learning methods formalizing such principles usually take as input a protein-protein interaction network represented as a graph and use methods such as matrix decomposition (Tang et al., 2011) and node embeddings (Grover & Leskovec, 2016) to develop a vector representation grouping neighboring nodes into a similar position.

Original: However, these methods do not take into account the rich information that can be learned by examining a protein’s primary sequence.
Revised: However, these methods do not take into account the rich information that can be learned by examining a protein’s primary sequence.

Original: We aim to synthesize the previous approaches, and also take more contextual information about the tissues in which proteins interact.
Revised: We aim to synthesize the previous approaches, and also take more contextual information about the tissues in which proteins interact.

Original: We use OhmNet (Zitnik & Leskovec, 2017) to include the tissue hierarchy and develop tissue-speciﬁc node embeddings taking into account local neighborhoods among proteins as well as local neighborhoods among tissues.
Revised: We use OhmNet (Zitnik & Leskovec, 2017) to include the tissue hierarchy and develop tissue-speciﬁc node embeddings taking into account local neighborhoods among proteins as well as local neighborhoods among tissues.

Original:  M ETHODS
Revised:  M ETHODS

Original: The main idea we present is to integrate information at different levels of the biological hierarchy into the learned representation of each protein.
Revised: The main idea we present is to integrate information at different levels of the biological hierarchy into the learned representation of each protein.

Original: We used information from two sources: the amino acid sequence and the tissue-speciﬁc protein-protein interaction network.
Revised: We used information from two sources: the amino acid sequence and the tissue-speciﬁc protein-protein interaction network.

Original: We combined these representations by concatenating them into a 128 dimensional vector and trained a linear classiﬁer to predict tissue-speciﬁc protein functions in a one vs all fashion.
Revised: We combined these representations by concatenating them into a 128 dimensional vector and trained a linear classiﬁer to predict tissue-speciﬁc protein functions in a one vs all fashion.

Original: That is, each classiﬁer is a binary classiﬁer to predict if a given protein plays a given role in a speciﬁc tissue.
Revised: That is, each classiﬁer is a binary classiﬁer to predict if a given protein plays a given role in a speciﬁc tissue.

Original: We measure the area under the curve for each classiﬁer and average it to have a tissue-speciﬁc AUROC.
Revised: We measure the area under the curve for each classiﬁer and average it to have a tissue-speciﬁc AUROC.

Original: 3. A MINO ACID SEQUENCE REPRESENTATION
Revised: 3. A MINO ACID SEQUENCE REPRESENTATION

Original: To represent the amino acid sequence, we leaned towards recent work using techniques from natural language processing to represent proteins. Indeed, recent works such as UniRep and SeqVec treat the amino acids as an alphabet and the amino acid sequence as a string in that discrete alphabet.
Revised: To represent the amino acid sequence, we used recent works such as UniRep and SeqVec treat the amino acids as an alphabet and the amino acid sequence as a string in that discrete alphabet.

Original: They learn representations by leveraging the millions of protein sequences available to train a machine learning model to predict the next amino acid given the previously seen amino acids.
Revised: They learn representations by leveraging the millions of protein sequences available to train a machine learning model to predict the next amino acid given the previously seen amino acids.

Original: Such training procedures have been found to develop powerful representation encompassing many known biochemical and structural properties.

Original: For our purposes, because we wanted a 64-dimensional embedding, we used the UniRep model trained with 64 hidden units, generating a 64 dimension output vector.
Revised: More speciﬁcally UniRep uses a multiplicative LSTM train to perform next amino acid prediction on 24 million UniRef50 amino acid sequences.

Original: We also considered an alternative: SeqVec, whose output protein summary vectors are 1024 dimensions. To obtain thedimension SeqVec, we included all proteins of interest in a matrix and projected them down todimensions using Principal Component Analysis.
Revised: The trained model is used to generate a single ﬁxed-length vector representation of the input sequence by globally averaging intermediate mLSTM numerical summaries. SeqVec works by training bi-directional language model ELMo on UniRef50.

Original: 3. T ISSUE - SPECIFIC PROTEIN NETWORK EMBEDDING
Revised: 3. T ISSUE - SPECIFIC PROTEIN NETWORK EMBEDDING

Original: For the second source of representation, we used the OhmNet representation.
Revised: For the second source of representation, we used two different methods: Ohmnet and Node2Vec.

Original: Ohmnet works by merging multiple sources of data. At the ﬁrst level of hierarchy, a given tissue has a tissue-speciﬁc protein-protein interaction networks which is represented as a graph where each node is a protein and there is an edge between nodes if they are in physical contact in the tissue of interest. Then, the tissues are grouped into a tissue hierarchy which represents another directed acyclic graph.
Revised: Node2vec learns a mapping of nodes to a low-dimensional space of features that maximizes the likelihood of preserving network neighborhoods of nodes.

Original: OhmNet encourages sharing of similar features among proteins with similar network neighborhoods and among proteins activated in similar tissues.
Revised: OhmNet encourages sharing of similar features among proteins with similar network neighborhoods and among proteins activated in similar tissues.

Original: Given that the task of tissue-speciﬁc protein function  is introduced in OhmNet and uses 128 dimensional vector to compare it with other methods, all of our vectors are also constructed to produce 128 dimensional vectors.
Revised: Given that the task of tissue-speciﬁc protein function prediction is introduced in OhmNet and uses 128 dimensional vector to compare it with other methods, all of our vectors are also constructed to produce 128 dimensional vectors.

Original: 3. D UMMY VECTORS
Revised: 3. D UMMY VECTORS

Original: To perform controlled experiments that ablate various sources of information, we constructed dummy vectors that we concatenated with either the amino acid sequence representation or the tissue-speciﬁc protein network embedding.
Revised: To perform controlled experiments that ablate various sources of information, we constructed dummy vectors that we concatenated with either the amino acid sequence representation or the tissue-speciﬁc protein network embedding.

Original: These vectors are: Random64, a 64 dimensional random vector where each dimension is generated by sampling from a uniform distribution in the [-1,1] interval.
Revised: These vectors are: Random64, a 64 dimensional random vector where each dimension is generated by sampling from a uniform distribution in the [-1,1] interval.

Original: Random128 is the corresponding 128 dimensional random vector.
Revised: Random128 is the corresponding 128 dimensional random vector.

Original: 0-pad, which simply pads the remaining dimensions with 0s.
Revised: 0-pad, which simply pads the remaining dimensions with 0s.

Original: [Table]

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: The goal of each experiment is to solve a multilabel binary classiﬁcation problem.
Revised: The goal of each experiment is to solve a multi-label binary classiﬁcation problem.

Original: Each label is binary and represents a speciﬁc function (more precisely a cellular function from the Gene Ontology) in a speciﬁc tissue.
Revised: Each label is binary and represents a speciﬁc function (more precisely a cellular function from the Gene Ontology) in a speciﬁc tissue.

Original: On each tissue, we aim to match every active protein with zero, one or more tissue-speciﬁc functions.
Revised: On each tissue, we aim to match every active protein with zero, one or more tissue-speciﬁc functions.

Original: Using a multi-output linear classiﬁer model, we then, for each tissue, use a separate linear classiﬁer to predict every single protein functional activation.
Revised: Using a multi-output linear classiﬁer model, we then, for each tissue, use a separate linear classiﬁer to predict every single protein functional activation.

Original: We evaluate and compare the protein representations from the original Ohmnet versus the augmented versions introduced in this paper.
Revised: We evaluate and compare the protein representations from the original Ohmnet versus the augmented versions introduced in this paper.

Original: At evaluation time, the protein embeddings are splitted between training set (75%) and validation set (25%) in a randomly stratiﬁed fashion. 
Revised: In this experiment, we run a 10-fold cross-validation with each method over 13 complex tissues (those mapped with more than one function in the Gene Ontology). Prior to that a random oversampling is run on the training data to make for the class imbalance present in almost all tissues. With each fold, the protein embeddings are split between training set (90%) and validation set (10%) in a randomly stratiﬁed fashion. This training/test split ration is done to reproduce the OhmNet setting.

Original: The task at hand is to predict the unseen validation set after ﬁtting the training set:
Revised: The task at hand is to predict the unseen validation set after ﬁtting the training set.

Original: • Ohmnet: We use Ohmnet algorithm to learn a multi-scale network where tissues ofinterest are on thelowest layers. The learned protein representations are 128-dimensional vectors. 
Revised: The name of the representations includes the data sources used to generate thedimensional vectors. More details including scores for speciﬁc tissues are available in the appendix. Out of the 13 tissues we’ve tried.

Original: • Ohmnet64-Unirep64: We use Ohmnet algorithm to learn 64-dimensional representations of proteins in all tissues. We also learn 64-dimensional representations of said proteins from the 64 hidden layers of a 64-unit Unirep network and build a ﬁnal 128 dimensional vector representation for every protein by merging these two together.
Revised: Some highlight results include:

Original: • Ohmnet64-Seqvec64: Using the previously learned Ohmnet 64-dimensional representations, we build a new set of 128-dimensional representations by mapping those embeddings with corresponding 64-dimensional sequence information learnt using SeqVec algorithm.
Revised: • Node2Vec-SeqVec outperforms Node2Vec 13/13 times

Original: • Ohmnet64-Random64: Instead of adding any signiﬁcant information to 64-dimensional Ohmnet representations, we pad all Ohmnet vectors with 64 dimensions of random noise generated from a uniform distribution in the open interval (-1, 1).
Revised: [Table]

Original: • Ohmnet64-0Padded: Instead of adding any signiﬁcant information to 64-dimensional Ohmnet representations, we pad all Ohmnet vectors with all-zeros 64 dimensions.
Revised: • Node2Vec-SeqVec outperforms SeqVec

Original: • Random128:
Revised: [Table]

Original: As as sanity check we also choose to evaluate the performance from completely random 128-dimensional representations. Out of the 13 tissues we’ve tried: • Ohmnet-SeqVec achieves best performance 6/13 times • Ohmnet-Unirep achieves best performance 4/13 times  • Either  therefore achieve best performance 10/13 times • On average Ohmnet-SeqVec has a 19% higher AUROC than pure OhmNet • On average Ohmnet-Unirep has a 13% higher AUROC than pure OhmNet
Revised: 13 times • Node2Vec-SeqVec achieves best performance 6/13 times • Ohmnet-Seqvec achieves best result 2/13 times and is a very close second 3/13 times. • Ohmnet-Seqvec scores consistently higher than both Ohmnet and Seqvec separately (respectively 11/13 times and 9/13 times) • Either hybrid representations (Node2Vec-SeqVec and Ohmnet-Seqvec) therefore achieve best performance 8/13 times • On average Ohmnet-SeqVec has a 14% higher AUROC than pure OhmNet • On average Node2Vec-SeqVec has a 11% higher AUROC than pure Node2Vec

Original: Looking at how Ohmnet-SeqVec and Ohmnet-Unirep perform shows that both Unirep and Seqvec add signiﬁcant and new information that’s not captured by tissue hierarchy  alone.
Revised: Looking at how Ohmnet-SeqVec and Node2Vec-SeqVec performs (a similar trend is observed for UniRep) shows that both Unirep and Seqvec add signiﬁcant and new information that’s not captured by tissue hierarchy or protein-protein interaction alone.

Original: The average AUROC score from Random is a big higher than what could be expected from such representations thanks to the spikes (Placenta, Epidermis) which might also result from the huge functional class imbalance within those two tissues which, given the uniformity of the data, gets them more often than not on the right side of the hyperplane.
Revised: The average AUROC score from Random is a big higher than what could be expected from such representations thanks to the spikes (Placenta, Epidermis) which might also result from the huge functional class imbalance within those two tissues which, given the uniformity of the data, gets them more often than not on the right side of the hyperplane.

Revised: Another explanation might be the low amount of data (respectiveley 35 and 72 active proteins) available on those two tissues !

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we have looked at how conceptually different representations of proteins could interact and complement each other for the task of predicting function.
Revised: In this work, we have looked at how conceptually different representations of proteins could interact and complement each other for the task of predicting function.

Original: We have shown that by merging information from two task-independent representations of proteins, we make consistently better tissue-speciﬁc function predictions in 13 complex tissues.
Revised: We have shown that by merging information from two task-independent representations of proteins, we make consistently better tissue-speciﬁc function predictions in 13 complex tissues.

Original: We have obtained higher scores than

Original: Ohmnet from either Ohmnet-Unirep or Ohmnet-Seqvec in 10 out of 13 tissues, with minimal increase in complexity of the classiﬁer.

Original: Our ablation analysis demonstrates  the improved results are a consequence of integrating information from different levels of the biological hierarchy.
Revised: Our ablation analysis demonstrates that the improved results are a consequence of integrating information from different levels of the biological hierarchy.

Original:  D ISCUSSION /F UTURE WORK
Revised:  D ISCUSSION /F UTURE WORK

Original: This work explores various ways of learning representations of proteins to understand protein function in its given biological context.
Revised: This work explores various ways of learning representations of proteins to understand protein function in its given biological context.

Original: One key takeaway is that combining representations from different level of biological abstractions leads to improved representations as judged by their ability to predict tissue-speciﬁc protein function.
Revised: One key takeaway is that combining representations from different level of biological abstractions leads to improved representations as judged by their ability to predict tissue-speciﬁc protein function.

Original: Recent work on developing representation from amino acid sequence enables us to take advantage of the vast amount of unlabeled sequences and work directly with proteins whether or not they have been aligned with existing sequences or annotated using known families.
Revised: Recent work on developing representation from amino acid sequence enables us to take advantage of the vast amount of unlabeled sequences and work directly with proteins whether or not they have been aligned with existing sequences or annotated using known families.

Original: In the current experimental setting, we only focused on 13 tissues which had more than 2 functions and between 90 and 1400 active proteins.
Revised: In the current experimental setting, we only focused on 13 tissues which had more than 2 functions and between 90 and 1400 active proteins.

Original: Further work can be done by looking at a more comprehensive set of tissues and functions.
Revised: Further work can be done by looking at a more comprehensive set of tissues and functions.

Original: Additionally, we trained relatively simply classiﬁers in a one vs. all manners; more powerful approaches using complex models should naturally be explored..
Revised: Additionally, we trained relatively simply classiﬁers in a one vs. all manners; more powerful approaches using complex models should naturally be explored.

Revised: Recent work has also developed embeddings encoding 3D protein structure.

Revised: These embeddings are currently missing in this work and could also be integrated in subsequent work to help understand the relative importance of sequence, structure and protein interaction network to predict tissue-speicifc function.

Original: We hope that our work spurs more research in representations that integrate information from multiple levels of the biological hierarchy and provide insight into the function of proteins and cells. 
Revised: We hope that our work spurs more research in representations that integrate information from multiple levels of the biological hierarchy and provide insight into the function of proteins and cells. A A PPENDICES t i ss u e P r o t e i n s

Revised: O h m n e t

Revised: N 2v ec

Revised: O h m n e t

Revised: N 2v ec

Revised: O h m n e t - r a nd

Revised: O h m n e t - s e q v ec

Revised: O h m n e t - un i re p b

Revised: l

Original: {+ood 0 .0 .0 .0 .0 .0 .
+}
Revised: {+ood 0 .0 .0 .0 .0 .0 .
+}

Revised: .b

Revised: l ood v e ss e l 0 .0 .0 .0 .0 .0 .0 .bon

Original: {+e 0 .0 .0 .0 .0 .0 .
+}
Revised: {+e 0 .0 .0 .0 .0 .0 .
+}

Revised: .b

Revised: r a i n 0 .0 .0 .0 .0 .0 .0 .ca

Original: {+r til a g e 0 .0 .
+}
Revised: {+r til a g e 0 .0 .
+}

Revised: .n

Original: {+a n 0 .0 .
+}
Revised: {+a n 0 .0 .
+}

Revised: .ce

Revised: n t r a l n e r vou s s y s t e m 0 .0 .0 .0 .0 .0 .0 .e

Revised: p i

Revised: d e r m i s 0 .0 .0 .0 .0 .0 .0 .e

Revised: y e 0 .0 .0 .0 .0 .0 .0 .k

Original: {+i dn e y 0 .0 .0 .0 .0 .0 .
+}
Revised: {+i dn e y 0 .0 .0 .0 .0 .0 .
+}

Revised: .l

Revised: e uko c y t e 0 .0 .0 .0 .0 .0 .0 .l

Revised: y m ph o c y t

Original: {+e 0 .0 .0 .0 .0 .0 .
+}
Revised: {+e 0 .0 .0 .0 .0 .0 .
+}

Revised: .n

Revised: e r vou s s y s t

Original: {+e m 0 .0 .0 .0 .0 .0 .
+}
Revised: {+e m 0 .0 .0 .0 .0 .0 .
+}

Revised: .p

Original: {+l ace n t a 0 .0 .0 .0 .
+}
Revised: {+l ace n t a 0 .0 .0 .0 .
+}

Revised: .n

Revised: a n 0 .

Revised: A vg .

Revised: AUR

Revised: O C 0 .0 .0 .0 .0 .0 .0 .

Revised: S t d .

Revised: A

Revised: UR

Revised: O C 0 .0 .0 .0 .0 .0 .0 .

Revised: T a b l e 2 :

Revised: A v e r a g e t e s t s c o r e s p

Revised: e r ti ss

Revised: u e f o r eac

Revised: h

Revised: r e p r e s e n t a ti on a f t e r 10 -f o l d c r o ss - v a li d a ti on r un t i ss u e

Revised: P r o t e i n s

Revised: N 2v ec - s e q v ec

Revised: N 2v ecun i re p

Revised: S e q v ec

Revised: U n i re p

Original: {+R a nd o m b l ood 0 .0 .0 .0 .
+}
Revised: {+R a nd o m b l ood 0 .0 .0 .0 .
+}

Revised: .b

Revised: l ood v e ss e l 0 .0 .0 .0 .0 .bon

Original: {+e 0 .0 .0 .0 .
+}
Revised: {+e 0 .0 .0 .0 .
+}

Revised: .b

Revised: r a i n 0 .0 .0 .0 .0 .ca

Original: {+r til a g e 0 .0 .0 .0 .
+}
Revised: {+r til a g e 0 .0 .0 .0 .
+}

Revised: .ce n t r a l n e r vou s s y s t e m 0 .0 .0 .0 .0 .e

Revised: p i

Revised: d e r m i s 0 .0 .0 .0 .0 .e

Revised: y e 0 .0 .0 .0 .0 .k

Original: {+i dn e y 0 .0 .0 .0 .
+}
Revised: {+i dn e y 0 .0 .0 .0 .
+}

Revised: .l

Revised: e uko c y t e 0 .0 .0 .0 .0 .l

Revised: y m pho c y t

Revised: e 0 .0 .0 .0 .0 .n

Revised: e r vou s s y s t

Original: {+e m 0 .0 .0 .0 .
+}
Revised: {+e m 0 .0 .0 .0 .
+}

Revised: .p

Revised: l ace n t a 0 .0 .n

Revised: a n 0 .n a n

Revised: A vg .

Revised: AUR

Revised: O C 0 .0 .0 .0 .0 .

Revised: S t d .

Revised: AUR

Revised: O C 0 .0 .0 .0 .0 .

Revised: T a b l e 3 :

Revised: A v e r a g e t e s t s c o r e s p

Revised: e r ti ss

Revised: u e f o r eac

Revised: h

Revised: r e p r e s e n t a ti on a f t e r 10 -f o l d c r o ss - v a li d a ti on r un (r e s t )