Prediction of Enzyme Specificity using Protein Graph Convolutional Neural Networks
Authors: Changpeng Lu
Samuel Z Stentz
Joseph H Lubin
Sijian Wang
Sagar D Khare
Venue:  (2021)
Keywords: graph convolutional neural networks
protease specificity
proteins
Rosetta energy function

Abstract: Specific molecular recognition by proteins, for example, protease enzymes, is critical for maintaining the robustness of key life processes.  The substrate specificity landscape of a protease enzyme comprises the set of all sequence motifs that are recognized/cut, or just as importantly, not recognized/cut by the enzyme. Current methods for predicting protease specificity landscapes rely on learning sequence patterns in experimentally derived data with a single enzyme, but are not robust to even small mutational changes. A comprehensive evaluation of specificity requires consideration of the three-dimensional structure and energetics of molecular interactions. In this work, we present a protein graph convolutional neural network (PGCN), which uses a physically intuitive, structure-based molecular interaction graph generated using the Rosetta energy function that describes the topology and energetic features, to determine substrate specificity. We use the PGCN to recapitulate and predict the specificity of the NS3/4 protease from the Hepatitic C virus. We compare our PGCN with previously used machine learning models and show that its performance in classification tasks is equivalent or better. Because PGCN is based on physical interactions, it is inherently more interpretable; determination of feature importance reveals key sub-graph patterns responsible for molecular recognition that are biochemically reasonable. The PGCN model also readily lends itself to the design of novel enzymes with tailored specificity against disease targets.

Under review as a conference paper at ICLR 2021
PREDICTION OF ENZYME SPECIFICITY USING PROTEIN GRAPH CONVOLUTIONAL NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Specific molecular recognition by proteins, for example, protease enzymes, is crit- ical for maintaining the robustness of key life processes. The substrate specificity landscape of a protease enzyme comprises the set of all sequence motifs that are recognized/cut, or just as importantly, not recognized/cut by the enzyme. Current methods for predicting protease specificity landscapes rely on learning sequence patterns experimentally derived data with a single enzyme, but are not robust small mutational changes. A comprehensive evaluation of specificity re- quires consideration of the daree-dimensiona, structure and energetics of molecu- lar interactions. In this work, we present protein graph convolutional neural net- work (PGCN), which uses a physically intuitive, structure-based molecular inter- action graph generated using the Rosetta energy function that describes the topol- ogy and energetic features, to determine substrate specificity. We use the PGCN to recapitulate and predict the specificity of the NS3/4 protease from the Hepatitic C virus. We compare our PGCN with previously used machine learning models and show that its performance in classification tasks equivalent or better. Because PGCN is based on physical interactions, it is inherently more interpretable; de- termination of feature importance reveals key sub-graph patterns responsible for molecular recognition that are biochemically reasonable. The PGCN model also readily lends itself to the design of novel enzymes with tailored specificity against disease targets.
INTRODUCTION
Selective molecular recognition between biomolecules e.g. protein-protein, DNA-protein (Tainer & Cunningham 1993), and protein-small molecule interactions, is key for maintaining the fidelity of life processes. Multispecfficity i.c. the specific recognition and non-recogntion of multiple targets by biomolecules. is critical for many biological processes, for example the selective recognition and cleavage of host and viral target sites by viral protease enzymes is critical for the life-cycle of many RNA viruses including SARS-CoV-2 (Vizovišek et al.) 2018). Prospective prediction of the sequence motifs corresponding protease enzyme target sites (substrates) is therefore important goal with broad implications. Elucidating the target specificity of viral protease enzyme can be used for the design of inhibitor anti-viral drug candidates. The ability to accurately and efficiently model the landscape of protease specificity i.e. the set of all substrate sequence motifs that are recognized (and not recognized) by a given enzyme and its variants would also enable the design of proteases with tailores specificities to degrade chosen disease-related targets.
Most current approaches for protease specificity prediction involve detecting and/or learning patterns in known substrate sequences using techniques ranging from logistic regression to deep learning- However, these black-box approaches do not provide any physical/chemical insight into the under- lying basis for particular specificity profile, that often arise in the course of evolution. A comprehensive model of protease specificity requires the consideration of the -imeesiimnii structure of the enzyme and the energetics of interaction between enzyme and various substrates such that substrates that are productively recognized (i.e. cleaved) by the protease lower in energy than those that not.
they robust changes in the protease enzyme

---

Under review as a conference paper at ICLR 2021
To encode the topology and energetic features, here we develop Protein Convolutional Neural Net- works (PGCN). PGCN uses experimentally derived data and a pyysicalll-intuitiii structure-baser molecular interaction energy graph to solve the classification problem for substrate specificity. Pro- tease and substrate residues are considered as nodes, and energies of interactions are obtained from pairwise decomposition of the energy of the complex calculated using the Rosetta energy func- tion. These energies are assigned as (multiple) node and edge features. We find that PGCN is as good as or better than other previously used machine learning models for protease specificity pre- diction. However, it is more interpretable and highlights critical sub-gres sub-graph patterns responsible for observed specificity patterns. As it is based on physical interactions, the PGCN model is capable of both prospective prediction of specificity of chosen protease enzymes and generating novel designed enzymes with tailored specificity again chosen targets.
RELATED WORK
In this work, develop graph- deep learning technique for protease enzyme specificity prediction. Here we provide a brief review previously developed predictive methods for protease specificity landscape prediction and applications of graph-based convolutional neural networks on protein-related problems.
2.1 PREDICTION OF PROTEASE SPECIFICITY LANDSCAPE
Current methods to discriminate the specificity landscape of one more types of protease enzymes, could be classified into two categories, machine learning approaches and scoring- matrix based ap- proaches (Li et al. 2019). Methods
machine learning methods such logistic regression, ran- dom forest, decision tree, support vector machine (SVM) to predict substrate specificity. The most popular tool is SVM among them. c.g. PROSPER (Song et al.. 2012), iProt-Sub (Song al. 2018)). CASVM (Wee et al.] 2007). Cascleave (Song et al. 2010). Besides, NeuroPred (Southey et al.) 2006) and PROSPERous (Song et al. 2017) applied logistic regression to predict specific neuropep- tide specificity (Neuropred) and 90 different proteases (PROSPERous). Pripper (Pùppo et al. 2010) provided three different classifiers based on SVM, decision tree and random forest. Procleave (Li et al. 2020b) implemented probabilistic model trained with both sequence and structure feature information. DeepCleave (Li et al. 2020a) is a tool to predict substrate specificity by using con- volutional neural network (CNN). None of methods mentioned above energy-related features, however we note that some energy terms for the interface were considered in (Pethe et al. 2017) and (Pethe al. 2018).
2.2 GRAPH CONVOLUTIONAL NEURAL NETWORK ON PROTEIN-RELATED PROBLEMS
There are several works proposing or implementing a graph-based convolutional neural network model to solve various protein modeling-relater problems. BIPSPI (Sanchez- et al.) 2019) made use of both hand-crafted sequence and structure features to predict residue-residue contacts in proteins. Gligorijevic et al. (2020) proposed novel model that generated node features by using LSTM to learn genetic information and edge adjacency matrix from contact maps to classify dif- ferent protein functions. Graph convolutional neural networks are also applied to drug discovery, for either node classification or energy score prediction (Sun et al. 2019). Fout et al. (2017) pro- posed graph-based model to encode a protein and drug into each graph, which considered local neighborhood information from each node and learned multiple edge features for edges between neighbor residues and nodes.
& Crivelli (2019) addressed their model to learn sequence- and structure- based information more efficiently than 2D/3D-CNN for protein structure classification. Moreover, [Cao & Shen (2019) and Sanyal et al. (2020) aimed at improving the energy functions used for protein model evaluation by using molecular graphs. Unlike previous work, approach uses per-residue and residue-ree residue pairwise energies as features for predicting molecular function.
2

---

Under review as a conference paper at ICLR 2021
(a) Molecular recognition site
F Node Feature NxF
Edge Feature NXNxM
Weighted
Sum
GCN Layer
Batch Norm
GCN Layer
Butch Norm
N
Dropeut
Linear Transform
NxNx1
Output 1 x Y
(b) Graph generation
(c) PGCN model
Figure 1: Schematic diagram of protease with substrate sample, graph generation and protein graph convolutional network (PGCN) Protease substrate diagram with sequence logo plot for the specificity landscape of P2-P6 sites. Here, the substrate labeled in blue is 11-amino-acid long, from P7 at N-terminus to P4' at C-terminus and is cleaved between P1 and P1'. The site labeled in yellow represents all neighbor residues which generation from the substrate residues (blue dots) and its neighbor residues (yellow dots). Grey lines denote edges. (c) PGCN training scheme. In the node matrix, residues listed in the order of residues starting from substrate residues followed by neighboring residues. In the node matrix, residues are listed in the order of residues from N-terminus C-terminus, with substrate residues followed by protease residues.
angstrom around the substrate. (b) Graph
METHODS
Overview We provide protein graph convolutional network (PGCN), which models protein- substrate complexes as fully connected graph structures conditioned on terms of both sequences and interaction energies. We generate energy-related features using Rosetta (Leaver-Fay) 2011), software suite for protein modeling and design, for either single residues or pairwise residues. Sub- strate 7-residue stretch (7-letter string) of amino acids sequences being recognized by a protease active site (Figure la). The goal is predict all motifs that are (are not) efficiently and correctly bound by the enzyme resulting in cleavage (non-clcavage).
3.1 PROTEIN GRAPH REPRESENTATION
The protein complex encoded as fully connected graph G (V,E), from which substrate and neighboring amino acids make up the nodes, and residue pairwise interactions make edges (Figure lb). Each node V; contains the one-body features single residue, while each edge Gij E & contains relational features between pair of residues. Since residues that are far away from the substrate are less likely to influence specificity and more likely to introduce noise into the model, we only consider sub-graph G', including the substrate and neighboring residues to specified distance around the substrate. The graph g' consists of two sets: nodes and edges. The N F matrix, where N is the number of nodes, F is the number
features of nodes encoded of node features. The features of edges are encoded as an N x x M adjacency tensor. (Summary of node and edge features shown in Table A.4

---

Under review as a conference paper at ICLR 2021
3.2 NORMALIZATION
Since the stability of the structure depends on the minimum potential energy, we normalize each energy-related feature with the exponential of negative potential amplify lower energy values and reduce higher energy values, which could be denoted as Qi =exp(- exp(-P,); where P; is äth node/edge potential energy.
3.3 MULTIPLE EDGE FEATURES
Traditionally, the adjacency matrix is used to represent whether pairs of residues is adjacent not, which is denoted as A E NxN (Wu et al. 2020) with Ai = if eij E E and A. = if egg & E. This representation is not able to handle multplee edge features, however, and in this work incorporating distances and multiple energetic components into the edge features, which Rosetta combines as weighted sum (Alford al. 2017). Thus it was necessary for us to flatten the edge feature tensor into a weighted adjacency matrix by summing pseudo total energy for each edge as shown below.
E - -- w. Eg
(1)
where E인 denotes total energy for an edge, w, denotes learned weight parameter for the jth energy terms and Esy denotes one of the edge energies for the edge. The matrix form of it is shown in Eq later.
3.4 PROTEIN GRAPH CONVOLUTIONAL NETWORK
PGCN learns features of nodes and edges and receives both tuned and learned parameter sets from training data, which is composed of graphs generated from Roselta-modeled plexes with known class labels (cleaved/uncleaved) PGCN feeds batch of graphs as input every step, calculates the loss hetween estimated and true labels, and updates learning parameters by gra- dient descent during back propagation.
com-
Here we describe the details of the PGCN architecture. First, PGCN multiplies the adjacency tensor of each graph from training data with M-long learning weight vector WA flatten multiple edge features using the weighted sum method. Second, the flattened adjacency matrix N x N. together with the node feature matrix, are fed into a graph convolutional (GC) layer shown as Eq Third, the GC layer is followed by BatchNorm layer (Toffe & Szegedy 2015), which aims at avoiding slow convergence. Next, the output from BatchNorm goes through another GC layer and BatchNorm layer to continue refining learning parameters. Then, PGCN drops out proportion of hidden nodes over nodes to avoid the overfitting problem (Srivastava et al.] (0044). Unlike normal multi-layer neural networks, PGCN does not count a proportion of hidden nodes out over all nodes using the standard dropout strategy. Instead, PGCN mutes different combinations of hidden nodes for different nodes. Finally, PGCN transforms the output into Y-dimensional vector, which shows the probabilities that the graph is classified into each class over classes. Here, the set of tuning hyperparameterss is made up of batch size, learning rate, dropout rate and weight decay coefficient. The weight decay coefficient is part of the 1.2 regularization term, that multiplies sum of learned weights for the anti-overfitting problem. Learning parameters keep updated through epochs. The trained PGCN model is used for testing, in which test data pass through each layer of the PGCN model but skip the dropout process.
The mathematical expression of PGCN model for one graph is shown below,
A' AWA
(2)
11 = D'D - XWx
(3)
H₂ = -''
(4)
where WA E Mx1 Wx E Fxc, Wi E C2x are learning weight matrix, C1 C2 are numbers of hidden nodes for two convolution layers, Á' -V=At
Á' + IN and D' is a diagonal matrix with D/ = E, Eq 2 shows the formulation of adjacency matrix A' from weighted sum of adjacency tensor A over features. In Eq and Eq 4 D' + D' - denotes normalization of the adjacency matrix, mentioned in Normalization section.
4

---

Under review as a conference paper at ICLR 2021
4 EXPERIMENTS
We use PGCN to discriminate protease specificity reflected on substrate cleavage, based on two sets of features: a hybrid set which contains both sequence (amino acid types) and energy information, and set which contains only energy information with categorical features. Cleaving, not cleaving are two main possibilities of pootess-ssubsraan interaction. We also consider ternary classification, wherein prbstase-Substrate pair exhibited very low cleavage activity in experiments, and can be said to partially cleave a substrate.
We then compare our results with SVM models from (Pethe et al. 2017) and (Pethe et al. 2018). and also compare with currently used machine learning models as mentioned in Related Work section. Furthermore, make analysis importance of nodes/edges to address PGCN's contribution to potential valuable enzyme design problems.
4.1 DATA
We have lists of 5-amino-acid- long substrate sequences (from P2-P6) that were determined previous experiments (Pethe al.] (017)) be cleaved, not cleaved, or partially cleaved by the wild type Hepatitis C (HCV) protease, or one of three mutants. HCV mutants had few substitu- tions, and were homology mod- eled have similar backbone ge- ometry. Despite the fact that the mutants have only 1-3 substitu- tions, they have significantly altered specificity landscapes. The database consists of 66,441 protein- substrate complexes and we trained 46,505 samples into four separate models for three classes, and trained on 27,475 samples of all training samples into another four PGCN models for two classes, see Table for details. The proportion of training and testing are the same among all models, which is 70%: 30%.
Table 1: Number of samples for different data
# Samples
Two Classes
Three Classes
Train
Test
Train
Test
WT
5139
2203
6106
2621
A17IT
9246
3962
15169
6501
D183A
8304
3560
12350
5294
R170K/AI71T//DISS
183A
4786
2052
12880
5520
4.2 PROTEIN COMPLEX MODELING
We modeled all P6-P2 substrate sequences in the context of an 11 amino acid (P7-P4') peptide with all non-variable residues consistent with the canonical HCV substrate sequence, bound to the HCV protease active (cleaving) conformation, based on the crystal structure (PDB:3M5L) (Romano) ct al.) 2010 from RCSB Protein Data Bank (PDB) (http://www.rcsb.org)) (Berman et al.) (000). Producing the models was done with Pyrosetta (Chaudhury et al., 2010), a Python- interface to Rosetta, and involved changing the side chains of the substrate to match the experimental sequence, then minimizing the complex using FastRelax (Tyka et al. 2011.
4.3 PROTEIN GRAPHI GENERATION
We considered residues of the core substrate (P2-P6), and neighbor residues within 10Å shell of the core substrate as nodes of our input graph (34 nodes total). All graphs are in the format of PyTorch 1.4.0 FloatTensor (Paszke et al. 2019).
PGCN used a more informative and concise feature set than other machine learning methods. PGCN with hybrid feature encoding mode includes all features described in Table A.4 and PGCN with energy-only features is the same set, excluding the one hot encoders of amino acids. All the other methods have hybrid feature encoding mode to include one hot encoders for P6-P2 amino acid types of substrates and energy terms for them are four coarse- graiieed potential energy terms instead (Pethe) et al. 2017).
5

---

Under review as a conference paper at ICLR 2021
4.4 TRAINING
As we mentioned above, PGCN with GC layers is used predict substrate specificity landscapes. Each layer has 20 hidden nodes, and non-linear ReLU term (Glorot et al. 20011) follows each graph convolution layer after implementing BatchNorm (loffe & Szegedy, 2015). For the training process of PGCN, weight decay and dropout rate. We compared feature encoding of PGCN with that other five machine learning methods. We used the Scikit-learn 0.20.1 (Pedregosa al. 2011) implement logistic regression (Ir), random forest (rf), decision tree (dt) and support vector machine (SVM) classification, and Tensorflow 1.13.1 (Abadi et al., 20T6) for artificial neural network (ANN). The ANN model in this experiment is one fully connected layer with 1024 hidden nodes and allows dropout rate between 0.1-0.9.
entropy loss function, Adam optimizer (Kingma & Ba 2015) and nonzero
4.5 NODE/EDGE IMPORTANCE
The determinants of protease specificity have not been isolated from the set of many contributing forces; for example, the most stable complexes do necessarily correlate with substrate recogni- tion. In order to derive biological insights about important residues or relationships between pairs of residues that contribute to discrimination, we perturbed the values of each node/edge term over all test samples and inspected how much the test accuracy drops, enabling us to efficiently determine the relative importance each node and edge. To be specific, we perturbed values of each node feature Fi, simultaneously across all samples for the importance detection of each node i and perturbed val- ues of each edge Mi,,, simultaneously across all samples for the importance detection of each edge (1,j). The upper and lower triangular of the adjacency matrix should change simultaneously with the same randomness scheme, since it is a symmetric matrix. Therefore, the number of total pertur- bations should be N + N(N-1) N(N+1) if considering all N nodes and edges. The formula for measuring accuracy loss given by Relative Acc = (Original Acc Perturbed Acc) (are)/Origiaa Acc.
2.
Table 2: Accuracy table for models based on feature settings for binary classification
Methods
Wild Type
A171T
D183A
Triple
Hybrid
E-only
Hybrid
E-only
Hybrid
E-only
Hybrid
E-only
Logistic Reg
92.19
76,08
95.96
79.38
89.21
69.27
92.2
70.71
Random Forest
91.68
73.62
95.85
73.25
87.7
65.63
91.05
65.63
Decision Tree
86.79
74.63
91.54
76.58
83.88
68.29
87.77
69.54
SVM
92.87
75.85
95.84
78.85
89.44
69.66
92.79
70.57
ANN
93.19
76.49
96.44
79.86
89.55
69.75
93.08
70.61
PGCN
92.73
90.46
96.19
95.41
88.88
88.17
92.30
90.64
Table 3: Accuracy table for models based feature settings for ternary classification
Methods
WT
A171T
D183A
Triple
Hybrid
E-only
Hybrid
E-only
Hybrid
E-only
Hybrid
E-only
Logistic Reg
79.97
63.3
80.08
52.45
68.45
47.26
71.85
65.65
Random Forest
80.57
56.34
81.53
52.49
69.22
48.22
72.14
61.31
Decision Tree
70.97
62.3
73.4
52.47
62.67
62.03
63.97
SVM
82.18
63.68
83.69
53.76
71.23
46.88
69.2
59.67
ANN
84.32
64.75
84.63
53.76
71.99
47.56
73.73
65.80
PGCN
82.93
78.50
83.59
79.71
71.08
69.04
73.71
72.34

---

Under review as a conference paper at ICLR 2021
5 RESULTS
5.1
FEATURE SET GENERATION
We compare different feature coding modes of PGCN (energy-only and energy+scquence "hybrid" fea- tures) with that of current five ma- chine learning methods for both two classes and three classes. Accuracy based on the hybrid feature set not significantly different across different models for each data, except that de- cision tree reaches at least 9% lower accuracy. ANN always reaches the highest accuracy among all data, for example, up to 96.4% for two classes of HCV with A17IT mutation predicted correctly and up to 84.6% for three classes of it.
VNC
10.6
9M
S4Y90M Dyme
0AAITT
When it comes to energy-only encod- ing, PGCN always performs the best, up to 95. 4.4% for two classes of A171T HCV, about 15% higher than the best accuracy among other machine learn- ing methods, see Table the Figure 2 and A.5 (in Appendix), it is casy to see that accuracy values of energy-only feature encoding drop heavily down to similar level for all five machine learning methods, which shows that coarse-grained energies used in other machine learnign models are not informative enough for label classification. PGCN based energy-only feature encoding almost recovers the accuracy based on hybrid feature encoding.
Figure 2: Accuracy of different models based different features for binary classification. The length of an orange bar denote the closeness of accuracy.
3 From
145
174
17lbys
Les
Wild type, hybrid, two classes
(b) Wild type, energy, two classes
(b) Wild type. energy, two classes
189
124bys
(k) DI83A, hybrid, three classes
(1) DI83A, energy, three classes
(c) Wild type, energy, three classes
(a) Importance diagram.
Figure 3: (a) Parts of the importance diagram for wild type HCV, binary classification and D183A HCV, ternary classification. Sizes of nodes, widths of edges proportional to importance scores. See The full importance diagram (b) Real protein structure examples with highlighted important nodes. Important nodes labeled in the form of "amino acid type, residue index".
""Triple" equivalent with "R "RI70K/A17 1TTN//A11TT/11831" "Logistic Reg" equivalent with "Logistic regression"

---

Under review as a conference paper at ICLR 2021
5.2 NODE/EDGE IMPORTANCE FOR PROTEASES
When comparing network diagrams for hybrid (sequence+energy) feature encoding with those for energy-only feature encoding, the energy-only PGCN found the largest contributions were from substrate nodes, except that two classes for wild type HCV in and three classes for D183A HCV have two significant edges, P2-72 and P4-173. Among all important edges, edges that connect residues of the substrate with residues of the protein stand out be the main contribution of importance. First, models detect P2-72 as an important edge, always within top 2 of all edges. Residue 72 is the catalytic residue, (see Figure (b). Next, a set important connections is between the substrate and the residues of beta-sheet of the enzyme, such as P4-173, P5-174, P2-171. We suspect that the importance of these interactions are due to the fact that this beta sheet serves as a template for the substrate and aids in positioning the scissile bond in the active site. In most cases, edges between substrate notes were lower influence. Morcover, several edges between residues both from the protease show importance well, such 170-183 of the wild type binary model, 138-183 of the wild type ternary model, and 171-183 of the A17IT ternary model. These residues may not interact directly with the substrate, but as they form the secondary shell around the binding pocket, they likely impact the stability of substrate binding.
6 DISCUSSION
In general, PGCN performs impressively well in recapitulating specificity profiles, especially in models using only energy-based features. Following are some avenues for further improvement of our model.
More divergent imput samples To generalize PGCN beyond the Hepatitis NS3/4 protease, more extensive and specific set for other proteases, such as TEV protease (Li et al. 2017 Packer et al., 2017) could be useful. These will allow sampling different mozyme-substrer sequence space as well as chemical enviormments, likely leading to a more robust model.
Imbalanced data The number of samples for each class are imbalanced and proportions of number of samples in classes vary especially for ternary classification. For example, the proportion of num- ber of samples in cleaved, partially cleaved and uncleaved classes for wild type HCV is 1:1:3, while the proportion for RI /110K/ I7TT/D A is 1:3:1. This may arise problem because many machine learning models assume balanced data as the input. In this case, the model may underestimate minor- ity class(es) (Mirza al. 2019). We tried oversampling strategy (Ke et al., (008)) various machine learning models, and found that it somewhat improves accuracies of certain classes. We would like to further explore different class imbalance learning (CIL.) strategies see if they improve PGCN performance.
7 CONCLUSION
In this work, we implemented a protein graph convolution network (PGCN) to classify protease- substrate pairs either yielding substrate cleavage, partial cleavage, or non-cleavage. Using Rosetta, we generated a structural model for each protease-substrate complex, which we con- verted into fully connected graph that encoded potential energies for each single residue and each pair of residues. Using the subgraph that includes the bound peptide and neighboring pro- tease residues, we trained the PGCN to predict the behavior of the interaction. We found that the PGCN reaches equivalent accuracy of other machine learning methods using the combination of sequence and energetic features. Furthermore, we demonstrated that variable importance analysis on the PGCN could be used to identify the nodes and edges most influential in determining pro- tease specificity. This method has the potential to enable better prediction and eventually design of engineered proteases with targeted substrate specificity. Codes for this work are available at http://ggihab..oom///ddeeee222/p pentpe--gnnnp..ccch....
ACKNOWLEDGMENTSS
Thanks to Wenfa Lu, Xiao Li for helpful discussions and supports.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenherg, Rajat Monga, Sherry Moore, Derek Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. Tensorflow: system for large-scale machine learning this paper is included in the proceedings of the 12th usenix symposium on operating systems design and implementation (osdi '16). tensorflow: A system for large-scale machine learning, 2016.
Rebecca F. Alford, Andrew Leaver-Fay, Jeliazko R. Jeliazkov, Matthew J. O'Meara, Frank P. Di- Maio, Hahnbeom Park, Maxim V. Shapovalov, P. Douglas Renfrew, Vikram K. Mulligan, Kalli Kappel, Jason W. Labonte, Michael S. Pacella, Richard Bonneau, Philip Bradley, Roland L. Dun- brack, Rhiju Das, David Baker, Brian Kuhlman, Tanja Kortemme, and Jeffrey Gray. The rosetta all-atom energy function for macromolecular modeling and design. Journal of Chemical Theory and Computation, 13:3031-3048, 2017.
Helen M. Berman, Jhon Westbrook, Zukang Feng, Gary Gilliland, T Bhat, Helge Weissig, Ilya N Shindyalov, and Philip E Bourne. The protein data bank. Nucleic Acids Research, 28:235-244 01 2000. doi: 10. 10.1003/nar/28.1.2
Yue Cao and Yang Shen. Energy-based graph convolutional networks for scoring protein docking models, 12 2019.
Sidhartha Chaudhury, Sergey Lyskov, and Jeffrey J. Gray. Pyrosetta: A script-based interface for im- plementing molecular modeling algorithms using rosetta. Bioinformatics, 26:689-688 03 2010.
Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. Protein interface prediction using graph convolutional networks, 2017.
Vladimir Gligorijevic, P. Douglas Renfrew, Tomasz Kosciolek, Julia Koehler Leman, Daniel Beren- berg, Tommi Vatanen, Chris Chandler, Bryn C. Taylor, Ian M. Fisk, Hera Vlamakis, Ramnik J. Xavier, Rob Knight, Kyunghyun Cho, and Richard Bonneau. Structure-based protein function prediction using graph convolutional networks, 2020.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks, 2011.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift, 2015.
Xiongwen Ke, Josef Dick, Dr Quoc, and Thong Le Gia. Learning with imbalanced data. PhD thesis, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2015.
Tyka Michael Lewis Steven M. Lange Oliver F. Thompson James Jacak Ron Kaufman Kristian W. Renfrew P. Douglas Smith Colin A. Sheffler Will Davis Ian W. Cooper Seth Treuille Adrien Mandell Daniel J. Richter Florian Ban Yih-En Andrew Fleishman Sarel J. Corn Jacob E. Kim David E. Lyskov Sergey Berrondo Monica Mentzer Stuart Popović Zoran Havranek James Karanicolas John Das Rhiju Meiler Jens Kortemme Tanja Gray Jeffrey J. Kuhlman Brian Baker David Bradley Philip Leaver-Fay, Andrew. Rosetta3: An object-oriented software suite for the simulation and design of macromolecules. Methods in Enzymology, 487:545-574 2011.
Fuyi Li, Yanan Wang, Chen Li, Tatiana T. Marquez-Lago, André Leier, Neil D. Rawlings, Gho- lamreza Haffari, Jerico Revote, Tatsuya Akutsu, Kuo-Chen Chou, Anthony Purcell, Robert N. Pike, Geoffrey Webb, A. Ian Smith. Trevor Lithgow, Roger J. Daly, James C. Whisstock, and Jiangning Song. Twenty years of bioinformatics research for protease-speciff substrate and cleav- age site prediction: A comprehensive revisit and benchmarking of existing methods. Briefings in Bioinformatics, 20:21 50-2166, 11 2019.
Fuyi Li, Jinxiang Chen, André Leier, Tatiana Marquez-Lago. Quanzhong Liu, Yanze Wang. Jerico Revote, A. Ian Smith, Tatsuya Akutsu, Geoffrey I. Webb, Lukasz Kurgan, and Jiangning Song. Deepeleave: deep learning predictor for caspase and matrix metalloprotease substrates and cleavage sites. Bioinformaticx, 16:1057-1065. 02 2020a.

---

Under review as a conference paper at ICLR 2021
Fuyi Li, Andre Leier, Quanzhong Liu, Yanan Wang, Dongxu Xiang, Tatsuya Akutsu, Geoffrey I Webb, A lan Smith, Tatiana Marquez-Lago, Jian Li. and Jiangning Song. Procleave: Predict- ing protease-specine substrate cleavage sites by combining sequence and structural information. Genomics, Proteomics Bioinformaticx, 18:52-64, 2020b.
Qing Li, Li Yi, Kam Hon Hoi, Peter Marek, George Georgiou, and Brent L. Iverson. Profiling protease specificity: Combining yeast er sequestration screening (yess) with next generation se- quencing. ACS Chemical Biology, 12:510-518, 01 2017.
Bilal Mirza, Wei Wang, Jie Wang, Howard Choi, Neo Christopher Chung, and Peipei Ping. Machine learning and integrative analysis of biomedical big data. Genes, 10, 01 2019.
Michael S. Packer, Holly A. Rees, and David R. Liu. Phage-assisted continuous evolution of pro- teases with altered substrate specificity. Nature Communications, 8, 10 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, Sigh-performance deep learning library, 2019.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, and Matthieu Perrot Andédouard Duch- esnay. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12: 2825-2830, 2011.
Manasi A. Pethe, Aliza B. Rubenstein, and Sagar D. Khare. Large-scale structure-based prediction and identification of novel protease substrates using computational protein design. Journal of Molecular Biology, 429:220-236, 01 2017.
Manasi A. Pethe, Aliza B. Rubenstein, and Sagar D. Khare. Data-driven supervised learning of vi- ral protease specificity landscape from deep sequencing and molecular simulations. Proceedings of the National Academy of Sciences, 116:168-176. 12 2018.
Mirva Piippo. Nina Lietzén, Olli S Nevalainen, Jussi Salmi, and Tuula A Nyman. Pripper: Predic- tion of caspase cleavage sites from whole proteomes. BMC Bioinformatics, 11, 06 2010.
Keith Romano, Akbar Ali, William E. Royer, and Celia A. Schiffer. Drug resistance against hev ns3/4a inhibitors is defined by the balance of substrate recognition versus inhibitor binding. Pro- ceedings of the National Academy of Sciences of the United States of America, 107:20986-20909 2010.
Ruben Sanchez-Garcin C. O. S. Sorzano, J. M. Carazo, and Joan Segura. Bipspi: A method for the prediction of partner-specific protein- interfaces. Bioinformatics, 35:470-477, 02 2019.
Soumya Sanyal, Ivan Anishchenko, Anirudh Dagar, David Baker, and Partha Talukdar. Proteingen: Protein model quality assessment using graph convolutional networks, 2020.
Jiangning Song. Hao Tan, Hongbin Shen, Khalid Mahmood, Sarah E. Boyd, Geoffrey Webb, Tat- suya Akutsu, and James C. Whisstock. Cascleave: Towards more accurate prediction of caspase substrate cleavage sites. Bioinformatics (Oxford, England), 26:752-760. 03 2010.
Jiangning Song, Hao Tan, Andrew J. Perry, Tatsuya Akutsu, Geoffrey I. Webb, James C. Whisstock, and Robert Pike. Prosper: An integrated feature-based tool for predicting protease substrate cleavage sites. PLOS ONE, 7:e50300, 11 2012.
Jiangnng Song, Fuyi Li, André Leier, Tatiana Marquez- Tatsuya Akutsu, Gholamreza Haf- fari, Kuo-Chen Chou, Geoffrey I Webb, and Robert N Pike. Prosperous: High-throughput pre- diction of substrate cleavage sites for 90 proteases with improved accuracy. Bioinformatics, 34: 684-687, 10 2017.
10

---

Under review as a conference paper at ICLR 2021
Jiangning Song. Yanan Wang, Fuyi Li, Tatsuya Akutsu, Neil D Rawlings, Geoffrey I Webb, and Kuo-Chen Chou. Iprot-sub: comprehensive package for accurately mapping and predicting protease-specific substrates and cleavage sites. Briefings in Bioinformatics, 20:638-658, 04 2018.
Bruce R. Southey, Andinet Amare, Tyler A. Zimmerman, Sandra L. Rodriguez-Zas, and Jonathan V. Sweedler. Neuropred: A tool to predict cleavage sites in neuropeptide precursors and provide the masses of the resulting peptides. Nucleic Acids Research. 34, 2006.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 19:9999-9958 2014.
Mengying Sun, Sendong Zhao, Coryandar Gilvary, Olivier Elemento, Jiayu Zhou, and Fei Wang. Graph convolutional networks for computational drug development and discovery. Briefings in Bioinformaticx, 21, 06 2019.
John A. Tainer and Richard P. Cunningham. Molecular recognition in dna-binding proteins and enzymes. Current Opinion Biotechnology. 4:474 483, 1993.
Michael Tyka, Daniel A. Keedy, Ingemar André, Frank DiMaio, Yifan Song, David C. Richard- son, Jane S. Richardson, and David Baker. Alternate states of proteins revealed by detailed energy landscape mapping. Journal of Molecular Biology, 405:607-618, 01 2011.
Matej Vizovišek, Robert Vidmar, Marcin Drag, Marko Fonović, Guy S. Salvesen, and Boris Turk. Protease specificity: Towards in vivo imaging applications and biomarker discovery. Trends in Biochemical Sciences, 43:829-844, 10 2018.
Lawrence J. K. Wee, Tin Wee Tan, and Shoba Ranganathan. Casvm: Web server for svm-based prediction of caspase substrates cleavage sites. Bioinformatics, 23:3241-3243, 2007.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. A comprehensive survey graph neural networks. IEEE Transactions Neural Networks and Learning Systems, pp. 1-21, 2020.
Rafael Gamora-Resendie and Silvia Crivelli. Structural learning of proteins using graph convolu- tional neural networks, 04 2019.

---

Under review as a conference paper at ICLR 2021
A
APPENDIX
Table A.4: Features for nodes and edges
TYPE FEATURES
DESCRIPTION
aa
One hot encoders for amino acid type
fa_atr fa.rep
Lennard-Jones attractive potential
Lennard-Jone repulsive potential
Node
fa sol
Lazaridis Lazaridis-Karplus solvation potential
Ik ball wtd fa.elec. hbond
Asymmetric solvation potential
Coulombic clectrostatic potential
Hydrogen bonding potential
is_substrate fa intra sol xover4 fa_intra_rep
if the node belongs to the substrate; otherwise, 0
Intra-residue LK solvation energy
Lennard-Jones repulsive energy between pairwise residues
rama-prepro
Ramachandran preferences of backbone angles Omega dihedral of the backbone
Edge
omega
p_aa_pp
Probability of amino acid type at backbone angles Side-chain conformation potential
fa_dun
ref
Reference potential of pairwise residues
pairwise residues form a covalent bond; otherwise, 0
covalent_bond
one residue from the substrate, the other from the protein; otherwise, 0
intramolecular
12

---

Under review as a conference paper at ICLR 2021
Hyteal
Pure
Pue
70
60
46
46-
40
40
10
30
3VM
XVM
ANN
(4) Type
(1)-1
nylrri
Byleri
Pure
Puze
70
70
10
(6) DI8SA
(6) R170K//11/TDDIB8
Figure A.4: Accuracy of logistic regression (lg), random forest (rf), decision tree (dt), support vector machine (svm), artificial neural network (ann) and PGCN based on either "hybrid" or energy-only feature encoding for ternary classification. See the caption of Figure 2 for annotations.
13

---

Under review as a conference paper at ICLR 2021
Gal Wild type lybrid, two dlanes
(b Wild cleaes
(c) WT, bolvid dwez clanci
(3) WT. tiree closes
(e) AITIT, hyfrid two cbese
()) AITIT, energy. two theos
(g) A171T, Byfeid, claous
(b) A171T, anergy throe obese
DI83A, lybrid classs
di DI8JA,
(3) DESSA, byhrld, clases
DI8TA, clasess
(mi Triplis, hybrid own closs
(e) Triple, enengy, Iro clanci
(o) Tripke, iyhrid firee clases
(pi Triple, energy, trre cianee
Figure A.S: The whole importance diagram for PGCN WT, A17IT, DI83A RI70K/AI71T/DI8AA proteases for either hybrid features or energy-only features.
14


Reviews:
[{"review_id": "zJ-ZmYp3lsF", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThe paper proposes a new method for calculating the specificity of proteases towards their substrates. The method constructs a graph with node and edge features from the Rosetta molecular energy function, and uses this as the basis for a graph convolutional neural network. The authors report performance on-par on better than existing methods, and highlight interpretability as a potential advantage of the method.\n\n**Strengths**\n o The manuscript addresses and important problem\n o The method claims to produce state-of-the-art performance with a new method.\n o By basing the method on physical features derived from an established energy function, the method can provide some degree of explanation for the predictions made.\n\n**Weaknesses**\n o The paper is primarily an application of existing machine learning methodology, rather than introducing a new method.\n o It is unclear whether the baselines in the paper truly reflect the current state-of-the-art, and a proper train/validation/test split was employed during training.\n o The notation used in the paper is not consistent. \n\n**Recommendation**\nI recommend this paper be rejected. In my view, this paper reads more like a machine learning application than a methodological contribution. The presented technique is based on established graph neural network methods, and in the areas were it potentially deviates from this standard it provides insufficient motivation and discussion (see examples below). Since there is no systematic exploration of the methodological contributions, I do not feel that there are sufficient lessons learned in this paper to constitute a valuable contribution to a Machine Learning conference.\n\n**Supporting arguments for recommendation**\nExamples with insufficient discussion/motivation of methodological choices:\n o An identify matrix seems to be added to the edge-feature matrix A' (page 4, bottom), but it is unclear why this choice was made - and why it makes sense - since these matrices contain weighted sums of (normalized) energies.\n o A custom dropout technique is introduced, where combinations of hidden nodes are \"muted\". It is not discussed why this choice was made, and how it compares to standard dropout.\n o Results are reported for two scenarios: A hybrid scenario and a pure energy scenario. However, these choices are not motivated.  Are there cases where we know the energies but not the sequences?  Does Rosetta even allow for calculation of energies without knowing the sequence?\n\nThe language used in the paper is also confusing at times. For instance, the term \"adjacency matrix\" is used to refer to an edge-feature matrix, and they discuss generalization as an \"anti-overfitting problem\". Also, the edge features that they introduce seem to be referred to by at least three different names, E (eq 1), A (eq 2), and Q/P (top of page 4), and edges are referred to by both single and double indices.\n\nI'm also not confident about how significant the reported results are. While Page 5 mentions a training and test set on Page 5, it is not clear how the hyper-parameters were tuned (no mention of a validation set). Finally, while they do compare their method to several methods from the literature, it was not quite clear to me whether their reimplementations were *identical* to the methods from the literature - or simplified versions.\n\nFinally, it is quite striking that the simple ANN baseline performs so well in the \"Hybrid scenario\". The authors should at least discuss why this might be.\n\n**Questions to the authors**\nPage 5. \"We use PGCN... based on two sets\"\nWhat is the motivation for studying these two cases. Is the amino acid sequence not always available? \n\nPage 5. \"We then compare our results with SVM models from...\"\nAre these the exact same implementations, or did you merely use *some* SVM implementation. Have you confirmed that your implementations can reproduce the originally reported results of these competing methods?\n\nPage 2. \"None of methods mentioned above use energy-related features.\"\nPage 5. \"All the other methods have bybrid feature encodings model to include ... and energy terms...\"\nIt seems that these two statements are contradictory. Could you clarify?\n\nPage 5: \"The proportion of training and testing...\"\nWhich dataset to you use for optimizing the hyperparameters of the model. Do you have a validation set?\n\n**6. Additional feedback**\nPage 1: \"non-recogntion\" (typo)\nPage 2, line 2: \"Protein Convolutional Neural Networks (PGCN)\". Considering writing \"Protein Graph Convolutional Network\" here, so that it fits with the abbreviation.\nPage 2, \"Methods use machine learning methods...\". Is there something missing in this sentence?\nPage 2, \"None of methods mentioned above\" -> \"None of the methods mentioned above\"\nPage 2, \"using LSTM\" -> \"using an LSTM\"\nFigure 1 caption. The last line was a bit confusing to me - since it described the order of the \"node matrix\" twice. Should one of these have been the \"edge matrix\"?\nPage 3, \"Substrate\" -> \"The substrate\"\nPage 4, \"\\tilde A' = \\tilde A' + I_N\". I assume the first element to the right should be A'? \nPage 6. \"non linear ReLU term\". This is not really a \"term\", is it?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "o0AT4oUuhbK", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper uses a structure-based molecular interaction graph generated from the Rosetta interaction energy function to develop protein graph convolutional neural networks (PGCN) that predict enzyme substrate specificity. \tThe authors clearly describe the goal of being able to accurately model the substrate specificity of enzymes such as proteases, including both a description of those substrates that the enzyme recognizes in addition to those substrates that it does not recognize. They propose that this model should capture the energetics of interactions between the enzyme and potential substrates, such that substrates recognized by the enzyme are assigned lower energies by the model than substrates that are not recognized by the enzyme. To address this challenge, they propose a protein graph convolutional neural network, in which the enzyme and substrate residues are modeled as nodes, while interaction energies obtained via a pairwise decomposition of the energy of the enzyme-substrate complex are considered as node and edge features.  \t\n\nThe authors provide some survey of recent literature that covers protein-substrate interactions. They can improve by discussing the contributions of papers on this subject from a wider range of research labs - the majority of papers cited in this section feature a common author. Similarly in the related work on graph convolutional networks for protein-related problems the authors should cite work such as e.g. the graph convolutional models for protein ligand interaction prediction from Torng and Altman 2019 among others.  \n\nThe authors present results for both 'hybrid' and 'energy-only' models. The energy-only feature encoding for the other machine learning models is not clearly described, and it is not clear why the energy-only feature encoding is of interest - the authors do not describe any context in which this encoding would be used in preference to the hybrid sequence and energy feature encoding, which performs better. However, for the energy-only feature encoding the model developed in this paper always performs the best - it is important for the authors to explain why this result is of interest to the reader, since the performance using this encoding is always worse than the performance of multiple other models using the hybrid encoding. For the hybrid encoding and overall, either the ANN or SVM models perform best. \n\nAn important question is whether the considerably more detailed energy feature encoding of the PGCN model effectively contains information about the amino acid sequence, making the one-hot sequence encoding that is in present in the hybrid encoding but absent from the energy-only encoding redundant. This would explain why PGCN does comparatively well in the energy-only setting, compared to the much simpler coarse grained energy terms used by the other models. Overall it is very unclear what value the PGCN model and featurization adds. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ivAGI8VdDg8", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work presents a protein graph convolutional neural network (PGCN) which feeds features from Rosetta through a graph CNN to predict substrate specificity. \n\nStrengths\n- Related work is concise but explanatory \n- Incorporating features from Rosetta into a graph-based neural network is creative. This is a contribution / idea that could be used in other paper as well.\n \nWeaknesses\n- The architecture is described well but there are not any ablations or explanations of how the authors converged on this architecture. Could the results be improved by iterating on the architecture?\n- The tested baselines are logistic regression, random forest, decision tree, SVM, and ANN. However, there are no baselines from previous literature, so it is difficult to place this work in the context of the field.\n- In Table 2 and Table 3, the default ANN from Tensorflow performs better than the PGCN. In that case, what is the advantage of the PGCN?\n- The authors compare a \"hybrid\" (energy + sequence) to a \"pure\" approach (energy). The PGCN performs better than other methods in the \"pure\" setting, but all models perform similarly in the \"hybrid\" setting. Is there a practical reason to ever use to the \"pure\" setting? If we are able to get energies in Rosetta, than we likely have the sequence available to us. If the authors can answer this, my next question would be why is PGCN barely affected by incorporating the sequence whereas the other methods see a great improvement?\n- From the abstract and introduction, it seems that the authors set up the problem in this way so that they could generalize to new enzymes. However, this is not tested in the paper.\n\nAdditional\n- In the introduction, the authors should be more clear about \"substrate sequence motifs\" - do they mean primary or tertiary structure?\n- In equation 1, why is a simple weighted sum of edge features used? The individual components could be useful to the network. Why not provide the features directly?  \n\nOverall, I was excited to see Rosetta features incorporated into a neural network. However, the evaluation, results, and model development are weak. These would need to be improved if the paper is to be accepted to ICLR.", "extra_scores": {}, "extra_text": {}}, {"review_id": "XQu3BDHepXC", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In the work, the authors applied graph convolutional neural networks to predict enzyme specificity. The problem is very important in biology, which even can be used for designing new drugs. Essentially, the authors generated energy-related features using Rosetta and then built the graph using the molecular structure. Then, they applied the existing GCN to the problem. In the evaluation section, they compared with Logistic Regression, Decision Tree, Random Forest, SVM, ANN, to show the superiority of the method. \n\nHowever, the manuscript has the following major flaws.\n1. There is no novelty in the methodology part. They only applied the existing GCN model. The manuscript is not for the audience of ICLR. \n2. The baselines are too weak. They only compared with the classic ML methods. They did not compare with the state-of-the-art methods to predict the binding affinity. The authors can find a lot of related literature.\n3. Even compared with the classic algorithms, the proposed method did not outperform them by a large margin. \n4. They provide the link to the Github repository, which makes the submission not anonymous anymore. That's the reason that I did not provide related works in comment #2.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All four referees have indicated reject. Severe points of criticism have been raised, concerning the lacking novelty, the  experimental setup and the significance and interpretation of results. I fully agree with the reviewers in all important points, so I recommend rejection.
Decision: Reject