Geometric Transformers for Protein Interface Contact Prediction
Authors: Alex Morehead
Chen Chen
Jianlin Cheng
Venue: ICLR 2022 Poster (2022)
Keywords: Geometric Deep Learning
Graph Transformers
Protein Bioinformatics
Invariance

Abstract: Computational methods for predicting the interface contacts between proteins come highly sought after for drug discovery as they can significantly advance the accuracy of alternative approaches, such as protein-protein docking, protein function analysis tools, and other computational methods for protein bioinformatics. In this work, we present the Geometric Transformer, a novel geometry-evolving graph transformer for rotation and translation-invariant protein interface contact prediction, packaged within DeepInteract, an end-to-end prediction pipeline. DeepInteract predicts partner-specific protein interface contacts (i.e., inter-protein residue-residue contacts) given the 3D tertiary structures of two proteins as input. In rigorous benchmarks, DeepInteract, on challenging protein complex targets from the 13th and 14th CASP-CAPRI experiments as well as Docking Benchmark 5, achieves 14% and 1.1% top L/5 precision (L: length of a protein unit in a complex), respectively. In doing so, DeepInteract, with the Geometric Transformer as its graph-based backbone, outperforms existing methods for interface contact prediction in addition to other graph-based neural network backbones compatible with DeepInteract, thereby validating the effectiveness of the Geometric Transformer for learning rich relational-geometric features for downstream tasks on 3D protein structures.

Published as a conference paper at ICLR 2022
GEOMETRIC TRANSFORMERS FOR PROTEIN INTERFACE CONTACT PREDICTION
Alex Morehead, Chen Chen, & Jianlin Cheng
Department of Electrical Engineering & Computer Science University of Missouri Columbia, MO 65211, USA (acmwbb, chen. chen, cheng @mi ssouri edu
ABSTRACT
Computational methods for predicting the interface contacts between proteins come highly sought after for drug discovery as they can significantly advance the accuracy of alternative approaches, such protein-protein docking, protein function analysis tools, and other computational methods for protein bioinfor- matics. In this work, present the Geometric Transformer, novel geometry- evolving graph transformer for rotation and translation-ing innslation-invariant protein inter- face contact prediction, packaged within DeepInteract, end-to-end prediction pipeline. DeepInteract predicts partner-specific protein interface contacts (i.e., inter-protein residue contacts) given the 3D tertiary structures of two pro- teins as input. In rigorous benchmarks, DeepInteract, on challenging protein com- plex targets from the 13th and 14th CASP-CAPRI experiments as well Docking Benchmark 5, achieves 14% and 1.1% top L/5 precision (L: length of a protein unit complex), respectively. In doing so, DeepInteract, with the Geomet- ric Transformer as its graph-based backbone, outperforms existing methods for interface contact prediction in addition to other graph-based neural network back- bones compatible with DeepInteract, thereby validating the effectiveness of the Geometrie Transformer for learning rich relational -eomeerri features for down- stream tasks on 3D protein structures
INTRODUCTION
Interactions of proteins often reflect and directly influence their functions in molecular processes, so understanding the relationship between protein interaction and protein function is of utmost impor- tance to biologists and other life scientists. Here, we study residue-residue interaction between two protein structures that bind together to form binary protein complex (i.e., dimer), to better understand how these coupled proteins will function in vivo. Predicting where two proteins will interface in silico has become appealing method for measuring the interactions between proteins since a computational approach saves time, energy, and resources compared to traditional methods for experimentally measuring such interfaces (Wells & McClendon (2007)). A key motivation for determining these interface contacts to decrease the time required to discover new drugs and to advance the study of newly designed proteins (Murakami et al. (2017)).
Existing approaches to interface contact prediction include classical machine learning and deep learning-based methods. These methods traditionally hand-crafted features to predict which inter-chain pairs of amino acid residues will interact with one another upon the binding of the two protein chains, treating each of their residue pairs being independent of one another. Recent work on interface prediction (Liu et al. (2020). however, considers the biological insight that the interaction between two inter-chain residue pairs depends not only on the pairs' features themselves but also other residue pairs ordinally nearby in terms of the protein complex's 's sequence. As such, the problem interface contact prediction became framed as one akin to image segmentation
'Training and inference code http::::///tttuub.ooo/////io/o
well pre-trained models are available
www.eerannnni nteract

---

Published as a conference paper at ICLR 2022
Figure 1: A Mol* (Sehnal et al. (2021) visualization of interacting protein chains (PDB ID: 3H11).
or object detection, opening the door to innovations in interface contact prediction by incorporating the latest techniques from computer vision.
Nonetheless, up to now, no works on partner-specific protein interface contact prediction have lever- aged two recent innovations to better capture geometric shapes of protein structures and long-range interactions between amino acids important for accurate prediction protein- prrciin-rrotennne interface con- tacts: (1) geometric deep learning for evolving proteins' geometric representations and (2) graph- based self-attention similar to that of Vaswani et al. (2017). Towards this end, we introduce Deep- Interact, end-to-end deep learning pipeline for protein interface prediction. DeepInteract houses the Geometric Transformer, a new graph transformer designed to exploit protein crructure-speciffe geometric properties, as well as dilated convolution-based interaction module adapted from Chen et al. 202T) to predict which inter-chain residue pairs comprise the interface between the two protein chains. In response to the exponential rate of progress being made predicting protein structures in silico, we trained DeepInteract end-to-end using DIPS-Plus (Morehead et al. (2021)), to date the largest feature-rich dataset of protein complex structures for machine learning of protein interfaces, to close the gap on a proper solution to this fundamental problem in structural biology.
2 RELATED WORK
Over the past several years, geometric deep learning has become an effective means of automati- cally learning useful feature representations from structured data (Bronstein et al. (202T)). Previ- ously, geometric learning algorithms like convolutional neural networks (CNNs) and graph neural networks (GNNs) have been used to model molecules and to predict protein interface contacts. Schütt et al. (2017) introduced a deep tensor neural network designed for molecular tasks in quan- tum chemistry. Fout al. (2017) designed siamese GNN architecture learn weight-tied feature representations of residue pairs. This approach, in essence, processes subgraphs for each residue in each complex and aggregates node-level features locally using nearest-neighhors approach. Since this partner-specific method derives its training dataset from Docking Benchmark 5 (DB5) (Vreven) (2015). is ultimately data-limited. Townshend al. (2019) represent interacting protein complexes by voxelizing each residue into 3D grid and encoding in each grid entry the presence and type of the residue's 's underlying atoms. This partner-speciff encoding scheme captures static geometric features of interacting complexes, but it is not able to scale well due to its requiring spatial resolution of the residue voxels achieve good results.
computationally-expenss
Continuing the trend of applying geometric learning to protein structures, Gainza et al. (020)) developed MaSIF to perform partner-indeppendet interface region prediction. Likewise, Dai & Bailey-Kellogy (2021) do so with attention-based GNN. These methods learn to perform binary classification of the residues both complex structures to identify regions where residues from both complexes are likely to interact with one another. However, because these approaches pre- dict partner-indee independdddee interface regions, they are less likely to be useful in helping solve related tasks such drug-protein interaction prediction and protein-protein docking (Ahmad & Mizuguchi
2

---

Published as a conference paper at ICLR 2022
2011)). Liu al. a02 created graph neural network for predicting the effects of mutations on protein-prooe protein binding affinities, and, more recently, Costa et al. (022) introduced a Euclidean equivariant transformer for protein docking. Both of these methods may benefit from the availability of precise interface predictors by using them to generate contact maps as input features.
To date, of the best result sets obtained by any model for protein interface contact prediction comes from Liu et al. (2020) where high-order (i.e. sequential and inevolution-based interactions between residues are learned and preserved throughout the network in addition static geomet- ric features initially embedded in the protein complexes. However, this work, like many of those preceding it, undesirably maintains the trend of reporting model performance in terms of the me- dian area under the receiver operating characteristic which is not robust to extreme class imbalances as often occur in interface contact prediction. In addition, this approach data-limited as it uses the DB5 dataset and its predecessors to derive both its training data and makes use of only each residue's carbon-alpha (Ca) atom in deriving its geometric features, ignoring important geometric details provided by all-atom view of protein structures.
Our work builds on top of prior works by making the following contributions:
We provide the first example of graph self-attention applied to protein interface contact prediction, showcasing its effective use in learning representations of protein geometries to be exploited in downstream tasks.
We propose the new Geometric Transformer which can be used for tasks 3D protein structures and similar biomolecules. For the problem of interface contact prediction, we train the Geometric Transformer to evolve a geometric representation of protein structures simultaneously with protein sequence and coevolutionary features for the prediction of inter-chain residue-residue contacts. In doing so, we also demonstrate the merit of the recently-released Enhanced Database of Interacting Protein Structures (DIPS-Plus) for in- terface prediction (Morehead et al. (202T))
Our experiments on challenging protein complex targets demonstrate that our proposed method, DeepInteract, achieves state-of-the-art results for interface contact prediction.
DATASETS
The current opinion in the bioinformatics community is that protein sequence features still carry important higher-order information concerning residue-residue interactions (Liu et al. (2020)). In particular, the residue-residue coevolution and residue conservation information obtained through multiple sequence alignments (MSAs) has been shown to contain powerful information concerning intra-chain and even inter-chain residue-residue interactions as they yield a compact representation of residues" coevolutionary relationships (Jumper et al. (20IT).
Keeping this mind, for our training and validation datasets, we chose to use DIPS-Plus (Morehead et al. (2021),, to knowledge the largest feature-rich dataset of protein complexes for protein in- terface contact prediction to date. In total, DIPS-Plus contains 42, binary protein complexes with positive labels (i.e., 1) for each inter-chain residue pair that are found within 6 Å of each other in the complex's 's bound (i.e., structurally-conffrmee] state. The dataset contains variety of rich residue- level features: (1) an 8-state one-hot encoding of the secondary structure in which the residue is found; (2) a scalar solvent accessibility; (3) a scalar residue depth; (4) a X 6 vector detailing each residue's 's protrusion concerning its side chain; (5) a 42 vector describing the composition of amino acids towards and away from each residue's 's side chain; (6) each residue's coordinate number conveying how many residues to which the residue meets significance threshold, (7) a 1 x 27 vec- tor giving residues' emission and transition probabilities derived from HH-suite3 (Steinegger et al.) 2019)) profile hidden Markov models constructed using MSAs; and (8) amide plane normal vectors for downstream calculation of the angle between each intra-chain residue pair's amide planes.
To compare the performance of DeepInteract with that of state-of-the-art methods, we select 32 homodimers and heterodimers from the test partition of DIPS- Plus to assess each method's 's compe- tency in predicting interface contacts. We also evaluate each method on 14 homodimers and 5 het- erodimers with PDB structures publicly available from the 13th and 14th sessions of CASP-CAPRI (Lensink et al. (2019). Lensink et al. (202T)) these targets considered by the bioinformatics
3

---

Published as a conference paper at ICLR 2022
DeepInteract
3
Trensfarce
Interla
Figure 2: DeepInteract overview. Our proposed pipeline separates interface contact prediction into two tasks: (1) learning new node representations da and hg for pairs of residue protein graphs and (2) convolving over h& and hg interleaved together to predict pairwise contact probabilities.
community to be challenging for existing interface predictors. For any CASP-CAPRI test complexes derived from multimers (i.e., protein complexes that can contain more than two chains), to represent the complex we chose the pair of chains with the largest number of interface contacts. Finally, we use the traditional 55 test complexes from the DB5 dataset (Fout et al. (2017); [Townshend et al. 2019); Liu et al. (2020)) to benchmark each heteromer-compatibe method.
To expedite training and validation and to constrain memory usage, beginning with all remaining complexes not chosen for testing, we filtered out all complexes where either chain contains fewer than 20 residues and where the number of possible interface contacts is more than 2562, leaving us with an intermediate total of 26,504 complexes for training and validation. In DIPS-Plus, binary protein complexes are grouped into shared directories according to whether they are derived from the same parent complex. As such, using per-directory strategy, these complexes for training and 20% for validation to restrict overlap between our cross-validation datasets. After choosing these targets for testing, validation partitions of DIPS-Plu PPSS-Plu that contain any chain with over 30% sequence identity to any chain in any complex in our test datasets. This threshold of 30% sequence identity is commonly used in the bioinformatics literature (Jordan et al. (012),, Yang et al. (2013) to prevent large evolutionary overlap between a dataset's cross-var cooss-vlldation partitions. However, most existing works for interface contact prediction do not employ such filtering criteria, so the results reported in these works may be over-optimistic by nature. performing such sequence-based filtering, we are left with 15,618 and 3,548 binary complexes for training and validation, respectively.
randomly designate 80% of
then filter complexes from our training and
METHODS
4.1 PROBLEM FORMULATION
Summarized in Figure we designed DeepInteract, our proposed pipeline for interface contact pre- diction, frame the problem of predicting interface contacts in silico two-part task: The first part is to use attentive graph representation learning to inductively learn new node-level represen- tations Hy & and hg E RxC for a pair of graphs representing two protein chains. The second part is to channel-wise interleave hş and h into an interaction tensor II E Rxxxxx2 where IR and B E are the numbers of amino acid residues in the first and second input protein chains, respectively, and C E R is the number of hidden channels in both hş and hg. We use in- teraction tensors such as Il as input to our interaction module, a convolution-based dense predictor of inter-graph node-node interactions. We denote each protein chain in input complex G with edges E between the k-nearest neighbors of its nodes N, with nodes corresponding to the chain's is amino acid residues represented by their Ca atoms. In this setting, we let A: = 20 served favorable cross entropy loss on our validation dataset with this level connectivity. We note that this level of graph connectivity has also proven be advantageous for prior works developing
graph
ob-
4

---

Published as a conference paper at ICLR 2022
Geometric Transformer
Rom
O
Rom
(0)
Hobi
22
040)
Pullicn
So
eloti))
Pastioad Boonleg
Figure 3: Geometric Transformer overview. Notably, our final layer of the Geometric Transformer removes the edge update path since, in our formulation of interface prediction, only graph pairs' node representations h2 and hi are directly used for the final interface contact prediction.
deep learning approaches for graph-based protein representations FFout et al. (201)) Ingraham et al. 2019
4.2 GEOMETRIC TRANSFORMER ARCHITECTURE
Hypothesizing that self-attention mechanism that evolves proteins' physical geometries is key component missing from existing interface contact predictors, we propose the Geometric Trans- former, graph neural network explicitly designed for capturing and iteratively evolving protein geometric features. As shown in Figure 3 the Geometric Transformer expands upon the existing Graph Transformer architecture (Dwivedi Bresson (2021) ) by introducing (1) an edge initializa- tion module, (2) an edge-wise positional encoding (EPE), and (3) geometry-evolving conformation module employing repeated geometric feature gating (GFG) (see Sections A.6 A.7 and A.8 for ra- tionale). Moreover, the Geometric Transformer includes subtle architectural enhancements to the original Transformer architecture (Vaswani et al. (017)) such as moving the network's 's first nor- malization layer precede any affinity score computations for improved training stability (Hussain et al. (202T)). ) To our knowledge, the Geometric Transformer is the first deep learning model that applies multi-head attention the task of partner-specific protein interface contact prediction. The following sections serve to distinguish our new Geometric Transformer from other Transformer-like architectures by describing new neural network modules for geometric self-attention.
4.2.1 EDGE INITIALIZATION MODULE
To accelerate its training, the Geometric Transformer first embeds each edge E with the initial edge representation
= < P2 0 (m:j || Ae) (f) (I (Fa ll of (fa)) Gij = (p)
(1)
(2)
where Ø refers to the i'th edge information update function such as multi-layer perceptron; denotes channel-wist concatenation; P1 and P2, respectively, are trainable one-hot vectors indexed by P. and P,. the positions of nodes and nodes in the chain's 's underlying amino acid sequence; m, are any user-preet predefinde features for e (in our case the normalized Euclidean distances between nodes and nodes j); A. edge-wise sinusoidal positional encodings sin(P: : Pý) for e; f, f2, fa.
5

---

Published as a conference paper at ICLR 2022
Geometric Transformer
Con.formation Module
Neighlore
Ad
42
48
3x Readloook
3 Headioook
+ (fyb
db
ReeDlook
*
Direction Sy
Orientation (0700
Angla
+lla;
xill)
o
e(Cs,
Figure 4: Conformation module overview. The Geometric Transformer uses conformation module in each layer to evolve proteins graphs geometric representations via repeated gating and final series of residual connection blocks.
and f4, in order, are the four protein-specific geometric features defined in Section A.3 and pe and P are feature addition and channel-wis gating functions, respectively.
4.2.2 CONFORMATION MODULE
The role of the Geometric Transformer's subsequent conformation module, illustrated in Figure is for it to learn how to iteratively evolve geometric representations of protein graphs by applying repeated gating our initial edge geometric features f1, fz, fz, and f,. To do the conformation module updates Gij by introducing the notion geometric neighborhood of edge e, treating e as pseudo-node. Precisely, Eg, the edge geometric neighborhood of e, is defined the 2n edges
1g (emgis Engil (M1, n2 E N₂) and (n1, n2 # i,j)),
where N₂ N are the source nodes for incoming edges on edge e's source and destination nodes. The intuition behind updating each edge according to its 2n nearest neighboring edges is that the geometric relationship between residue pair, described by their mutual edge's features, can be influenced by the physical constraints imposed by proximal residue-residue geometries. As such, use these nearby edges during geometric feature updates. In the conformation module, the iterative processing of all geometric neighborhood features for edge can be represented
O₂
Σ (<) 25 (n.)). KEB
(4)
Cij = 2 x (lssBlocy(o + ResBlock,
(o (e.) Ozy))
(5)
where F' are the indices of the geometric features (f1 fg, fg. f defined in Section A.3 A.3 @ is element- wise multiplication; E neighboring edge ex 's representation after gating with fn and 2 Res ResBlook; represents the i'th application two unique, successive residual blocks, each defined as Res Block())(x)
_ (d (x)) - I. Described in Section A.3 by way of their construction, each of our selected edge geometric features is translation and rotation invariant the network's input space. As discussed in Section A.5] we couple these features with our choice of node-wise positional encodings (see Section 4.2.3) attain canonical invariant local frames for each residue to encode the relative poses of features in our protein graphs. In doing so, we leverage many of the benefits of employing equivariant representations while reducing the large memory requirements they typically induce, to yield robust invariant representation of each input protein.
4.2.3 REMAINING TRANSFORMER INITIALIZATIONS AND OPERATIONS
For the initial node features used within the Geometric Transformer, include each of DIPS-Plus' residue-level features described succinetly in Section Additionally, we append initial min-max normalizations of each residue's index in P; each node node-wise positional encodings. For

---

Published as a conference paper at ICLR 2022
the remainder of the Geometric Transformer's operations, the network's order of operations closely follows the definitions given by Dwivedi & Bresson (2021) for the Graph Transformer, with an exception being that the first normalization layer now precedes any affinity score calculations.
4.3 INTERACTION MODULE
Upon applying multiple layers of the Geometric Transformer to each pair of input protein chains, we then channel-wise interleave the Geometric Transformer's learned node representations ha and by into IT to serve as input to our interaction module, consisting of dilated ResNet module adapted from Chen et al. (02(). The core residual network component in this interaction module consists of four residual blocks differing in the number of internal layers. Each residual block is comprised of several consecutive instance normalization layers and convolutional layers with 64 kernels of size 3 x 3. The number of layers in each block represents the number of 2D convolution layers in the corresponding component. The final values of the last convolutional layer are added to the output of a shorteut block, which is convolutional layer with 64 kernels of size 1 x A squeeze-and- excitation (SE) block (Hu et al. (2018) is added at the end of each residual block to adaptively recalibrate its channel-wis. feature responses. Ultimately, the output of the interaction module is probability A x IB matrix that can be viewed as an inter-chain residue binding heatmap.
5 EXPERIMENTS
5.1 SETUP
For all experiments conducted with DeepInteract, we used 2 layers of the graph neural network chosen for the experiment and 128 intermediate GNN and CNN channels to restrict the time required to train each model. For the Geometric Transformer, we used an edge geometric neighborhood of for each edge such that each edge's geometric features are updated by their 4-nearest incoming edges. In addition, we used the Adam optimizer (Kingma & Ba (0144), a learning rate of le-3, weight decay rate of 1 dropout (i.e., forget) rate of 0.2, and batch size of 1. We also employed 0.5- 0.5-threshold gradient value clipping and stochastic weight averaging (Izmailov et al. (2188). With an early-stopping patience period of epochs, we observed most models converging after approximately 30 training epochs on DIPS- For our loss function, we used weighted cross entropy with a positive class weight of to help the network overcome the large class imbalance present in interface prediction. All DeepInteract models employed 14 layers of our dilated ResNet architecture described in Section 4.3 and had their top-k metrics averaged over three separate runs, each with a different random seed (standard deviation of top-k metrics in parentheses). Prior to our experiments on the DB5 dataset's 55 test complexes, we fine-tume tine-tuned each DeepInteract model using the held-out 140 and 35 complexes remaining in DB5 for training and validation, respectively. Employing a similar training configuration as described above, in this context we used a lower learning rate of 1e-5 facilitate smoother transfer learning between DIPS-Plus and DB5.
size
5.2 HYPERPARAMETER SEARCH
To identify our optimal set of model hyperparameters, we performed manual hyperparameter search over the ranges of [le-
Le Le and [Le 11 learning rate and weight decay rate, respectively. In doing so, we found a learning rate of 1e and weight decay rate of 1e-2 to provide the lowest loss and the highest metric values on our DIPS-Plus validation dataset. We restricted our hyperparameter search to the learning rate and weight decay rate of our models due to the large computational and environmental costs associated with training each model. However, this suggests further improvements our models could be found with more extensive hyperparameter search over, for example, the models' dropout rate.
2, 1e_
1e Le +1 for the 1, 1e
5.3 SELECTION OF BASELINES
We considered the reproducibility and accessibility of a method to be the most important factors for its inclusion in our following benchmarks to encourage the adoption of accessible and transparent benchmarks for future works. As such, we have included the methods BIPSPI (an XGBoost-based algorithm) (Sarchez-Garcia et al. (2018), DeepHomo (a CNN for homodimers) (aa & Huang
7

---

Published as a conference paper at ICLR 2022
Table 1: The average top-k precision on two types of DIPS-Plus test targets.
16 (llamo
(lleteeo)
Method
10
L/10
1./5
10
L/10
1,5
B1
0
0
0.02
0.02
0.02
DH
0.13
0.12
0.09
CC
0.17
0.16
0.15
(GUN)
0.22 (0.U6)
0:20 (0.07)
(0.04)
0.08 (b.01)
(0.01)
U.D7 (0.022
DI (GT)
0.27 (0.06)
0:24 (0.04)
0.21 (0.04)
0.10 (0.04)
0.09 (0.04)
0.08 (0.04)
DI (GeoT wio EPE)
0.28 (0.05)
0.24 (0.01)
0.23 (0.03)
0.11 (0.05)
0.10 (0.04)
0.09 (0.03)
(Géool wio GIRG)
0.27 (0.08)
0.24 (0.08)
(0.08)
0.10 (0.02)
0.09 (0.02)
0.09 (0.01)
(GooT))
0.25 (0.03)
0.25 (0.03)
0.23 (0.02)
0.15 (0.04)
0.14 (0.03)
0.11 (0.04)
Table 2: The average top-k precision and recall on DIPS-Plus test targets of both types.
32 (Both Types)
Method
P@10
L/10
L0L/5
R@L
R@L/2
L@
B1
0.01
0.01
0.01
0.01
0.004
0.003
(GCN)
0.15 (0.08)
0.16
(0.02)
0.10 (0.02)
0.06 (0.01)
(0.003)
(GTT)
0.18
0.16 (0.04)
0.15 (0.04)
0.13 (0.02)
0.07 (0.01)
0.04 (0.01)
DI (GeoT wio BPE)
0.19 (0.04)
0.18 (0.03)
0.16 (0.00)
0.14 (0.02)
0.08 (0.02)
6.04 (0.02)
DI (GooT wis (IRC)
0.18 (0.05)
0.16 (0.04)
0.15 (0.04)
0.14 (0.02)
0.08 (0.02)
0.04 (0.01)
(GeoT)
0.20
(0.01)
(0.02)
(0.008)
(0.004)
(0.002)
2021)) ) and ComplexContact (a CNN for heterodimers) Zeng et al. (2018) since they are either easy to reproduce or simple for the general public to use make predictions. Each method predicts interfacing residue pairs subject to the (on average) 1:1000 positive-negatin
class imbalance im- posed by the biological sparsity of true interface contacts. We note that we also considered adding more recent baseline methods such as those of Townshend et al. (2019) and Liu et al. (2020). How- ever, for both of these methods, we were not able to locate any provided source code or web server predictors facilitating the prediction of inter inter-protein residue-residus contacts for provided FASTA or PDB targets, so they ultimately did not meet our baseline selection criterion of reproducibility (c.g., an ability to make new predictions). We also include two ablation studies (e.g.. DI (Geo' w/o GFG)) to showcase the effect of including network components unique to the Geometric Transformer.
Our selection criterion for each baseline method consequently determined the number of complexes against which we could feasibly test each method, thereby restricting the size of our test datasets to 106 complexes in total. In addition, not all baselines chosen were originally trained for both types of protein complexes (i.e., homodimers and heterodimers), so for these baselines we do not include their results for the type of complex for which they are not respectively designed.
For brevity, in all experiments, we refer to BIPSPI, DeepHomo, ComplexContact, and DeepInteract as BI, DH, CC, and DI, respectively. Further, we refer to the Graph Convolutional Network of Kipf & Welling (2016), the Graph Transformer of Dwivedi & Bresson (202T), and the Geometric Transformer as GCN, GT, and GeoT, respectively. To assess models' ability to correctly select residue pairs in interaction upon binding of two given chains, all methods are scored using the top-k precision and recall metrics (defined in Section A.2) commonly used for intra-chain contact prediction (Chen et al. (2021)) as well recommender systems (Jiang et al.] (2020)), where k {10, L/10, L/5, 2 with being the length of the shortest chain in a given complex.
5.4 DISCUSSION
demonsstesss that DeepInteract outperforms or achieves competitive results compared to ex- isting state-of-tth--rt methods for interface contact prediction on DIPS-Plus with both types of pro- tein complexes, homodimers (homo) where the two chains are of the same protein and heterodimers (hetero) where the two chains are of different proteins. Table 2 shows that, when taking both types of complexes into account, DeepInteract outperforms all other methods" predictions on DIPS-Plus Since future users of DeepInteract may want predict interface contacts for either type of complex, we consider a method's 's type-averaged top-k metrics as important metrics for which to optimize.
Likewise, Tables 3 and 4 present the average top-k metrics of DeepInteract 19 challenging protein complexes (14 homodimers and heterodimers) from the 13th and 14th rounds of the joint CASP- CAPRI meeting. In them, we again see DeepInteract exceed the precision of state-of-l of-th-of-th-art in-
8

---

Published as a conference paper at ICLR 2022
Table 3: The average top-k precision on dimers from CASP-CAPRI 13 & 14.
Method
10
1/10
L.5
10
L/10
1,65
BI
0
0.04
0
0.03
DH
0.02
0.02
0.02
CC
0.06
0.08
0.05
(GUN)
0.12 (0.04)
(0.03)
0.13 (0.02)
0.10 (U.U7)
(0.08)
(b.04)
DI (GT)
0.08 (0.03)
0.09 (0.05)
0.08 (0.03)
0.14 (0.02)
0.14 (0.02)
0.12 (0.03)
DI (GeeT weo EPE)
0.11 (0.01)
0.12 (0.02)
0.11 (0.01)
0.18 (0.07)
0.20 (0.09)
0.18 (0.04)
(Gioo GHG)
(0.02)
(0.02)
(0.02)
0.14 (0.03)
(0.03)
(0.02)
(GroT)
(0.05)
0.13 (0.03)
0.11 (0.02)
(0.08)
(0.07)
0.24 (0.04)
Table 4: The average top-A precision and recall across all targets from CASP-CAPRI 13 & 14.
(Both Types)
Methad
Pir L/10
L//
k#
R#L/D
DI (OCN)
0.12 (0.04) 04)
0.10 (0.05)
0.09 (0.04)
0.11 (0.001)
0.06 (0.01)
0.02 (0.01)
DI (CT)
0.10 (0.03)
0.09 (0.03)
0.08 (0.02)
0.11 (0.02)
0.06 (0.01)
0.02 (0.01)
DI (GeeT EPE)
(0.02)
0.14
(0.02)
0.12 (0:01)
0.07 (0.01)
0.03 (0.01)
(GeuT CFG)
(0.01)
(0.01)
0.03 01)
DI (GeoT)
0.21 (0.01)
0.19 (0.01)
0.14 (0.01)
0.13 (0.02)
0.08 (0.01)
0.04 (0 003)
Table 5: The average top-k precision and recall on DB5 test targets.
55 (Hetero)
Mathed
PD
1,/00
Pot 1./5
R L.
R L,/2
R 1./5
0.002
0.001
0.003
0. 0.001
0.0004
CC
0.002
0.003
0.003
0.007
0. 0.003
0.001
(GCN)
0.005 00.0
0. 0..066/0 (0.001)
0.007 (0. (0.001)
0.013 (0.002)
0,008 0010
0,003 (0.001)
(0. (0.004)
0.008 (0.005)
0.008 (0.004)
0.010 (0.005)
0.006 003;
0.003 0000
wis FPF)
0.011 (0. 00-0044
0.009 (0.004)
0.011 (0.002)
0.018 (0.01)
0.010 004)
0.0034 (0.002)
DI (GeeT wi GFG)
0.008 (0. (0.001)
0. 0.008// (0.001)
0.009 (0. 002)
0.014 (0.01)
0.006 (0. 002)
0.003 (0 001)
DI (GeeT)
0.013 (0.001)
0.009 (0.008)
0.011 (0.001)
0.018 (0.001)
0.010 (0.001)
0. 0.0034 (0. 00
terface contact predictors for both complex types. In particular, we see that combining DeepInteract with the Geometric Transformer offers improvements to the majority of our top-k metrics for both homodimers and heterodimers compared to using either GCN Graph Transformer-basee GNN backbone, notably for heteromeric complexes with largely asymmetric inter-chain geometries. Such result supports our hypothesis that the Geometric Transformer's 's geometric self-attention mecha- nism can enable enhanced prediction performance for downstream tasks on geometrically-intican 3D objects such as protein structures, using interface contact prediction case study.
Finally, in Table 5 we observe that, in predicting the interface contacts between unbound protein chains in the DB5 test dataset, the Geometric Transformer enables enhanced top-k precision and recall (definition in A.2) compared to all other baseline methods, including GCNs and Graph Trans- formers paired with DeepInteract. Such as result confirms, to a degree, the Geometric Transformer's ability to predict how the structural conformations occurring upon the binding of two protein chains influence which inter-chain residue pairs will interact with one another in the complex bound state.
6 CONCLUSION
We presented DeepInteract which introduces the geometry-evolving Geometric Transformer for pro- tein structures and demonstrates its effectiveness in predicting residuc-residue interactions in protein complexes. We foresee several other uses of the Geometric Transformer in protein deep learning such as quaternary structure quality assessment and residue disorder prediction, to name a few. One limitation of the Geometric Transformer's 's current design is the high computational complexity sociated with its dot product self- attention mechanism, which we hope to overcome using efficient alternatives to self-attention like that of the Nyströmformer (Xiong al. (2021)).

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
The project is partially supported by two NSF grants (DBI 1759934 and IIS IIS 1763246), one NIH grant (GM093123), three DOE grants (DE-SC0020400, DE-AR0001213, and DE-SC0021303), and the computing allocation on the Summit compute cluster provided by Oak Ridge Leadership Computing Facility (Contract No. DE-ACO5-000R227255)
ETHICS STATEMENT
DeepInteract is designed to be used for machine learning of protein molecular data. It makes use of only publicly available information concerning biomolecular structures and their interactions. Con- sequently, all data used to create DeepInteract models do not contain any personally identifiable information or offensive content. As such, we do not foresee negative societal impacts as a conse- quence of DeepInteract the Geometric Transformer heing made publicly available. Furthermore, future adaptions or enhancements to DeepInteract may benefit the machine learning community and, more broadly, the scientific community by providing meaningful refinements transparent and extensible pipeline for geometric deep learning of protein-protein interactions.
REPRODUCIBILITY STATEMENT
To enable this work to reproducible by others, we have thoroughly documented two methods for running predictions with DeepInteract using provided PyTorch Lightning model checkpoint (Fal- con (2199) and one method for training DeepInteract models in an associated GitHub repository. DeepInteract Docker container. The second method, for both training and prediction, details how users can piece- wise install DeepInteract and its data and software dependencies on Linux-based operating system.
The convenient prediction method describes how
platform-agnostic
REFERENCES
Shandar Ahmad and Kenji Mizuguchi. Partner-aware prediction of interacting residues in protein- protein complexes from sequence data. PloS one, 6(12):c29 104, 2011.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021.
Chen Chen, Tianqi Wu, Zhiye Guo, and Jianlin Cheng. Combination of deep neural network with at- tention mechanism enhances the explainability of protein contact prediction. Proteins: Structure, Function, and Bioinformatics, 8966)69777707 2021. doi: http::///dbi..ogg//..00//ppppppp URL htLlhttps://nnt ine wwweellibrry.... www.arr.wwwly.............. com doi /aeei//bs//ie 10 wt..a///......////.......
26052
Jiang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp. 801-818, 2018.
Taco S. Cohen and Max Welling. Steerable cnns. CoRR, abs/1612.08448,, 2016. URL http:
7/arxiv. org/ aba/ 612.66.. 08498
Allan Costa, Pranam Chatterjee, Manvitha Ponnapati, Suhas Bhat, Kalyan Palepu, Joseph Jacobson, and Iddo Drori. End-to-end euclidean equivariant transformers for protein docking. NeurlPS Workshop on Learning Meaningful Representations of Life, 2021.
Bowen Dai and Chris Kellogg. Protein Interaction Interface Region Prediction by Geometric Deep Learning, Bioinformatiis, 03 2021. ISSN 1367-4803, doi: 10. 1001000biiiioommatss/baabbbblll URL httths:////ioooooee org http:///00000/iiooooooaaaaaaaa/a
btab154.
Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks graphs, 2021.
et al. Falcon, WA. Pytorch lightning. GitHub. Note: http://ghhhaacooo////ooollliiinh lightning, 3, 2019.
10

---

Published as a conference paper at ICLR 2022
Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. Protein interface prediction using graph convolutional networks. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30, pp. 6530-6539. Curran Associates, Inc., 2017. URL
neurips.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. tr(3)-transsormmrr:: 3d roto- translation equivariant attention networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1970-1981. Curran Associates, Inc., 2020. URL
ngg .nwuriss...... cc/ 789d1 3b 3b7222c5c9553334444 -aaperpppr... pdf
Pablo Gainza, Freyr Sverrisson, Frederico Monti, Emanuele Rodola, D Boscaini, MM Bronstein, and BE Correia. Deciphering interaction fingerprints from protein molecular surfaces using geo- metric deep learning. Nature Methods, 17(2):144-13) 2020.
Albert Gu, Caglar Gulcehre, Thomas Paine, Matt Hoffman, and Razvan Pascanu. Improving the gat- ing mechanism of recurrent neural networks, In International Conference on Machine Learning, pp. 3800-3809. PMLR, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excinee conference on computer vision and pattern recognition, pp. 7132-7141, 2018.
networks. In Proceedings of the IEEE
Md Shamim Hussain, Mohammed J Zaki, and Dharmashankar Subramanian. Edge-augmented graph transformers: Global self-attention is enough for graphs. arXiv preprint arXiv:2108.03345 2021.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019. URL http:///ppppocoeegggnnonnnssssss
pdf
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv: 054000 2018.
Jyun-Y, Yu Jiang, Patrick H Chen, Cho-Jui Hsieh, and Wei Wang. Clustering and constructing user coresets to accelerate large-scale top-k recommender systems. In Proceedings of The Web Con- ference 2020, pp. 2177-2187, 2020.
Rafacl A Jordan, EL-Manzalawy Yasser, Drena Dobbs, and Vasant Honavar. Predicting protein- protein interface residues using local surface structural similarity. BMC bioinformatics, 13(1): 1-14, 2012.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, pp. 1-11, 2021.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988, 2014.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. arXiv preprint arXiv: (099002977 2016.
Mare Lensink, Guillaume Brysbaert, Nurul Nadzirin, Sameer Velankar, Raphaël AG Chaleil, Tereza Gerguri, Paul A Bates, Elodie Laine, Alessandra Carbone, Sergei Grudinin, et al. Blind prediction of hnd hetero-protein complexes: The casp13-capri exprrriimeeepeeeeeeee Proteins: Struc- fure, Function, and Bioinformatics, 87(12): 1200 2019.
11

---

Published as a conference paper at ICLR 2022
Mare F Lensink, Guillaume Brysbaert, Théo Mauri, Nurul Nadzirin, Sameer Velankar, Raphael AG Chaleil, Tereza Clarence, Paul A Bates, Ren Kong, Bin Liu, et al. Prediction of protein as- semblies, the next frontier: The casp14-capri experiment. Proteins: Structure, Function, and Bioinformatics, 2021.
Xianggen Liu, Yunan Luo, Pengyong Li, Sen Song, and Jian Peng. Deep geometric representa- tions for modeling effects of mutations on protein-protein binding affinity. PLoS computational biology, 17(8)e 1009284, 2021a.
Yi Liu, Hao Yuan, Lei Cai, and Shuiwang Ji. Deep learning of high-order interactions for protein interface prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 679-687, 2020.
Yi Liu, Limei Wang, Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d graph networks. arXiv preprint arXiv:2102.05013, 2021b.
Alex Morehead, Chen Chen, Ada Sedova, and Jianlin Cheng. Dips-plus: The enhanced database of interacting protein structures for interface prediction, 2021.
Yoichi Murakami, Lokesh P Tripathi, Philip Prathipati, and Kenji Mizuguchi. Network analysis and in silico prediction of protein-protein interactions with applications in drug discovery. Current opinion in structural biology, 44:134-142 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. F d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL http: http::::/ppppp.aeee www.ppppppp.....ss.pppp.p.....pp wwt.:urrr.....pppppp.pppepepe 9015 Sal5--yytrcc--ao an an- an-immmaatiiivee styieveeesseleeeeeee niihh-perforannce wew.:/ommmmmmceeeeeee-eeaeeaeaaa Library. pdf
Ruben Sanchez-Garein COS S Sorzano. M Carazo, and Joan Segura. BIPSPI: method for the prediction of speccie protein-protein interfaces. Bioinformatiis, 35(3):470-477,077 2018. ISSN 1367-4803. doi: 10. 100.0/ooinffrmmiic//h/6477 (ttppttts:///ci. http://006...00100100000003
Stoinformaticssbttyy..
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural net- works. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Con- ference on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pp. 9323-9332. PMLR, 18-24 Jul 2021. URL https: http:///wwcceedeoo...eeeeeeeeeee satorras2la.
mlr.
Kristof T Schütt, Farhad Arbabzadah, Stefan Chmicla, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemice insights from deep tensor neural networks. Nature communications, 8(1):1-8, 2017.
David Sehnal, Sebastian Bittrich, Mandar Deshpande, Radka Svobodová, Karel Berka, Václav Bazgier, Sameer Velankar, Stephen Burley, Jaroslav Koča, and Alexander S Rose. Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures. Nucleic Acids Research, 40(W1):431-W437)
2021. ISSN 0305-1048. doi: 10. 109
gkab314. BtLhttts::/ /oi. org.i.orrg// 10. .100..1933300 1093/nar http///arggkkkaa//// gkab314
Martin Steinegger, Markus Meier, Milot Mirdita, Harald Vöhringer, Stephan J Haunsberger, and Johannes Söding. Hh-suite3 for fast remote homology detection and deep protein annotation. BMC bioinformatics, 20(1):1-15, 2019.
Raphael Townshend, Rishi Bedi, Patricia Suriana, and Ron Dror. End-to-end learning on 3d protein structure for interface prediction. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buus Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, pp. 15642-15651 Curran Associates, Inc., 2019. URUAttpss:/prroceeiiioo newriin...nnnnnn.....o.o.... httppaapr/2219/555 e dee f2 3
12

---

Published as a conference paper at ICLR 2022
Guido Van Rossum and Fred L. Drake. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA, 2009. ISBN 1441412697.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need, 2017.
Thom Vreven, lain H Moal, Anna Vangone, Brian G Pierce, Panagiotis L Kastritis, Mieczyslaw Torchala, Raphael Chaleil, Brian Jiménez-García, Paul A Bates, Juan Fernandez-Reci, et al. Updates the integrated and affinity benchmark version 2. Journal of molecular biology, +27(19):3031-304) 2015.
interaction benchmarks: docking benchmark version
James A Wells and Christopher I. McClendon. Reaching for high-hanging fruit in drug discovery protein-protein interfaces. Nature, 45007172):1001-1009, 2009
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and
Vikas Singh. Nystr)," omformer: A nystr)." om-based algorithm for approximating self-attention. arXiv preprint arXiv:2102.0390)) 2021.
Yumeng Yan and Sheng- You Huang. Accurate prediction of inter-protein residue-residee contacts for homo-oligomeric protein complexes. Briefings Bioinformatics, 2021.
Jianyi Yang, Ambrish Roy, and Yang Zhang. mooeiin-ligamn binding site recognition using comple-
mentary binding- specific substructure comparison and sequence profile alignment. Bioinformat- ics, 29(20) 2595, 2013.
Hong Zeng, Sheng Wang, Tianming Zhou, Feifeng Zhao, Xiufeng Li, Qing Wu, and Jinbo Xu. Complexcontact: web server for inter-protein contact prediction using deep learning. Nucleic acids research, W.WWW:W332-W477,
13

---

Published as a conference paper at ICLR 2022
Figure 5: The network's softmax contact probabilities (leftmost column), 0.5 positive probability- thresholded predictions (middle column), and ground-truth labels (rightmost column), respectively, for PDB ID: 4HEQ (first row) and 6TRI (second row), two of the complexes in our test datasets.
A APPENDIX
A.1 SAMPLE INTERFACE CONTACT PREDICTIONS
In the first row of Figure 5 we see predictions made by DeepInteract for homodimer complex from our test partition of DIPS-Plus (i.e., PDB ID: 4HEQ). The leftmost image represents the softmax contact probability map. The center image corresponds to the same contact map after having 0.5 probability threshold applied to such that residue pairs with at least a 50% probability of being in interaction with each other have their interaction probabilities rounded up to 1.0. The rightmost
image the ground-truth contact map. Similarly, in the second row of Figure [ we are shown the cropped predictions made by DeepInteract for CASP-CAPR test heterodimer (i.e., PDB ID: 6TRI).
A.2 TOP-K TEST PRECISION AND RECALL OF BOTH COMPLEX TYPES IN DIPS-PLUS AND CASP-CAPRI
Formally, definitions of a model's top-k precision prec and recall rec where T paez represents the number of true positive residue pairs selected from model's 's top-k most probable pairs and Toos corresponds to the total number of true positive pairs in the complex, are
Tposa k
preck
(6)
and
Tposk
reck
T Tpos
After defining top-k recall as such, in Tables and we provide the results of each model's top-k recall in the same set of experiments given in Section
A.3 DEFINITION OF EDGE GEOMETRIC FEATURES
Similar to Ingraham et al. (2019) we construct local reference frame (i.e., orientation O,) for each protein chain graph's 's residues. Representing each residue by its Cartesian coordinates æ,, we formally define
14

---

Published as a conference paper at ICLR 2022
Table 6: The average top-k recall on two types of DIPS-Plus test targets.
(Homo)
(Heter))
Method
R@
R@
R@L/
R@L
R@L/2
RØ
DH
0.07
0.04
0.02
CC
0.17
0.12
0.07
DI (GCN)
0.14 (0.03)
0.08 (0.01)
0.04 (0.01)
0.08 (0.02)
0.05 (0.02)
0.02 (0.01)
DI (GT)
0.17 (0.01)
0.10 (0.01)
0.05 (0.01)
0.09 (0.02)
0.05 (0.02)
0.03 (0.01)
(GeoT
(U.02)
(0.01)
(0.01)
(9.03)
(0.02)
(0.02)
DI (GeoT w/ GFG)
0.19 (0.04)
0.11 (0.03)
0.05 (0.02)
0.09 (0.01)
0.05 (0.02)
0.03 (0.01)
(GeoT)
(0.004)
0.12 (0.004)
0.06 (0.003)
0.12 (0.003)
0.07 (0.01)
(0.01)
Table 7: The average top-k recall on dimers from CASP-CAPRI 13 & 14.
14 (Homo)
(Hetero)
Method
R@L. I.
R@ L/2
R@
R@L
R@ L/2
R@/6
BI
0.02
0.01
0
0.01
0
0.02
CC
0.03
0.01
0.01
DI (GCN)
0.10 (0.01)
0.07 (U.01)
0.04 (U.02)
0.08 (0.04)
0.04 (0.02)
0.02 (0.01)
DI (GT)
0.10 (0.01)
0.06 (0.01)
0.02 (0.01)
0.10 (0.01)
0.05 (0.01)
0.02 (0.01)
DI (Geo' w/o BPE)
0.11 (0.01)
0.07 (0.01)
0.04 (0.01)
0.12 (0.02)
0.07 (0.01)
0.03 (0.01)
(GeuT w/o GFG)
(0.01)
(0.01)
(0.02)
(0.01)
(0.01)
DI (GeoT)
0.12 (0.03)
0.07 (0.01)
0.04 (0.01)
0.15 (0.02)
0.09 (0.01)
0.04 (0.01)
2;
3;
U U++1
b, U ui+1
U
n2:
(8)
3 x;-1
|ee x us+il
|e: ui+1l
with n. being the unit vector normal to the plane formed by the rays (00-11 - ox and (0::41 - x);)
and b; being the negative bisector of this plane. We then define O,
O
lb: n. b, n.).
Having defined the orientation O for each residue that describes the local reference frame (æ;, O,)- To provide the Geometric Transformer with an alternative notion of residue-residue orientations, we define the unit vector normal to the amide plane for residue i as
U, = (a - #C31) x (#CB .N:))
(10)
where CCa. &C and NNi; are the Cartesian coordinates of the residue's carbon-alpha (Ca), carbon-beta (C3), and nitrogen (N) atoms, respectively.
Finally, we relate the reference frames for residues i and by describing their edge geometric fea- tures
r r(112, - x.))), O.T I; x;
q0O Oz), aU,, U,)
(11)
x;
with the first term r() being distance encoding of 16 Gaussian RBFs spaced isotropically from 0 20 Å, the second term describing the relative direction of æ, with respect reference frame (æ,, O,), the third term detailing orientation encoding q() of the quaternion representation of the rotation matrix OFO,, O, representing each quaternion with respect to its vector of real coefficients, and the fourth term a() representing the angle between the amide plane normal vectors U, and U,-
Our definition of these edge geometric features makes use of the backbone atoms for each residue. As such, the graph representation of protein chains we use with the Geometric Transformer encodes not only residue-level geometric features but also those derived from an atomic view of protein structures. We hypothesized this hybrid approach modeling protein structure geometries would have a noticeable downstream effect on interface contact prediction precision via the node and edge representations learned by the Geometric Transformer. This hypothesis is confirmed in Section 5.4
15

---

Published as a conference paper at ICLR 2022
Table 8: The protein complexes selected from DIPS-Plus for testing interface contact predictors.
PDB ID
Chain
Chain 2
Type
PDB ID
Chain
Chain
Type
IBHN
B
D
Homo
LAON
R
S
Hetero
IKPT
A
B
Homo
IBE3
Hetero
ISDU
A
B
Homo
1GK8
K
M
Hetero
IUZN
A
B
Homo
1OCZ
V
Hetero
2B4H
A
B
Homo
IUWA
Hetero
2G30
C
E
Homo
3A6N
E
Hetero
DIDD
2GLM
AANAAA
E
F
Homo
3ABM
K
Hetero
2IUO
D
J
Homo
3JRM
Hetero
3BXS
B
Homo
3MG6
Hetero
3CT7
E
Homo
3MNN
F
Hetero
DEB
3NUT
D
Homo
3TIY
Hetero
3RE3
C
Homo
3TUY
GGU
E
Hetero
4HEQ
B
Homo
3VYG
Hetero
4LIW
BBB
Homo
4A3D
C
L
Hetero
4OTA
DB
Homo
4CW7
G
Hetero
4TO9
D
Homo
4DR5
G
Hetero
Table 9: The CASP-CAPRI 13-14 protein complexes selected for testing interface contact predic-
tors.
PDB ID
Chain
Chain
Тype
5W6L
A
B
Homo
6D2V
A
B
Homo
6Е4B
A
B
Homo
6FXA
C
D
Homo
6HRH
A
B
Homo
6MXV
B
Homo
6N64
B
Homo
6N91
B
Homo
6NQ1
B
Homo
6QEK
ANKANA
Homo
6UBL
B
Homo
6UKS
B
Homo
BB
6YA2
Homo
7CWP
D
Homo
C
6CP8
Hetero
6D7Y
B
Hetero
B
6TRI
BB
Hetero
6XOD
Hetero
A
7M5F
Hetero
A.4 PROTEIN COMPLEXES SELECTED FOR TESTING
To facilitate reproducibility of the results presented in Section 5.4 Table 8 displays the PDB and chain IDs of DIPS-Plur Plus protein complexes chosen for testing. Likewise, in Table we provide the PDB and chain IDs of CASP-CAPRI 13-14 targets chosen for testing. These two tables describe precisely which targets were selected and ultimately used in our RCSB-derived benchmarks. For full data provenance, the targets selected from the Docking Benchmark 5 dataset (Vreven et al. (2015)) for benchmarking are the same protein heterodimers used for testing in works such as that of Fout al. (2017), Townshend et al. (2019), and Liu et al. 20200
16

---

Published as a conference paper at ICLR 2022
A.5 INVARIANCE OR EQUIVARIANCE?
In our view, natural question to ask concerning a deep learning architecture designed for a specific task whether equivariance to translations and rotations in IR should be preferred over invariance to transformations in such geometric space. The benefits of employing equivariant representations cemmeery--reeserveee
in a deep learning architecture primarily include such the coordinates representing an object in R and the derivation of invariant relative feature poses for type-O features such as scalars (Cohen & Welling (2016)). However, equivariant represen- tations, particularly those derived with self-attention mechanism, typically induce large memory requirements for training and inference. In contrast, in the context of data domains such as ordered sets proteins where there exists canonical ordering of points, invariant representations may be adopted to simultaneously reduce memory requirements and provide many of the benefits of using equivariant representations such as attaining these relative poses of type-0 features (Ingraham et al. (219)) Jumper et al. (202T)) ). As such, in the context of the Geometric Transformer, we decided to pursue invariance over equivariance, to reduce the network's 's effective memory requirements and to improve its learning efficiency and generalization capabilities (Bronstein et al. (202I). How- ever, for applications such as protein-proteii docking that may more directly rely on type-1 tensors for network predictions (Costa al. 2021 ), designing one's 's network architecture to preserve full translation and rotation equivariance in R is, in our perspective, a worthwhile research direction to pursue as many promising results on molecular datasets have already been demonstrated with equivariant neural networks such as architectures such the Equivariant Graph Neural Network (Satorras al. (202T)
updates to type-I tensors
(Fuchs et al. (2020) and lightweight graph
A.6 RATIONALE BEHIND THE NODE INITIALIZATION SCHEME
DIPS-Plus residue-level features are initially embedded in our protein chain graphs to accelerate the network's training. However, we also initially append node-wis wise min-max positional encodings in our network's 's operations. We do this to initialize the Geometric Transformer with information con- cerning the residue ordering of the chain's 's underlying sequence as such ordering is important to un- derstanding downstream protein structural, interactional, and functional properties of each residue.
A.7 RATIONALE BEHIND THE EDGE INITIALIZATION MODULE'S DESIGN
For the edge initializer module's four protein geometric features, we sought to include enough geo- metric information for the network to be able uniquely determine the Euclidean positions of each node's neighboring nodes. For this reason, we adopt similar distance, direction, and orientation descriptors as Ingraham et al. (0119. We concatenate the protein backbone-geomeen vided by inter-residue distances, directions, and orientations with the angles between each residue pair's amide plane normal vectors. This done ultimately to apply gating to each edges' messages, distances, directions, orientations, and amide angles separately to encourage the network to learn the importance of specific channels in each of these input features. Gating is a technique that has previously been shown to encourage neural networks to not become over-reliant on any particular input feature (Gu et al. (2200) and, as such, in the Geometric Transformer can be seen as a form of channel-wist dropout for single feature sets. By also employing residual connections from original edge representations to gating-learned edge representations, the network module can operate more stably in the presence of multiple neural network layers (He al. (2016)). Furthermore, in the edge initialization module, we introduce edge-wise sinusoidal position encodings to provide the network with a directional notion of
features pro-
distances in protein chains' underlying sequences.
A.8 RATIONALE BEHIND THE CONFORMATION MODULE'S DESIGN
The conformation module's 's design was inspired by SphereNet (Liu et al. (022bb) and similar graph neural network architectures designed for learning on 3D graphs. What distinguishes our confor- mation module from the works of others is its introduction of the notion of 2n edge geometric neighborhoods when updating edge representations as well as its incorporation of geometric in- sights specific to large biomolecules such as proteins. Namely, by including the residue-residue distances, residue-resien residue local reference frame directions and (quaternion) orientations, and amide plane-amide plane angles, the network is provided with enough information to ascertain the rela-
17

---

Published as a conference paper at ICLR 2022
tive coordinates of each neighboring residue from a given residue's local reference frame (Liu al. (021b)), thereby ensuring the network's 's capability of adequately learning from 3D structures.
A.9
ALTERNATIVE NETWORKS WITHIN THE INTERACTION MODULE
We, like Liu al. 2020), that the task of interface prediction bears striking similarities to dense prediction tasks in computer vision (e.g., semantic segmentation). In this train of thought, we experimented with several semantic segmentation models as replacements for our interaction module's dilated ResNet, one namely being Deepl V3 (Chen et al. (2018)). We observed a strong propensity of such semantic segmentation models to identify interaction regions well but to do with low pixel-wise precision. We hypothesize this is due to the downsampling and up- sampling methods often employed within such architectures that invariably degrade the original input tensor's s representation resolution. We also experimented with several sate-of-the-arr Vision Transformer and MLP-based models for computer vision but ultimately found their algorithmic complexity, memory usage, input shape requirements to be prohibitive for this task, since our test datasets' input protein complexes can vary greatly in size to contain between 20 residues and over 2,000 residues in length. As such, for the design of DeepInteract's interaction module, experi- mented primarily with convolution based architectures that do not employ such sampling techniques or pose limited input size constraints.
A.10 HARDWARF USED
The Oak Ridge Leadership Facility (OLCF) the Oak Ridge National Laboratory (ORNL) is an open science computing facility that supports HPC research. The OLCF houses the Summit compute cluster. Summit, launched in 2018, delivers 8 times the computational performance of Titan's 's 18,689 nodes, using only 4,608 nodes. Like Titan, Summit has a hybrid architecture, and each node contains multiple IBM POWER9 CPUs and NVIDIA Volta GPUs all connected with NVIDIA's high-speed NVLink. Each node has over half terabyte of coherent memory (high bandwidth memory DDR4) addressable by all CPUs and GPUs plus 800GB of non-volatile RAM that be used as burst buffer or as extended memory. To provide a high rate of I/O throughput, the nodes are connected non- blo-bloc fat-tree using a dual-rail Mellanox EDR InfiniBand interconnect. We used the Summit compute cluster to train all our models.
A.11 SOFTWARE USED
In addition, we used Python 3.8 (Van Rossum & Drake (209)), PyTorch 1.7.1 (Paskke et al. (219)), and PyTorch Lightning 1,4.8 (Falcon) (2019)
deep learning experiments. PyTorch Lightning was used to facilitate model checkpointing, metrics reporting, and dis- tributed data parallelism across 72 Tesla V100 GPUs.
in-depth description of the software environment used to train and predict with DeepInteract models can be found at
18


Reviews:
[{"review_id": "uEqvA70WXRB", "reviewer": "Reviewer_HqpY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The strengths of this work stem from the architecture used, consisting of attention mechanisms and invariant representations.\n2. This work is uses a novel geometric neighborhood representation which considers edges as pseudo-nodes, with geometric feature updates.\n3. The exposition and presentation are clear, and the work makes available the data, models, and code. \n\nWeaknesses:\n1. The performance metrics and benchmarks are lacking:\n- Specifically, the evaluation metric is precision; whereas recall is missing.\n- The results are not compared with standard benchmarks, and require porting different results for comparison.\n2. The network trained is relatively small and may not scale well.\n3. The work does not predict distograms or a sequence of pose updates.\n\nMissing is a reference to recent related work by Costa et al. from MIT:\nEnd-to-end Euclidean equivariant Transformers for protein docking, NeurIPS Workshop on Learning Meaningful Representations of Life, 2021.\n\nMinor typo: Section 5.2 should read \"To identify..", "summary_of_the_paper": "This paper proposes a state of the art deep learning architecture for predicting mutual contacts between proteins.\nSpecifically, this work presents a Geometric Transformer for rotation and translation invariant protein interface contact prediction,\ngiven 3D protein structures.\n\n", "summary_of_the_review": "In summary, the proposed deep learning architecture for protein interface contact prediction is somewhat new,\nthough aspects of this contribution exist in related work. The performance metrics used and baselines may be improved.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "fB2zwdqfp5", "reviewer": "Reviewer_uZ6F", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength : \n- The approach is original and performs better than a naive GNN implementation.\n- Using a more comprehensive data set to train the methods should lead to increased results, it would be an interesting avenue in general to retrain former data-driven methods over this larger data set\n- The tackled problem is interesting and the results presented here are claimed to be sota.\n\nWeaknesses : \n- Even though the approach is novel, there is not enough evidence that it is needed. Several choices seem to be counter-intuitive and make the approach hard to follow, even with explicit equations. For instance, the attention mechanism is split into successive attention computations. The authors build several geometrical invariant descriptors, why cannot they be simply concatenated into one vector and compute a single attention over this pairwise geometric vector ? In this formulation, I think the approach would more or less reduce to graph transformers.\n- The claim that this method could be used on  \"other 3D graphs\" is too strong because to my understanding, the invariant encoding of the geometric information heavily depends on the ordering of the points. A canonical ordering is not available in general.\n- A general equivariant graph based already exists.\"SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks\" Fuchs et al. Another more recent and simpler equivariant method can be found in \"E(n) Equivariant Graph Neural Networks\" (Sattoras et al.). These methods seem to be applicable in your setting here.\n\n- The data used looks very relevant, but since the learning leverages geometric features, I would suggest conducting the splitting of your dataset based on geometric similarity as was done for instance in Maasif (you could use CATH classes or TM-score splitting).\n- The choice of the methods against which the tools is benchmarked should be expanded and these methods properly introduced. I think it should include PINET (\"Protein interaction interface region prediction by geometric deep learning\" Dai et al.) that is to my knowledge the state of the art for this task. Despite what was stated, to my understanding PINET does target-specific prediction. The fact that the performance of the concurrent methods might be corrupted because of improper splitting should be addressed by authors, for instance by removing from the test sets interfaces too close to the training set (see structural splits above).", "summary_of_the_paper": "This paper addresses contact prediction for PPI by proposing a geometric deep learning framework. They use a k nearest neighbor representation for each protein and compute geometric-based attention scores to convolute the messages over this graph. They validate this approach on the PPI task, using the DIPS-plus dataset. ", "summary_of_the_review": "This paper pursues two complementary aims. A first one is building a predictor for contact residues using the structures of both proteins interacting. To do so, the authors propose leveraging a larger data set along with a novel architecture. However the choice of the methods included in the benchmark is not motivated enough and should include more recent state of the art.\n\nThe second one introduces a novel invariant geometry-based attention for use in graph transformers. It also includes using edges as pseudo-nodes in this graph. These ideas seems very interesting and could be validated onto other classical tasks involving protein structure such as binding site prediction. However the chosen formulation sounds a bit complicated and could be motivated by comparing it to a simpler version. Such an analysis should be complemented by a more extensive comparison to existing equivariant methods. I think such a contribution could be a standalone contribution, but the current version of the paper is not motivating this approach enough.\n\nI would advise to reject the paper in the current form, and encourage authors to pursue the validation of their paper along these two avenues.\n\n\n########################\n\nAfter an interesting discussion with the authors and inclusion of further results, I changed my evaluations twice and now recommend this paper for acceptance. I would advise the authors to work more on the writing of the paper using the supplementary for the equations and emphasizing more the novelties of their methods compared to previous work (backed up by ablation studies disproving the usefulness of these technical choices). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cP3zDFvrsop", "reviewer": "Reviewer_Qnz5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nWeakness:\n\n1. The key contribution of this paper is not clear. The author should clarify the difference between the proposed method and the graph transformer (Dwivedi & Bresson (2021). Several related works should also be discussed, such as [1] and [2].\n\n2. The method section as well as the figures is hard to read. The author should use equations to describe the computations of the DL model.\n\n3. More than two important baselines (such as [2] and [3]) are not included. The author should also conduct the experiments on the DB5 dataset, which is widely used for protein-protein contact prediction.\n\n \n[1] Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature communications, 8:13890, 2017.\n\n[2] Liu, X., Luo, Y., Li, P., Song, S., & Peng, J. (2021). Deep geometric representations for modeling effects of mutations on protein-protein binding affinity. PLoS computational biology, 17(8), e1009284.\n\n[3] Yi Liu et al. “Deep learning of high-order interactions for protein interface prediction”. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, pp. 679–687.\n\nOverall, I am inclined to reject. The proposed architecture seems not novel, and I have some serious concerns regarding the experiments.\n", "summary_of_the_paper": "\nThis paper aims to improve the prediction performance of the protein-protein interaction, which involves predicting partner-specific protein interface contacts (i.e., inter-protein residue-residue contacts) given the 3D tertiary structures of two proteins. In this paper, the authors present a graph transformer to encode the protein geometric features and perform interface contact prediction. The proposed method is validated on two benchmarking datasets, i.e., DIPS-Plus and CASP-CAPRI datasets.\n", "summary_of_the_review": "Considering the noticeable improvement made by the authors, I reconsider my original view and agree with the other reviewers on the significance of the proposed approach.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "c2QqA9wHwMg", "reviewer": "Reviewer_rPiF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "##########################################################################\n\nPros:\n\n- The paper provided the first example of graph self-attention applied to protein interface contact prediction, which is an interesting idea for the protein bioinformatics area.\n- The proposed Geometric Transformer can be used for tasks on 3D protein structures and other 3D graphs. The author trains the Geometric Transformer to evolve a geometric representation of protein structures simultaneously with protein sequence and coevolutionary features for the prediction of interchain residue-residue contacts, demonstrating the merit of the recently released Enhanced Database of Interacting Protein Structures (DIPS-Plus) for interface prediction.\n- The paper is well-written and the design decisions are clearly explained. The comparison of benchmark methods is also interesting to read. \n\n##########################################################################\n\nCons:\n\n- The core idea of this paper is Geometric Transformer, however, it seems that the difference between Geometric Transformer and Graph Transformer is not very big except for the last layer, because the essence of these two models is multi-head attention. Although the author stated in the paper the new modules of Geometric Transformer, including EDGE REPRESENTATION INITIALIZATION, CONFORMATION MODULE, NODE REPRESENTATION INITIALIZATION, and INTERACTION MODULE, it will be more convincing if the author can compare with Graph Transformer.\n- In the model description, the author lacks the statement of the objective function and loss function.\n- In the experiment comparison, if the author can add an ablation study experiment, it will be more convincing. In addition, from the experimental results in Table 1 and Table 2, Geometric Transformer does not seem to be significantly improved compared to Graph Transformer. Can the author provide more explanations?\n- From Figure 1, the author provided the visualization of interacting protein chains, can the author also provide visualization of the predicted results? Such as qualitative analysis.", "summary_of_the_paper": "This paper proposed a novel geometry-evolving graph transformer for protein interface contact prediction, named DeepInteract. In particular, it predicts partner-specific protein interface contacts (i.e., inter-protein residue-residue contacts) given the 3D tertiary structures of two proteins as input, showcasing its effective use in learning representations of protein geometries to be exploited in downstream tasks. The experiments on challenging protein complex targets also demonstrated the proposed method achieves SOTA results for interface contact prediction.", "summary_of_the_review": "Considering the above pros and cons, my recommendation of the paper is marginally above the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a novel neural network architecture to predict interacting residues among two interacting proteins, and evaluates its performance on benchmarks. While the reviews were initially mixed, there has been a productive discussion and significant improvements in the paper during the discussion, including in particular much needed clarifications about the proposed methods, and more experimental results with an ablation study to better assess the benefits of various design choices. While no reviewer is willing to champion this paper as a "strong accept", due to the relatively modest novelty compared to existing methods, there is a consensus towards "weak accept" given the final quality of the work presented and potential usefulness of the method for the problem tackled.
Decision: Accept (Poster)