Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a SE(3) equivariant graph matching network for end-to-end rigid protein docking. They propose a novel optimal transport loss to approximate binding pocket and a differentiable Kabsch algorithm to predict the docking pose. They achieve significant running time improvements over existing protein docking software with competitive results, and do not rely on heavy sampling, structure refinement, or templates 

main_review: ## Strength
The proposed method is quite novel. Built upon Graph Matching Networks [1] and E(3)-Equivariant Graph Neural Networks [2], they propose a SE(3) equivariant graph matching network. The keypoint selection with optimal transport and rotation prediction with Kabsch algorithm are also interesting to me. The proposed method achieves a promising speedup of 80-500x.
The proposed method seems general and can be applied to other rigid docking scenarios, i.e., predicting a rotation and translation matrix. The algorithm can potentially be adapted for protein-ligand docking, which is also an important task in computational chemistry. 
The paper is well written and the mathematical formulation of SE(3) equivariant rigid protein docking is very nice. The authors also promise they will release the code and datasets after reviewing the process.

## Weakness
Although the proposed method achieves a significant speedup compared to previous docking software, it does not always outperform the baselines as acknowledged by the authors. It is also unclear how structure refinement, or templates can be combined with the current method to further improve the performance, which are used in many other protein structure prediction algorithms, e.g., alphafold2.
The DIPS dataset does not contain protein structures in their unbounded form. It is unclear how the authors tackle this bias in the dataset.
The authors include Surface Aware Node Features into their model. I agree that surface contact modeling is very important for protein docking. It is therefore interesting to see an ablation study over the features.

Reference
[1] Graph matching networks for learning the similarity of graph structured objects
[2] E(n)-equivariant graph neural networks.

summary_of_the_review: I vote for accepting this manuscript for the reasons listed above. However, several important issues remain to be addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose a method for the “rigid body” docking of protein-protein complexes, i.e., a complex in which conformational changes in protein structures are not allowed. This method, called independent E(3)-equivariant graph matching networks (IEGMNs), finds the optimal rotation and translation to place proteins in a manner that the distances between residues in the binding site is minimized. A core feature of this method is SE(3) invariance, which means the optimal solution is invariant to the rotations and translations of the two proteins. This is achieved in an elaborate manner employing several ideas and technics:

- A k-NN graph is built for each protein using its CA coordinates and a set of additional features are also extracted
- Message-passing neural network (MPNN) is used for graph-matching
- Binding pocket residues or “keypoints” are identified
- A differentiable Kabsch model is used for superimposing is the “keypoints”
- Optimal transport is used for the alignment of residues in the binding pocket 

This method, as reported by the authors, runs very fast and can identify the protein complex efficiently. 

main_review: The authors have put a significant amount of time into designing, developing, and testing their proposed method, and it introduces and adapts several interesting ideas. In particular, the core contribution is their proposed transformation, which guarantees pairwise independent SE(3)-equivariance for the two sets of 3D coordinates. My comments are as follows:

- Since no conformational changes are allowed, the residues buried in the hydrophobic core of the proteins play no role in predicting the binding conformation. Calculating solvent-accessible surface area is one of the most well-studied problems in structural genomics. The authors could use one of such methods, e.g., one based on the Voronoi procedure [PMID: 12376381], to significantly reduce the problem size and render identifying keypoints easier. 
- The authors have a loss term to avoid point cloud intersections, but there are no terms for preventing steric clashes by enforcing Van der Waals radii of proximal atoms. Have the authors checked for steric clashes in the binding pocket?
- Analyzing performance metrics needs more elaboration. For example, the distribution of C/I RMSDs could be plotted, and/or scatter plots could be added. Moreover, statistical tests could be performed to reliably show whether the proposed method outperforms the other two. The same applies to inference times and a box/violin plot might be a better option. 
- Judging by the results in Table 1, one might infer that HDOCK performs better than IEGMN. It is true that IEGMN has significantly lower inference times, but in practice, experimentally determining protein structures is time-intensive (in the order of months) and costly, and run times in the order of minutes or even hours do not matter.
- It is good to see that the authors have used two different RMSD definitions; however, the authors should consider looking at CAPRI criteria (critical assessment of predicted interactions criteria) [PMID: 12784359] and standardize their evaluation method.
- The authors employ several ideas and their contribution to the final performance is not clear. For example, what would happen if the “surface-aware node features” are not used? 

Some minor notes:
- SE(3)-equivariance and SE(3)-invariance are used interchangeably. 
- In p6, what does “normalized keypoints” mean, zero-mean?

summary_of_the_review: The proposed method is interesting, but it combines too many ideas together without quantifying their contribution. Moreover, performance evaluation and comparisons with the existing methods are lacking.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes an end-to-end deep learning architecture to model the rigid body protein docking problem. By incorporating the inductive biases of SE(3)-invariance of the final docking position and commutativity, the proposed method avoids the millions of sampling and achieves a competitive performance with much faster speed. In addition, they discover and align keypoints by the attention-based selection algorithm and use optimal transport to predict the binding pocket location based on those selected points. The main contribution of this paper is the combination of the novel graph matching networks and keypoint selection algorithm to predict the 3D position of the docking model. Results are shown on the task of 1) Protein docking complex prediction 2) Runtime
main_review:  Pros:
The whole paper is constructed and written very well.
The authors nicely show the proof of SE(3)-invariant and commutativity for the rigid body docking problem.
Following the invariance property, the paper introduces the novel graph matching network and avoids sampling step based on it.
Using a soft loss function avoids point cloud intersection.

Cons:
As much as we appreciated the approach presented here the performance just seems unacceptable compared to other methods
The author uses two baselines which were developed 4 years ago. The reason they didn’t use other methods, no local packages, is somewhat problematic too. Still, their performance is much worse than one of the methods and hypothesizing the test data was used to train those models does not resolve this issue.
Their runtime is much faster. However, HDOCK, which is the best performance model among those three, uses a bearable amount of time and achieves much better performance. 
Three models are ran on different hardware, which makes runtime comparison problematic.

Lack of result analysis
It may have been helpful to show the method proposed can provide comparable results on more downstream tasks. Even though its rigid docking prediction is not good enough, maybe the prediction has other properties which can help the other perspectives. 

We also noted the method uses soft constraints to discourage point cloud intersections. However, the authors do not show/discuss if it really prevents this from happening and why not use hard constraints directly (stability during training?).

Other Comments:
It is not clear how the number of keypoints K was chosen in the “Keypoints for Differentiable Protein Superimposition” section.

Also we noticed that the whole section of proofs for propositions lacks citations but they are not claimed as novelty at the beginning. Are they part of the contributions or not? How does these parts relate to previous works?

Update:
===========
The improvements implemented in the revised manuscript and clarifications supplied by the authors elevated many of the above concerns and we therefore updated the evaluation to reflect this.
summary_of_the_review: Overall we found the paper to be clearly written and offer a nice formulation with a matching modeling approach for the important problem of rigid body protein docking. In that sense we definitely liked the work. However, the results are just unimpressive and no other insights/utility are offered beyond improved running time compared to standard techniques available.
review_title: null

Meta-review: This paper introduces a novel SE(3) equivariant graph matching network, along with a keypoint discovery and alignment approach, for the problem of protein-protein docking, with a novel loss based on optimal transport. The overall consensus is that this is an impactful solution to an important problem, whereby competitive results are achieved without the need for templates, refinement, and are achieved with substantially faster run times.
Decision: Accept (Spotlight)