Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work focuses on representation learning for molecules with stereochemistry and designs an SE(3)-invariant model that processes torsion angles of a 3D molecular conformer. In particular, the authors explicitly model conformational flexibility by integrating a novel
type of invariance to rotations about internal molecular bonds into the architecture, mitigating the need for multi-conformer data augmentation. 
main_review: Overall the model sounds reasonable and the results on multiple tasks are promising. I have several minor questions about empirical studies.
- “ Although SE(3)-invariant 3D GNNs, such as the recently proposed SphereNet (Liu et al., 2021) and GemNet (Klicpera et al., 2021), can
in theory learn chirality, their expressivity in this setting has not been explored.” Give that both SphereNet and GemNet can learn chirality, why only SphereNet is compared in experiments? It is better to compare with both methods.
- “ We emphasize that we view this classification task as necessary but not suf￾ficient to demonstrate that a model can learn a meaning￾ful representation of chiral molecules.” Can you establish a bit more about this? Why not sufficient? 
- SphereNet is comparable to Chiro in Table 1, but much worse in Table 2. Is this mainly because the differences between two tasks?  I’d like to see some analysis here. Why is Chiro much better in the second task? It would be great if you can show several case studies to support Chiro.
summary_of_the_review: Overall it is a nice work, but I’d like to see some deep analysis on experiments, e.g., why the proposed model is better, and in which cases, etc.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a novel SE(3)-invariant GNN model for predicting 3D geometry-dependent physicochemical properties of molecules. In particular, this method focuses on an important issue of how to handle the chirality of molecules in molecular GNNs. When a molecule takes a 3D shape, we have a degree of freedom to rotate every C-C single bond. To distinguish enantiomers (mirror-image molecule pairs), we also need to see whether two molecule pairs are superposable by these single-bond rotation operations. The developed method proposed a torsion-angle encoder having 1) an invariance to rotations about internal molecular bonds and 2) the ability to learn molecular chirality. Several empirical results are also reported.

main_review: Overall I liked the paper's idea. It is well written, easy to follow, and the provided list of related work was relevant. Even though technically it might be incremental, it might be able to provide an interesting insight and informative approach to see. 

However, a big concern is: it is unclear which parts are technically novel, and actually brings any new flexibility because the paper combines several existing ideas. I understand providing a new effective combination of existing ideas is also an important contribution, but for this paper, I'm still not sure on this point of the significance of the provided combinations. The current form just seemed a minor straightforward modification by combining existing GNN design patterns to achieve SE(3) invariance. It is also unclear why other referred models such as SphereNet (Liu+, 2021), GemNet (Klicpera+ 2021), GeoMol (Ganea+ 2021) are insufficient for this targeted problem.

Technically speaking, the components are (1) 2D graph encoders, (2) bond distance and bond angle encoders, and (3) torsion encoders. 

- (1) are a standard design pattern by EdgeConv and GAT layers
- (2) largely relies on the idea by Winter et al (2021)
- (3) largely relies on the idea by GeoMol (Ganea+ 2021) that considers torsional geometric handlings. 

(1) are standard design patterns by EdgeConv + GAT layers, and (2) follows Winter et al (2021) to drop out any information on atom-atom distances and the like. To handle the chirality, the most important aspect would be torsion angles of (3), but comparisons to the original GeoMol (Ganea+ 2021) are presented only in "A.8 Additional Ablations" with a confusing statement of "It is not strictly necessary that ChIRo learn the weight coefficients or phase shifts in order to distinguish enantiomers".

So, the presented model seemed a minor modification of existing ideas to achieve SE(3)-invariance. The novelty is somewhat still unclear. Several natural questions are

- For example, can we include GeoMol (Ganea+ 2021) in Table 2?
- If (2) is from Winter et al (2021), and (3) is mainly from Ganea+ (2021), can we consider a direct combination of these two as a basline in all experiments? OR this work is exactly doing this...? If this won't work, why? 
- Achieving SE(3)-invariance itself is the purpose? I understood SchNet and DimeNet will fail, but cited existing models such as SphereNet (Liu+, 2021), GemNet (Klicpera+ 2021), GeoMol (Ganea+ 2021) are also insufficient for some reasons? How they are different from the presented work?


*Remarks*: For ML practitioners who try to apply GNNs to life science problems or chemistry problems, the chirality of molecules is very important issue but also quite a big technical hurdle. When a molecule takes a 3D shape, we have a degree of freedom to rotate every C-C single bond. However, we have enantiomers, that is, non-superposable isomers by these operations but indistinguishable by 2D chemical formulas. These mirror-image molecules are very important both in life science (many drugs are enantiomers because interacting molecules in living cells also have chiral centers) and chemistry (to produce such drug compounds efficiently we need asymmetric synthesis producing only one of such enantiomers). 

summary_of_the_review: The idea and the focused problem of the paper is very interesting and can be impactful. However, technically speaking, the current form seemed quite incremental, and a minor modification of existing ideas to achieve SE(3)-invariance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper focuses on improving the capacity of GNN models, using chirality identification as the case study. As an important character to represent the geometry in molecules, chirality has a fundamental impact on the molecule properties and downstream applications. It is a major extension of [1]. The distinguishing features are mainly based on the sinusoid transformations of torsion angles, which are invariant to the bond rotations.
main_review: 
**Strength**

1. Good efforts towards incorporating inductive bias from science perspectives into model designs, have a quite lot insights.

**Weakness**

1. Rationale of the problem setting. Base on my experiments (using the models and data from [1]), using "chiral atom tags" with the vanilla GIN model will reach 100% test accuracy. So it would be nice to elaborate why "omit chiral atom tags and bond stereochemistry tags"(page 4) is a rational setting.
2. For the last part of "Torsion Encoder", I understand that it is complete to include necessary independent torsion angles, but how would the model perform if we feed it more, redundant and self-consistent features? It would be helpful to check on this :)


**Comments**

1. Why the scales of the embeddings are quite different in Figure 3 (1e^2 to 1e^8)?
2. I feel the term "contrastive learning" more close to "instance discriminative" instead of triplet loss, it is a bit appropritate forr the term usage. If you have some literatures to back this, please let me know.

**Open Questions**

Not addressing the following comments won't have negative effects on my rating. But it would be nice if the authors can make a few clarifications, which also helps with my understandings!

1. Have you tried SE(3)-equivariant models (Fuchs et al., 2020; Thomas et al., 2018)? It is a bit sceptical that they can actually distinguish this geometry chirality (i.e, considering coupled torsions in the message updates).
2. Can it be incorporated into other models as a simple add-on/module?
3. Do improving model's capacity on chirality identification have positive impacts on the performance on benchmarks such as QM9?

**Reference**:

[1] "Message passing networks for molecules with tetrahedral chirality"
[2] 
summary_of_the_review: This paper is well written and focuses on a simple yet important problem. I tend to accept and am happy to adjust my assessment based on the rebuttal.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: 
This paper proposes a 3D-GNN molecular representation model  (ChIRo) which:

1. can perceive chirality information to improve expressivity;
2. is invariant to bond rotations and mitigates the need for data augmentation.
main_review: [Novelty]

From my point of view, if a molecular representation model wants to achieve good expressivity, it should:

1. perceive more specific molecular properties, such as chirality utilized to separate enantiomers;
2. be invariant to abundant information for better generalization.

Traditional models, such as 2D-GNNs, E(3)-Invariant ones and SE(3)-Invariant ones, can't meet the requirements above , so their expressivity is limited. Contrastively, ChIRo well tackles these two issues by defining and implementing (SE(3)+InterRoto)-Invariance and is proved to be more expressive.

[Theory]

ChIRo uses a series of trigonometric functions to encode the torsion angles. To demonstrate that such encoding is invariant to bond torsion but sensitive to chirality, this paper gives a proof in Appendix. This proof is clear but Figure 2 makes me confused before reading the proof. (This will be elaborated in Cons.[Illustration].) Despite this flaw, the theoretical foundation of this paper is very solid.

[Reproduction]

In Section 3, it gives a very clear description on the structure of ChIRo. And implementation details are presented in Appendix.

[Representation]

This paper is very clear and easy to understand.



Cons:

[Model Design]

As far as we know, most molecular representation models considering 3D structure use the spatial information when passing messages among the atoms and bonds, such as 3DGCN, DimeNet and PhysChem(https://openreview.net/forum?id=Uxi7X1EqywV). However, ChIRo consists of a 2D-GNN and an individual NN to embedding D/Phi/Psi found in molecular conformation. We think the way ConfAE (the method ChIRo follow) processes the spatial information will lose graph topology, because D/Phi/Psi are detached from message passing and treated as "bags of words". This reason of adopting such structure is "to be invariant to the order of presenting the atoms", but actually the output of MPNN is invariant to input atom order if only order-independent aggregation functions are used (such as sum/mean pooling). Therefore, we think the structure design of ChIRo can be further improved.

[Illustration]

In Figure 2, the contents in 3rd and 4th columns are confusing before the readers fully understand the method and its demonstration (e.g. what's the relationship between the solid sinusoids and dotted ones, what does "r" mean and so on). We hope the illustrations can be better arranged.

In Figure 3, the illustrations of "Original OMEGA Conformers" and "Original+Reflected Conformers" are almost the same, we can't figure out what the difference is between them.

[Experiment]

Although ChIRo is evaluated in 4 tasks, its application value remains suspicious.

Firstly, in the first 3 tasks (Contrastive Learning to Distinguish Stereoisomers, Classifying Tetrahedral Chiral Centers as R/S and Predicting Enantiomers’ Signs of Optical Rotation), the targets are strongly correlated to the chirality of enantiomers. So, these tasks seem more like sensitivity tests of "if ChIRo can actually perceive chirality information" rather than "if ChIRo can handle existing tasks". 

Secondly, ChIRo is labeled as a "molecular representation model", so it's necessary to demonstrate its effectiveness on various popular tasks on molecules, such as QM7/8/9 with given conformations (see MoleculeNet(https://pubs.rsc.org/en/content/articlelanding/2018/sc/c7sc02664a)), or other molecular property prediction tasks (FreeSolv, TOX21, ...) assisted with RDKit-generated conformations (see HamNet(https://openreview.net/forum?id=q-cnWaaoUTH)). Besides the 3 tasks mentioned in last paragraph, this paper only evaluate ChIRo on one dataset of molecule-protein binding. Therefore, we think ChIRo has not yet been proved to be a SOTA molecular representation model.

Thirdly, in Ablation Study, we see "adding chiral tags to our unablated model does not improve performance". We think that's caused by overfitting to abundant tags (as chirality information has been well embedded thru torsion encoder), and we hope a validation of it can be given in this paper rather than just a statement of this abnormal phenomenon.
summary_of_the_review: This paper proposes ChIRo to improve the expressivity in molecular representation by well tackling chirality. It has a solid theoretical foundation and a clear and relatively well-designed model structure. Although the experiment is not yet sufficient to demonstrate its application value, we believe that ChIRo's effectiveness can be claimed after being evaluated on diverse tasks on molecular representation, such as molecular property prediction datasets in MoleculeNet.

In summary, it's a good piece of work but needs some improvement.
review_title: null

Meta-review: This work presents ChIRo, a method that incorporates 3D torsion angles of a molecular conformer to specifically handle chirality. Specifically ChIRo uses trigonometric functions to encode the torsion angles, which are invariant to bond torsion but sensitive to chirality, thus capable of distinguishing between enantiomers.  Overall, although not groundbreaking, we found the idea presented in the paper to be sufficiently novel and its ability to handle chirality to be of significant practical values.
Decision: Accept (Poster)