Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose ComENet, a new network for molecular property prediction with 3D information. The authors define a new message passing way (see equation (1)) and a new network backbone Figure 3.
strengths_and_weaknesses: ## Originality

1. The general framework is similar to PaiNN and DimeNet++, which is not novel to me.

2. In Eqn.(1), seems more features are used in mesage passing. The method is new but not surprising. 


## Significance

1. The accuracy in Table 3 and 4 are not as good as baselines. 

2. On OC20, the authors did not compare with the state-of-the-art algorithms.

3. If efficiency is the most important part of this paper, I am not sure whether we can directly apply Eqn(1) to existing backbones like DimeNet++?

## Clarify

1. Section 2: What are the key points to deliver? And how & why they help the models?
questions: Please refer to the ``Strengths And Weaknesses'' section.
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces a message-passing paradigm on 3D molecular graphs with better completeness (bijectivity) and additionally other advantages including operating within a 1-hop neighborhood, lower computational complexity, and faster training and inference. Analysis and experiments show that the proposed method can perform comparably well on multiple datasets with obviously increased efficiency.
strengths_and_weaknesses: Strengths:
- The paper brings up the problem of local completeness != global completeness which is kinda overlooked in prior works.
- The paper proposes a useful message-passing paradigm on molecular graphs with better completeness and efficiency.
- Systematical analysis and extensive experiments on multiple tasks under different evaluation metrics are provided.
- Good experimental results and analysis support the effectiveness of the proposed method.

Weaknesses:
- To my understanding, the main arguments of the paper are "completeness" and "efficiency" of the proposed method, and my feeling is that the latter is better addressed than the former in the experiments. As stated in Section 2.3, completeness is most important in identifying structures like conformers. More specific experiments on this (in addition to the general tasks) or some relevant details (e.g. how many conformers exist in the dataset) for the datasets could make the argument stronger, as a higher number on a generic task can be affected by many factors.
questions: Questions:
- As "unless otherwise specified, the values for baseline methods are taken from the referred papers" (line 320), can I also assume that the (macro) network architectures for different baselines are not necessarily the same? I'd be interested in a comparison of the message-passing paradigm itself to other message-passing paradigms (e.g. the one in SphereNet[34]) under the same (macro) network architectures. If such a comparison is hard to conduct, some other details such as the network size of the baselines and ComENet may also help make the statements more convincing.

Suggestions:
- I may suggest first introducing the message passing scheme in Section 2.5 (e.g. Equation 1) and then showing its completeness (Section 2.2-2.4) -- reading the deductions with an overview of the final formulation and a summary of notations may help with faster comprehension. But just my two cents.
limitations: I probably missed it but I haven't found discussions of limitations and potential negative societal impact in the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes ComENet designed for the representation learning on 3D molecular graphs. The framework involves the message passing schemes to capture local completeness and the rotation angle to help capture the global completeness. The paper shows the computational advantage of the proposed model over the existing one and the experimental results show its effectiveness.
strengths_and_weaknesses: Strengths:
1. Instead of the intuition, the paper defines the global completeness and provides theoretical ground for the model to capture the global completeness.
2. The experiments show the effectiveness of the model.

Weakness:
1. There are a lot of places that need to be further clarified:
    a) Line 33, in O(nk^2), what are n and k? They should be explicitly defined before being used.
    b) Line 82: what is m in m x d?
    c) Line 88: What are the function R() and SE(3)? I would suggest the author to add those in the main text of the appendix to make the paper more readable.
    d) Proposition 1: what is the definition of "strongly connected"? Does it mean the line 199 that two arbitrary nodes are connected by at least one path?
    e) Line 245-246: it would be more clear to say "node feature vector v".
    f) Line 113: What is the definition of L_{ij, j\in N_i}? It seems that its notation is different from previously defined L_i.
2. A tiny typo: line 94: It should be achieve rather that achieves.
3. I'm confused about how to rotation angle. In the example of Figure 1 (b), the angle is calculated as the angle between two planes indicated in the figure. But why is it computed between plane f_{i\j}-i-j and f_{j\i}-i-j rather than other planes that contain the edge i-j (e.g., the plane formed by i, j and the node at the top right corner)? How to choose plane to calculate the angle?
questions: I hope the author could address the concerns raised in the above section.
limitations: The paper does not discuss the limitation and the social negative impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a new neural network architecture to learn representations of 3D graphs, primarily
oriented towards applications to molecules.  The method uses a message passing scheme
operating in 1-hop neighborhoods of each node to propagate information throughout the graph efficiently.
It is said to guarantee global completeness, i.e., distinct conformers of the same molecule yield
distinct representations.
strengths_and_weaknesses: The proposal is sound and appears to have good reference value for practitioners.
The method is tested with large public datasets and compared to prior art to demonstrate its effectiveness.

At places the writing of the paper requires some clarification.

questions: p.2, line 83: need a reference for SchNet at first mention; how is the distance generated for an edge?

p.3, line 89: missing definition/reference for SE(3).

p.3, line 91: what is included in "differences in structures"? what does "structure" refer to?

p.3, line 94: need a reference for DimeNet at first mention.

p.3, line 95: how large is a local neighborhood with guaranteed completeness by SphereNet?

p.6, Figure 3: how is the single-atom unit shown in the figure applied to the entire molecule?
It will be useful to show an example with, say, one of the conformers in Figure 2(b).
Is each atom in a molecule to be passed through it one by one in turn?  Or are they concatenated
to serve as input to another network for a "downstream" prediction task?

p.8, p.9: the list of Properties in Tables 4 and 5 requires spelled out names and also references for their definitions.


limitations: The datasets used in the experiments contain relatively small molecules (# nodes/graph < 100).
Is it possible to directly apply this method to much larger molecules?
Some comments on the challenges and limitations there are needed.
review_title: null

Meta-review: metareview: This paper presents useful MP for molecular graphs with better completeness properties. 
I encourage the authors to consider addressing the reviewers' comments while preparing their revision. 
In particular, discuss relation to previous work such as PaiNN, DimeNet++. Clarify splits. Consider adding "completeness". Address exposition issues (eaXa, SSbD). 
confidence: Certain
Decision: Accept