Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: * The paper proposes to use a type of networks that can be shown to be universal approximators for functions that are permutation equivariant, rotationally and translationally invariant.
* This property is required to model functions that work on molecular structures. In essence, this provides a very strong prior on the functions we want to learn.
* The experiments show that by enforcing this structure learning on molecular graphs can be improved (higher quality/label efficiency) compared to previous work. While, the experimental section is quite compact and heavily compressed nothing stood out as obviously wrong but I might be missing a detail.

Overall I like the high level idea behind the paper and the results seem good. While, at a high level I am able to understand the paper, I am really struggling to understand the details and I do not feel like I would be able to reproduce the results or understand the nuances of the architecture. I am aware that the code is released, but this should not be a replacement for a clearly written paper. For this reason I suggest a score of marginally below the acceptance threshold. In the main review I have highlighted the issues and points were I got stuck. 


main_review: 
## Section 3
The tensor field network definition is a notational mess and I am struggling to properly understand the actual structure/meaning.
* Eq. 1: K and D are not defined K(D) properly, $K,D\inN$ is provided but not what they stand for.
* Eq. 2: what does $l_o$ and $m_o$ stand for?
* Eq. 2: indexing of the clebsch-gordan coefficients is not clear to me. (And a wikipedia search does not help either since it has a different notation). I digged deeper in the references in this paper but unfortunately the TFN paper refers to Griffiths (Introduction to quantum mechanics) which is 147 dollars on amazon at the day of the review and a bit to expensive/cumbersome to order and read for this review. Given that many people at Neurips might be in a similar situation I would recommend the authors to actually explain this properly. 
* Eq. 3. $F_{sphere}=\sum_{l,m}$ what is l and m? Is R^l a part of the learned polynomial function.  Can you better describe how I should think about Y.  Looking at this is seems similar to a fourier basis but for functions defined on the sphere? It is not immediately obvious how to think about the function on the sphere. Eq. 3 depends on a direction $\hat{r}$ and $x_b-x_a$, which is then reduced into a direction and a magnitude component. Providing an intuition for how to think about such a function would be helpful for the general ML researcher to better understand this paper.

## Section 4
* Eq. 5 what is $F_2$? What is $r_o$ and $r_i$.

## Section 5:
* There is a definition of $r_o$ and $r_i$. This would have been helpful in the previous section.
* Figure 1, right hand side is not helpful. The flat arrow going in a circle does not really mean much to me. Why is a and a,b darker?.

## Figure 2
* Residual is not defined. are all the W's the same? What is the atom emb. What is W_arrowup, what is W_arrowdown. What is z. How is the first mca initialized? 



limitations_and_societal_impact: Clarity
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Many GNN models have recently been proposed for the problem of predicting molecular dynamics. These GNNs rely on rotationally invariant predictions since the underlying physics of the problem does not change with rotations. While such models have obtained good performance, there is limited theoretical understanding of these methods.

This paper makes two contributions to this task: first, it addresses the theoretical limitations by proving that GNNs with directed edge embeddings and two-hop message passing are universal approximators; second, it presents a new model called GemNet (an extension of Dimenet++ that uses two hop message passing), that obtains excellent results on COLL and MD17 datasets.
main_review: ## Originality
The theoretical contributions in the paper appear novel and interesting. The GemNet model builds upon the Dimenet and Dimenet++ models, but introduces some novel ideas, particularly the use of dihedral angles.

## Quality
The paper appears technically sound. The theoretical claims made by the authors have been proven rigorously and the new model is shown to outperform previous methods experimentally on MD17 and COLL datasets. The new model is a natural extension to the theoretical claims, which provides a good backing for the theory.

Stabilizing activation variance and efficient interactions that the authors use in the GemNet paper are orthogonal to the other improvements and they can also be used in Dimenet++. It would be good to include ablations that incorporate these improvements into Dimenet++ so we can more clearly see the benefits of including dihedral angles.

## Clarity
The paper was difficult to follow and suffers from mathiness. I would recommend the authors to try and simplify the presentation. 

## Significance
The methods presented in the paper are likely to be useful for many molecular chemistry problems.
limitations_and_societal_impact: The authors include a discussion of the limitations of their method.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a model Geometric Message Passing Neural Network (GemNet) which leverages directed edge embeddings and a new two-hop message passing scheme. The model is more expressive and performs convincingly better than the most related baseline (DimeNet) as well as other baselines across a large range of datasets.
main_review: The model contributions seem theoretically sound (subject to my limited expertise on the topic) and in a way the model feels like a natural extension to DimeNet to include torsion angles. The paper is well written, with sufficient information for reproducibility, and the results compared to previous baselines (and specially compared to DimeNet) are compelling. While I did not go into the details, I appreciate the proof of universality of representations for rotationally invariant models, which is great way to qualify the expressivity of the model. Finally, the problem it tackles has very direct applications in science problems, and is of very high interest in state of the art research in rotationally invariant models.

I think my main criticism is about the comparison between GemNet-Q and GemNet-T. Authors say “This suggests that one-hop message passing is expressive enough for some practical use cases, but two-hop message passing gives an advantage for the more challenging task of fitting multiple molecules at once”. As a potential user, it would be great to have a bit more insight (e.g. some rule of thumb) of when to use one or the other. Perhaps some investigation using synthetic datasets would have helped conveying this. For example considering that GemNet-Q, is technically more expressive than GemNet-T, in the cases where GemNet-Q performs worse, is this due to overfitting, or due to optimization problems?

--------------

Post rebuttal:

Thank you for your responses and additional experiments and clarification on the comparison between GemNet-Q and GemNet-T. I would like to maintain my original rating, which was already very positive.

limitations_and_societal_impact: See main review.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes GemNet, an extension to DimeNet++ for predicting molecular properties. The additions include two-hop message passing, symmetric message-passing and efficient bilinear layers. GemNet achieves state of the art results on the MD17 and COLL benchmarks.
main_review: The paper is well motivated and structured. Some aspects could be explained in more detail, in particular the "symmetric message-passing" scheme in Sec 5. The experiments and ablation studies demonstrate the performance and careful design choices of GemNet.

The sections on the universality of spherical  and directional message-passing seem a bit misleading to me. If I'm not mistaken, the proofs are only valid for general functions $F_\text{sphere}$ / $F_\text{TFN}$. However, in practice one needs to introduce a cutoff. In this case, the "Picasso problem" mentioned in the introduction becomes an issue again, if one only uses rotationally-invariant representations. Thus, the central issue is not just the universality of the architecture in theory, but the tradeoff between rotational symmetries of the representation (angular momentum of spherical harmonics) and locality. A discussion on this and how this might be connected to the two-hop scheme would be interesting.

Another minor issue is the discussion of inference time and model size:  the authors state that those are "on par" with previous models. While this is true for DimeNet, it is not the case for NequIP and PaiNN, which are much faster and smaller, while outperforming DimeNet. The timing of Toluene in Table 9 for NequIP (100ms, bs 4) also seems a bit high compared to the original paper (16ms, bs 1). It would be interesting to see also the accuracy and performance of a scaled-down GemNet with a comparable number of parameters.
limitations_and_societal_impact: Clarify practical implications of universality proof (size of cutoff etc., see above).
review_title: null

Meta-review: This paper was generally well received. Reviewers unanimously agreed that the paper was interesting and impactful, especially given the widespread interest in message passing neural networks to model atomistic systems. It seems clear that NeurIPS is an appropriate venue for this publication. 

The authors provided a number of clarifications and further experiments in response to the reviews that helped to address several concerns (in particular regarding baselines against existing models such as NequIP and ablation studies). The main issues that could not be addressed during the rebuttal period were the clarity of exposition which several reviewers commented on. I would encourage the authors to work on simplifying the paper as much as possible in the time leading up to the camera ready. 
Decision: Accept (Poster)