Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors consider the problem of capturing non-local effects in molecular modelling. To address this, they construct a model based on:

- Spherical Harmonic Coordinates (SPHC): A wavelet transform with different orders of spherical harmonic is used to represent the molecule geometry in a SPHC space. An equivariance relation between rotations in the euclidean input and Wigner-D transformations in SPHC gives the model the right symmetries, and the intention is that non-local interactions in Euclidean space may become local in SPHC.
- An adapted message passing scheme which passes messages in local neighbourhoods both in Euclidean and SPHC space while preserving equivariance.

The authors provide demonstration of their method on an educational example of cumulene twisiting as well as larger benchmarking on QM7-X250 and MD17.
strengths_and_weaknesses: Overall the authors have come up with a principled approach for including long ranged geometric features into a message passing architecture. I think the method is interesting and sufficiently novel for publication, and the results that are presented are favourable but perhaps slightly incomplete (see significance section below).

Originality

The idea to perform some sort of convolution with spherical harmonic wavelets to provide additional geometric input is not new, and nor is the discussion of the equivariance of this operation. However, the interpretation of these SPHC features as living in a coordinate space where interesting neighbourhoods can be learned for message passing that correspond to very distant communication in Euclidean space is an interesting new insight that I had not thought about before.

Quality

The main takeaway for me was that the proposed model is (1) solving the specific case of cumulene highlighted in literature before (2) achieving similar performance on MD17 as a number of other methods, but with a small number of parameters and higher train/test speed. I have comments on (1) in the Questions section below and (2) in the significance section below. However I think overall the quality of results meets the publication bar because the lightweight nature of the model seems a promising direction.

Clarity

The paper is easy to follow.

Significant

I feel like the significance of the paper suffers slightly from a couple of missing baselines:

(1) I’m not an expert on ML forcefields, but I think SpookyNet makes more of a deliberate effort to model non-local behaviour than NequIP, so I was not sure why NequIP was chosen as the baseline in fig 2. A more considered description of how models like SpookyNet, PhysNet, sGDML etc capture non-locality and what sort of non-locality SPHC is particularly adept (or not!) at capturing might be more educational than the somewhat obvious (and already published) statement that a NequIP model cannot capture effects larger than it can see.

(2) I’m not sure I know how to evaluate the differences between the numbers on MD17: Although the statement that the errors are low and the parameter counts are low are true, all the models in table 1 have ~10^6 parameters in order of magnitude and all have broadly similar performance on MD17. Do these differences matter in practice? The timings of So3krates in fig 1d seem the most convincing difference to existing methods, but some of the baseline models are missing from this plot and DimeNet (the closest competitor in timing) is missing from the rest of the paper.

(3) SpookyNet has ~10-20 meV/A on the full QM7-X set, but this is not mentioned so I was not sure why So3krates was compared with sGDML rather than something like SpookyNet. Also I think I missed why So3krates is only tested on a subset of QM7-X.

I also flag a slight concern that this paper mainly addresses the cumulene problem which is not of broad interest to NeurIPS, but I personally found this interesting so I did not factor this into my score.

questions: The proposed SPHC solution is well suited to the specific angle-dependent challenge of cumulene, however there are no other examples of non-local interaction such as dipole-dipole or charge-dipole interaction highlighted in the paper. Is there another case study where SPHC is the correct representation to capture the non-local effects and a 6A cutoff is so clearly deficient? How does So3krates do at modeling water clusters or the Au2-MgO system considered in the SpookyNet paper or other examples originating from Ko et al?
limitations: Locality is often built-in to neural-network based force fields to both (1) limit computational cost and (2) enable generalisation from smaller sub-systems to larger simulations. Do we lose these when we use a non-local model? To what extent can So3krates be scaled to something like the 100 million atom system recently run on 128 GPUs using Allegro [1]. Perhaps such an experiment is hard to run, but some evidence that the speed benefit of So3krates on MD17 translates to a large scale simulation would help the case for non-local model development.

[1] https://arxiv.org/pdf/2204.05249.pdf
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a new ML force field based on higher order interactions and equivariant self-attention. The model represents atomic features using spherical harmonic coordinates that are updated during the forward pass through the network, and spherical neighborhoods are constructed using these representations. This helps the model learn relations between distant atoms. As a result, the proposed method is computationally efficient, requires few parameters and achieves comparable performance to recent methods like Nequip.
strengths_and_weaknesses: Strengths
* SPHC representation helps the model capture global dependencies in a natural way.
* The entire model is equivariant, which is a desirable property to have for its data efficiency.
* The proposed method requires very few parameters to achieve good performance.
* It is also very fast for inference, which is an important property to have for many applications like MD simulations where millions of inference steps are needed.

Weaknesses
* The paper does not present results from multiple, diverse datasets (unlike many other papers in this area). So, it is unclear how generalizable the results are.
* The results in table 1 show that Nequip obtains better results (though at higher training & inference cost). Therefore, this method is only suitable when computational efficiency is more important than accuracy.
questions: 1. How does the performance vary with the number of nodes? Spherical neighborhoods computation and self-attention could be very expensive for large systems like bio-molecules and materials.
2. MD17 is a small, relatively homogenous dataset. Do the authors have a sense for how the model would perform on a large and diverse dataset like OC20 (opencatalystproject.org)?

limitations: Seems adequate.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes to use separate representations of spherical harmonic coordinates 
jointly with representations of atomic features in learning models of molecular force fields.
This allows message passing to local neighborhoods in the spherical harmonic coordinates
which translate to non-local neighborhoods in Euclidean space. As a result the method
is able to model long-range interaction effects that are prohibitively expensive to model with
existing, hop-limited message-passing graph neural networks.  It is also helpful to the
low-data cases since the fewer number of parameters in the network requires less training
data to estimate.
strengths_and_weaknesses: The proposal is sound and convincing, and echoes the long established methods
in signal processing.  Experimental demonstration and comparison to prior art are convincing.

questions: The idea of using harmonic functions like wavelets in deep learning has been
pursued before [for example, see "Wavelet Neural Networks: 
With Applications in Financial Engineering, Chaos, and Classification", 
Antonios K. Alexandridis, Achilleas D. Zapranis, John Wiley & Sons, 2014].
How does the current proposal compare to those uses?

limitations: The authors discuss the limits of the current applications to materials
discovery, while projecting a wider scope of application to other spatial modeling needs
like in computer vision.  However, in those wider areas, there have been more active
considerations of other representations to capture spatio-temporal dependences.
Some review of where the proposal approach could distinguish itself can add to the
argument.
review_title: null

Meta-review: metareview: The reviewers all agreed that this was a novel, interesting and effective innovation. It seems clear that, with the modifications to the paper that the authors agreed to, this paper should be accepted.
confidence: Certain
Decision: Accept