MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
Authors: Ilyes Batatia
David Peter Kovacs
Gregor N. C. Simm
Christoph Ortner
Gabor Csanyi
Venue: NeurIPS 2022 Accept (2022)
Keywords: GNN
graph neural network
equivariance
higher order
message passing neural network
point clouds
molecules

Abstract: Creating fast and accurate force fields is a long-standing challenge in computational chemistry and materials science. Recently, Equivariant Message Passing Neural Networks (MPNNs) have emerged as a powerful tool for building machine learning interatomic potentials, outperforming other approaches in terms of accuracy. However, they suffer from high computational cost and poor scalability. Moreover, most MPNNs only pass two-body messages leading to an intricate relationship between the number of layers and the expressivity of the features. This work introduces MACE, a new equivariant MPNN model that uses higher order messages, and demonstrates that this leads to an improved learning law. We show that by using four-body messages, the required number of message passing iterations reduces to just one, resulting in a fast and highly parallelizable model, reaching or exceeding state of the art accuracy on the rMD17 and 3BPA benchmark tasks. Our implementation is available at https://github.com/ACEsuit/mace.

Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
MACE:
Tless Batatia
Dávid Péter Kováes Engineering Laboratory, University of Cambridge Cambridge, CB2 1PZ UK
Engineering Laboratory. University of Cambridge Cambridge, CB2 1PZ UK Department of Chemistry, ENS Paris-Saclay, Université Paris-Saclay 91 190 Gif-sur- Yettee France
ilyes. waw.saaatttaaaeeeaaaeeeeeee pateees-ppiiisssssaaee fr
Gregor N. C. Simm Engineering Laboratory, University of Cambridge Cambridge, CB2 1PZ UK
Christoph Ortner Department Mathematics University of British Columbia Vancouver, BC, Canada V6T 1Z2
Gábor Csányi Engineering Laboratory, University of Cambridge Cambridge. CB2 1PZ UK
Abstract
Creating fast and accurate force fields is long-standing challenge in computational chemistry and materials science. Recently, several equivariant message passing neural networks (MPNNs) have been shown to outperform models built using other approaches in terms of accuracy. However, most MPNNs suffer from high computational cost and poor scalability. We propose that these limitations arise because MPNNs only pass two-body messages leading to a direct relationship between the number of layers and the expressivity of the network. In this work, we introduce MACE, new equivariant MPNN model that uses higher body order messages. In particular, we show that using four-body messages reduces the required number of message passing iterations to just two, resulting in a fast and highly parallelizable model, reaching or exceeding state-of-the-ar accuracy on the rMD17, 3BPA, and AcAc benchmark tasks. We also demonstrate that using higher order messages leads to an improved steepness the learning curves.
1 Introduction
The earliest approaches for creating force fields (interatomic potentials) using machine learning tech- niques were using local atom-centered symmetric descriptors and fced-forward neural networks (6] Gaussian Process regression (2 or linear regression [44] [7]. The first attempts to use graph neural net-
works model the potential energy of atomistic systems had only limited success. The DTNN [2).
SchNet [11, HIP-NN [35], PhysNet [88] or DimeNet (20 29 approaches could only come close
but not improve upon the atomic descriptor-baat based methods terms of computational efficiency
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

and accuracy on public benchmarks. Furthermore, most MPNN interatomic potentials use 2- 2-body invariant messages, making them non-universal approximators [3]
The MACE architecture presented here allows for the efficient computation of equivariant messages with high body order. As result of the increased body order of the messages, only two message passing iterations are necessary to achieve high accuracy unlike the typical five or six iterations of MPNNs, making it scalable and parallelizable. Finally, our implementation has remarkable computational efficiency, reaching state-of-the-an results on the 3BPA benchmark after 30 mins of training on NVIDIA A100 GPUs.
We summarise our main contributions as follows:
We introduce MACE, novel architecture combining equivariant message passing with efficient many-body messages. The MACE models achieve state-of-the-art performance challenging benchmark tests. They also display greater generalization capabilities other approaches on extrapolation benchmarks.
We demonstrate that many-body messages change the power of the empirical power-law of the learning curves. Furthermore, show experimentally that the addition of equivariant messages only shifts the learning curves but does not change the power law when higher order messages used.
We show that MACE does not only outperform previous approaches in terms of accuracy but also does so while being significantly faster to train and evaluate than the previous most accurate models.
2 Background
2.1 MPNN Interatomic Potentials
MPNNs 22 are type of graph neural network (GNN, 40
27
51
that parametrises
mapping from a labeled graph to a target space, either a graph or a vector space. When applied to parameterise properties of atomistic structures (materials molecules), the graph is embedded in 3-dimensional (3D) Euclidean space, where each node represents an atom, and edges connect nodes if the corresponding atoms are within given distance of each other. We represent the state of each node i in layer of the MPNN by a tuple
0, (ri, 21, (1))
where E R is the position of atom i, 2, the chemical element, and h are its learnable features. forward pass of the network consists multiple message construction, update, and readout steps. During message construction, a message m is created for each node by pooling over its neighbors:
m
Mr(o .0)
(2)
JEN(O)
where Mj is learnable message function and 4 (E)(() a learnable, permutation invariant pooling operation over the neighbors of atom (e.g., a sum). In the update step, the message m, is transformed into new features
h (+++) = (o ("),,
(3)
where U, is learnable update function. After T message construction and update steps, the learnable readout functions R, map the node states o to the target, in this case the site energy of atom
E
(o:
2.2 Equivariant Graph Neural Networks
In equivariant GNNs, internal features (9) transform in specified way under some group action 1
12 32 46. 49 When modelling the potential energy of atomic structure, the group of interest is

---

O(3), specifying rotations and reflections of the particles We call GNN O(3) equivariant if it has internal features that transform under the rotation Q E ((3) as
h (1) (Q- (.....,NN) D(Q) (TI...,TN)
(5)
where Q- (fl,..,, denotes the action of the rotation on the set of atomic positions and D(Q) is matrix representing the rotation Q, acting on message h; In general, elements of the feature vector can be labeled according to the irredueible representation they transform with. We will write 1 to indicate collection of features on atom i, indexed by k that transform according to
I,LLM
(0 LikLM (Q ((1))(11.-.. (F1,..... (01.....:N) rN))
EDir D (Q) L.LM (r1, ......,,,... rN), M
(6)
where (Q) E ((2L+))x(2++1)) is a Wigner D-matrix of order L. A feature labelled with L = 0
describes an invariant scalar. Features labeled with L > O, describe equivariant features, formally corresponding to equivariant vectors, matrices or higher order tensors. The features of invariant models, such as [4T and DimeNet (2), transform according to D{Q) = 1, the identity matrix. Models such as NequlP [5), equivariant transformer 445, PaiNN 43], SEGNNs [8), in addition to invariant scalars, employ equivariant internal features that transform like vectors or tensors.
3 Related Work
ACE - Higher Order Local Descriptors In the last few years, there have been two significant breakthroughs in machine learning force fields. First, the Atomic Cluster Expansion (ACE) [16] provided a systematic framework for constructing high body order complete polynomial basis functions (features) been shown that ACE includes many previously developed atomic environment representations as special cases, including Atom Centred Symmetry Functions (6), the Smooth Overlap of Atomic Positions (SOAP) descriptor (2). Moment Tensor Potential basis functions [44), and the hyperspherical bispectrum descriptor (2) used by the SNAP model [47). These local models are limited by their cutoff distance and their relatively rigid architecture compared to the overparametrised MPNNs, leading to somewhat lower accuracy, in particular, for molecular force fields.
constant cost per basis function, independent of body order [17]. It has also
Equivariant MPNNS The second breakthrough was using equivariant internal features in MPNNs. These equivariant MPNNs, such Cormorant M. Tensor Field Networks [46). EGNN [3)] PaiNN 43 Equivariant Transformers 45 SEGNN g NewtonNet 223 and NequlP 5 were able
to achieve higher performance than previous local descriptor-based models. However, they suffer from two significant limitations: first, the most accurate models used L = 3 spherical tensors as messages and 4 to 6 message passing iterations [5] which resulted in relatively high computational cost. Second, using this many iterations significantly increased the receptive field of the network, making them difficult parallelise across multiple GPUs (36
Higher Order Message Passing Most MPNNS use a message passing scheme based on two-body messages, meaning they simultaneously depend on the states of two atoms. It has been recognised that it can be beneficial to include angles into the features, effectively creating 3-body invariant messages [29]] This idea has also been exploited in other invariant MPNNs, in particular, by SphereNet (34] and GemNet (30]. Even though these models improved the accuracy compared to the 2-body message passing, they were limited by the computational cost associated with explicitly summing over all triplets or quadruplets to compute the higher order features.
Recently, multi-ACE has been proposed as a unifying framework of -quivvanuaa atom- interatomic potentials, extending the ACE framework to include methods built on equivariant MPNNs (3). A similar unifying theories were also put forward by
Multi-ACE Framework
37 and 77. The idea is to parameterise the message m (t) in terms of invariant or equivariant ACE models. This framework sets out design space in which each model can be characterised
in terms of: (1) the number of layers, (2) the body order of the messages, (3) the equivariance (or
invariance) of the messages, and (4) the number of features in each layer. The framework highlights the relationship between the overall body order of the models and message passing, also previously
Translation invariance is trivially incorporated through the use of relative distances.
3

---

discussed in Kondor [11 Most previously published models achieved high accuracy by either using
4 to 6 layers [ 3
or increasing the local body order with single layer [3 36. With our model,
we fall in between these two extremes by combining high body order with message passing.
4 The MACE Architecture
Our MACE model follows the general framework of MPNNS outlined in Section 2 Our key innovation is a new message construction mechanism. We expand the messages m; body order expansion,
(t) in a hierarchical
m; (4)
(6) Σ u1 (of"oof)) + Σ u2 (o!" oo
Σ uu (a (0 (1)
(1)
where the u functions are learnable, the sums run over the neighbors of i, and v a hyper-parameter corresponding the maximum correlation order, the body order minus 1, of the message function with respect to the states. Even though we refer to the message as (v + 1)-body with respect to the states, the overall body order with respect the positions can be larger depending on the body order of the states themselves. Crucially, by writing Lj....... which includes self-interaction (e.g., j1 = j2), we will later obtain a tensor product structure with a computationally efficient parameterisation, that allows us circumvent the seemingly exponential scaling of the computational cost with the correlation order v. This contrasts with previous models, such as DimeNet (2 29), that compute 3-body features via the more standard many-body expansion Ej <......" Below, we describe the MACE architecture in detail. To better understand the architecture, we report in A.4 a table of the introduced tensors along with their shapes.
Message Construction At each iteration, we first embed the edges using learnable radial basis klyllss a set of spherical harmonics Y,,"), and learnable embedding of the previous node features (()
R(()
M.Maa using weights (t(t) The A! ( are obtained by pooling over the neighbours (ii) &klz" to obtain permutation invariant 2-body features whilst, crucially, retaining full directional information, and thus, full information about the atomic environment:
A Aihlgma
Σ
Clamms Climy.ggm
Σ
R Riiiiss (19s)) (f Σ
((O (1) kkiz j.hlams
L damm2
JEN(()
where Chama Cimm Jamp are the standard Clebsch-Gordas coefficients ensuring that A maintain the correct equivariance, Tyi is the (scalar) interatomic distance, and fys is the corresponding unit vector.
p(!) khgl obtained by feeding a set of radial features that embed the radial distance r; using Bessel
functions multiplied by smooth polynomial cutoff (cf. Ref. (29) ) to multi-layer perceptron (MLP).
See Section A.5 for details. In the first layer, the node features hitt) correspond to the (invariant) chemical element 2j. Therefore, (8) can be further simplified:
Σ JEN(1)
RL R (r::)) (P)) kej"
This simplified operation is much cheaper, making the computational cost of the first layer low.
The key operation of MACE the efficient construction of higher order features from the A features. This is achieved by first forming tensor products of the features, and then symmetrising:
B I Bm..
KLM
Ecth
nu Im
Σ
w
(e) kkd,
(0
i.kleme" Im = (4,m,...,,,m
(10)
Im
E=1
where the coupling coefficients CL-M
corresponding to the generalised Clebsch-Gordan coefficients LLM are L-equivariant, the weights (() are mixing the channels "kkd
(details A.3) ensuring that (()
(k) of A and v is a given correlation order. Cli.M is very sparse and can be pre-computed such that 1T) can be evaluated efficiently (see Appendix A.3.3). The additional index y) simply enumerates all possible couplings of (1...,,y ly features that yield the selected equivariance specified

---

by the L index. The ( features are constructed up some maximum v. This variable in (T)) is the order of the tensor product, and hence, can be identified as the order of the many-body expansion terms in (7). The computationally expensive multi- dittesinaan sums over all triplets, quadruplets, etc., are thus circumvented and absorbed into and
The message m ( can now be written as a linear expansion
(0) m L
ΣΣ The
w zikLon
B L
(11)
where W((()
s,kL. is learnable weight matrix that depends the chemical element of the receiving atom and message symmetry L. Thus, we implicitly construct each term u in by a linear combination of B(O) i. features of the corresponding body order.
&.M can be interpreted as a complete basis of many-body interactions (17), which can be computed at cost comparable to pairwise interactions. Because this, the expansion (TT) is systematic. can in principle be converged represent any smooth (v + 1) --boyy equivariant mapping in the limit of infinitely many features (proof in [17]).
Under mild conditions the two-body bases A the higher order features B
Update
In MACE, the update is linear function of the message and the residual connection 25
(++1) L.LLM = Uo m.
Σ wll A +
w(0 (0) 2,kL,K K.LM
(12)
In the readout phase, the invariant part of the node features mapped to hierarchical decomposition of site energies via readout functions:
Readout
E, = E
(1)
(TT)
where
Σx k w
1 <
readout, & 3,000
(13)
E R2 (n!
MLP(*)
readout (( ((r(2 }
t=
The readout only depends on the invariant features 1,000 to ensure that the site energy contributions E(") are invariant as well. To maintain body ordering, we use linear readout functions for all layers except the last, where we use one-layer MLP.
Results
5.1 Effect of Higher Order Messages
Number of Layers In this section, we investigate the effect of using higher order messages. Many MPNN architectures [41 48) exclusively pass two-body invariant messages resulting in incomplete representation of the local environment [8]. Equivariant
paassagepassingggg schemes [5] lift the degeneracy of most structures by containing directional information in the messages. MPNNs that only employ two-body messages at each layer can increase the body order either by stacking layers 31 which simultaneously increases the model's receptive field or by using non-linear activation functions, generate only a subset of all possible higher order features. By constructing higher order messages using the MACE architecture, disentangle the increase body order from the increase of the receptive field.
43.8
In Figure I we show the accuracy of MACE, NequlP, and BOTNet 3 on the 3BPA benchmark 33 as a function of the number of message passing layers. Approaches employing 2-body message passing require up five iterations for their accuracy to converge. By constructing many body messages, the number of required layers to converge in accuracy reduces to just two. all subsequent experiments,
two-layer MACE models.
Furthermore, we compare BOTNet, which does not use any non-linearities the update step to NequIP, which does. Otherwise, the two models very similar. We observe that the increase in body
5

---

300K
600K
1200K
10a
10a
RMSE [meV/Al
MACE
102
NequiP
102
BOTNet
102
10a
10a
RMSE [meV]
102
10a
102
10a
10
2
5
2
3
5
2
3 5
Number of layers
Number of layers
Number of layers
Figure 1: Energy and force errors of BOTNet, NequlP, and MACE (I. 2) on the 3BPA dataset at different temperatures as a function of the number of layers.
order through non-liieasisies within the update provides only marginal improvement, highlighting the difference between an increase in body order through non-linearities (NequlP) and higher order symmetric messages (MACE). Consequently, higher order message passing allows one to reduce the number of layers, thereby increasing speed and ease of parallelizatio over multiple GPUs. We note that MACE does not improve after two layers as the diameter of the 3BPA molecule is about 9 Å and radial cutoff in each layer is Å.
Learning Curves We study how higher order message passing affects the learning curves. A recent study of the NequlP model [5] showed that the inclusion of equivariant features results in enhanced data efficiency, increasing the slope of the log-log plot of predictive the dataset size. They showed that adding equivariance not only shifts the learning curves, but also changes the powers in the empirical power law of the learning curves, which is usually constant for given dataset [26].
function of
On the left panel of Figure we replicate the experiments of [5] by training a series of invariant MACE models with increasing correlation order on the aspirin molecule from the rMD17 17 dataset. We observe that adding higher order messages changes the steepness of the learning curves, without equivariant messages. The model with correlation order v = corresponds to two-layer 2-body invariant model, similar to SchNet. This model is the least accurate due to the incomplete nature 2-body invariant representations of the local environment (38]. The invariant messages with are akin to those in DimeNet, which explicitly puts angular information into the messages. We see that including higher order information significantly improves the model's accuracy. Finally, by going beyond any current message passing potential by setting to accuraee 2-body, equivariant MPNN while only using higher order invariant messages.
achieve similar performance
On the middle panel of Figure the symmetry order L the messages. While the slope remains nearly unchanged, the curves are shifted. In the right panel of Figure 2 we keep the correlation order fixed increase the symmetry order L of the messages. We see only marginal slope change when adding equivariant features, which could be attributable to the relatively low expressiveness of two-layer MACE restricted to correlation order 2-body MPNN models: creating higher correlation order messages incorporating equivariant messages. By exploiting both of these options, the MACE model achieves state-of-the-a accuracy.
keep the correlation order fixed
3 and gradually increase
1 and gradually
These results suggest two routes to improve invariant
5.2 Scaling and Computational Cost
Chemical Elements A significant limitation of existing atomic environment representations is that their size grows with the number of chemical elements S and correlation order as S". Data-driven compression schemes have been proposed [50] solve this issue, and MPNNs incorporate similar

---

10-1
MAE F [ev/A]
L=0,== 0,33
L- == 5--3
10
2,
1. 0.63
1-1
1- 0,,0-- 3,000-3,,00,0 3,6- 0.64
1- 2, 3, 0.64 064
1,2,--1,60005
10a
10a
10a
10a
10a
10a
Number of training data
Number of training data
Number of training data
Figure 2: Learning curve of force errors (MAE in eV AA for aspirin from the rMD17 dataset
for different models. Left: Two layers of invariant (L 0) MACE with increasing body order
(1,2,3. Center: Two layers of MACE with 3 and increasing equivariance L. {0. 1. Right: Two layers of MACE with // = 1 and increasing equivariance L E 1,,1. In each case the slope (s) indicated.
embeddings of the chemical elements into fixed-size vector space. MACE uses continuous species embedding and when constructing the higher order features in (1Op, it does not include the species dimension k in the tensor product resulting in O(1) scaling of the model with the number of chemical elements S.
Receptive Field A severe limitation of many previously published MPNNs was their large receptive field, making it difficult to parallelize the evaluation across multiple GPUs. In traditional MPNNs, the total receptive field of each node, which grows with each message passing iteration, can be up to 30 A. This scaling results in the number of neighbours being in the thousands in condensed phase simulation, preventing any efficient parallelization (36.. By decoupling the increase in correlation order of the messages from the number of message passing iterations, MACE only requires two layers resulting in much smaller receptive field. With a local radial cutoff of 4 to 5 Á, the overall receptive field remains small, making the model more parallelisable.
Computational Cost The computational bottleneck of equivariant MPNNs is the equivariant tensor product (8) This tensor product is evaluated edges. In MACE, we only evaluate this expensive tensor product once, within the second layer, and build up correlations through the tensor product of (IO). Importantly, this operation is carried out nodes. Typically the number of nodes is orders of magnitudes smaller than the number of edges resulting in computational advantage. In addition, developed loop tensor contraction algorithm for the efficient implementation of (TO) and detailed in Section A.3
We report evaluation times for BOTNet, NequlP, and multiple versions of MACE in Table 2 We observe that the invariant MACE (L = 0) is close to 10 times faster than BOTNet and NequlP while achieving similar accuracy at high temperatures. MACE with L and L 2 is 5 and times faster than BOTNet and NequlP, respectively, while outperforming them at every temperature. We acknowledge that accurate speed comparisons between codes hard to obtain, and further investigations need to be carried out. is also essential to consider training times. order to do a fair comparison, all the timings were realised using the mace code that implements all the above models. Models that are significantly faster to train are hetter suited for applications of active learning, which is typically how databases for materials science applications reported in Table takes approximately 30 mins to reach the accuracy of a converged BOTNet model, taking more than day to be trained on the 3BPA dataset using NVIDIA A100 GPUs.
built (13 The MACE model
5.3 Benchmark Results 2
5.3.1
rMD17: Molecular Dynamics Trajectory
The revised MD17 (rMD17) dataset contains train test splits randomly selected from a long molecular dynamics trajectory of ten small organic molecules [T]. For each molecule, the splits consist of 1000 training and test configurations. Table shows that MACE achieves excellent accuracy, improving
Training details and hyper- parameters for all experiments can be found in Appendix A.5
7

---

Table 1:
Mean absolute errors on the rMD17 dataset [TT]. Energy (E, me V) and force (F, meV)
errors of different models trained on 950 configurations and validated on 50. The models on the right of the first vertical line, DimeNet and NewtonNet, were trained on the original MD17 dataset [T]] The models on the right of the second (double) vertical line were trained on just configurations.
339
151
529
43.5
53
150
4.0
23.5
35.4
255
230
2.6
100
3.7
Viseel
154
43.0
612
Nakkealettek
115
08
13
48
4.1
123
122
16a
Nophitatio
1.
18
15
10
63
201
56
945
16.
IN7
700
4.
13
16
13
1/4
68
30
8.5
113
64
606
343
87
31
26
43
33
63
547
261
64
152
4.7
350
144
Rdwar
0.5
63
60
13
63
61
53
1.
31
51
15
23 186 214
191
660
801
354
the state of the art for some molecules, particularly those with the highest emros. As several methods achieve similar accuracy on the standard task predicting energies and forces based on the whole training set, we also trained MACE and NequlP, another accurate model, on just 50 configurations to increase the difficulty of the benchmark. In this case, we found that MACE outperformed NequlP for most molecules.
5.3.2 3BPA: Extrapolation to Out-of-domain Data
The 3BPA dataset introduced in 33 tests a model's extrapolation capabilities. Its training set contains 500 geometries sampled from 300 K molecular dynamics simulation of the large and flexible drug- like molecule 3- 5-(bezzyloxy (oooy)yyiiii---ammin 600 K, and 1200 K to assess and out-of-domain accuracy. A fourth test set consists optimized geometries, where two of the molecule's i dihedral angles are fixed, and third is varied between O and 360 degrees resulting so-called dihedral slices through regions of the PES far away from the training data.
The three test sets contain geometries sampled at 300 K,
The root-mean nottmmea-ssqared errors (RMSE) on energies and forces for several models are shown in Table It can be seen that MACE outperforms the other models on all tasks. In particular, when extrapolating 1200 K data, MACE with MACE with L = 2 outperforms the next best model, BOTNet, by 40% on energies for the dihedral slices. Finally, the MACE model with invariant messages (L = often nearly matches or exceeds the performance of competitive equivariant models.
outperforms NequlP and Allegro models by about 30%. Further,
Table Root-mean-square errors on the 3BPA dataset. Energy (E, m) and force meV/A) enrors of models trained and tested on configurations collected at 300 K of the flexible drug-like molecule (beezyyooy)pyrriiinn 2-mmine (3BPA). Standard deviations are computed over three runs and shown in brackets if available. In order to facilitate measuring the efficiency of architectures we implemented the NequlP and BOTNet architectures in the same code that we used for MACE and which is published together with this paper. For the precise specification of our NequlP implementa- All PyTorch timings were realised on an NVIDIA A100 GPU custom
tion see the Appendix A.5.2
implementations.
Allegro (1.=3)
NequIP (1-3)
NequIP (1=33
BOTNe (1.=3)
MACE (L=0)
MACK
MACE (l.22)
Code
allegro 3G
nequip S
300
12.98 (0.17)
(0.08)
10.8 (0.2)
(0.1)
11.3 (0.2)
(0.1)
11.0
3.1 (0.13)
4.5 (0:5))))
(4.6
10.3 (0.3)
34
8.8 (0.3)
(D.2)
600
12.07 (0.45)
(1.1
(0.1)
11.3 (0.31)
(0.3)
11.5 (0.6)
(0.29)
13.7 (0.16)
(1.35)
(1.6
9.9 (0.8)
9,7 (0.5)
(0.6)
1200 K
E
42,37 (1 46) 82.96 (1.77)
(1.6
(6.2(1.)
86.4 (1.5)
40.8 (1.3)
(1.1
(1.1
(3.89))
(0.8)
31.7 31.7(
67.8 (1.8)
62.0 (0.7)
29.8 (1
16.3
Dihadral Slices
23.1
20.0 (1.2)
26.1 (2.8)
19.3 (0.6)
16.5 (1.7)
Time latency [ms]]
103.5
101.2
10.5
17.5
24.3

---

MACE shows excellent results while also featuring low computational cost compared to many other models. The = model, which approaches previous models in terms of accuracy, outpaces them by nearly a factor of 10, whereas the I, = 2 model achieves state-of-the-arr accuracy and is around four times faster than other equivariant MPNN models. the table, we characterise the evaluation speed of the models by reporting the "latency" which is defined as the time it takes to compute forces on a structure, which is typically independent of the number atoms until GPU threads are filled (typically 10,000 atoms for these models an Nvidia A100 80GB GPU).
In Figure we compare the BOTNet, NequlP, and MACE (L 2) by inspecting their energy profile for three dihedral slices. Overall, it can be seen that all models produce smooth energy profiles and that, in general, MACE comes closest to the ground truth. The fact that MACE outperforms the
a= c=1100% p=120.0
a= 67.0670,,, 8-1.0,,81150000 150.00
a=151.0%, 8=180.0
50
60
50
DFT
40
40
NequIP
MACE
20
30
10
10
-40
-10
0
60
120
180
240
300
120 180
240
300
0
60
120
180
240
300
y["]
Y('
Figure 3: Energy predictions three through the potential energy surface of the 3- bhenzyylox)pyrrdin--aaiii (3BPA) molecule by BOTNet, NequlP, and MACE (L. = 2). The ground-trutth energy (DFT) shown in black. For each cut, the curves have been shifted vertically so that the lowest ground-truth energy is zero.
other methods in the middle panel, which contains geometries furthest from the training dataset [ suggests superior extrapolation capabilities.
5.3.3 AcAc: Flexibility and Reactivity
A similar benchmark dataset assessing a model's extrapolation capabilities to higher temperatures, bond breaking, and bond torsions of the acetylacetone molecule was proposed (3). In Table 3 we show that MACE achieves state-of-the-art results on this dataset as well. For details, see Ap- pendix 5.3.3
Table Root-mean-square errors on the acetylacetone dataset. Energy (E, mcV) and force (F, V/À) errors models trained on configurations of the acetylacetone molecule sampled at 300 and tested on configurations sampled at 300 K and 600 K. Standard deviations are computed over
three runs.
BOTNet
NequIP
MACE
0.89 (0.0)
0.81 (0.04)
0.9 (0.03)
300 K
6.3 (0.0)
5.90 (0.38)
5.1 (0.10)
(1.1)
6.04 (1.26)
4.6 (0.3)
F
29.8 (1.0)
27.8 (3.29)
22.4 20.9)
Parameters
2,756.416
3,190.488
2,803,984
Discussions
With MACE, we extend traditional (equivariant) MPNNs from 2-body many-body message passing in computationally efficient manner. Our experiments show that the approach reduces the required

---

number of message passing, leading to efficient and parallelizable models. Furthermore, we have demonstrated the high accuracy and good extrapolation capabilities of MACE, reaching sfate-of-the- art accuracy on the rMD17, 3BPA, and AcAc benchmarks. Future development should concentrate on testing MACE on larger systems, including condensed phases and solids.
Reproducibility Statements
We have included error bars via different seeds and various ablation studies wherever necessary and appropriate. We have stated all hyper-parameters and data description the Appendix A.5 Source code is available at atthtps:// //g github...b... com/ ACEsuit/mace
8 Ethical Statements
The societal impact of MACE is challenging to predict. However, better force fields have a positive impact on socicty by speeding up drug discovery and through helping to understand, control, and design new materials. However, machine learning force fields rely on generating ab initio training data leading to heavy computation and large energy consumption. Machine learned force fields do alleviate the costs of doing molecular modelling significantly when compared with using solely ab initio methods.
Acknowledgments and Disclosure of Funding
9
Acknowledgement
The authors acknowledge useful discussions with Simon Batzner, Albert Musaelian and William Baldwin. This work was performed using resources provided by the Cambridge Service for Data Driven Discovery (CSD3) operated by the University of Cambridge Research Computing Service (www.csd3.caml wessd..ca..c.... caa.ac..mk.a uk). provided by Dell EMC and Intel using Tier-2 funding from the Engineering and Physical Sciences Research Council (capital grant EP/T022 159/1). and DiRAC funding from the Science and Technology Facilities Council www..dirac.a..oo
DPK acknowledges support from AstraZeneca and the Engincering and Physical Sciences Research Council. CO is supported by Leverhulme Research Project Grant RPG-2017-191 and by the Natural Sciences and Engineering Research Council of Canada (NSERC) [funding reference number IDGR019381).
References
[1] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue Buc. E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpt:t:////:ceeeeeeeeee
weg.gss.eurip.... wwwurips.cc/ppppr.../ cc/paper/2019/ pdf
[2] Albert P Bartók, Mike Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett., 104(13): 136403, April 2010.
[3] Ilyes Batatia, Simon Batzner, Dávid Péter Kovács, Albert Musaelian, Gregor N. Simm, Ralf Drautz, Christoph Ortner, Boris Kozinsky, and Gábor Csányi. The design space of B(3)- equivariant atom-centered interatomic potentials, 2022. URL /httt::t///aatt org/abs. abs/ 0665.66433
[4] Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzale Vinícius Flo- res Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Çaglar Gülçehre, H. Francis Song, Andrew J. Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matthew M. Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. CoRR, abs/ wtt./800.0000 806.01261, 2018. URL http: / arxiiiii org/ abs 1tte//180600222 01261
10

---

[5] Simon Batzner, Albert Musaclian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13(1):
2453, 2022.
[6] Jörg Behler and Michele Parrinello. Generalized neural-network representation of high- dimensional potential-energy surfaces. Phys. Rev. Leit., 98:146401, Apr 2007. doi: 10.1103/ PhysRevLeett99. 146401. URL httt/http:///iiin aps 146401
[7] Anton Bochkarev, Yury Lysogorskiy, Christoph Ortner, Gábor Csányi, and Ralf Drautz. Multi- layer atomic cluster expansion for semi-local interactions, 2022. URL https:::: http::htt///xiiiii //tp:::://x/x/o///////
abs abs/2205. 2t5p//205500.17
[8] Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J Bekkers, and Max Welling. Geometric and physical quantities improve e(3) equivariant message passing, 2021. URL
https://arxiv wwt.:/orgga/bb//teeeoo 02905
[9] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021. URL https:t/aaxiiii wwwrxii.orggabb/21111 13478
[10] Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller, Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5):e1633015 2017.
[11] Anders S. Christensen and O. Anatole Lilienfeld. On the role of gradients for machine learning of molecular energies and forces. Machine Learning: Science and Technology, 1, 2020. ISSN 23318422. doi: 10. 21100///632221///abbaa
[12] Taco S. Cohen and Max Welling. Group equivariant convolutional networks, 2016. URL
https://arxiv.orgg
[13] Volker L Deringer, Chris J Pickard, and Gábor Csányi. Data-driven learning of total and local energies in elemental boron. Physical review letters, 120(15):156001, 2018.
[14] Volker I. Deringer, Miguel A Caro, and Gábor Csányi. A general-purpose machine-learning force field for bulk and nanostructured phosphorus. Nature communications, 11(1):1-11, 2020.
[15] Volker L Deringer, Albert Bartók, Noam Bernstein, David M Wilkins, Michele Ceriotti, and Gábor Csányi. Gaussian process regression for materials and molecules. Chemical Reviews, 121(116::0033--0144 2021.
[16] Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B Condens. Matter, 99(1):11410,, January 2019.
[17] Genevieve Dusson, Markus Bachmayr, Gabor Csanyi, Ralf Drautz, Simon Etter, Cas van der Oord, and Christoph Ortner. Atomic cluster expansion: Completeness, efficiency and stability. Journal of Computational Physics, page 10046,, 2022.
[18] Felix A. Faber, Anders Christensen, Bing Huang, and O. Anatole von Lilienfeld. Alchemical and structural distribution based representation for universal quantum machine learning. The Journal of Chemical Physics, 148(24):241717, 2018. doi: 10.1063//.5020777 URL hhtps ridoi.org//0.1. 1063 5022070
[19] Xiang Gao, Farhad Ramezanghorbani, Olexandr Isayev, Justin S. Smith, and Adrian E. Roitberg. Torchani: free and open source pytorch-basee deep learning implementation of the ani neural network potentials. Journal of Chemical Information and Modeling, 60(7):3408 3415, 2020. doi: 10. 1aa1002/cccs. hat.s.ccci..c0004555 URL httts:///oi..rrg 100 1021/acc jeim. 0c00451 PMID: 32568524.
[20] Johannes Gasteiger, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules, 2020. URL
attps://arxis org/ abs/2011 14115

---

[21] Mario Geiger and Tess Smidt. c3nn: Euclidean neural networks, 2022. URL https://arxiv org/abs/ 2207 .04453
[22] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry, 2017. URL
[23] Mojtaba Haghighatlari, Jie Li, Xingyi Guan, Oufan Zhang, Akshaya Das, Christopher Stein, Farnaz Heidar- Zadeh, Meili Liu, Martin Head-Gordon, Luke Bertels, Hongxia Hao, Itai Leven, and Teresa Wead-Gordon Newtonnet: A newtonian message passing network for deep learning of interatomic potentials and forces, 2021. Uttpsts: org/ ht///bb//2//0 02913
[24] Mojtaba Haghighatlari, Jie Li, Xingyi Guan, Oufan Zhang. Akshaya Das, Christopher J. Stein, Farnaz Heidar-Zadeh, Meili Liu, Martin fordon, Luke Bertels, Hongxia Hao, Itai Leven, and Teresa -ordon. Newtonnet: A newtonian message passing network for deep learning of interatomic potentials and forces, 2021.
[25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[26] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically, 2017. Uttps::::
http://xxxxvooog////// http//bb///100 1712.00409
[27] Thomas N. Kipf and Max Welling. Sumi-supervised classificaion with graph convolutional networks, 2016. httptttss://aaw.oorrr//aaaaa
02907
[28] Johannes Klicpera, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and uncertainty-awarc directional message passing for non-equilibrium molecules. CoRR, abs/2011.14111 2020. URL htttt:///axxii>>> //arxv////xiv////aa orgo abs/ /abb:/2011.111
[29] Johannes Klicpera, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs, 2020.
[30] Johannes Klicpera, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https //openreview net htt.///ommmiiddde _aasddds
[31] Risi Kondor. N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials. arXiv preprint arXiv: 1803.01588 01588, 2018.
[32] Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups, 2018.
1802.03690
[33] Dávid Péter Kováes, Cas van der Oord, Jiri Kucera, Alice F. A. Allen, Daniel J. Cole, Christoph Ortner, and Gábor Csányi. Linear atomic cluster expansion force fields for organic molecules: Beyond rmse. Journal of Chemical Theory and Computation, 17(12) 2021. doi: 10. (a002///ssetteee000000 34735161.
URL
10.1/ wtt.://////...cccc....ooo 1c00647 1c0044
PMID:
[34] Yi Liu, Limei Wang, Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d graph networks, 2021. URL jttps://://aaxx www./rgg/////eeeeee atsp//bb/22222 05013
[35] Nicholas Lubbers, Justin S Smith, and Kipton Barros. Hierarchical modeling of molecular energies using a deep neural network. The Journal of chemical physics, 148(24):241715, 2018.
[36] Albert Musaclian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J. Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics, 2022. URL htttt::t//aarttt
org/ ab// 05249
[37] Jigyasa Nigam, Sergey Pozdnyakov, Guillaume Fraux, and Michele Ceriotti. Unified theory of atom-centered representations and message-passing machine-learning schemes. The Journal of Chemical Physics, O(a))null,, O. doi: 10. 10.80622500087442. URL, http::htt///et htwddddi.oog////// 1063/
5. 5.0087042
12

---

[38] Sergey N Pozdnyakov and Michele Ceriotti. Incompleteness of graph convolutional neural networks for points clouds in three dimensions. arXiv preprint arXiv:2201.07136, 2022.
[39] Victor Garcia Satorras, Emiel Hoogebbom,, and Max Welling. E(n) equivariant graph neural networks, 2021. URL https:///a/a .rg/ abs/ /202 2w...00022020 09844
[40] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2009. doi: 10.1 1..1109/NN..0088205555 2005600..
[41] Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Rober Müller. Schnet: A continuous filter convolutional neural network for modeling quantum interactions. In I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: http::/:s//.....cenng.s.ossos neuripp pdf
[42] Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature com- munications, 8(1):1-8, 2017.
[43] Kristof T. Schütt, Oliver T. Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. CoRR, abs/2102.03150, 2021. URL https://arxiv 010
[44] Alexander V. Shapeev. Moment tensor potentials: A class of systematically improvable
interatomic potentials. Multiscale Modeling Simulation, 14(3):1153-117 2016. doi:
10.1137/15M1054188. URL htthht:t:/doo/ org/ httt:://ao..oorr000111///////010 1137/15M1054183
[45] Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. International Conference on Learning Representations, 2022. URL
https://openreviess www.:nnttooumrm mett:/orrmm.dddxlll222222
[46] Nathaniel Thomas, Tess Smidt, Steven M. Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation-equiivrrann neural networks for 3d point clouds. CoRR, abs/1802.08211 2018. Utttttp:://arxi org/abs/1802..om 08219
[47] A.P. Thompson, L.P. Swiler, C.R. Trott, S.M. Foiles, and G.J. Tucker. Spectral neigh- bor analysis method for automated generation of quantum-accuraal interatomic potentials.
Journal of Computational Physics, 285:316-330, 2015. ISSN 0021-9991. doi: https:
(abbi:ogg//011000//j.....1111111 att.cce//p111/00211999110000
URL
[48] Oliver T Unke and Markus Meuwly. Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. Journal of chemical theory and computation, 15(6): 3678-3693 2019.
[49] Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco Cohen. 3d steerable cnns: Learning rotationally equivariant features in volumetric data, In NeurlPS, pages 10402-10413, 2018. URL http::/papers. nips www.iiippp........eeeee
[50] Michael J Willatt, Félix Musil, and Michele Ceriotti. Feature optimization for atomistic machine learning yields a data-driven construction of the periodic table of the elements. Physical Chemistry Chemical Physics, 20(47):29611-29668 2018.
[51] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang. Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: review of methods and applications, 2018. URL httpt:tts/s:xiiii htrg/abs/18122 08434
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyper-parameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're roe're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "zgTvFlhS9c3", "reviewer": "Reviewer_FTod", "summary": "The authors propose a new ML force field based on GNNs that uses higher-order equivariant messages. Previous methods like Dimenet++ and Gemnet use higher-order messages (triplets or quadruplets), but they enumerated the various triplets / quadruplets, which made them computationally expensive, and limited the order of interactions. In the present work, the authors use a tensor product formulation to make the framework efficient. Moreover, the authors empirically show that only 2 MP layers are needed to get good performance with their model, which makes it quite fast. SO3 equivariance is achieved using spherical harmonics representations, similar to previous methods. The authors also perform scaling laws analysis as well as extrapolation to OOD data, and show that their model performs well.", "questions": "* How does the cost of tensor product scale with number of nodes? In other words, can the method be applied for larger systems like materials or proteins?\n* The experiments in the paper are based on small datasets. Do the authors have a sense for how MACE would perform on a much larger dataset like OC20 (opencatalyst.org)?", "limitations": "Seems adequate.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* The authors present a new MPNN using higher order messages. The network is equivariant which is a desirable property, for its data efficiency. The model performs well on a variety of small molecular datasets.\n* The model is very fast thanks to the tensor formulation of higher order messages, and the fact that it only requires 2 layers. Efficiency for ML force fields is very useful in practice for running long MD simulations or performing structure relaxations.\n* Scaling laws analysis shows desirable scaling behavior.\n* The model works well for OOD data.\n\nWeaknesses\n* In section 4, the paper claims that the higher order features B_i,eta,K,L,M can be interpreted as a complete basis. Is there a reference / proof for this? The paper does not mention any.\n* Experiments don't compare against newer methods like Gemnet that also use higher order message passing.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8CqAc2Ng5y6", "reviewer": "Reviewer_2Lfx", "summary": "The paper describes a message passing NN approach that efficiently handles many-body interactions (where classical methods typically handle 2 body interactions). The method is based on a standard steerable/equivariant edge embedding and aggretation, leading to equivariant atomic environment embeddings. These per-node embeddings are then pulled through some higher order CG tensor product (like an equivariant polynomial) by which the pair-wise features start interacting with each other and effectively a higher body order is achieved. Notably, this interaction takes place on the node level (not over all possible many body combinations) which makes the method highly efficient. \n\nThe paper then systematically shows what components of equivariant architectures are important (spherical harmonic degree or max body order, or both?). To a large degree it is the max body order, and spherical harmonic degree helps but to a lesser extend. The paper further shows that with high-body order, the number of layers can be greatly reduced (to 2) whilst reaching state of the art.", "questions": "1. What can there be said about which observations would hold in general, in which would be limited to single (and small?) molecule analysis, versus systems of molecules (maybe like OC20?) or large molecules like proteins?\n2. Dimenet and other higher-order methods like simplicial neural networks maintain a representation of the “many bodies” (e.g. dimenet evolves edge features). Another difference is the encoding of higher order interactions via geometric invariants s.a. angles, whereas MACE relies on products of invariants and covariants. I have the feeling that this product structure  (instead of e.g. summing over bodies) is important, but cannot quite put my finger on it. It is evident that the presented approach works well, but what precisely makes it successful? E.g. if Dimenet uses 3 body interactions, and this work does as well, MACE (mu=2) seems to be better, but why? Related to my puzzlement w.r.t. Dimenet comparison, if L=0 and mu=2, we get 3-body interactions through multiplication (?) of invariants like distances. Is there any interpretation to this?\n3. Equation 10 reminds me of polynomical activation functions, akin to consideration in Kondor [31] that such (higher order) tensor products are non-linearities in themselves (viewing them as activation functions). I am used to the idea of such equivariant polynomial activation functions (through works like [F21]), simply as necessary non-linearities for neural networks to work. But should I change my viewpoint and consider it more than just a non-linearity, but rather see it as a way to achieve higher-body interactions?\n\n[F21] Franzen, D., & Wand, M. (2021). General Nonlinearities in SO (2)-Equivariant CNNs. Advances in Neural Information Processing Systems, 34, 9086-9098.\n", "limitations": "Experiments are very well organized and conclusions are appropriate. It is however slightly unclear how well this method generalizes to other types of problems in computational chemistry (involving more or large molecules).", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n* The paper is very well written.\n* Literature review is coherent and the storyline is clear\n* Despite though material, the paper does an excellent job in presenting the method in a concise, precise and comprehendible way\n* Experiments are systematically setup as to allow to specifically test individual components and in order to draw appropriate conclusions.\n\n**Weakness**\n* I found it hard to assess the impact of some of the ablation studies, in particular those pertaining to MD17.  I wonder if anything can be set about how well these results generalize to large datasets and molecules of larger size? I am wondering how representative is the aspirin molecule for the large amount of challenging problems in computational chemistry?\n* The work could still benefit from a more in depth discussion on handling higher-order interactions (e.g. via simplicial neural networks or via Dimenet-like methods). It is clear that these approaches are different, is there anything to say about what difference is decisive? (e.g. eq 10 is based on products\n\n**Small notes**\nI do not see the point of the MLP in equation 13. All are linear layers, except for the last one, why? The last one is apparently a one layer MLP, is the only difference then an activation function? (otherwise a one-layer MLP == a linear layer)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4lDo4dFDh5", "reviewer": "Reviewer_GriX", "summary": "This work proposes a new equivariant MPNN model named MACE for force field prediction. In particular, by combining equivariant message passing with efficient many-body messages, MACE achieves both state-of-the-art performance on several benchmarks and considerable computational efficiency.", "questions": "* For the evaluation benchmarks, what do you mean by the **revised** MD17 dataset?  Is there any difference between rMD17 and MD17 in other works? Why do the authors choose the 3BPA dataset to analyze the effect of higher order messages and the computational efficiency of the method, instead of the larger scale MD17 dataset?\n* I encourage the authors to reorganize the methodology section to make it more readable and highlight their technical contributions.\n* The authors should further clarify the potential negative impact of small receptive fields. If any, how do you deal with such impact?\n* Can the authors provide a more formal mathematical description for the computation complexity for MACE and other baseline methods? This would be more direct to exhibit the advantage of MACE.\n* Is there any typo in Eqn. (3)? The authors introduce the new symbol $h_i^{t+1}$ in line55 but don't interpret it in subsequent paragraphs.", "limitations": "Limitations are well discussed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The proposed method achieves state-of-the-art performance on several benchmarks.\n* The proposed method is faster to train and inference than previous models, which is meaningful for modeling some macromolecular systems.\n* The analysis of the impact of many-body messages on learning curves is interesting, which may inspire future works in balancing message correlation orders and network depth while designing architectures.\n\nWeaknesses:\n* The connection between tensor product and the standard many-body expansion is unclear to me. The authors should provide clearer background and technical details while presenting this core module. \n* The novelty of the proposed method is not well clarified. As mentioned in related works (lin99-109), both the theories and implementation strategies of the Multi-ACE framework have been put forward by previous works. There is no clear clarification of non-trivial modification in the proposed method, making it difficult to justify the technical contribution of the paper.\n* The description of evaluation benchmarks is unclear. For example, the meaning of rMD17 is not well explained. \n* The claim about receptive fields is misleading. In line 220 of section 5.2, the authors claim that \"By decoupling the increase in correlation order of the messages from the number of message passing iterations, MACE only requires two layers resulting in a much smaller receptive field.\" There is no doubt that a small receptive field will make the model more parallelisable. However, the big receptive field also captures more global (long-range) interactions than the small receptive field, which usually leads to better empirical performance (see Figure 1 in this paper). It seems that the authors think the correlation order of body messages is more important than the size of the receptive field, so the negative impact of small receptive fields is ignored. But there is a lack of convincing discussion about this point. Correct me if I misunderstand something.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept