Efficiently incorporating quintuple interactions into geometric deep learning force fields
Authors: Zun Wang
Guoqing Liu
Yichi Zhou
Tong Wang
Bin Shao
Venue: NeurIPS 2023 poster (2023)
Keywords: Machine learning force field
graph neural network
many-body interactions

Abstract: Machine learning force fields (MLFFs) have instigated a groundbreaking shift in molecular dynamics (MD) simulations across a wide range of fields, such as physics, chemistry, biology, and materials science. Incorporating higher order many-body interactions can enhance the expressiveness and accuracy of models. Recent models have achieved this by explicitly including up to four-body interactions. However, five-body interactions, which have relevance in various fields, are still challenging to incorporate efficiently into MLFFs. In this work, we propose the quintuple network (QuinNet), an end-to-end graph neural network that efficiently expresses many-body interactions up to five-body interactions with \emph{ab initio} accuracy. By analyzing the topology of diverse many-body interactions, we design the model architecture to efficiently and explicitly represent these interactions. We evaluate QuinNet on public datasets of small molecules, such as MD17 and its revised version, and show that it is compatible with other state-of-the-art models on these benchmarks. Moreover, QuinNet surpasses many leading models on larger and more complex molecular systems, such as MD22 and Chignolin, without increasing the computational complexity. We also use QuinNet as a force field for molecular dynamics (MD) simulations to demonstrate its accuracy and stability, and conduct an ablation study to elucidate the significance of five-body interactions. We open source our implementation at https://github.com/Zun-Wang/QuinNet.

QuinNet: Efficiently Incorporating Quintuple Interactions into Geometric Deep Learning Force Fields
Zun Wang Mierosoft Research Al4Science Beijing, China, 100084 zunwang@microsoff. com
Guoqing Liu
Microsoft Research A14Science Beijing, China, 100084
Yichi Zhou
Tong Wang Microsoft Research Al4Science Beijing, China, 100084
Microsoft Research A14Science Beijing, China, 100084
Bin Shao
Microsoft Research A14Science Beijing, China, 100084
ginshao@microsoft.com
Abstract
Machine learning force fields (MLFFs) have instigated groundbreaking shift in molecular dynamics (MD) simulations across a wide range of fields, such as physics, chemistry, biology, and materials science. Incorporating higher order many-body interactions can enhance the expressiveness and accuracy of models. Recent models have achieved this by explicitly including up to four-body interactions. However, five-body interactions, which have relevance in various fields, are still challenging incorporate efficiently into MLFFs. In this work, we propose the quintuple network (QuinNet), an end-to-end graph neural network that efficiently expresses many-body interactions up five-body interactions with ab initio accuracy. By analyzing the topology of diverse many-body interactions, we design the model architecture efficiently and explicitly represent these interactions. We evaluate QuinNet on public datasets of small molecules, such as MD17 and its revised version, and show that is compatible with other state-of-the- models on these benchmarks. Moreover, QuinNet surpasses many leading models on larger and more complex molecular systems, such as MD22 and Chignolin, without increasing the computational complexity. We also use QuinNet as a force field for molecular dynamics (MD) simulations to demonstrate its accuracy and stability, and conduct ablation study to elucidate the significance of five-body interactions. We open source our implementation at htthtts/// : github. com/ Zun Haan-Wang//aa BaangQuuineet
Introduction
6 have brought about transformative shift in molecular dynamics (MD) simulations across diverse fields, including physics, chemistry, biology, and material science. MLFFs enable superior accuracy and reliability by leveraging ab initio and experimental data to learn interatomic interactions, while simultaneously keeping the low cost of empirical force fields, As a result, MLFFs provide powerful tool for accurately modeling molecular
Machine learning force fields (MLFFs) 12 4
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

interactions, and have the potential to revolutionize our understanding of complex systems at both the atomistic and macroscopic scales.
Recent geometric deep learning potentials, represent atoms in a system as nodes and chemical bonds edges in a graph while incorporating physical symmetries. 'hese methods have been proven to be data-efficient and effective in modeling many-body interactions 77. The development track
of geometric deep learning potentials similar to that of empirical force fields, which follow the
many-body expansion theory 8 9 [0] Previous studies M 13
15
[8] [9 20 have
primarily adopted a hierarchical approach model truncated energy correction terms, enabling the incorporation of higher-order interactions. As higher-order many-body interactions play critical role
in accurately modeling complex molecular systems, this approach has shown promise in leveraging
graph neural networks (GNNs) 21 to construct more expressive and accurate models.
Nowadays, GNN models have incorporated many-body interactions up four-body interactions explicitly [17 18 19 20 and incorporating higher order many-body interactions is still a challenge. The neglect of higher-order many-body interactions may result incomplete molecular representa- tions [22] and limited accuracy in certain cases [23 [4]. For example. five-body interactions play an important role in various fields such coarse-grained protein force field [25], organic molecules 26
crystal vibrations 27 28), electrostatic interaction potentials [29]. and so on.
In this paper, we develop MLFF based on GNNs, called quintuple network (QuinNet), address the challenge of incorporating higher order many-body interactions, up to quintuple, into molecular representations without introducing more computational complexity. To achieve this, analyze the topology of diverse many-body interactions and design QuinNet to explicitly represent these interactions efficiently. Our approach outperforms existing methods in terms of accuracy on several large molecular systems (e.g., MD22 and Chignolin dataset) while remaining comparable on small molecules (e.g., MD17 and revised MD17 dataset). Additionally, computational complexity analysis demonstrates the efficiency of our model. Our work has the potential to advance the field of MLFFs by providing methods to incorporate higher order many-body interactions.
2 Related Work
The field of MLFFs has witnessed significant progress since the introduction of the first neural network-based framework by Behler and Parrinello [30]. However, the inherent indistinguishabiiity of atoms has posed challenge in ordering them, which has limited the success of this approach. To overcome this challenge, GNNs have emerged as a promising solution, leveraging the power of graph formalism to focus on relationships among entities rather than individual node properties.
Currently, two mainstream GNN-based methods have been developed for constructing force fields: group theory-based methods and ftrection-based methods. Group theory-based meth- ods preserve physical symmetries by imposing trainable weights as the representation of related groups 122 31]. These equivariant models, including Tensor Field Networks (32), Cormorant [33), SE(3)-Transfommers [34], NequlP 77. Equiformer [35]. and Allegro [36), have demonstrated high performance across diverse tasks. Furthermore, the MACE (37, which was developed hierarchically using a body order expansion manner, has been proposed as a unifying framework of B(3) Ee3)eequia
atom-centered interatomic potentials, extending the ACE [38 framework to include methods built on equivariant message passing neural networks (MPNNs).
On the other hand, direction-based methods, such SchNet 12. DimeNet (DimeNet+++) 13 [4]
PaiNN [5), FT [6). GemNet [17, SphereNet [8], ComENet 19. and ViSNet 20,, explicitly model interatomic interactions by incorporating more geometric information. This progress has motivated us to investigate the potential performance enhancements of explicitly incorporating higher-order interactions into GNN-based force fields.
3 Preliminary
3.1 GNN-based force fields
The key idea of graph neural networks boils down to a message-passing framework (39], wherein each node iteratively updates its embedding by aggregating messages from its neighboring nodes.

---

With hy and eve denoting node features and edge features respectively, the message- passsing layer is defined as
1+1
E wEN(v)
Ma Nas cow);
my
h1 U,(,, m
where My and U2 are a message function and update function, respectively. In the realm of GNN- based force field, physical symmetries such as translation, rotation, and permutation symmetries have been demonstrated to be intrinsic and highly effective for model design.
Group equivariance Let C:
Y be function that maps from input space X' to output space ). We say that L is equívariant with respect to a group G if for all group element G,
Lo (g) = ())
where D is the representation of group G in the input vector space X'
Group equivariant convolutional layer In recent years, there has been growing interest in designing group equivariant neural networks due to the importance of respecting relevant symmetries as inductive bias. Specifically, the group equivariant convolutional layer is defined as follows:
Cllo) acmo (Fas Vil Volm..)
Σ YE 1.
cllomooo (p.m)))mm
Σ BES
Fll (Fas) (leb))
(Fab)
(3)
where the node features denoted by V. and Fab represents the relative position vector between point a and point b. The subscripts c, i, f, and denote channel, input, filter, and output, respectively, Additionally, 1 and correspond to the orbital quantum number and magnetic quantum number, respectively. This definition of the group equivariant convolutional layer is an important contribution to the development of neural networks that effectively handle physical symmetries.
3.2 Empirical force fields
In classical molecular dynamics simulations, force fields play crucial role in describing the potential energy (r*N of a system, where N represents the positions of N particles. A common functional form for force fields y (Ir") = Eranted + Enon-bonded- The bonded energy term captures the deviation of bonds and angles from their equilibrium values, while the non-bonded interactions include long-range forces, such as electrostatic and van der Waals interactions. To represent the bonded interactions hierarchically, bond stretching, angle bending, bond rotation, and other factors can be taken into account.
Improvement of torsion potential The torsion potential is crucial aspect of molecular modeling, representing the energy associated with the rotation of dihedral angles between bonded atoms. While many of the torsion terms are comprised of only one term from the cosine series expansion [40), it has been discovered that including more than one term is necessary for certain bonds, such as gauche conformations 41 [2). This involves the use of higher order cosine series, represented by cos(nw- - >]], where w denotes the torsion angle and V,, is the barrier height.
Improper torsions and out-of- (a)
plane bending motions Experi-
mental observations have shown that in certain molecular systems, the atom connected to a ring tends to re- main in the same plane due to the ø- bonding energy. However, the force field discussed in the previous sec- tion predicts an equilibrium structure with the atom out of plane. In order to correct this discrepancy, an addi-
Figure Two methods to represent out-of of---nee bending terms: (a) calculating the angle between bond from the central atom and the plane defined by the central atom and the other two atoms and (b) calculating the height of the central atom above a plane defined by the other three atoms.
tional term or terms need to incorporated into the force field to ensure that the sp² carbon and its three bonded atoms remain in the same plane. One simple approach is to include an "out-of-plane"
3

---

bending term 433 Fig. I illustrates two methods that can be used to model the out-of- olane bending contributions. The first method involves calculating the angle between a bond from the central atom and the plane defined by the central atom and the other two atoms, as shown in Fig. 1 (a). The second method calculates the height of the central atom above a plane defined by the other three atoms, as shown in Fig. (b).
3.3 Challenge of incorporating higher order many-body interactions into GNNs explicitly
Higher-order many-body interactions could be learned implicitly by stacking more MPNN layers, while incorporating these interactions explicitly into GNNs would improve the performance sig- nificantly. Incorporating many-body interactions explicitly into GNN models heavily relies on the selection of appropriate physical quantities, such as bond lengths, bond angles, and dihedral angles. However, for higher order many-body interactions, finding suitable quantity poses a significant challenge. Moreover, calculating such physical quantity increases the computational complexity as the order of interactions increases. Therefore, solving this problem while simultaneously reducing computational complexity is of utmost importance.
4 Methods
To explicitly incorporate five-body interactions into GNNs, we first analyze the topology of three-, four-, and five-body interactions to identify the key physical quantities required to describe these many-body interactions. Based on this analysis, we design the model architecture effectively capture and represent these interactions.
4.1 Topology of five-body interactions
Five-body interactions have been shown play critical role in various cases, such reproducing specific phenomena in protein systems [25]. While three-body and four-body interactions can be effectively captured by incorporating angular and dihedral angular information, which physical quantity is suitable for representing the five-body interactions explicitly is still question.
In this work, propose to use dihedral angles unified physical quantity describe five-body interactions. Fig. illustrates the topology of three-body, four-body, and five-body interactions, respectively. Fig. 2 (a) and (c) show that hody interactions and improper torsion in four- body interactions could be represented as angles; and Fig. 2 (b) implics that torsion in four-body interactions could be represented as dihedral angles. There are only three different topologies for five-body interactions (Fig. (d)-(f) and all of them could be represented as dihedral angles. To calculate dihedral angles, first compute the cosine of the angle between two planes formed by vectors connecting neighboring atoms. For instance, in Fig. 2 (d), atoms ý1, J2, ja, and j4 form neighborhood around atom i, and the normal vector of the plane composed of vectors T'j and Tija can be computed using their outer product, i.e. (Tan, x Tij.) (Fija x F. The dihedral angle
(a)
(b)
(c)
(d)
(0)
(f)
j
fa
J1
Figure 2: A schematic diagram that describes the topology of (a) three-body interaction (angles), (b) body interaction (torsions), (c) four-body interaction (improper torsions), and (d-f) five-body interactions. The marker O on a plane represents the normal vector of this plane.

---

between these two planes can be easily determined as the supplementary angle to the angle between their normal vectors, which can be calculated by taking their inner product. Further details on the relationships between different many-body interactions refer to the Supplementary Materials.
4.2 Architecture of QuinNet
Inspired by the topology analysis of many-body interactions, design the model architecture of QuinNet. The overall schematic diagram of QuinNet is shown in Fig. 3 (a), which explicitly incorporates many-body interactions and reduces computational complexity simultaneously. In particular, we design the QuinNet module, as shown in Fig. (b), where different many-body (i.e., from 3-body to 5-body) interactions are handled separately and linearly combined the output of the module.
Three-body interactions The three-body interaction, which is related to angular information as depicted in Fig. 2 (a), is handled using the PaiNN [5] approach (refer to Fig. (c)). The equation for the three-body interaction expressed as:
Pe Σ
Σ J.KE
COS O'jiko
JENr
J.KEM
where Pij denotes the normalized vector of relative position bewteen atom and j. Fij. and (:s is inner product. This approach incorporates the representation of bond angular information into node i.
Four-body interactions (torsion) The incorporation of torsion interactions in force fields neces- sitates the use of dihedral angular terms, depicted in Fig. 2 (b). To address this, adopt the ViSNet [20] model, which explicitly accounts for torsion energy, as shown in Fig. 3 (b). This is achieved through vector rejection of the direction unit of node i, v. which is equivalent to Gram- Schmidt normalization. In 3-dimensional space, the process reduces to calculating the cross product of the normal vector of two planes, as what we summarized in the last section. The torsion angles then represented via
Σ Pej Pats ) JEN,
F(t))
Σ AENS
PyR (-PWs)
Σ SCA. RENS
(5)
where Miyk, is the normal vector of the plane composed of nodes i, j. and kı (i. j, and h non-colinear). This approach constructs hidden states with dihedral angular information for edge eij.
Four-body interactions (improper torsion) As ViSNet does not incorporate improper torsion term, the QuinNet module incorporates the -ft-le- bending term by computing it in the same manner Fig. 2 (c), as depicted in Fig. 3 (e). Specifically, the term is computed using the equation
)
Σ Ai
Σ JEN.
agtis
Σ JEN.
3 33))))
Σ
Voiis (Fijar (ania))
JEN,
where the different coefficients aj and 3, are introduced to prevent the term from being zero due the properties of the cross product and the is the coefficient from the product of and B. The out-of-plane bending term is an essential component of nodes' embedding.
Five-body interactions@) @1 The evaluation the five-body @ term could be represented dihedral angles (depicted Fig. 2 (d) and Fig. (f)):
Σ agfis JEN,
x
Σ JEN
3
Σ Tiiiisii (Mijia Rijass)
As noted before, different coefficients aj and 33 are assigned to avoid the term becoming zero and the term provides node embedding. Moreover, the term encompasses a portion of the description for improper interactions, and further details can be found in the Supplementary Materials.
5

---

(a)
(b)
{2}
Embedding
|
LN
QuinNe:
S1
RBF
Yi(+)
QuinNet Module
3-body (PaiNN)
4-hody (ViSNes)
4-body (Impenper)
body
5 boddyygllll
QuinNet Module
QuinNet Module
An
LN
D==
(c)
(d)
YI
(e)
17
3-body
Rojio
T
(PaiNN)
de
Rey
4-bady (Improper)
split
4-body (ViSNel)
(f)
(g)
(h)
W'
W
W ++b
5-body@il
5-body
S.bod @
Figure 3: A schematic diagram of QuinNet. (a) The overall architecture QuinNet, where {2} represents the set of atomic numbers. The initial node features are vectors of fixed dimension. The QuinNet module inputs scalar and vectorial node features 8; and e, radial basis function (RBF) scalar edge feature eij. and relative position vector 7i vectorial edge feature, and then outputs an updated scalar and vectorial node feature and sealar edge feature for the next layer. The set of spherical harmonic functions, Y,m, denoted as Y, The final scalar node features summed to form the graph feature, and the target prediction is obtained. (b) The QuinNet module consists of six modules that handle (c) 3-body interactions. (d) torsion terms in 4-body interactions, (e) improper terms in 4-body interactions, and (f)-(h) five-body interactions, respectively. The normal vector corresponding node represented by n7. linear layer, the weights are referred as W. while the term signifies the bias.

---

Five-body interactions@11 To compute the five-body interaction @ II term (refer to Fig. 2 (e) and Fig. 3 (g)), we use the following equation to construct hidden states on edges:
Σ ReW
Pukk x
Σ Batkin
Σ
Th (Tijk Tisska)
(8)
REN
SIEN,
where 1 E NN indicates that the values should be chosen according the source nodes of node
Five-body interactions@ III Fig. 3 (h), is computed as follows,
The five-body interaction @ IIll term, illustrated in Fig. 2 (f) and
)
( KEN;
Σ agfis JEA;
Σ Byfrs Bdf JEN;
axffK Σ REN;
Berga
Σ
JEN;
JEN
which constructs the representation of edges.
Higher order cosine series Since it is beneficial to include higher order cosine series terms in the force field 41 42), we utilize the vector addition theorem of spherical harmonic functions. Although all the angular calculations in Eq. 4 Eq. Only consider one order of cosine terms, the higher-order cosine terms could be incorporated easily by using the following equation:
4T
Pi 0)
Σ Yim (i)
(10)
21 +
where 0 denotes the angle between two unit vectors û and é, and P/ and Y, represent Legendre polynomials and spherical harmonic functions 1-th order, respectively. Y, is the complex conjugate of Yj. As increases, higher-order cosine terms are incorporated into the calculation, enabling us to improve the accuracy of our model and make suitable for a broader range of applications. Additional proof can be found in the Supplementary Materials.
4.3 Complexity analysis
Inspired by the design of PaiNN (15), QuinNet incorporates many-body interactions by calculating relevant physical quantities during the message passing layer, eliminating the need for calculations beforehand. The operations within the QuinNet module can be decomposed into eross product and inner product, which are independent of the number nodes' neighbors. As a result, the computational complexity of each many-body interaction is significantly reduced to ((W))) where [W] represents the number of neighbors for each node. This approach allows QuinNet to efficiently model complex interactions, enhaneing its overall performance. A more comprehensive analysis refers to the Supplementary Materials.
Results
We conduct comprehensive evaluations on different datasets to demonstrate the effectiveness of the QuinNet model. First, we benchmark our model on the public MD17 (2) and revised MD17 [44] small molecular datasets, where the impact of five-body interactions relatively weak. Despite this, we show that the QuinNet model shows comparable accuracy with the state-of-the-an models in these datasets. Moreover, QuinNet has also been benchmarked on the QM9 dataset [45] 46). the details of
which can be found in the Supplementary Materials. Furthermore, we evaluate the performance of
QuinNet on the larger molecular systems of MD22 (7 and Chignolin datasets (88) These complex systems are known to exhibit more pronounced influence of five-body interactions. Our evaluation demonstrates that QuinNet accurately models these interactions and achieves higher accuracy in energy and force prediction compared to other models.
7

---

[3)05
[b]..
(6)
(d)o
Reference Prediction
Reference
Reference
Reference Prediction
0.4
Prediction
2.0
55.22
0.4
0.1
0.2
0.0
0.0
10
[AA
1.0
Reference
19)
Referencs Prediction
0.5
Prodiction
0.8
Prediction
4.6
0.6
0.4-
0.2
0.2
0.0
0.0
10
10
(1A)
1A3
1A0
Figure 4: The distribution of interatomic distances fr(r) for (a) aspirin, (b) ethanol, (c) malonaldchyde, (d) naphthalene, (e) salicylic acid, (f) toluene, and (g) uracil in MD17 dataset. The insets display the ball-and-stick representations of these molecules.
5.1
MD17 Dataset and MD Simulation
The MD17 dataset 2 contains diverse range of sizes, spanning from 150k to nearly 1M conforma- tional geometries. Each trajectory was computed at temperature of 500 K with a resolution of 0.5 fs. The total energy and force labels for each dataset were computed using the PBE+vdW-TD electronic structure method [4 50). In Table we present the mean absolute errors (MAEs) of QuinNet for molecules in the MDI 17 dataset, along with comparisons to nine other models. QuinNet outperforms other tested models in nearly 57.14% of tasks, especially the performance on forces. Additionally, we perform MD simulations using trained QuinNets as force fields and plot the distribution of interatomic distances h(r) for these 7 molecules in Fig. Further details regarding additional settings can be found in the Supplementary Materials.
Table I: Comparison of the MAEs between several benchmarked models and QuinNet trained on MD17 dataset using 950 training samples and 50 validation samples (energies in kcal/mol and forces in kcal/(mol-Aa A)). The lowest values are highlighted in bold.
SANNet [HB
DincNec [3
PuNN [5]
Spools/ei [3]
ET [1c
GeatNet P
NoqulP (1x))
SOJKRATES [X
ViBNei 20
QuinNet
Aspinn
Baergy
0.318
0.533
Edazol
Energy
Foce
0.39
0.06
01230
8064
0.234
0384
0.54
0832
8083
0.109
0008
0851
0.011
0.86
0052
0051
0/860
0.069
Mahenakihyyde
Piror
0.66
0.581
0.319
0.167
0.189
0.139
6147
0.100
Nagttalezee
Esery
Pooze
0.16
0.58
0.122
01255
0.116
0.07
0.59
0.136
0,085
0.003
0051
0.113
0J83
0.115
0.76
0,885
0/139
IL039
0.101
Saleelic soid
Frezz
0.85
0.504
0.188
0.100
0.120
0.12
0.000
0.145
0.084
0.088
Tolusse
Earry
Fooze
0.12
0.002
0.885
0.894
0.084
0.074
0,892
0.095
0.033
0.074
0.000
Deaci
Freez
0.56
0.501
0.19
0.106
0.119
0.005
0.097
0.104
0,076
0.111
0.100
0.612
0,055
0.062
5.2 Revised MD17 Dataset
Christensen and von Lilienfeld discovered that the energy the original MD17 dataset was noisy Therefore, 100,000 structures of each molecule were selected from the original dataset to create the
44
revised MD17 dataset, and the energies and forces were recalculated the PBF/def2-ST SVP level of
theory using very tight SCF convergence and very dense DFT integration grid (4 53 54). As
shown in Table 2 QuinNet matches the performance of popular test models on the revised MD17 dataset. Supplementary Materials provide additional information on the relevant settings.
5.3
MD22 Dataset
The MD22 dataset 47 is recently introduced collection of MD trajectories, which encompasses four major classes of biomolecules and supramolecules, ranging from small peptide consisting
8

---

Table 2: Comparison of the MAES between several benchmarked models and QuinNet trained on revised MD17 dataset using 950 training samples and 50 validation samples (energies in kcal/mol and forces in kcal/(mol-A) A)). The lowest values are shown in bold.
UNITE S1
GemNet (TRQ) Lid
NoqulP (fm3)
MACE X7
Allegro 76
BOTNet
ViSNer 20
QuinNet
Energy
0,0530
0.0507
0.0445
Aspirin
Force
0.175
0.2191
0.1891
0.1522
0.1684
0.1960
0.1520
0.1429
Azobenzene
Fnergyy
Force
0.097
0.025
0.0161
0.0669
0.0692
0.0277
0.0600
04027
0.1611
0.0761
0.0585
0.9156
0.0994
0,0513
Benzene
Energy
Force
0.002
0.077
15
0.0009
0.0069
0.0052
0.0009
0.0069
0.0046
0.0007
0.0699
0.0077
0.0056
0.0077
0.0096
Edhanol
Energy
Force
0.014
0.083
0.0092
0,0616
0,0484
0,0032
0.0092
0.0484
0.0092
0.0078
0.0516
0.0096
Malonaldehyde
Energy Farce
0.025
0.152
0.1522
0.01176
0.0184
0.0946
0.0855
0.0188
0,0830
0.1338
0.183
0.0893
0.0132
0.0875
0.0168
Naphihalene
Energy
0,0046
0,1153
0,0046
0.0046
0.0174
Force
0.0488
0,0300
0.0300
0,0208
0.0415
0.0291
0,0242
Parccetamol
Energy
0.044
0.0323
0.0300
0.0346
0.0500
0.0258
0.0362
Force
0,161
0.1361
0,1107
0.1130
0.1538
0,1029
0.0979
Salicylie
Energy
Farce
0.088
0,1222
0.0161
0.0922
0.0208
0.0715
0,0208
0.0669
0.0992
0.0795
0.0771
Toluene
Energy
0.010
0,0507
0.0369
0,0069
0.0350
0.0115
0,0092
0.0415
0.0069
0,0059
0.0264
0.0244
0.0139
Uracil
Energy
0.013
0.0092
0.01
0.0138
0.0092
0.0069
0,0149
Force
0.088
0.0576
0.0669
0.0484
0,0415
0.0738
0.0495
0.0487
Table 3: Comparison of the MAEs between several benchmarked models and the lowest values marked in bold (energies in kcal/mol per atom and forces in ())-)) Furthermore, the data splitting of the training set and validation set for training QuinNet on MD22 dataset is shown.
Vaais
#GDML 67
VSRN
ViSNee 20.50
MACE:
MACC:
57
MADE (5A) 57
QunNee
Ae-A
5500/500
Flece
Raznyy
01991
0.79
IJ1016
0,1922
0.0019
0,0972
010144
0.1753
0.3930
0.0080
0.0015
0.0876
0,0000
RJ681
DHA dincosaesoencicc aii))
7500500
Eacrgy
0.023
0.75
0.594
0.068
0,0027
0.0003
11441
0.0002
0.5419
00646
0.004
0.0021
IJ8188
Stachyine
35008900
Barrgy
0.68
0.0859
R.1543
AFAP
15000000
Energy
Fecee
00012
0.08
0,1070
0,022
00226
1.3067
BJ0036
0.3435
0.0015
01991
0U0024
RU667
AT-AT-CG-C
1500/50
Fazngy
0.70
IL0810
0.1864
0.1555
6/0017
0.1153
0.011
0.1273
0,032
Buckyball cocher
55050
Earrgy
Firoz
0.68
0.1006
0,0030
0.1518
0.901
0.5120
0.0033
0,0833
0,0058
1.10
Diuile-walled memeeube
750550
Fiees
Barrgy
0.52
0.3391
B10099
0.3059
0.4188
09132
010005
0.2767
0.2473
of 42 atoms to a wolble- nanotube containing 370 atoms. The trajectories were sampled at temperatures between 400 and 500 K at a resolution of fs. and the corresponding potential energy and atomic forces were calculated using the PBE+MBD 49 50 level of theory. Table presents the MAEs of the QuinNet model for seven systems within the MD22 dataset, along with a comparison to six benchmark models. Our results show that QuinNet outperforms the vast majority of benchmark models in predicting atomic forces, and since the MD22 dataset claims that long-range interactions are crucial, this could be the reason why the ViSNet-L.SHE model that incorporates long-range interactions and the models with chosen cutoff radius have more accurate energy predictions. Further details on the relevant settings refer to the Supplementary Materials.
5.4 Ablation study on Chignolin Dataset
The Chignolin, consisting of 166 atoms, serves as the simplest artificial protein for folding and unfolding studies. The dataset includes diverse set of folding and unfolding states of Chignolin, comprising total of 9,543 representative conformations 48). As Wang 25) noted that incorporating five-body interactions could enhance the performance of models on Chignolin, we performed an ablation study evaluate the impact of various many-body interactions. As shown in Fig. 5, the explicit inclusion of five-body interactions up to part II led to a significant decrease in MAES. is worth noting that the energy errors exhibit some fluctuations when introducing five-body interactions up to part I. Considering that the five-body interactions up to part I to some extent describe the four-body improper terms, as demonstrated in the Supplementary Materials, we speculate that these two different representations of the same interaction may cause antagonism. To test this,

---

Exerg
0.400
1.7
0.38
MAV5 (nmol A0)
MARe (boa)/oo))
0.36
1.5
1.4
0.34
00322
1.2
0:30
0.25
1.0
34ooyy (PaiNT) (:T)
Abody (VelNe)
4-bydy (Iopeappa)
Shodyytt
5-body inll
Bobodytil1
Figure The ablation study on Chignolin dataset.
removed the four-body (improper) terms and retained only the five-body interactions up to part for training on the Chignolin dataset. The resulting energy MAE 1.2906 + 0.0267 kcal/mol, and the force MAE 0.2974 0.002417 kcal/(mol-A) A). These values are lower than the case where both four-body (improper) and five-body interactions up to part coexist, thus confirming These results demonstrate the importance of incorporating five-body interactions for accurately modeling the folding and unfolding conformations of Chignolin. Comparisons of the MAEs between different models and the detailed values of the ablation study refer to the Supplementary Materials.
hypothesis.
Conclusion
In this work, we propose the QuinNet architecture, which efficiently incorporates many-body inter- actions up to whole five-body in graph neural networks for molccular dynamics simulations. Our experiments on several public datasets, including MD17, revised MD17, MD22, and Chignolin, demonstrate that QuinNet achieves high accuracy without significantly increasing computational com- plexity. Notably, ablation study Chignolin highlights the significance of five-body interactions in accurately modeling complex bio-molecular systems.
References
[1] Albert P Bartók, Mike C Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett., 104(13):136403, 2010.
[2] Stefan Chmicla, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Glluss-Rbber Müller. Machine learning of accurate enorgy-conserving molecular force fields. Sci. Adv., 3(5):e 2017.
[3] Stefan Chmiela, Huziel E Sauceda, Robus-Robert Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun., 91):38887 2018.
[4] Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and Weinan E. Deep potential molec- ular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett., 120(14):14300) 2018.
[5] Jonathan P Mailoa, Mordechai Kornbluth, Simon Batzner, Georgy Samsonidze, Stephen Lam, Jonathan Vandermause, Chris Ablitt, Nicola Molinari, and Boris Kozinsky. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat. Mach. Intell., 1(10):471-477 2019.
[6] Yu Xie, Jonathan Vandermause, Lixin Sun, Andrea Cepellotti, and Boris Kozinsky. Bayesian force fields from active learning for simulation of inter-dimensiooal transformation stanene. Npj Comput. Mater., 7(1):40, 2021.
10

---

[7] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornlluth, Nicola Molinari, Tess Smidt, and Boris Kozinsky. (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun., 13(1 )::15,,453, 2022.
[8] RK Nesbet. Atomic Bethe-Goldsttons equations. III. correlation energies of ground states of Be, B, C, N, O, F, and Ne. Phys. Rev. 175(1):2, 1968.
[9] D Hankins, JW Moskowitz, and FH Stillinger. Water molecule interactions. J. Chem. Phys., 53(12))454444554 1970.
[10] Mark S Gordon, Dmitri G Fedorov, Spencer R Pruitt, and Lyudmila V Slipchenko, Fragmenta- tion methods: A route to accurate calculations on large systems. Chem. Rev. 113(2):632-672 2012.
[11] Kristof Schütt, Picter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: A continuous-filter convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
[12] Kristof T Schütt, Huziel Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. SchNet-a deep learning architecture for molecules and materials. J. Chem. Phys., 148(24):24122,, 2018.
[13] Johannes Gasteiger, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations, 2019.
[14] Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. Preprint at http: // arziv. org/ abs/ 2011. 14115 2020.
[15] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377-9388. PMLR, 2021.
[16] Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations, 2022.
[17] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. GemNet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790-6802, 2021.
[18] Benjamin Coors, Alexandru Paul Condurache, and Andreas Geiger. SphereNet: Learning spher- ical representations for detection and classification in omnidirectional images. In Proceedings of the European conference on computer vision (ECCV), pages 518-533, 2018.
[19] Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3D molecular graphs. Alice Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[20] Yusong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, Tong Wang, and Tie- Yan Liu. ViSNet: scalable and accurate geometric deep learning potential for molecular dynamics simulation. Preprint at http: // arziv. org/ abs/ 2210. 16518 2022.
[21] Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. Preprint http: // arriv. org/ abs/ 2301. 09308 2023.
[22] Albert P Bartók, Risi Kondor, and Gábor Csányi. On representing chemical environments. Phys. Rev. B, 15111) 2013,

---

[23] Roberto Rivelino, Puspitapallab Chaudhuri, and Sylvio Canuto. Quantifying multiple-body in- teraction terms in H-bonded HCN chains with many-body perturbation/coupled theories.
J. Chem. Phys., 1052330:059310000 2003.
[24] Patricia RP Barreto, Alessandra F Albernaz, Federico Palazzetti, Andrea Lombardi, Gaia Grossi, and Vincenzo Aquilanti. Hyperspherical representation of potential energy surfaces: intermolecular interactions in tetra-atomic and penta-atoma systems. Phys. Scr., 84(2):028111 2011.
[25] Jiang Wang, Nicholas Charron, Brooke Husic, Simon Olsson, Frank Noé, and Cecilia Clementi. Multi-body effects in a coarse-grained protein force field. J. Chem. Phys., 154(16):16411,, 2021.
[26] Dávid Péter Kovács, Cas van der Oord, Jiri Kucera, Alice EA Allen, Daniel Cole, Christoph Ortner, and Gábor Csányi. Linear atomic cluster expansion force fields for organic molecules: beyond RMSE. J. Chem. Theory Comput., 17(12)7696-7711 2021.
[27] AW Solbrig Jr. Valence force potentials for calculating crystal vibrations in silicon. J. Phys. Chem, Solids, 1761)177, 1971.
[28] SL Altmann, A Lapiccirella, KW Lodge, and N Tomassini. A valence force field for the silicon crystal. J. Phys. C: Solid State Phys., 15(27):5581, 1982.
[29] Clifford E Dykstra. Electrostatic interaction potentials in molecular force fields. Chem. Rev., (3(7):2339-2352 1993.
[30] Jörg Behler and Michele Parrinello. Generalized neural-network representation of high- dimensional potential-energy surfaces. Phys. Rev. Lett., 98(14):146401, 2007.
[31] Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch- Gordan Nets: fully fourier space spherical convolutional neural network. Advances in Neural Information Processing Systems, 31, 2018.
[32] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equiiarian neural networks for 3d point clouds. Preprint at http::::: // arwiv. org/ abs/ 1802. 08219 2018.
[33] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks, Advances neural information processing systems, 32, 2019.
[34] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE (3)-transformers: 3D Roto-translation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970-19999 1981, 2020.
[35] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3D atomistic graphs. In The Eleventh International Conference on Learning Representations, 2023.
[36] Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun., 14(1):579, 2023.
[37] Ilyes Batatia, David Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35:1 1423-1 1436, 2022.
[38] Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B, 99(1):014104, 2019.
[39] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry, International conference on machine learning, pages 1263-1272. PMLR, 2017.
[40] Junmei Wang, Romain M Wolf, James W Caldwell, Peter A Kollman, and David A Case. Development and testing of a general amber force field. Comput. Chem., 25(9):1157-117 2004.
12

---

[41] Charles W Bock, Philip George, Mendel Trachtman, and Murray Zanger. A reappraisal of the structure of the second stable conformer of buta- 3-diene. Chem. Soc., Perkin trans., (1):26-34, 1979.
[42] Andrew R Leach and Andrew R Leach. Molecular modelling: principles and applications. Pearson education, 2001.
[43] Thomas A Halgren. Merck molecular force field. III. molecular geometries and vibrational frequencies for MMFF94. J. Comput. Chem., (7(5-6):553-586 1996.
[44] Anders S Christensen and O Anatole Von Lilienfeld. On the role of gradients for machine learning of molecular energies and forces, Mach. Learn.: Sci. Technol. 1(4):045018, 2020.
[45] Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. Chem. Inf. Model., 52(11 ):2864 2875, 2012.
[46] Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data, 1(1):-77, 2014.
[47] Stefan Chmiela, Valentin Aassilev-Galindo, Oliver T Unke, Adil Kabylda, Huziel E Sauceda, Alexandre Tkatchenko, and Klaus-Robert Müller. Accurate global machine learning force fields for molecules with hundreds of atoms. Sci. Adv., 9())eadd08873 2023.
[48] Zun Wang, Hongfei Wu, Lixin Sun, Xinheng He, Zhirong Liu, Bin Shao, Tong Wang, and Tie- Yan Liu. Improving machine learning force fields for molecular dynamics simulations with fine fine-grained force metries. J. Chem. Phys., (39(3), 2023.
[49] John P Perdew, Kieron Burke, and Matthias Emzerroff Generalized gradient approximation made simple. Phys. Rev. Lett., (7(18)3886,, 1996.
[50] Alexandre Tkatchenko, Robert A DiStasio Jr. Roberto Car, and Matthias Scheffler. Accurate and efficient method for many-body van der Waals interactions. Phys. Rev. Lett., 108(23):236402, 2012.
[51] Oliver Unke, Stefan Chmiela, Michael Gastegger, Kristof TT Schütt,, Huziel E Sauceda, and Klaus-Robert Müller. Spookynet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nat. Commun., 12(1):7273, 2021.
[52] Thorben Frank, Oliver Unke, and Klaus-Robert Müller. So3krates: Equivariant attention for interactions on arbitrary length-scale in molecular systems. Advances in Neural Information Processing Systems, 11:29400--2413, 2022.
[53] Frank Neese. Software update: the orca program system, version 4.0. Wiley Interdiscip. Rev. Comput. Mol. Sci., 8( 8(1)xe132) 2018.
[54] Florian Weigend and Reinhart Ahlrichs. Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy. Phys. Chem. Chem. Phys., 7(18)332773305 2005.
[55] Zhuoran Qiao, Anders S Christensen, Matthew Welborn, Frederick R Manby, Anima Anandku- mar, and Thomas Miller III. Informing geometric deep learning with electronic interactions to accelerate quantum chemistry. Proc. Natl. Acad. Sci. U.S.A., 119(31))e20522211 119, 2022.
[56) Yunyang Li, Yusong Wang, Lin Huang, Han Yang, Xinran Wei, Jia Zhang, Tong Wang, Zun Wang, Bin Shao, and Tie- Yan Liu. Long-short-rand framework to capture non-local interaction for scalable molecular dynamics simulation. Preprint at http: // arviv. org/ abs/ 2304. 13542 2023.
nessage-passing physics-informed
[57] David Peter Kovacs, Ilyes Batatia, Eszter Sara Arany, and Gabor Csanyi. Evaluation of the mace force field architecture: from medicinal chemistry materials science. Preprint at http: // arwiv. org/ abs/ 2305. 14247 2023.
13


Reviews:
[{"review_id": "gCu396cgKT", "reviewer": "Reviewer_YYfR", "summary": "This paper introduces a machine learning force field that is a neural network with explicit interactions for up to 5-body terms.  The authors evaluate the model on a couple of public datasets and show demonstrate the competence or superiority of this new model compared to the state of the art in this field.", "questions": "Could the authors add the presentation of the QM9 quantities estimated in the recent publication for Allegro?  (https://www.nature.com/articles/s41467-023-36329-y Table 3)\n\nHow long and how stable were the actual MD simulations?  What were the exact codes/protocols used?\n\nWhat is the practical performance of the model during evaluation?", "limitations": "No potential negative societal impacts from this work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper provides an important addition to a series of ever-improving machine learning potentials.  The contribution is clear and simple to understand at the high level, though the details are often unclear.  The benchmarks were compared against a set of reasonably strong published methods in this area.  In my opinion, if this work was presented in an unambiguously clear fashion and accompanied by code, it could be a strong contribution to this conference.  \n\n[The paper improved significantly following the first round of feedback from reviewers, so I'm raising my rating to a 7.]", "weaknesses": "The complexity analysis is very limited.  How many total interactions did the typical molecule have as a function of their atoms, and how did the practical experimental complexity scale for the evaluation of these molecules.  One of the main reasons that 5-body terms were not used in traditional MD simulations was the poor scaling of the number of interactions one would need to calculate.\n\nThe MD simulation mentioned in section 5.1 and Fig 4 are not described anywhere.  The following sentences suggest that there would be some explanations in the supplement, but I couldn't find them: \"Additionally, we perform MD simulations using trained QuinNets as force fields and plot the distribution of interatomic distances h(r) for these 7 molecules in Fig. 4. Further details regarding additional settings can be found in the Supplementary Materials.\" \n\nThese sentences in the supplement, page2, are confusing or wrong: \"Similarly, five-body@III interaction (Fig. S1 (c)) is a special case of six-body interaction when nodes i and k4 in Fig. S1 (d) superpose each other. Thus, the QuinNet model captures all five-body interactions and a portion of six-body interactions, making it a versatile and comprehensive tool for modeling complex molecular systems.\"  There is no six-body interaction if two of the bodies are the same, and there is no physically acceptable case where two different atoms could superpose each other.\n\nThe code is not provided, so it is not possible for me to assess the reproducibility of this method.  The diagram in Figure 3 seems reasonable at the very high level, but it lacks the definitions of most of the terms annotated in the figure, thus rendering it confusing.  (What is $Y_l$? is it the set of all spherical harmonics $Y_{lm}$ for a given angular momentum $l$? What is $n_j$? What is $s_j$?  $W$?...)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for his/her comments and will address each point in our response accordingly.\n\n### Weakness 1:\n* We address your concern through both theoretical and practical analyses. In the official comment part, we present the time complexity analysis and comparisons of inference time and model parameters. In empirical force fields, interactions are computed explicitly using physical quantities. For instance, two-body interactions are represented as bonds, with the order of the number of two-body interactions being $O(NN_b)$, where $N$ and $N_b$ denote the number of atoms and the number of neighbors, respectively. Three-body interactions are represented as angles, with an order of $O(NN_b^2)$. Four-body and five-body interactions are depicted as dihedral angles, with complexity orders of $O(NN_b^3)$ and $O(NN_b^4)$, respectively. Taking aspirin as an example, and setting the radius cutoff at 5Å, the number of two-body interactions, three-body interactions, four-body interactions (torsion), four-body interactions (improper torsion), five-body interactions@I, five-body interactions@II, and five-body interactions@III are 306, 2202, 38553, 10369, 35635, 628197, and 517155, respectively. **The number of five-body interactions is significantly greater than that of other many-body interactions.** This is why 5-body terms are typically not used in traditional MD simulations. However, in some cases, five-body interactions play a crucial role in various fields such as coarse-grained protein force fields, organic molecules, crystal vibrations, and electrostatic interaction potentials. The method introduced in QuinNet may provide an efficient solution in these cases.\n\n### Weakness 2:\n* We apologize for the lack of detail in the Supplementary Materials. Simulations were carried out for each model and molecule, starting from the first frame configurations and spanning 300 ps. A 0.5 fs time step was employed, and the temperature was maintained at 500 K using a Nosé-Hoover thermostat. The distribution of interatomic distances, h(r), was computed as the ensemble average of distance statistics within the trajectories. The relevant descriptions will be defeinitely added to the Supplementary Materials.\n\n### Weakness 3:\n* As shown in Equation 9, there are six indices, i.e., $i$, $j$, $j_1$, $j_2$, $k_1$, and $k_2$, where $j, j_1, j_2\\in \\mathcal{N}_i$ and $k_1, k_2 \\in \\mathcal{N}_j$. Therefore, Equation 9 describes six-body interactions. However, since $i \\in \\mathcal{N}_j$, $k_1$ or $k_2$ might be the same index as $i$, causing the equation to describe 5-body interactions, as there will be only five indices in the equation. This is why we state that the five-body@III interaction is a special case of six-body interaction. We will modify the sentence as \"Similarly, five-body@III interaction (Fig. S1 (c)) is a special case of six-body interaction (Fig. S1 (d)) when the index i and k4 are the same in the Equation 9. Thus, the QuinNet model captures all five-body interactions and a portion of six-body interactions, making it a versatile and comprehensive tool for modeling complex molecular systems.\"\n\n### Weakness 4:\n* The code will be released upon the paper's acceptance. To avoid confusion, we will add descriptions for the notations in the caption of Figure 3. $Y_l$ is indeed the set of $Y_{lm}$. $n_j$ represents the normal vector associated with node $j$, and $s_j$ denotes the scalar embedding of node $j$. $W$ refers to the weights in a linear layer, while $b$ indicates the bias.\n\n### Questions 1:\n* We have conducted benchmarks on the QM9 dataset, and the preliminary results (Table 3) can be found in the official comment part. Due to time constraints, the experiments have not yet fully converged. Therefore, the table presents our current results, and we will update it once the experiments are completed. Additionally, we will include these results in the manuscript.\n\n### Questions 2:\n* The MD simulations for molecules in the MD17 dataset were performed over 300 ps, and all seven trajectories exhibited stability. As shown in Refs. [1] and [2], simulations using most of the models remain stable for the MD17 dataset. Moreover, the code for the simulation was implemented using the ASE Python package and adapted from Ref. [1].\n\n[1] Fu X, Wu Z, Wang W, et al. Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations[J]. Transactions on Machine Learning Research, 2023.\n\n[2] Wang Z, Wu H, Sun L, et al. Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics[J]. The Journal of Chemical Physics, 2023, 159(3).\n\n### Questions 3:\n* In terms of practical performance, we conducted MD simulations for seven molecules in the MD17 dataset over 300 ps, and all trajectories exhibited full stability. Regarding accuracy, the distributions of interatomic distances h(r) calculated using the trained QuinNet model closely matched the results obtained from DFT calculations. As for efficiency, we have included additional analysis (Time complexity section) in the official comment part to demonstrate the performance of the QuinNet model."}, {"review_id": "5cxcdLaObP", "reviewer": "Reviewer_8RW7", "summary": "This paper aims to incorporate 5-body interactions into geometric deep learning models. They first analyze the topology of 5-body interactions and identify three 5-body angles. Then they propose an efficient way to incorporate these 5-body information into models. The complexity of the proposed QuinNet is still O(|N|), the same as many previous 2-body methods like PaiNN. The results are comparable to previous SOTA methods.", "questions": "1. About the motivation:   \n this paper aims to incorporate 5-body interactions into geometric deep learning models. However, based on my understanding, using up to 4-body (torsions) interaction is already complete [1][2] in terms of capturing the geometric structures. If this is correct, then why do we need these 5-body angles? In addition, if we can incorporate 5-body interactions, do we also need to incorporate 6-body interactions?\n\n2. About the complexity:   \nin Section 4.3, the authors claim that the complexity is O(|N|), as efficient as many 2-body methods like SchNet and PaiNN. But I think this complexity is not well explained. Using pseudocode/algorithm may be better to analyze the complexity. In addition to the analysis, I suggest the authors use some results to empirically verify the great efficiency compared to other baseline methods, e.g. the inference time, used memory, etc.\n\n3. About the tasks:   \nthis paper focuses on MLFFs, how about other molecular property prediction tasks, such as QM9 and OC20? I am wondering if this method is specially designed for MLFFs, or can be used on all 3D molecule tasks. In other words, why do the authors emphasize MLFFs? Is there any significant difference between MLFFs and other molecule property prediction tasks?\n\n4. Other related papers: many-body [3], MLFFs [4]\n\n5. The j, k in Figure 2 are confusing to me. For example, in (f), why not be i, j1, j2, j3, and k1?\n\n[1] ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs.  \n[2] GemNet: Universal Directional Graph Neural Networks for Molecules.  \n[3] On the Expressive Power of Geometric Graph Neural Networks.  \n[4] Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations.", "limitations": "None", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper is well-written and easy to follow.\n\nThe experimental results show that the proposed method can perform well on most tasks. The ablation study in Section 5.4 and Figure 5 show that the proposed 5-body information indeed helps to model.", "weaknesses": "See details in the Question part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for his/her comments and will address each point in our response accordingly.\n\n### Questions 1:\n* **4-body interactions are not complete for modeling molecular interactions.** As stated in Ref [1], \"it is unclear how many descriptor elements are actually needed in order to make the descriptor complete and thus able to uniquely specify an atomic environment of the N neighbors.\" Moreover, from the perspective of \"Many-Body Expansion\" theory [2], representing the energy of an entire system in a hierarchical level requires including all many-body interactions for completeness. ComENet demonstrates its geometric completeness for a strongly connected 3D graph, proving that torsion angles are indeed adequate for capturing geometric structures. However, determining the identical nature of two 3D graphs does not guarantee the ability to model molecular interactions accurately. Furthermore, while incorporating 5-body interactions may be over-complete for capturing geometric structures, our results indicate their effectiveness with only a minor increase in complexity. The inclusion of 6-body interactions could potentially lead to increased computational costs if deemed necessary. Nevertheless, due to the computational complexity and the challenge of identifying an appropriate physical quantity to describe many-body interactions, this area warrants further investigation. We hope our work can offer valuable insights for future studies.\n  \n[1] Bartók A P, Kondor R, Csányi G. On representing chemical environments[J]. Physical Review B, 2013, 87(18): 184115.\n\n[2] Collins M A, Bettens R P A. Energy-based molecular fragmentation methods[J]. Chemical reviews, 2015, 115(12): 5607-5642.\n\n### Questions 2:\n* Time Complexity section presented in the official comment part, illustrates the complexity of calculating relevant physical quantities explicitly and within the QuinNet framework. Additionally, the inference time and model parameters are detailed in the Time Complexity section of the official comment part.\n\n### Questions 3:\n* Machine learning force fields are indeed an important scientific problem. Our network has the potential to naturally extend to molecular property prediction tasks. In response to your suggestion, we have conducted experiments on the QM9 dataset, and the preliminary results are displayed in Table 3 of the official comment. Due to time constraints, the experiments have not yet converged. Therefore, the table presents the current results, and we will update it once the experiments are completed. We will also incorporate these results into the manuscript. In comparison to molecular property prediction tasks, which typically predict a single value, machine learning force fields demand a more significant trade-off between accuracy and efficiency. This is the rationale behind designing our network with a complexity of $O(|N|)$.\n\n### Questions 4:\n* We will add them into references.\n\n### Questions 5:\n* $j$ and $k$ represent the neighbor atoms of $i$ and $j$, respectively. We will revise the figures and symbols to enhance the clarity of the manuscript."}, {"review_id": "ZSw2aBiuqA", "reviewer": "Reviewer_YmDt", "summary": "In this work, the authors propose to incorporate features from five-body interaction into machine-learning force field models and develop QuinNet. To efficiently incorporate such high-order information, the authors are motivated by the topology of many-body interactions and design sophisticated components. Experiments on several benchmarks are conducted to demonstrate the performance of QuinNet.", "questions": "Please refer to the Weakness section to address the concerns.", "limitations": "The authors did not discuss the limitations of this work.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The target problem of this paper, the development of machine learning force field models, is of great significance.", "weaknesses": "1. **The motivation for the designed components of many-body interaction is puzzling**. As introduced in Section 4, the development of four-body interaction (improper torsions) and five-body interactions are based on the topology. First, such analysis is purely qualitative. The authors did not provide further completeness proof or quantitative evidence about these interaction schemes in real-world data. Second, the reasons for deriving Eq (4)-(9) are not well explained. It is suggested to clarify how these components are motivated according to the topology analysis.\n\n\n2. **On the experimental evaluation**. Additionally, there are several aspects of the experiments that are concerned:\n    - The empirical performance is not consistently better than other baselines. Among the evaluated benchmarks, the proposed QuinNet cannot outperform the baselines significantly. For example, in MD17, the newly developed five-body interaction modules do not significantly improve performance. In rMD17, the best performance is diversely distributed among the compared models. Overall, the experimental evaluation does not well demonstrate the power of newly developed modules.\n    - The computation efficiency evaluation is missing. Although the authors provide complexity analysis, it is better to further show the time/memory cost comparison between the proposed QuinNet and baselines. Besides, the model parameters should also be provided for all compared models.\n    - The scale of the chosen benchmarks is rather small. Both the dataset size and sample size (number of atoms) are limited. It is suggested to further evaluate the proposed QuinNet on large-scale benchmarks, e.g., Open Catalyst Project [1].\n    - The ablation study. First, as shown in Figure 5, the inclusion of Five-body@I even induces further errors, which would make readers curious about whether such a phenomenon generally exists. Second, as introduced in VisNet, the improper angle was also considered. The authors should add further discussions and empirical comparisons between it and the newly proposed four-body interaction (improper torsion).\n\n\n3. **The writing does not meet the requirement of an acceptable paper in this conference**. First, Section 3.2 can be thoroughly extended (e.g., in the appendix) to introduce the background of force fields and highlight the importance of torsion potential, improper torsions, and higher-order many-body interactions. Second, there lack of formal descriptions of QuinNet. Figure 3 can hardly be understood by readers that are not familiar with the related works in this area. \n\n[1] Chanussot L, Das A, Goyal S, et al. Open catalyst 2020 (OC20) dataset and community challenges[J]. Acs Catalysis, 2021, 11(10): 6059-6072.\n    - ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for his/her comments and will address each point in our response accordingly.\n\n### Weakness 1:\n* In our experiments, we employ Chignolin, a protein system [1,2], which offers quantitative evidence regarding the impact of five-body interactions, aligning with the conclusions in Ref [3]. As stated in Ref [4], \"it is unclear how many descriptor elements are actually needed in order to make the descriptor complete and thus able to uniquely specify an atomic environment of the N neighbors.\" Moreover, from the perspective of \"Many-Body Expansion\" theory [5], representing the energy of an entire system in a hierarchical level requires including all many-body interactions for completeness.\n\n[1] van der Spoel, David, and M. Marvin Seibert. \"Protein folding kinetics and thermodynamics from atomistic simulations.\" Physical review letters 96.23 (2006): 238102.\n\n[2] Satoh, Daisuke, et al. \"Folding free‐energy landscape of a 10‐residue mini‐protein, chignolin.\" FEBS letters 580.14 (2006): 3422-3426.\n\n[3] Wang, Jiang, et al. \"Multi-body effects in a coarse-grained protein force field.\" The Journal of Chemical Physics 154.16 (2021).\n\n[4] Bartók A P, Kondor R, Csányi G. On representing chemical environments[J]. Physical Review B, 2013, 87(18): 184115.\n\n[5] Collins M A, Bettens R P A. Energy-based molecular fragmentation methods[J]. Chemical reviews, 2015, 115(12): 5607-5642.\n\n* Thanks for your suggestion. As illustrated in Figure 2 of the manuscript, three-body interactions can be represented as angles (Fig. 2a). To efficiently calculate the physical quantity, we can adopt the method used in PaiNN. Similarly, the approach employed in ViSNet can be utilized to compute dihedral angles by calculating the normal vectors of planes $ijk_1$ and $ijk_2$, and the inner product of these two normal vectors will yield the torsion angle (Fig. 2b). For improper angles, we can first compute the normal vector of the plane $ij_1j_2$, then find the inner product of $ij_3$ and the normal vector (Fig. 2c). Five-body interactions follow a similar process, involving the calculation of normal vectors for planes $ij_1j_2$ and $ij_3j_4$ in Fig. 2d; planes $ij_1k_1$ and $ij_2k_2$ in Fig. 2e; and planes $ik_1k_2$ and $ijk_3$ in Fig. 2f. The inner product of these two normal vectors will then provide the dihedral angle. We will include more details in the manuscript to better explain our methodology.\n\n### Weakness 2:\n* MD17 and revised MD17 are small molecular datasets in which the influence of five-body interactions is relatively minor. Nevertheless, five-body interactions have been demonstrated to be crucial in various scenarios, such as replicating specific phenomena in protein systems. Consequently, QuinNet is not only compatible with other state-of-the-art models on MD17 and rDM17 benchmarks without any sacrifices but also outperforms numerous leading models on larger molecular systems, such as the MD22 and Chignolin datasets.\n* Time Complexity section in the official comment showcases the complexity of explicitly calculating the relevant physical quantities, as well as the calculations performed in QuinNet. Additionally, the inference time and model parameters are presented in the Time Complexity section of the official comment.\n* As highlighted in the introduction section, to demonstrate the significance of five-body interactions, it is essential to test large-scale systems. The mean system size in OC20 is 77.75 (as reported in ComENet), whereas the size of supramolecules in MD22 ranges from 42 to 370, and the size of the Chignolin protein is 166. Experimental results indicate that five-body interactions have a more substantial impact on larger molecules than on smaller ones, such as those in the MD17 and rMD17 datasets. Furthermore, in response to other reviewers' suggestions, we have included the performance results for the QM9 dataset in Table 3 of the official comment section to demonstrate our model's effectiveness with varying dataset sizes. Due to time constraints, the experiments are not yet fully converged. Therefore, the table presents the current results, and we will update it once the experiments are completed. We will also incorporate these results into the manuscript.\n* First, we arrange the 5-body interactions as depicted in Figure 2(d)-(e) since 5-body interactions@I encompass a portion of 4-body (improper) interactions and 5-body interactions@III include a part of 6-body interactions. According to our ablation study, when incorporating the 5-body@I component, the energy errors increase, whereas the force errors remain comparable to the results of 4-body (improper) interactions. Although 5-body interactions@I can partially represent the improper term with dihedral angles, using dihedral angles for this purpose may not be suitable, as the descriptions for the improper term in empirical force fields shown in Figure 1 typically involve height or angles. These inappropriate pieces of information may potentially damage performance to some extent. Second, it should be noted that ViSNet incorporates the improper angle in its latest version on arXiv, which was released after the NeurIPS submission deadline and has not yet undergone peer review. Consequently, the related discussion was not included.\n\n### Weakness 3:\n* We will thoroughly revise the paper to improve its clarity and readability. The revisions will include:\n  * Providing a more detailed introduction to the background, which will help readers who are not familiar with the field to gain a better understanding of the concepts.\n  * Offering a more comprehensive context description for Figure 3, ensuring that the information presented is clear and easily interpretable."}, {"review_id": "7tgx5R1bqA", "reviewer": "Reviewer_um1j", "summary": "The paper introduces a new method for molecular modeling, QuinNet, which incorporates five-body interactions using only dihedral angles. The authors first introduce relevant concepts related to machine learning force fields and related work in the field related to a variety of equivariant models. Next, the paper describes pertinent definitions of force fields, group equivariance, and methods for calculating empirical force fields. In the methods section, the authors describe their approach for integrating five-body terms into the architecture of QuiNet using only dihedral angles and incorporating model designs from prior work (PaiNN for 3-body interactions, ViSNet for 4-body interactions) and new definitions for different topologies of 5-body interactions. In addition to the architectural description, the authors provide relevant mathematical formulations and a complexity analysis. In their results, the authors showcase QuiNets performance on a low (MD17) and high complexity (MD-22) dataset in terms of energy and force modeling, including an ablation for different body terms in Figure 5.", "questions": "* Could you provide additional details on the limitations of QuinNet? E.g. Is it limited to modeling mainly molecular systems? What sizes of molecules do you think QuinNet can be effective in and why?\n* Do you have data that supports your compute complexity analysis compared to other methods? If so, what kind of speedup do you generally find, if any?", "limitations": "The authors do not provide a discussion on limitations, which I raised as a weakness. I would like to see a discussion of limitations in future versions and/or during the discussion period.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper has the following strengths:\n* Originality: The proposed architecture incorporates relevant terms for molecular modeling that are physically relevant, but have not been incorporated before.\n* Quality: The method and experimental design showcase relevant cases for applying GNN models for molecular modeling with the idea behind the architecture being well-motivated.\n* Clarity: The paper presents a cohesive formulation of their method, both in figures and mathematics, and experiment descriptions with relevant takeaways.\n* Significance: The proposed architecture shows improved modeling performance, especially in forces, and provides a potential framework for incorporating physical interactions into GNNs.", "weaknesses": "The paper could be improved by the following:\n* Providing a clear and concise discussion of limitations. [Quality, Significance]\n* Adding more context for the results in Figure 4. The MD simulations are only briefly described in Section 5.1, which is on a different page then the figure and easy to miss. [Clarity]\n* A description of the case in which a greater set of many-body interactions is beneficial. This is briefly mentioned in the discussion between MD17 and MD22, but it would be good to put in greater context in terms of the experimental results and could serve as part of the conclusion. [Clarity]", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for his/her comments and will address each point in our response accordingly.\n\n### Weakness 1:\n* As the experimental results indicate, five-body interactions do not have a substantial impact on small molecules. To demonstrate the significance of these interactions, it is essential to examine larger molecular systems.\n\n### Weakeness 2:\n* We apologize for not providing detailed information on the MD settings. Simulations were conducted for each model/molecule, covering a duration of 300 ps and starting from the initial frame configurations. With a 0.5 fs time step and a maintained temperature of 500 K, the simulations were controlled using a Nosé-Hoover thermostat. The distribution of interatomic distances, h(r), was computed as the ensemble average of distance statistics within the trajectories. We will include these relevant details in the Supplementary Materials section.\n\n### Weakness 3:\n* As evidenced by the experimental results, higher-order many-body interactions become more remarkable for larger molecules. We appreciate the reviewer's suggestion and will incorporate additional discussion on this topic in the conclusion section.\n\n### Questions 1:\n* As previously mentioned in response to weakness 1, our experimental results indicate that five-body interactions do not have a significant impact on small molecules. To demonstrate the importance of five-body interactions, larger molecular systems should be investigated. While our current experiments primarily focus on molecular systems, we plan to extend our methods to more complex systems, such as periodic structures. As we have stated, to highlight the significance of five-body interactions, larger systems must be examined, with the largest system in our experiments containing 370 atoms. The results show an improvement when incorporating five-body interactions. Based on the findings from the MD22 and Chignolin dataset, molecules with nearly 100 atoms exhibit improvements when five-body interactions are included.\n\n### Questions 2:\n* Time Complexity section in the official comment illustrates the complexity of calculating related physical quantities explicitly, as well as the calculations performed in QuinNet. The inference time and model parameters are further elaborated in the time complexity section of the official comment. We will expand our discussion on complexity, considering both theoretical and practical system perspectives. Additionally, we will include details regarding the inference time and memory usage in the manuscript."}]

Author rebuttal:

Meta-review: The paper received mixed ratings before the rebuttal.  Some common concerns shared by reviewers include inadequate complexity analysis, unclear motivation and some unclear model descriptions. The authors provide detailed complexity analysis, additional experimental results, and model clarifications. The authors’ response clarified some of the shared concerns and steered reviewer **8RW7** towards positive. 

Though the paper could benefit from further clarification of the motivation and model description, the reviewers in general appreciate the paper’s effort in studying an important problem (including the reviewer **YmDt**, who expresses lukewarm negativity) and the convincing empirical model evaluations. The ACs tend to be on the positive side and agree that the paper could contribute valuable insights to the machine learning community, thus recommending acceptance. In the final version, wherever possible, the clarity should be further improved by incorporating well-articulated motivation, and model description.
Decision: Accept (poster)