Title: Efficiently incorporating quintuple interactions into geometric deep learning force fields
Venue: NeurIPS 2023
Accepted: yes
Keywords: Machine learning force field

graph neural network

many-body interactions
Abstract: Machine learning force fields (MLFFs) have instigated a groundbreaking shift in molecular dynamics (MD) simulations across a wide range of fields, such as physics, chemistry, biology, and materials science. Incorporating higher order many-body interactions can enhance the expressiveness and accuracy of models. Recent models have achieved this by explicitly including up to four-body interactions. However, five-body interactions, which have relevance in various fields, are still challenging to incorporate efficiently into MLFFs. In this work, we propose the quintuple network (QuinNet), an end-to-end graph neural network that efficiently expresses many-body interactions up to five-body interactions with \emph{ab initio} accuracy. By analyzing the topology of diverse many-body interactions, we design the model architecture to efficiently and explicitly represent these interactions. We evaluate QuinNet on public datasets of small molecules, such as MD17 and its revised version, and show that it is compatible with other state-of-the-art models on these benchmarks. Moreover, QuinNet surpasses many leading models on larger and more complex molecular systems, such as MD22 and Chignolin, without increasing the computational complexity. We also use QuinNet as a force field for molecular dynamics (MD) simulations to demonstrate its accuracy and stability, and conduct an ablation study to elucidate the significance of five-body interactions. We open source our implementation at https://github.com/Zun-Wang/QuinNet.

QuinNet: Efficiently Incorporating Quintuple Interactions into Geometric Deep Learning Force Fields

Abstract

Machine learning force fields (MLFFs) have instigated a groundbreaking shift in molecular dynamics (MD) simulations across a wide range of fields, such as physics, chemistry, biology, and materials science. Incorporating higher order many-body interactions can enhance the expressiveness and accuracy of models. Recent models have achieved this by explicitly including up to four-body interactions. However, five-body interactions, which have relevance in various fields, are still challenging to incorporate efficiently into MLFFs. In this work, we propose the quintuple network (QuinNet), an end-to-end graph neural network that efficiently expresses many-body interactions up to five-body interactions with ab initio accuracy. By analyzing the topology of diverse many-body interactions, we design the model architecture to efficiently and explicitly represent these interactions. We evaluate QuinNet on public datasets of small molecules, such as MD17 and its revised version, and show that it is compatible with other state-of-the-art models on these benchmarks. Moreover, QuinNet surpasses many leading models on larger and more complex molecular systems, such as MD22 and Chignolin, without increasing the computational complexity. We also use QuinNet as a force field for molecular dynamics (MD) simulations to demonstrate its accuracy and stability, and conduct an ablation study to elucidate the significance of five-body interactions. We open source our implementation at https://github.com/Zun-Wang/QuinNet .

1 Introduction

Machine learning force fields (MLFFs) [1, 2, 3, 4, 5, 6] have brought about a transformative shift in molecular dynamics (MD) simulations across diverse fields, including physics, chemistry, biology, and material science. MLFFs enable superior accuracy and reliability by leveraging ab initio and experimental data to learn interatomic interactions, while simultaneously keeping the low cost of empirical force fields. As a result, MLFFs provide a powerful tool for accurately modeling molecular

interactions, and have the potential to revolutionize our understanding of complex systems at both the atomistic and macroscopic scales.

Recent geometric deep learning potentials, represent atoms in a system as nodes and chemical bonds as edges in a graph while incorporating physical symmetries. These methods have been proven to be data-efficient and effective in modeling many-body interactions [7]. The development track of geometric deep learning potentials is similar to that of empirical force fields, which follow the many-body expansion theory [8, 9, 10]. Previous studies [11, 12, 13, 14, 15, 16, 17, 18, 19, 20] have primarily adopted a hierarchical approach to model truncated energy correction terms, enabling the incorporation of higher-order interactions. As higher-order many-body interactions play a critical role in accurately modeling complex molecular systems, this approach has shown promise in leveraging graph neural networks (GNNs) [21] to construct more expressive and accurate models.

Nowadays, GNN models have incorporated many-body interactions up to four-body interactions explicitly [17, 18, 19, 20] and incorporating higher order many-body interactions is still a challenge. The neglect of higher-order many-body interactions may result in incomplete molecular representations [22] and limited accuracy in certain cases [23, 24]. For example, five-body interactions play an important role in various fields such as coarse-grained protein force field [25], organic molecules [26], crystal vibrations [27, 28], electrostatic interaction potentials [29], and so on.

In this paper, we develop a MLFF based on GNNs, called quintuple network (QuinNet), to address the challenge of incorporating higher order many-body interactions, up to quintuple, into molecular representations without introducing more computational complexity. To achieve this, we analyze the topology of diverse many-body interactions and design QuinNet to explicitly represent these interactions efficiently. Our approach outperforms existing methods in terms of accuracy on several large molecular systems (e.g., MD22 and Chignolin dataset) while remaining comparable on small molecules (e.g., MD17 and revised MD17 dataset). Additionally, computational complexity analysis demonstrates the efficiency of our model. Our work has the potential to advance the field of MLFFs by providing methods to incorporate higher order many-body interactions.

2 Related Work

The field of MLFFs has witnessed significant progress since the introduction of the first neural network-based framework by Behler and Parrinello [30]. However, the inherent indistinguishability of atoms has posed a challenge in ordering them, which has limited the success of this approach. To overcome this challenge, GNNs have emerged as a promising solution, leveraging the power of graph formalism to focus on relationships among entities rather than individual node properties.

Currently, two mainstream GNN-based methods have been developed for constructing force fields: group theory-based methods and direction-based methods. Group theory-based methods preserve physical symmetries by imposing trainable weights as the representation of related groups [22, 31]. These equivariant models, including Tensor Field Networks [32], Cormorant [33], SE(3)-Transformers [34], NequIP [7], Equiformer [35], and Allegro [36], have demonstrated high performance across diverse tasks. Furthermore, the MACE [37], which was developed hierarchically using a body order expansion manner, has been proposed as a unifying framework of E(3)-equivariant atom-centered interatomic potentials, extending the ACE [38] framework to include methods built on equivariant message passing neural networks (MPNNs).

On the other hand, direction-based methods, such as SchNet [11, 12], DimeNet (DimeNet++) [13, 14], PaiNN [15], ET [16], GemNet [17], SphereNet [18], ComENet [19], and ViSNet [20], explicitly model interatomic interactions by incorporating more geometric information. This progress has motivated us to investigate the potential performance enhancements of explicitly incorporating higher-order interactions into GNN-based force fields.

3 Preliminary

3.1 GNN-based force fields

The key idea of graph neural networks boils down to a message-passing framework [39], wherein each node iteratively updates its embedding by aggregating messages from its neighboring nodes.

With h v and e vw denoting node features and edge features respectively, the message-passing layer is defined as

where M t and U t are a message function and an update function, respectively. In the realm of GNNbased force field, physical symmetries such as translation, rotation, and permutation symmetries have been demonstrated to be intrinsic and highly effective for model design.

Group equivariance Let L : X → Y be a function that maps from input space X to output space Y . We say that L is equivariant with respect to a group G if for all group element g ∈ G ,

where D X is the representation of group G in the input vector space X .

Group equivariant convolutional layer In recent years, there has been a growing interest in designing group equivariant neural networks due to the importance of respecting relevant symmetries as an inductive bias. Specifically, the group equivariant convolutional layer is defined as follows:

where the node features are denoted by V , and ⃗ r ab represents the relative position vector between point a and point b . The subscripts c , i , f , and o denote channel, input, filter, and output, respectively. Additionally, l and m correspond to the orbital quantum number and magnetic quantum number, respectively. This definition of the group equivariant convolutional layer is an important contribution to the development of neural networks that can effectively handle physical symmetries.

3.2 Empirical force fields

In classical molecular dynamics simulations, force fields play a crucial role in describing the potential energy V ( r N ) of a system, where r N represents the positions of N particles. A common functional form for force fields is V ( r N ) = E bonded + E non-bonded . The bonded energy term captures the deviation of bonds and angles from their equilibrium values, while the non-bonded interactions include long-range forces, such as electrostatic and van der Waals interactions. To represent the bonded interactions hierarchically, bond stretching, angle bending, bond rotation, and other factors can be taken into account.

Improvement of torsion potential The torsion potential is a crucial aspect of molecular modeling, representing the energy associated with the rotation of dihedral angles between bonded atoms. While many of the torsion terms are comprised of only one term from the cosine series expansion [40], it has been discovered that including more than one term is necessary for certain bonds, such as gauche conformations [41, 42]. This involves the use of higher order cosine series, represented by ∑ N n =0 V n 2 [1 + cos( nω -γ )] , where ω denotes the torsion angle and V n is the barrier height.

Improper torsions and out-ofplane bending motions Experimental observations have shown that in certain molecular systems, the atom connected to a ring tends to remain in the same plane due to the π -bonding energy. However, the force field discussed in the previous section predicts an equilibrium structure with the atom out of plane. In order to correct this discrepancy, an addi-

(a)

(b)

tional term or terms need to be incorporated into the force field to ensure that the sp 2 carbon and its three bonded atoms remain in the same plane. One simple approach is to include an "out-of-plane"

bending term [43]. Fig. 1 illustrates two methods that can be used to model the out-of-plane bending contributions. The first method involves calculating the angle between a bond from the central atom and the plane defined by the central atom and the other two atoms, as shown in Fig. 1 (a). The second method calculates the height of the central atom above a plane defined by the other three atoms, as shown in Fig. 1 (b).

3.3 Challenge of incorporating higher order many-body interactions into GNNs explicitly

Higher-order many-body interactions could be learned implicitly by stacking more MPNN layers, while incorporating these interactions explicitly into GNNs would improve the performance significantly. Incorporating many-body interactions explicitly into GNN models heavily relies on the selection of appropriate physical quantities, such as bond lengths, bond angles, and dihedral angles. However, for higher order many-body interactions, finding a suitable quantity poses a significant challenge. Moreover, calculating such a physical quantity increases the computational complexity as the order of interactions increases. Therefore, solving this problem while simultaneously reducing computational complexity is of utmost importance.

4 Methods

To explicitly incorporate five-body interactions into GNNs, we first analyze the topology of three-, four-, and five-body interactions to identify the key physical quantities required to describe these many-body interactions. Based on this analysis, we design the model architecture to effectively capture and represent these interactions.

4.1 Topology of five-body interactions

Five-body interactions have been shown to play a critical role in various cases, such as reproducing specific phenomena in protein systems [25]. While three-body and four-body interactions can be effectively captured by incorporating angular and dihedral angular information, which physical quantity is suitable for representing the five-body interactions explicitly is still a question.

In this work, we propose to use dihedral angles as a unified physical quantity to describe five-body interactions. Fig. 2 illustrates the topology of three-body, four-body, and five-body interactions, respectively. Fig. 2 (a) and (c) show that three-body interactions and improper torsion in fourbody interactions could be represented as angles; and Fig. 2 (b) implies that torsion in four-body interactions could be represented as dihedral angles. There are only three different topologies for five-body interactions (Fig. 2 (d)-(f)) and all of them could be represented as dihedral angles. To calculate dihedral angles, we first compute the cosine of the angle between two planes formed by vectors connecting neighboring atoms. For instance, in Fig. 2 (d), atoms j 1 , j 2 , j 3 , and j 4 form a neighborhood around atom i , and the normal vector of the plane composed of vectors ⃗ r ij 1 and ⃗ r ij 2 can be computed using their outer product, i.e. ( ⃗ r ij 1 × ⃗ r ij 2 ) · ( ⃗ r ij 3 × ⃗ r ij 4 ) . The dihedral angle

(a)

(b)

(c)

(d)

(e)

(f)

J3

1

H

θ

j1

21

j2

k1

ji1

J2

k1

j2

J3

O

2

21

j4

J3

J2

k1

between these two planes can be easily determined as the supplementary angle to the angle between their normal vectors, which can be calculated by taking their inner product. Further details on the relationships between different many-body interactions refer to the Supplementary Materials.

4.2 Architecture of QuinNet

Inspired by the topology analysis of many-body interactions, we design the model architecture of QuinNet. The overall schematic diagram of QuinNet is shown in Fig. 3 (a), which explicitly incorporates many-body interactions and reduces computational complexity simultaneously. In particular, we design the QuinNet module, as shown in Fig. 3 (b), where different many-body (i.e., from 3-body to 5-body) interactions are handled separately and linearly combined as the output of the module.

Three-body interactions The three-body interaction, which is related to angular information as depicted in Fig. 2 (a), is handled using the PaiNN [15] approach (refer to Fig. 3 (c)). The equation for the three-body interaction is expressed as:

where ˆ r ij denotes the normalized vector of relative position bewteen atom i and j , ⃗ r ij , and ⟨· , ·⟩ is inner product. This approach incorporates the representation of bond angular information into node i .

Four-body interactions (torsion) The incorporation of torsion interactions in force fields necessitates the use of dihedral angular terms, as depicted in Fig. 2 (b). To address this, we adopt the ViSNet [20] model, which explicitly accounts for torsion energy, as shown in Fig. 3 (b). This is achieved through vector rejection of the direction unit of node i , ⃗ v i , which is equivalent to GramSchmidt normalization. In 3-dimensional space, the process reduces to calculating the cross product of the normal vector of two planes, as what we summarized in the last section. The torsion angles are then represented via

where ⃗ n ijk 1 is the normal vector of the plane composed of nodes i , j , and k 1 ( i , j , and k 1 are non-colinear). This approach constructs hidden states with dihedral angular information for edge e ij .

Four-body interactions (improper torsion) As ViSNet does not incorporate improper torsion term, the QuinNet module incorporates the out-of-plane bending term by computing it in the same manner as Fig. 2 (c), as depicted in Fig. 3 (e). Specifically, the term is computed using the equation

where the different coefficients α j and β j are introduced to prevent the term from being zero due to the properties of the cross product and the γ is the coefficient from the product of α and β . The out-of-plane bending term is an essential component of nodes' embedding.

Five-body interactions@I The evaluation the five-body@I term could be represented as dihedral angles (depicted in Fig. 2 (d) and Fig. 3 (f)):

As noted before, different coefficients α j and β j are assigned to avoid the term becoming zero and the term provides a node embedding. Moreover, the term encompasses a portion of the description for improper interactions, and further details can be found in the Supplementary Materials.

3-body

(PaiNN)

split

4-body

(ViSNet)

split

4-body

(Improper)

5-body

@Ⅰ

5-body

5-body

@Ⅲ

Embedding

RBF

QuinNet Module

QuinNet Module

QuinNet Module

LN

LN

3-body

(PaiNN)

4-body

(ViSNet)

4-body

(Improper)

5-body

@Ⅰ

5-body

5-body

@Ⅲ

QuinNet Module

split

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

{Z}

{Z}

Si

Yi(o)

Yi(Ti)

Sj

eij

q

kK

U

W·+W.+W·+

?

E.N

M

·+b

01

03

02

△sj

SA

+

L

13

S6

eij

②-E-F

Y(Ti)

Vi

Uj

qi

eij

W.

Rej(o,

Rej(o,

W·+b

W·+b

W·+b

du

dk

W·+b

M

M

M

Rej(,

S

2

n

n

M

M

W·+b

M

W·+b

IEN,

27

M

M

Rej(o,·

n

Five-body interactions@II To compute the five-body interaction@II term (refer to Fig. 2 (e) and Fig. 3 (g)), we use the following equation to construct hidden states on edges:

where ∣ ∣ j ∈ N i indicates that the values should be chosen according to the source nodes of node i .

Five-body interactions@III The five-body interaction@III term, as illustrated in Fig. 2 (f) and Fig. 3 (h), is computed as follows,

which constructs the representation of edges.

Higher order cosine series Since it is beneficial to include higher order cosine series terms in the force field [41, 42], we utilize the vector addition theorem of spherical harmonic functions. Although all the angular calculations in Eq. 4 - Eq. 9 only consider one order of cosine terms, the higher-order cosine terms could be incorporated easily by using the following equation:

where θ denotes the angle between two unit vectors ˆ u and ˆ v , and P l and Y l represent Legendre polynomials and spherical harmonic functions of l -th order, respectively. Y ∗ l is the complex conjugate of Y l . As l increases, higher-order cosine terms are incorporated into the calculation, enabling us to improve the accuracy of our model and make it suitable for a broader range of applications. Additional proof can be found in the Supplementary Materials.

4.3 Complexity analysis

Inspired by the design of PaiNN [15], QuinNet incorporates many-body interactions by calculating relevant physical quantities during the message passing layer, eliminating the need for calculations beforehand. The operations within the QuinNet module can be decomposed into cross product and inner product, which are independent of the number of nodes' neighbors. As a result, the computational complexity of each many-body interaction is significantly reduced to O ( |N| ) , where |N| represents the number of neighbors for each node. This approach allows QuinNet to efficiently model complex interactions, enhancing its overall performance. A more comprehensive analysis refers to the Supplementary Materials.

5 Results

We conduct comprehensive evaluations on different datasets to demonstrate the effectiveness of the QuinNet model. First, we benchmark our model on the public MD17 [2] and revised MD17 [44] small molecular datasets, where the impact of five-body interactions is relatively weak. Despite this, we show that the QuinNet model shows comparable accuracy with the state-of-the-art models in these datasets. Moreover, QuinNet has also been benchmarked on the QM9 dataset [45, 46], the details of which can be found in the Supplementary Materials. Furthermore, we evaluate the performance of QuinNet on the larger molecular systems of MD22 [47] and Chignolin datasets [48]. These complex systems are known to exhibit a more pronounced influence of five-body interactions. Our evaluation demonstrates that QuinNet accurately models these interactions and achieves higher accuracy in energy and force prediction compared to other models.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

5.1 MD17 Dataset and MD Simulation

The MD17 dataset [2] contains a diverse range of sizes, spanning from 150k to nearly 1M conformational geometries. Each trajectory was computed at a temperature of 500 K with a resolution of 0.5 fs. The total energy and force labels for each dataset were computed using the PBE+vdW-TS electronic structure method [49, 50]. In Table 1, we present the mean absolute errors (MAEs) of QuinNet for 7 molecules in the MD17 dataset, along with comparisons to nine other models. QuinNet outperforms other tested models in nearly 57.14% of tasks, especially the performance on forces. Additionally, we perform MD simulations using trained QuinNets as force fields and plot the distribution of interatomic distances h ( r ) for these 7 molecules in Fig. 4. Further details regarding additional settings can be found in the Supplementary Materials.

5.2 Revised MD17 Dataset

Christensen and von Lilienfeld discovered that the energy of the original MD17 dataset was noisy [44]. Therefore, 100,000 structures of each molecule were selected from the original dataset to create the revised MD17 dataset, and the energies and forces were recalculated at the PBE/def2-SVP level of theory using a very tight SCF convergence and a very dense DFT integration grid [49, 53, 54]. As shown in Table 2, QuinNet matches the performance of popular test models on the revised MD17 dataset. Supplementary Materials provide additional information on the relevant settings.

5.3 MD22 Dataset

The MD22 dataset [47] is a recently introduced collection of MD trajectories, which encompasses four major classes of biomolecules and supramolecules, ranging from a small peptide consisting

of 42 atoms to a double-walled nanotube containing 370 atoms. The trajectories were sampled at temperatures between 400 and 500 K at a resolution of 1 fs, and the corresponding potential energy and atomic forces were calculated using the PBE+MBD [49, 50] level of theory. Table 3 presents the MAEs of the QuinNet model for seven systems within the MD22 dataset, along with a comparison to six benchmark models. Our results show that QuinNet outperforms the vast majority of benchmark models in predicting atomic forces, and since the MD22 dataset claims that long-range interactions are crucial, this could be the reason why the ViSNet-LSRM model that incorporates long-range interactions and the models with chosen cutoff radius have more accurate energy predictions. Further details on the relevant settings refer to the Supplementary Materials.

5.4 Ablation study on Chignolin Dataset

The Chignolin, consisting of 166 atoms, serves as the simplest artificial protein for folding and unfolding studies. The dataset includes a diverse set of folding and unfolding states of Chignolin, comprising a total of 9,543 representative conformations [48]. As Wang et al.[25] noted that incorporating five-body interactions could enhance the performance of models on Chignolin, we performed an ablation study to evaluate the impact of various many-body interactions. As shown in Fig. 5, the explicit inclusion of five-body interactions up to part II led to a significant decrease in MAEs. It is worth noting that the energy errors exhibit some fluctuations when introducing five-body interactions up to part I. Considering that the five-body interactions up to part I to some extent describe the four-body improper terms, as demonstrated in the Supplementary Materials, we speculate that these two different representations of the same interaction may cause antagonism. To test this, we

·

3-body

(PaiNN/ET)

4-body

(ViSNet)

4-body

(Improper)

5-body@I

5-body@II

5-body@III

1

.

0

1

.

1

1

.

2

1

.

3

1

.

4

1

.

5

1

.

6

1

.

7

1

.

8

MAE

E

(kcal/mol)

0

.

28

0

.

30

0

.

32

0

.

34

0

.

36

0

.

38

0

.

40

MAE

F

(kcal/(mol

˚

A

))

Energy

Force

removed the four-body (improper) terms and retained only the five-body interactions up to part I for training on the Chignolin dataset. The resulting energy MAE is 1.2906 ± 0.0267 kcal/mol, and the force MAE is 0.2974 ± 0.002417 kcal/(mol · Å). These values are lower than the case where both four-body (improper) and five-body interactions up to part I coexist, thus confirming our hypothesis. These results demonstrate the importance of incorporating five-body interactions for accurately modeling the folding and unfolding conformations of Chignolin. Comparisons of the MAEs between different models and the detailed values of the ablation study refer to the Supplementary Materials.

6 Conclusion

In this work, we propose the QuinNet architecture, which efficiently incorporates many-body interactions up to whole five-body in graph neural networks for molecular dynamics simulations. Our experiments on several public datasets, including MD17, revised MD17, MD22, and Chignolin, demonstrate that QuinNet achieves high accuracy without significantly increasing computational complexity. Notably, our ablation study on Chignolin highlights the significance of five-body interactions in accurately modeling complex bio-molecular systems.

References

Albert P Bartók, Mike C Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. , 104(13):136403, 2010.

Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. , 3(5):e1603015, 2017.

Stefan Chmiela, Huziel E Sauceda, Klaus-Robert Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun. , 9(1):3887, 2018.

Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and Weinan E. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett. , 120(14):143001, 2018.

Jonathan P Mailoa, Mordechai Kornbluth, Simon Batzner, Georgy Samsonidze, Stephen T Lam, Jonathan Vandermause, Chris Ablitt, Nicola Molinari, and Boris Kozinsky. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat. Mach. Intell. , 1(10):471-479, 2019.

Yu Xie, Jonathan Vandermause, Lixin Sun, Andrea Cepellotti, and Boris Kozinsky. Bayesian force fields from active learning for simulation of inter-dimensional transformation of stanene. Npj Comput. Mater. , 7(1):40, 2021.

Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. , 13(1):2453, 2022.

RK Nesbet. Atomic Bethe-Goldstone equations. III. correlation energies of ground states of Be, B, C, N, O, F, and Ne. Phys. Rev. , 175(1):2, 1968.

D Hankins, JW Moskowitz, and FH Stillinger. Water molecule interactions. J. Chem. Phys. , 53(12):4544-4554, 1970.

Mark S Gordon, Dmitri G Fedorov, Spencer R Pruitt, and Lyudmila V Slipchenko. Fragmentation methods: A route to accurate calculations on large systems. Chem. Rev. , 112(1):632-672, 2012.

Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: A continuous-filter convolutional neural network for modeling quantum interactions. Advances in neural information processing systems , 30, 2017.

Kristof T Schütt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. SchNet-a deep learning architecture for molecules and materials. J. Chem. Phys. , 148(24):241722, 2018.

Johannes Gasteiger, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations , 2019.

Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. Preprint at http://arxiv.org/abs/2011.14115 , 2020.

Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning , pages 9377-9388. PMLR, 2021.

Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations , 2022.

Johannes Gasteiger, Florian Becker, and Stephan Günnemann. GemNet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems , 34:6790-6802, 2021.

Benjamin Coors, Alexandru Paul Condurache, and Andreas Geiger. SphereNet: Learning spherical representations for detection and classification in omnidirectional images. In Proceedings of the European conference on computer vision (ECCV) , pages 518-533, 2018.

Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3D molecular graphs. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Yusong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, Tong Wang, and Tie-Yan Liu. ViSNet: a scalable and accurate geometric deep learning potential for molecular dynamics simulation. Preprint at http://arxiv.org/abs/2210.16518 , 2022.

Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. Preprint at http://arxiv.org/ abs/2301.09308 , 2023.

Albert P Bartók, Risi Kondor, and Gábor Csányi. On representing chemical environments. Phys. Rev. B , 87(18):184115, 2013.

Roberto Rivelino, Puspitapallab Chaudhuri, and Sylvio Canuto. Quantifying multiple-body interaction terms in H-bonded HCN chains with many-body perturbation/coupled-cluster theories. J. Chem. Phys. , 118(23):10593-10601, 2003.

Patricia RP Barreto, Alessandra F Albernaz, Federico Palazzetti, Andrea Lombardi, Gaia Grossi, and Vincenzo Aquilanti. Hyperspherical representation of potential energy surfaces: intermolecular interactions in tetra-atomic and penta-atomic systems. Phys. Scr. , 84(2):028111, 2011.

Jiang Wang, Nicholas Charron, Brooke Husic, Simon Olsson, Frank Noé, and Cecilia Clementi. Multi-body effects in a coarse-grained protein force field. J. Chem. Phys. , 154(16):164113, 2021.

Dávid Péter Kovács, Cas van der Oord, Jiri Kucera, Alice EA Allen, Daniel J Cole, Christoph Ortner, and Gábor Csányi. Linear atomic cluster expansion force fields for organic molecules: beyond RMSE. J. Chem. Theory Comput. , 17(12):7696-7711, 2021.

AW Solbrig Jr. Valence force potentials for calculating crystal vibrations in silicon. J. Phys. Chem. Solids , 32(8):1761-1768, 1971.

SL Altmann, A Lapiccirella, KW Lodge, and N Tomassini. A valence force field for the silicon crystal. J. Phys. C: Solid State Phys. , 15(27):5581, 1982.

Clifford E Dykstra. Electrostatic interaction potentials in molecular force fields. Chem. Rev. , 93(7):2339-2353, 1993.

Jörg Behler and Michele Parrinello. Generalized neural-network representation of highdimensional potential-energy surfaces. Phys. Rev. Lett. , 98(14):146401, 2007.

Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch-Gordan Nets: a fully fourier space spherical convolutional neural network. Advances in Neural Information Processing Systems , 31, 2018.

Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. Preprint at http://arxiv.org/abs/1802.08219 , 2018.

Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. Advances in neural information processing systems , 32, 2019.

Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE (3)-transformers: 3D Roto-translation equivariant attention networks. Advances in Neural Information Processing Systems , 33:1970-1981, 2020.

Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3D atomistic graphs. In The Eleventh International Conference on Learning Representations , 2023.

Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun. , 14(1):579, 2023.

Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems , 35:11423-11436, 2022.

Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B , 99(1):014104, 2019.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263-1272. PMLR, 2017.

Junmei Wang, Romain M Wolf, James W Caldwell, Peter A Kollman, and David A Case. Development and testing of a general amber force field. J. Comput. Chem. , 25(9):1157-1174, 2004.

Charles W Bock, Philip George, Mendel Trachtman, and Murray Zanger. A reappraisal of the structure of the second stable conformer of buta-1, 3-diene. J. Chem. Soc., Perkin trans. , (1):26-34, 1979.

Andrew R Leach and Andrew R Leach. Molecular modelling: principles and applications . Pearson education, 2001.

Thomas A Halgren. Merck molecular force field. III. molecular geometries and vibrational frequencies for MMFF94. J. Comput. Chem. , 17(5-6):553-586, 1996.

Anders S Christensen and O Anatole Von Lilienfeld. On the role of gradients for machine learning of molecular energies and forces. Mach. Learn.: Sci. Technol. , 1(4):045018, 2020.

Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. , 52(11):2864-2875, 2012.

Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data , 1(1):1-7, 2014.

Stefan Chmiela, Valentin Vassilev-Galindo, Oliver T Unke, Adil Kabylda, Huziel E Sauceda, Alexandre Tkatchenko, and Klaus-Robert Müller. Accurate global machine learning force fields for molecules with hundreds of atoms. Sci. Adv. , 9(2):eadf0873, 2023.

Zun Wang, Hongfei Wu, Lixin Sun, Xinheng He, Zhirong Liu, Bin Shao, Tong Wang, and Tie-Yan Liu. Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics. J. Chem. Phys. , 159(3), 2023.

John P Perdew, Kieron Burke, and Matthias Ernzerhof. Generalized gradient approximation made simple. Phys. Rev. Lett. , 77(18):3865, 1996.

Alexandre Tkatchenko, Robert A DiStasio Jr, Roberto Car, and Matthias Scheffler. Accurate and efficient method for many-body van der Waals interactions. Phys. Rev. Lett. , 108(23):236402, 2012.

Oliver T Unke, Stefan Chmiela, Michael Gastegger, Kristof T Schütt, Huziel E Sauceda, and Klaus-Robert Müller. Spookynet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nat. Commun. , 12(1):7273, 2021.

Thorben Frank, Oliver Unke, and Klaus-Robert Müller. So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems. Advances in Neural Information Processing Systems , 35:29400-29413, 2022.

Frank Neese. Software update: the orca program system, version 4.0. Wiley Interdiscip. Rev. Comput. Mol. Sci. , 8(1):e1327, 2018.

Florian Weigend and Reinhart Ahlrichs. Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy. Phys. Chem. Chem. Phys. , 7(18):3297-3305, 2005.

Zhuoran Qiao, Anders S Christensen, Matthew Welborn, Frederick R Manby, Anima Anandkumar, and Thomas F Miller III. Informing geometric deep learning with electronic interactions to accelerate quantum chemistry. Proc. Natl. Acad. Sci. U.S.A. , 119(31):e2205221119, 2022.

Yunyang Li, Yusong Wang, Lin Huang, Han Yang, Xinran Wei, Jia Zhang, Tong Wang, Zun Wang, Bin Shao, and Tie-Yan Liu. Long-short-range message-passing: A physics-informed framework to capture non-local interaction for scalable molecular dynamics simulation. Preprint at http://arxiv.org/abs/2304.13542 , 2023.

David Peter Kovacs, Ilyes Batatia, Eszter Sara Arany, and Gabor Csanyi. Evaluation of the mace force field architecture: from medicinal chemistry to materials science. Preprint at http://arxiv.org/abs/2305.14247 , 2023.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new method for molecular modeling, QuinNet, which incorporates five-body interactions using only dihedral angles. The authors first introduce relevant concepts related to machine learning force fields and related work in the field related to a variety of equivariant models. Next, the paper describes pertinent definitions of force fields, group equivariance, and methods for calculating empirical force fields. In the methods section, the authors describe their approach for integrating five-body terms into the architecture of QuiNet using only dihedral angles and incorporating model designs from prior work (PaiNN for 3-body interactions, ViSNet for 4-body interactions) and new definitions for different topologies of 5-body interactions. In addition to the architectural description, the authors provide relevant mathematical formulations and a complexity analysis. In their results, the authors showcase QuiNets performance on a low (MD17) and high complexity (MD-22) dataset in terms of energy and force modeling, including an ablation for different body terms in Figure 5.\n\nSTRENGTHS:\nThe paper has the following strengths:\n* Originality: The proposed architecture incorporates relevant terms for molecular modeling that are physically relevant, but have not been incorporated before.\n* Quality: The method and experimental design showcase relevant cases for applying GNN models for molecular modeling with the idea behind the architecture being well-motivated.\n* Clarity: The paper presents a cohesive formulation of their method, both in figures and mathematics, and experiment descriptions with relevant takeaways.\n* Significance: The proposed architecture shows improved modeling performance, especially in forces, and provides a potential framework for incorporating physical interactions into GNNs.\n\nWEAKNESSES:\nThe paper could be improved by the following:\n* Providing a clear and concise discussion of limitations. [Quality, Significance]\n* Adding more context for the results in Figure 4. The MD simulations are only briefly described in Section 5.1, which is on a different page then the figure and easy to miss. [Clarity]\n* A description of the case in which a greater set of many-body interactions is beneficial. This is briefly mentioned in the discussion between MD17 and MD22, but it would be good to put in greater context in terms of the experimental results and could serve as part of the conclusion. [Clarity]\n\nQUESTIONS:\n* Could you provide additional details on the limitations of QuinNet? E.g. Is it limited to modeling mainly molecular systems? What sizes of molecules do you think QuinNet can be effective in and why?\n* Do you have data that supports your compute complexity analysis compared to other methods? If so, what kind of speedup do you generally find, if any?\n\nLIMITATIONS:\nThe authors do not provide a discussion on limitations, which I raised as a weakness. I would like to see a discussion of limitations in future versions and/or during the discussion period.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose to incorporate features from five-body interaction into machine-learning force field models and develop QuinNet. To efficiently incorporate such high-order information, the authors are motivated by the topology of many-body interactions and design sophisticated components. Experiments on several benchmarks are conducted to demonstrate the performance of QuinNet.\n\nSTRENGTHS:\n1. The target problem of this paper, the development of machine learning force field models, is of great significance.\n\nWEAKNESSES:\n1. **The motivation for the designed components of many-body interaction is puzzling**. As introduced in Section 4, the development of four-body interaction (improper torsions) and five-body interactions are based on the topology. First, such analysis is purely qualitative. The authors did not provide further completeness proof or quantitative evidence about these interaction schemes in real-world data. Second, the reasons for deriving Eq (4)-(9) are not well explained. It is suggested to clarify how these components are motivated according to the topology analysis.\n\n\n2. **On the experimental evaluation**. Additionally, there are several aspects of the experiments that are concerned:\n    - The empirical performance is not consistently better than other baselines. Among the evaluated benchmarks, the proposed QuinNet cannot outperform the baselines significantly. For example, in MD17, the newly developed five-body interaction modules do not significantly improve performance. In rMD17, the best performance is diversely distributed among the compared models. Overall, the experimental evaluation does not well demonstrate the power of newly developed modules.\n    - The computation efficiency evaluation is missing. Although the authors provide complexity analysis, it is better to further show the time/memory cost comparison between the proposed QuinNet and baselines. Besides, the model parameters should also be provided for all compared models.\n    - The scale of the chosen benchmarks is rather small. Both the dataset size and sample size (number of atoms) are limited. It is suggested to further evaluate the proposed QuinNet on large-scale benchmarks, e.g., Open Catalyst Project [1].\n    - The ablation study. First, as shown in Figure 5, the inclusion of Five-body@I even induces further errors, which would make readers curious about whether such a phenomenon generally exists. Second, as introduced in VisNet, the improper angle was also considered. The authors should add further discussions and empirical comparisons between it and the newly proposed four-body interaction (improper torsion).\n\n\n3. **The writing does not meet the requirement of an acceptable paper in this conference**. First, Section 3.2 can be thoroughly extended (e.g., in the appendix) to introduce the background of force fields and highlight the importance of torsion potential, improper torsions, and higher-order many-body interactions. Second, there lack of formal descriptions of QuinNet. Figure 3 can hardly be understood by readers that are not familiar with the related works in this area. \n\n[1] Chanussot L, Das A, Goyal S, et al. Open catalyst 2020 (OC20) dataset and community challenges[J]. Acs Catalysis, 2021, 11(10): 6059-6072.\n    -\n\nQUESTIONS:\nPlease refer to the Weakness section to address the concerns.\n\nLIMITATIONS:\nThe authors did not discuss the limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to incorporate 5-body interactions into geometric deep learning models. They first analyze the topology of 5-body interactions and identify three 5-body angles. Then they propose an efficient way to incorporate these 5-body information into models. The complexity of the proposed QuinNet is still O(|N|), the same as many previous 2-body methods like PaiNN. The results are comparable to previous SOTA methods.\n\nSTRENGTHS:\nThis paper is well-written and easy to follow.\n\nThe experimental results show that the proposed method can perform well on most tasks. The ablation study in Section 5.4 and Figure 5 show that the proposed 5-body information indeed helps to model.\n\nWEAKNESSES:\nSee details in the Question part.\n\nQUESTIONS:\n1. About the motivation:   \n this paper aims to incorporate 5-body interactions into geometric deep learning models. However, based on my understanding, using up to 4-body (torsions) interaction is already complete [1][2] in terms of capturing the geometric structures. If this is correct, then why do we need these 5-body angles? In addition, if we can incorporate 5-body interactions, do we also need to incorporate 6-body interactions?\n\n2. About the complexity:   \nin Section 4.3, the authors claim that the complexity is O(|N|), as efficient as many 2-body methods like SchNet and PaiNN. But I think this complexity is not well explained. Using pseudocode/algorithm may be better to analyze the complexity. In addition to the analysis, I suggest the authors use some results to empirically verify the great efficiency compared to other baseline methods, e.g. the inference time, used memory, etc.\n\n3. About the tasks:   \nthis paper focuses on MLFFs, how about other molecular property prediction tasks, such as QM9 and OC20? I am wondering if this method is specially designed for MLFFs, or can be used on all 3D molecule tasks. In other words, why do the authors emphasize MLFFs? Is there any significant difference between MLFFs and other molecule property prediction tasks?\n\n4. Other related papers: many-body [3], MLFFs [4]\n\n5. The j, k in Figure 2 are confusing to me. For example, in (f), why not be i, j1, j2, j3, and k1?\n\n[1] ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs.  \n[2] GemNet: Universal Directional Graph Neural Networks for Molecules.  \n[3] On the Expressive Power of Geometric Graph Neural Networks.  \n[4] Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a machine learning force field that is a neural network with explicit interactions for up to 5-body terms.  The authors evaluate the model on a couple of public datasets and show demonstrate the competence or superiority of this new model compared to the state of the art in this field.\n\nSTRENGTHS:\nThe paper provides an important addition to a series of ever-improving machine learning potentials.  The contribution is clear and simple to understand at the high level, though the details are often unclear.  The benchmarks were compared against a set of reasonably strong published methods in this area.  In my opinion, if this work was presented in an unambiguously clear fashion and accompanied by code, it could be a strong contribution to this conference.  \n\n[The paper improved significantly following the first round of feedback from reviewers, so I'm raising my rating to a 7.]\n\nWEAKNESSES:\nThe complexity analysis is very limited.  How many total interactions did the typical molecule have as a function of their atoms, and how did the practical experimental complexity scale for the evaluation of these molecules.  One of the main reasons that 5-body terms were not used in traditional MD simulations was the poor scaling of the number of interactions one would need to calculate.\n\nThe MD simulation mentioned in section 5.1 and Fig 4 are not described anywhere.  The following sentences suggest that there would be some explanations in the supplement, but I couldn't find them: \"Additionally, we perform MD simulations using trained QuinNets as force fields and plot the distribution of interatomic distances h(r) for these 7 molecules in Fig. 4. Further details regarding additional settings can be found in the Supplementary Materials.\" \n\nThese sentences in the supplement, page2, are confusing or wrong: \"Similarly, five-body@III interaction (Fig. S1 (c)) is a special case of six-body interaction when nodes i and k4 in Fig. S1 (d) superpose each other. Thus, the QuinNet model captures all five-body interactions and a portion of six-body interactions, making it a versatile and comprehensive tool for modeling complex molecular systems.\"  There is no six-body interaction if two of the bodies are the same, and there is no physically acceptable case where two different atoms could superpose each other.\n\nThe code is not provided, so it is not possible for me to assess the reproducibility of this method.  The diagram in Figure 3 seems reasonable at the very high level, but it lacks the definitions of most of the terms annotated in the figure, thus rendering it confusing.  (What is $Y_l$? is it the set of all spherical harmonics $Y_{lm}$ for a given angular momentum $l$? What is $n_j$? What is $s_j$?  $W$?...)\n\nQUESTIONS:\nCould the authors add the presentation of the QM9 quantities estimated in the recent publication for Allegro?  (https://www.nature.com/articles/s41467-023-36329-y Table 3)\n\nHow long and how stable were the actual MD simulations?  What were the exact codes/protocols used?\n\nWhat is the practical performance of the model during evaluation?\n\nLIMITATIONS:\nNo potential negative societal impacts from this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}