Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Authors: Jiahua Rao
Dahao Xu
Wentao Wei
Yicong Chen
Mingjun Yang
Yuedong Yang
Venue: ICML 2025 poster (2025)
Keywords: Molecular Property Prediction
Many-body System
Quadruple Attention
Molecular Force Fields
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: While Graph Neural Networks and Transformers have shown promise in predicting molecular properties, they struggle with directly modeling complex many-body interactions. Current methods often approximate interactions like three- and four-body terms in message passing, while attention-based models, despite enabling direct atom communication, are typically limited to triplets, making higher-order interactions computationally demanding. To address the limitations, we introduce MABNet, a geometric attention framework designed to model four-body interactions by facilitating direct communication among atomic quartets. This approach bypasses the computational bottlenecks associated with traditional triplet-based attention mechanisms, allowing for the efficient handling of higher-order interactions. MABNet achieves state-of-the-art performance on benchmarks like MD22 and SPICE. These improvements underscore its capability to accurately capture intricate many-body interactions in large molecules. By unifying rigorous many-body physics with computational efficiency, MABNet advances molecular simulations for applications in drug design and materials discovery, while its extensible framework paves the way for modeling higher-order quantum effects.

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Jiahua Rao
Dahao Xu
Wentao Wei!
Yicong Chen
Mingjun Yang? Yuedong Yang 13
Abstract
Traditional approaches, including structural fingerprints and descriptor- based approaches (Rogers & Hahn, 2010; Jaeger et al., 2018) 8), encode molecules using simplified heuristics that emphasize one- and two-body interactions (Song et al., 2020; Rao et al., 2022a), for instance, atomic connectiv- ity or pairwise bond relationships. While these represen- tations efficiently capture coarse structural patterns, they inherently neglect complex quantum mechanical phenom- ena. Such simplifications limit their utility in modeling properties rooted in the synergistic interplay of multiple atoms or bonds (Hansen et al., 2015).
While Graph Neural Networks and Transformers have shown promise in predicting molecular prop- erties, they struggle with directly modeling com- plex many-body interactions. Current methods of- ten approximate interactions like three- and four- body terms in message passing, while attention- based models, despite enabling direct atom com- munication, are typically limited to triplets, mak- ing higher-order interactions computationally de- manding. To address the limitations, we intro- duce MABNet, a geometric attention framework designed to model four-body interactions by fa- cilitating direct communication among atomic quartets. This approach bypasses the compu- tational bottlenecks associated with traditional triplet-based attention mechanisms, allowing for the efficient handling of higher-order interactions. MABNet achieves state-of-the-art performance on benchmarks like MD22 and SPICE. These improvements underscore its capability to accu- rately capture intricate many-body interactions in large molecules. By unifying rigorous many-body physics with computational efficiency, MABNet advances molecular simulations for applications in drug design and materials discovery, while its extensible framework paves the way for modeling higher-order quantum effects.
address these limitations, modern machine learning frameworks increasingly rely on quantum mechanical prin- ciples to predict molecular properties, necessitating explicit modeling of many-body interactions (Unke et al., 2021), These include not only classical covalent and non-covalent atom-atom interactions (two-body) but also higher-order terms such as bond-angle distortions (three-body), torsional potentials (four-body), and electronic delocalization effects (e.g., in conjugated r-systeem)) For example, the quan- tum level, properties such dipole moments, polarizability, and excitation energies emerge from entangled multi- aatti correlations that resist decomposition into pairwise terms. Ignoring these interactions can lead to inaccurate predictions of molecular properties (Yang & Zhou, 2008).
To mimic the many-body terms, Graph Neural Networks (GNNs) (Schütt et al., 2018; Gasteiger et al., 2020b) and Transformers (Liao & Smidt, 2023) have emerged as foun- dational architectures by leveraging node-edge frameworks to represent atoms and bonds as discrete entities. How- ever, their reliance on pairwise correlations inherently limits their capacity to encode multi-atom interactions, such as three-body angle dependencies or four-body torsional terms. While these architectures approximate higher-order effects through the iterative aggregation of local interactions, this in- ductive bias introduces systematic errors in capturing global quantum behaviors, particularly in systems with strong elec- tronic cooperativity or long-range correlations.
1. Introduction
Molecular representation is pivotal in cheminformatics, crit- ically influencing the accuracy of property prediction in drug discovery and materials science (Fourches et al., 2010).
Eqqual contribution 'School of Computer Science and Engi- neering, Sun Yat-sen University, China Shenzhen Jingtai Tech- nology Co., Lid. (XtalPi), Shenzhen, China "Key Laboratory of Machine Intelligence and Advanced Computing, Sun Yat- sen University, China. Correspondence to: Yuedong Yang <yangyd25@
Various architectures have recently been proposed to mit- igate these limitations. For example, SE3Set (Wu et al., 2024) operates on hypergraphs constructed from overlap- ping fragments to provide natural representation of many-
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
body phenomena. ViSNet (Wang al., 2024) and Quin- Net (Wang et al., 2023b) implicitly capture many-body geo- metric features in message passing, aligning with the force fields used in classical molecular dynamics (MD). Despite their strengths, these methods primarily rely on message- passing mechanisms that approximate multi-body interac- tions rather than explicitly modeling them, often introducing trade-offs in accuracy or computational efficiency.
demonstrating its ability to accurately model intricate many-hody interactions across diverse and complex molecular systems.
2. Related Work
2.1. Molecular Property Predictions
Graph neural networks (GNNs) (Gilmer et al., 2017; Song et al., 2020; Rao et al., 2022b; 2024a) have revolutionized molecular property predictions by enabling accurate mod- cling of atomic interactions. They (Schütt al., 2018; Gasteiger et al., 2020b) naturally encode atomic systems as graphs, preserving permutation invariance through mes- sage passing to predict molecular energies directly from quantum states. However, their limited ability to incorpo- rate geometric symmetries (e.g.. SE(3)E((3 equivariance) restricts generalization across conformations and coordinate systems.
To address these challenges, we propose MABNet (MAny- Body interaction Network), novel four-body direct com- munication attention mechanism specifically designed for quantum many-body systems. Unlike traditional methods that rely on approximations or implicit representations of high-order interactions, our approach explicitly models com- plex interactions involving up to four bodies. This explicit modeling not only captures the intricate geometric and struc- tural dependencies within molecular systems but also en- sures computational efficiency. making it suitable for larger molecular simulations, These geometric features also en- hance the E(3) equivariant message-passing process by pro- viding a more expressive and accurate representation of the molecular system's 's symmetries. By explicitly integrating higher-order interactions into the architecture, our approach overcomes the limitations of existing methods, paving the way for more precise molecular property predictions.
Recent advances address this by integrating geometric in- ductive biases (Satorras et al., 2021; Thölke & Fabritiis, 2022; Rao et al., 2024b). Equivariant architectures (Schütt et al., 2021) use irreducible representations to enforce symmetry-aware tensor transformations, while invariant methods (Montavon al., 2012) augment features with de- scriptors like distances or angles. Graph transformers (Liao & Smidt, 2023) have also been adapted to geometric do- mains: Equiformer combines equivariant irreps with at- tention mechanisms, achieving accurate force and energy predictions.
Our approach achieves state-of-the-art (SQTA) performance on challenging benchmarks, including MD22 and SPICE, showcasing its capability to accurately capture intricate many-body interactions, even in larger and more complex molecular systems. By excelling across diverse molec- ular structures, our method highlights its robustness and adaptability, addressing the limitations of existing models that struggle with high-order interactions. This exceptional performance not only validates its effectiveness but also underscores its transformative potential in computational chemistry. By providing a more accurate and physically grounded framework for modeling complex systems, our method paves the way for advances in molecular dynamics simulations, quantum chemistry, and related fields.
2.2. Many-body Interactions Modeling
Traditional approaches approximate many-body interac- tions via pairwise decompositions (atom-atom, atom- etc.) (Unke et al., 2021), but struggle to capture coopera- tive phenomena like dipole moments, polarizability, and excitation energies, which require explicit multi-atom corre- lation modeling. Neural networks must preserve SE(3)E(3)) equivariance while directly representing hierarchical rela- tionships, avoiding reliance on pairwise approximations.
atom-bond, bond,
The contributions can be summarized as follows:
Recent methods show partial progress: SE3Set (Wu et al., 2024) employs hypergraphs to model delocalized effects but introduces computational overhead from fragment decompo- sition, while VisNet/QuinNet (Wang et al., 2024; 2023b) ap- proximate multi-body terms via geometric message passing, inheriting classical force fields' limitations. Emerging ap- proaches like TGT (Hussain et al., 2024) enable direct triplet communication but lack ability limited three-node interactions. Other strategies, such as GEM-2's s (Liu et al., 2022) axial attention over edge pairs and AlphaFold's (Jumper al., 2021) ccalar-bssd triangular updates, either constrain interaction granularity to triplets or rely on scalar approximations. Collectively,
We introduce a novel attention mechanism that explicitly models complex higher-order interactions, enabling the direct communication of four-body interactions in molec- ular quantum many-body systems.
Our model generates rich geometric features through the four-body direct communication attention mechanism in the E(3) equivariant paassigg process, improving the expressivity and accuracy of our model.
and seal-
The proposed method achieves SOTA performance on challenging benchmarks, including MD22 and SPICE,
2

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
puting interactions as:
Table 1. Comparison of different many-hody interactions.
Say = Σ R
(aittUgh)) = Softmax ( T kyk),
(2)
Methods
Axial Attn.
Tria. Update
Triplet Attn.
MABNet
where g Kÿt queris and keys for edge pairs. Though this extends complexity to O(LV)), it neglects critical 3rd- order geometric features (e.g., the angle 0xyk) and the direct influence of the (i, k) pair. GEM-2 (Liu et al., 2022) par- tially addresses this by 3rd-order positional encodings, but it remains limited to implicit angular dependencies, reducing accuracy in many-body systems.
Ops.
3-body
3-body x
3-body V
4-body "
Attn.
V
Equi.
X
Values
Vectors O(LN))
Scalars ((
Vectors O(AA7))
Vectors ((NA)))
Comp.
these methods fall short of enabling explicit and equivari- ant modeling of many-body effects (e.g.. torsions, covalent hybridization), In general, these limitations underscore the need for architectures that unify E(3) -quivaarieee preserva- tion with scalable many-body interaction modeling.
Triangular Update (AlphaFold) AlphaFold's triangu- lar update aggregates scalar features (Jumper al., 2021; Lu et al., 2022; Abramson et al., 2024) from edge triplets (ij, jk, ik) via tensor products, achieving com- plexity ((1))..37)) through optimized matrix multiplica- tion (Ambainis et al., 2015). However, this method lacks attention- -asdd gating: all triplets contribute equally, re- gardless of chemical relevance. Additionally, scalar fea- tures discard directional information (e.g., vectorial forces), and unbounded summation over variable-sized graphs intro- duces instability.
3. Preliminaries
3.1. Many-body Interactions
The properties of molecular system, such as energy, can be expressed as the sum of contributions from various factors, including bonds, bond angles, torsion angles, and nonbond- ing interactions (such as electrostatic and van der Waals forces) (Unke et al., 2021). These include three-body in- teractions (e.g., bond angles, where the energy depends on three linked atoms), four-body interactions (e.g., tor- sion angles, involving four sequentially bonded atoms), and oigeer-order collective effects (e.g., electronic polarization).
Triplet Graph Transformers (TGT) Triplet- meth- ods (Hussain et al., 2024) explicitly model 3rd-order inter- actions (i, k), enabling direct information flow between edges (i.j) and (j.k). k). The attention mechanism becomes:
Sij Σ 8yk Ouf = Softmax( (4, +0(0r))),
Therefore, the total energy E of an N-atom system is:
where Th kyt are queries and keys for the triplet of nodes, and ((0iz)) encodes the angular features. While this im- proves expressivity, the ((VV)) complexity persists, and equivariance is sacrificed by scalarizing geometric features.
+
Σ Kjjc
E Gk
+
Σ
EX
Ghi
+.... (1)
+
where F(n) denotes n-body terms. Classical force fields approximate E(3), EX4) with handcrafted functions, while MLFFs learn them directly.
Proposed Many-body Attention Network (MABNet) Our method generalizes attention to four-body interactions (i.j, k. I), critical for modeling torsional angles Φiskt and non-local polarization effects. To mitigate the combinato- rial explosion ((W])), we employ: (1) spatial sparsity: restrict interactions to quartets within a cutoff radius rei (2) trantum-inspiree pruning: prioritizes edges critical for multi- body interactions (e.g., dihedrals).
3.2. Complexity Analysis
Node-to-Node Attention (Pairwise) Standard attention mechanisms (Vaswani, 2017; Chen et al., 2021) in Trans- formers operate on node pairs (i.)), yielding computa- tional complexity of (( where N is the number of atoms. While efficient, this approach restricts expressivity to I-GWL (Graph Weisfeiler-Lehmar equivalence (Joshi et al., 2023). failing to distinguish molecular substructures requiring higher-order interactions. This limitation arises be- cause pairwise attention cannot explicitly model multi-body geometric relationships like angles or torsions.
The resulting complexity scales as O( [E'), where E < 6| is the average neighborhood size. Crucially, our attention retains SE(3)--quivariance by operating on vector-valued queries, keys, and values:
Sij
Σ k.
aijki Uzhl,
(4)
Axial Attention
Axial attention (Liu et al., 2022) gener- alizes self-attention to pairs of edges () and (j. k), com-
< 9 KjN + (obgaa))))
aijk = Softmax
(5)
Vd
3

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
(a) Model Overview
Product
Node
Embeoding
Sum
Message Passing Block
Neightbor
Embaadinn
Equivariont Graph
Sigmold
x
Output Module
energy
Scalar Feature
Radial Basis Expansion
3D mol.
pos
Edge Eineedding
forces
Vector Feature
Edge Feature
X N
Equivariant Feature
(b) Equivariant Graph Message Passing Block
(c) Many-body Attention
Heads
Nodi-Ediee Edge
NW
Many bodyy
Attomiia
X
dx
ONN
Rts
Softmax
Node=
FWE
Scalar Scala--Vent Veco- Interaction
dv
+
101
Layer Narn
K jkc
DEL
muskk
pNR
Pooling
Edge Updbtte
Box
cttn
e/11
jkl
Pigure 1. The overall framework of our method.
where (b((sjz)) encodes torsional features via spherical har- monics. This framework combines the expressivity of ex- plicit four-body terms with the efficiency of equivariant sparsity, addressing the limitations of prior works (Table I).
E represents the number of edges, I. (fmax 1)2 1
corresponds to the spherical harmonics expansion level, and D is the feature dimensionality.
Scalar and Vector Node Feature Embedding Scalar features are derived through two sequential stages: (I) atomic embedding directly encodes atomic types into high- dimensional vectors, followed by (2) neighborhood aggre- gation that integrates adjacent atom data via radial basis function (RBF)-expanded distances, filtered through cutoff function and combined with neighbor features via element- wise multiplication. The scalar features are generated by concatenating the initial atomic embeddings with the aggre- gated neighborhood representations:
4. Methodology
4.1. Model Architecture
Figure illustrates our framework. The input is a 3D molec- ular structure defined by atomic types and coordinates. The model first embeds atomic and edge features into scalar features. vector features. and edge features. Inspired by VisNet (Wang et al., 2024), these features are iteratively up- dated through equivariant graph message passing (EquiMP) blocks, which enforce E(3)-equivariance during trans- formations. The key innovation to our model the many- body attention module within each EquiMP block, which explicitly models four-body interactions to capture complex quantum effects. Finally, the updated scalar and vector fea- tures are processed to predict total energy, and atomic forces are derived as the energy gradient with respect to atomic coordinates, ensuring physical consistency.
Σ (Ø(dy)) (Wad b,.) hg)) (6) JEN())
Jeuth
where 3 denotes element-wise multiplication. W. and b, are learnable parameters, and M( denotes the set of neighboring nodes of node i. The scalar feature of node is computed by concatenating its initial feature h; with the aggregated neighbor features Tnbho followed by a linear transformation:
4.2. Feature Embedding
23 Wh(hi knbn) b.,
As shown in Figure (()), the feature embedding layer trans- forms 3D molecular structures into three components: scalar features æ E TR. NDD vector features 7 E N and edge features e REXD where N is the number of atoms,
where the denotes the concatenation operation. W. and b, are learnable parameters. The vector features ił are ini- tialized zero tensors to preserve equivariance.

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Edge Feature Embedding Edge features are computed by combining the scalar features of connected nodes with edge attributes transformed via RBF expansion into higher- dimensional space. This process captures complex interac- tion patterns, enhancing the model's ability to understand intricate network relationships.
The final node update aggregates information from all neigh- boring components:
Σ JEN(() R.J
Σ
(13)
attn
where (2) denotes the neighborhood of node i. This multi- head attention formulation enables simultaneous modeling of both direct many-body (node-node, node-edge-node in- teractions, effectively capturing complex many-body re- lationships while maintaining permutation equivariance through invariant tensor operations.
Full implementation details for scalar, vector, and edge embeddings are provided in Appendix A.I.
4.3. Mody-body Attention Module
As shown in Fig. 1(b), the scalar features are updated through many-body attention module in each EquiMP layer. Let x! E R and e R denote the node and edge features at layer I, respectively. These features interact through two distinct relationship tensors:
4.4. Equivariant graph neural message passing
As shown in Fig. 1(b), each EquiMP layer performs feature updates through an additive residual scheme that preserves geometric equivariance:
NNN = a/
1yME = =1 E
a' AA'' tth - +
(14)
where N, F., and D correspond the number of atoms, edges, and feature dimensions. The node-node interac- tion tensor JNN captures pairwise atomic interactions (two- body terms), while the node-edge interaction tensor fN encodes three-body relationships through edge-mediated connections between node triples (j, k. l).
The update terms AZ', AF, Ad' are computed through equivariant transformation blocks and respect the geometric constraints.
This architecture ensures all feature updates transform equiv- ariantly under SF(3) transformations while enabling rich information flow between scalar and geometric represen- tations. The residual connections help preserve physical constraints while allowing deep feature integration across multiple interaction scales.
Inspired by the cross-attention mechanisms (Vaswani, 2017), establish asymmetric information fusion by designating different relationship tensors query (Q), key (K), and value (V) components. As shown in Fig. 1(c), the interaction process begins with layer normalization of both tensors,
followed by learned linear projections:
Scalar-Vector Interaction The many-body attention mod- ule is applied to the scalar features x to directly enable four-body communication for feature updates. The updated scalar feature Tatta of node i is divided into three compo- nents: z!, x7. and xद. Mcanwhile, the vector feature τ is split into two parts: T7 and uf. The updated scalar feature Ar, is derived by integrating u7!
9. = WW KyKt = W GiP
ViKi
W.
(9)
bakt= = WD Jikt 9aki = WO fiki
(10)
where Wº, W W BBB WG are learnable projection matrices. The bias term but and gating vector fakt enable adaptive modulation of attention patterns.
Az = (Wo1 Ja + O +
(15)
where W. and b,2 are learnable parameters, and @ denotes the Hadamard product.
For each attention head, the interaction weights are com- puted through a gated softmax operation:
The update of vector features employs the Hadamard prod- uet of the updated scalar feature 2a and the linearly trans- formed v in addition to incorporating the equivariant fea- ture n.z.
Kiykykt Vd
0ijkt Softmax
best)
G(gat),
(11)
2 Σ k.
(12)
A
(Wi2 1
of
+
bo2)
3 x
f:g.
pig = o
F
(16)
where ((gnt) (gaz) denotes the gate function. This gat- ing mechanism allows the model to selectively control the flow of information, enhancing its ability to learn complex, adaptive representations by emphasizing specific interac- tions.
Fl
where Wie and b,2 are learnable parameters. Fyy represents the position vector from nodes to j. The equivariant fea- ture f is derived from the Hadamard product of s with ūf
5

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
and the Hadamard product of 5² with the equivariant infor- mation, where s and x² are projections of the pre-update a used as residuals.
5. Experiments
5.1. Experimental Settings
In this section, we briefly introduce the benchmarks, base- lines, and implementation details related to our experiment. Further details are presented in the Appendix A.3-A.4.
Edge Update The update of edge features employs vector rejection, which calculates the component of node features perpendicular to the edge direction, thereby capturing the influence of relative node positions on edge features. The dot product wdot of vector rejections in two directions is then computed:
Datasets We consider two challenging benchmarks of MD22 and SPICE, following (Wang et al., 2024) and (East- man et al., 2023). MD22 was introduced by (Chmiela et al., 2023), containing 42-atom peptide to 370-atom nan- otube, with high-resolution sampling at 400-500 K using the PBE+MBD (Perdew et al., 1996; Tkatchenko et al., 2012) framework for energy and force computations. The training and testing splits used in MD22 are consistent with those in methods such as LSR-MP (Li et al., 2023). SPICE, collected by (Eastman al., 2023), offers approximately one million conformations for pharmaceutical molecules, dipeptides, and solvated amino acids. SPICE spans a wider range of chemical elements compared to MD22 and includes both covalent and non-covalent interactions, with energy and force calculations performed at the TZVPPD level of theory. Unlike MD22, which focuses training and testing within a single molecule, SPICE enables -ross-molecular training and testing, making it a more comprchensive benchmark for assessing the general- izability of molecular dynamics simulation models. The SPICE dataset follows a consistent 8:1:1 split for training, validation, and testing. For more details about the datasets, please refer to Appendix A.3.
Wdot = R(U, F(y)
R(ū -rg)::
(((e) Fig
(17)
(G), = L(
|
where T(5))) denotes a linear transformation of i7 The computation for R(ūj,- -Ryy) follows a similar process.
Finally, the edge feature Eij undergoes a nonlinear trans- formation, and the Hadamard product with the dot product result dot yields the updated edge feature Aey:
eay WW. Ctg + bb) ① Wdot:
(18)
where We and be are learnable parameters.
4.5. Output Module
Through iterative processing via EquiMP layers that inte- grate Many-body Attention mechanisms and Equivariant Graph Message Passing operations, the system generates updated scalar features r and geometrically aware vector features v. These dual-modal features emerge as disentan- gled latent descriptors encoding both invariant chemical properties and directional spatial interactions within the molecular system. These features are processed by the Out- put Module to ultimately predict the energy E and atomic forces F of the molecular system. The Output Module com- prises multiple equivariant output blocks, where scalar and vector features interact within each block as follows:
Baselines We compared our method with the state- of-the-art baselines, including both GNN/Transfomeers: based methods such as SchNet (Schütt et al., 201 8), sGDML (Chmiela et al., 2023). PaiNN (Schütt et al., 2021), ET (Thölke & De Fabritiis, 2022), So3krates (Frank et al., 2022), MACE (Batatia et al.), and Equiformer (Liao Smidt, 2023). as well as methods designed for modeling many-body interactions, such as ViSNet (Wang et al., 2024) and LS (Li et al., 2023). Additional baseline meth- ods and their detailed descriptions are provided in Ap- pendix A.4.
G = MLP (all + bo)).
(19)
Here, the MLP consists of two linear layers with a SiLU activation function, || denotes the concatenation operation, and W. and b, are learnable parameters. Following pooling across multiple output blocks, the final output of the Output Module yields the energy E. The force F; acting atom is computed as the negative gradient of E with respect to its initial input coordinates pos,, expressed as:
Implementation details All models are implemented in PyTorch (Paszke et al., 2019) and trained using the Adam optimizer (Kingma & Ba) with Mean Squared Error (MSE) loss, unless otherwise specified. Training begins with a lin- ear learning rate warm-up phase, followed by systematic reduction of the learning rate using a decay factor when- ever the validation loss stagnates. All experiments are con- ducted on either NVIDIA A800 Tensor Core GPUs. Our code is publicly available at https: / / com/ bi Looeddd22 AIIMABNMMABBB Additional details on hyperparam- eter settings are provided in Appendix A.5.
aE
F.
(20)
Opos,
6

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Table 2. Mean absolute errors of energy (kcal/mol) and force (kcal/mol/A) for 5 large-scale molecules on MD22.
Nc-e
AT-AT
AT-AT-CG-CG
DHA
Stachyose
Model
Energy
Forces
Energy
Forces
Energy
Forces
Energy
Forces
Energy
Forces
sGDML
0.3902
0.7968
0.7235
0.6911
1.3885
0.7028
1.3117
0.7474
4.0497
0.6744
PaiNN
0.1168
0,2302
0.1673
0.2384
0.2638
0.3696
0.1151
0.1355
0.1517
0.2329
ET
0.1121
0.1879
0.1120
0.2036
0.2072
0.3259
0.1205
0.1209
0.1393
0.1921
So3krates
0.337
0.224
0.178
0.216
0.345
0.332
0.379
0.242
0.442
0.435
Allegro
0.1019
0.1068
0.1428
0.0952
0.3933
0.1280
0.1153
0.0732
0.2485
0.0971
MACE
0.0620
0.0876
0.1093
0.0992
0.
0.1153
0.1317
0.0646
0.1244
0.0876
Equiformer
0.0828
0.0804
0.1309
0.0960
0.1510
0,1252
0,1788
0.0506
0.1404
0.0635
ViSNet
0.0796
0.0972
0.1688
0.1070
0.1995
0.1563
0.1526
0.0668
0.1283
0.0869
Equiformer-I.RMM
0.0780
0.0887
0.1007
0.0881
0.1335
0.1065
0.0878
0.0534
0.1252
0.0632
ViSNet-LSRM
0.0654
0.0902
0.0772
0.0781
0.1135
0.1063
0.0873
0.0598
0.1055
0.0767
MABNet
0.0534
0.0773
0.0637
0.0731
0.1862
0.1251
0.0618
0.0502
0.0895
0.0752
5.2. Results on MD22 Dataset
5.3. Results on SPICE Dataset
To evaluate the capacity of higher-order many-body interac- tion modeling in complex molecular systems, we first bench- marked our approach against the comprehensive MD22 dataset. As demonstrated in Table 2, MABNet achieved state-of-the-art performance, outperforming leading meth- ods such as ViSNet and LSR-MP with an average 20% reduction in mean absolute errors (MAEs) across both ergy and force predictions. Notably, MABNet attained the lowest MAEs in 7 of 10 metrics, including critical bench- marks for the glycoside DHA (56 atoms) and the 189-atom Stachyose system. This advancement arose from our many- body attention module, which directly encoded high-order interactions, bypassing the approximation errors inherent in existing frameworks that relied on sequential message- passing or aggregated lower-order expansions.
The SPICE dataset's diversity provided a rigorous test for evaluating a model's ability to generalize across molecular scales and interaction types. As shown in Table 3, our frame- work achieved state-of-the-art performance in all categories, demonstrating its capacity to unify many-body interaction modeling within a single architecture. Specifically, for dipeptides, where torsional flexibility and hydrogen bonding introduce complex many- effects, model achieved force MAE of 7.7 VeÀ, outperforming TensorNet by 67%. This improvement highlighted the limitations of meth- ods that approximate angular and torsional terms through sequential message-passing In solvated amino acids, a regime demanding explicit modeling of solvent-solute polar- ization, our framework reduced force errors by 52% (31.9 66.4 V/Ā) compared to models like DimeNet++ and ET. These baselines, which lacked explicit higher-order interaction terms, struggled to resolve the dynamic inter- play between solvent molecules and solute charge distri- butions. Critically, the cross- molsecll robustness of our approach stemmed from the many-body attention module, which adaptively resolved both local (e.g., covalent bonds) and noc-local (e.g., solvation shells) interactions through direct four-body communication. Unlike nessage-passing frameworks, our method avoided accuracy degradation in heterogeneous systems by explicitly encoding multi-body physics.
While MABNet achieved superior performance across most systems, its energy MAE for the 370-atom carbon nanotube (0. (0.1862 kcal/mol) trailed ViSNet-LSRM (0.1135 135 kcal/mol). We attributed this gap to the dominance of pairwise inter- actions in symmetrie systems with periodic geometries. In such cases, methods like LSR-MP, which prioritize scalable approximations of lower-order effects, gained an advantage by focusing on dominant pairwise terms.
MABNet achieved superior energy predictions across all sys- tems, but force MAEs occasionally lagged behind the base- lines (e.g.. Equiformer-L.SR on Stachyose forces: 0.0632 vs. MABNet's 0.0752). This divergence reflected the sen- sitivity of force calculations, which depended energy surface gradients, in high-order expansions. While our mod- ule captured global energy landscapes robustly, fine- ine-ggrined force accuracy might have benefited from local geometric descriptors with subtle atomic displacements.
5.4. Ablation Study
To better understand our model, we conducted ablation stud- ies focusing on model architecture and the combinations of many-body nodes. Table summarized performance across three chemically distinct systems: dipeptides, sol-
7

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Table 3.
Mean absolute errors of energy (meV) and force (meV/A) for 8 large-scale molecules on SPICE.
Molecular
SchNet
+imeNet++
ET
TensorNet
MABNet
Energy Forces
220.5
59.0
57.6
68.1
32.4
PubChem Set
150.7
59.1
88.3
49.4
26.5
Energy Forces
222.5
60.6
55.8
32.5
27.2
PubChem Set
144.1
58.5
80.8
39.2
21.7
Energy 217.0
47.7
45.2
34.1
28.9
PubChem Set
Forces
138.9
45,9
65.9
36.2
20.5
Energy Forces
299.9
44.6
40.6
32.8
26.7
PubChem Set 4
153.3
42.3
58.8
35.1
20.3
Energy Forces
266.7
53.3
45.6
31.4
29.2
PubChem Set
144.8
52.7
69.7
35.1
20.6
Energy 242.9
32.8
30.7
23.8
17.3
PubChem Set
Forces
135.6
31.2
43.7
27.7
14.1
Dipeptides
Energy 386.2
169.9
26.0
28.7
Forces
118.2
45.8
34.8
23.1
7.7
Energy Forces
1438.3
460.1
224.4
149.1
98.2
Solvated Amino Acids
318.1
173.7
118.2
66.4
31.9
Energy 411.8
116.0
65.7
50.1
33.7
Average
Forces 163.0
63.7
70.0
39.0
20.4
Table 4. Mean absolute errors of energy and force for different attention mechanisms on Dipeptides and Solvated Amino Acids (energy in mcV and force V/A, and for Ac-Ala3-NHMo (energy kcal/mol force in kcal/mol/A).
Dipeptides
Sol. Amino Acids
Ac-Ala3-NHMe
Methods
Energy
Forces
Energy
Forces
Energy
Forces
3-body Attn. (w/o Equi.)
151.5
59.4
3-body Attn.
12.3
10.1
130.2
48.4
0.061
0.082
body Attn. (1.5N Edges)
11.2
8.4
119.3
36.3
0.058
0.080
4- b-byyy Attn. (3N Edges)
10.0
7.9
98.6
32.5
0.055
0.078
4-body Attn. (6N Edges)
9.6
7.7
98.2
31.9
0.053
0.077
vated amino acids, and the tripeptide Ac-Ala3-NHMe.
vated systems. This showed that while even sparse 4-body graphs (1.5N edges) captured most many-body dependen- cies, denser connectivity refined accuracy by explicitly mod- eling weaker but chemically relevant interactions (e.g., van der Waals contacts in Ac-Ala3-NHMe). Crucially, our effi- cient attention mechanism minimized computational over- head, with 6N edges incurring only a 1.Ix runtime increase over 1.5N, shown in Table 5.
Removing equivariance from the 3-body attention module ("3-body Attn. (w/o Equi.)", similar to TGT) degraded force prediction accuracy on solvated amino acids by 23% (59.4 vs. 48.4 V/À), demonstrating that roto-translation equiv- ariance was critical for modeling cecentation-depended in- teractions like hydrogen bonding and solvation. Extending 4-body attention while maintaining equivariance further reduced force errors by 33% (48.4 vs. 32.5 me V/À), con- firming that higher-order terms were essential for resolving multi-center polarization effects.
5.5. Computational Cost and Analysis
further validate the efficiency of our approach, a tripep- tide Ac-Ala3-NHMe was used quantify computational efficiency. Table compared the efficiencies ('3-body Attn. (w/o Equi.)', similar to TGT), 3-body attention, and various edge counts of 4- 4-body attention in terms of memory con- sumption, training speed, and inference speed. The results
Increasing the edge density in 4-body attention from 1.5N to 6N proge--prgyyssivelly improved performance, with foree MAEs decreasing by 8% (7.9 vs. 7.7 V/À) for dipeptides and 2% (32.5 31.9 meV/A) for sol-
8

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
of the ablation study, presented in Table 4, revealed that the mean absolute error (MAE) for the energy and forces prediction using 4-body attention with 6N edges is markedly lower than that of the 3-body Attn. (w/o Equi.). Notably, this improvement in prediction accuracy came at a modest cost, with memory consumption increasing by only 6.8%, and no substantial degradation in training speed. Addition- ally, we performed a comparative analysis of training and inference costs with ViSNet. Our results indicated that the improvement in prediction accuracy came at modest cost, with no significant degradation in training speed. A de- tailed analysis of computational efficiency is provided in Appendix B.2.
Impact Statement
Our work aims to improve chemical property prediction, which can accelerate the discovery of new beneficial materi- als and drugs. This has the potential greatly benefit soci- ety by enabling the development of cheaper, safer, and more effective medicines. We would make the code open-source so that domain experts can further improve and validate the models before real-world deployment.
References
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pp. 1-3, 2024.
Table 5. Comparison of memory consumption, training speed for diffcrent attention mechanisms on the Ac-Ala3-NHMe dataset.
Ambainis, A., Filmus, Y., and Le Gall, F. Fast matrix multiplication: limitations of the senpersmmtthwiinngee method. In Proceedings of the forty-seventh annual ACM symposium on Theory of Computing, pp. 585-593, 2015,
Type of Many-body Attn.
Memory (MiB)
Training (it/s)
Methods
ViSNet
15962
4.44
3-body
16542
3.93
Batatia, I., Kovács, D. P., Simm, G. N., Ortner, C.,, and Csányi, G. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields (2022). URL https://arxiv. org/abs/2206.0769)
4-body (1,5N Edges)
17264
2.78
MABNet
4-body (3N Edges)
17400
2.66
4-body (6N Edges)
17758
2.35
Chen, J., Zheng, S., Song, Y., Rao, J., and Yang, Y. Learn- ing attributed graph representation with communicative message passing transformer. In Zhou, Z.-H. (ed.), Pro- ceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 2242-22444 In- ternational Joint Conferences on Artificial Intelligence Organization, 8 2021.
6. Conclusion
By explicitly modeling four-body interactions through a geometric attention mechanism, MABNet advances molec- ular property prediction with quantum-mechanical accuracy, as demonstrated by state-of-the-ee results on MD22 and SPICE. Future work will extend this framework to five- body and higher-order terms, enabling precise modeling of systems governed by entangled electronic states (e.g.. tran- sition metal catalysts) or cooperative solvation effects. To balance accuracy and scalability, we will explore adaptive quantum quantum-inspired pruning strategies that prioritize dominant interactions in large systems. These advancements could redefine computational chemistry workflows, accelerating drug discovery through simulations that close the gap be- tween empirical and ab initio accuracy.
Chmiela, S., Tkatchenko, A., Sauceda, H. E.,, Poltavsky, I., Schütt, K. T., and Müller, K.-R. Machine learning of ac- curate energy energy-conserving molecular foree fields. Science advances, 3(5):e1603015, 2017.
Chmicla, S., Vassiley-Galindo, V., Unke, O. T., Kabylda, A., Sauceda, H. E., Tkatchenko, A., and Müller, K.- R. Accurate global machine learning force fields for molecules with hundreds of atoms. Science Advances, (2)cadf08773, 2023.
Eastman, P., Swails, J., Chodera, J. D., McGibbon, R. T.,
Acknowledgements
Zhao, Y., Beauchamp, K. A., Wang, L.-P., Simmonett, A. C., Harrigan, M. P., Stern, C. D., et al. Openmm Rapid development of high performance algorithms for molecular dynamics. PLoS computational biology, 13(7): el e1005659, 2017.
This study has been supported by Guangdong S&T Pro- gram (202480101040005) Guangdong S&T Program (2023B11 10130002),, Shenzhen Science and Technology (10022222205142220000000
Plan Project Science and Technology Talent Support Program of Guangdong Precision Medicine Application Association ASSTGIDPMA2202502))
and Young
Eastman, P., Behara, P. K., Dotson, L., Galvelis, R., Herr, J. E., Horton, J. T., Mao, Y., Chodera, J. D., Pritchard, P., Wang, Y., et al. Spice, a dataset of drug-like
9

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
molecules and peptides for training machine learning potentials. Scientific Data, 10(1):11, 2023.
Kingma, D. P. and Ba, J. In Bengio, Y. and LeCun, Y. (eds.). ICLR (Poster).
Fourches. D., Muratov, E., and Tropsha, A. Trust, but verify: on the importance of chemical structure curation in cheminformatics and qsar modeling research. Journal of chemical information and modeling, 50(7):1189 2010.
Langley, Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Frank, T., Unke, O.,, and Müller, K.-R. So3krates: Equiv- ariant attention for interactions arbitrary length-scales in molecular systems. Advances in Neural Information Processing Systems, 35:29400-2994 29411,, 2022.
Li, Y., Wang, Y., Huang. L., Yang, H.,, Wei, X., Zhang, J., Wang, T., Wang, Z., Shao, B., and Liu, T.-Y. Long-short- range message- passing: A physics-informed framework to capture non-local interaction for scalable molecular dynamics simulation. arXiv preprint 2:444 2023.
Gasteiger, J., Giri, S., Margraf, J. T., and Günnemann, S. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv preprint arXiv:2011.14115. 2020a.
Liao, Y.-.. and Smidt, T. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Represen- tations, 2023. URL https: / openreview net/ forum?
Gasteiger, J., GroB, J., and Günnemann, S. Direc- tional message passing for molecular graphs. In In- ternational Conference on Learning Representations, 2020b. URL htttttt/pppeereiiieeee net/forum! id=BleWbxStPH.
fARgOTD..
Liu, L., He, D., Fang, X., Zhang, S., Wang, F., He, J., and Wu, H. Gem-2: Next generation molecular property prediction network by modeling full--ange many-body interactions. arXiv preprint arXiv:2208.c 05863, 2022.
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chem- istry. In International conference on machine learning, pp. 1263-1272. PMLR, 2017.
Liu, T., Lin, Y., Wen, X., Jorissen, R. N., and Gilson, M. K. Bindingdb: a web-accessible database of experimentally determined
binding affinities. Nucleic acids research, 35(suppl_ 11:1p11:::)12112000 2007.
Hansen, K., Biegler, F., Ramakrishnan, R., Pronobis, W., Von Lilienfeld, O. A., Muller, K.-R., and Tkatchenko, A. Machine learning predictions of molecular properties: Accurate many-body potentials and nonlocality in chem- ical space. The journal of physical chemistry letters, 6 (12):2326-2331 2015.
Lu, W. Wu, Q., Zhang, J., Rao, J., Li, C., and Zheng, S. Tankbind: Trigonometry-aware neural networks for drug- protein binding structure prediction. Advances in neural information processing systems, 35:7236-7225 7249, 2022.
Maier, J., Martinez, C., Kasavajhala, K., Wickstrom, L.,
Hussain, M. S., Zaki, M. J., and Subramanian, D. Triplet interaction improves graph transformers: accurate molec- ular graph learning with triplet graph transformers. In Proceedings of the s1 International Conference on Ma- chine Learning, ICML.24. 24. JMLR.org, 2024.
Hauser, K., and Simmerling ff14SB, C. Improving the accuracy of protein side chain and backbone parameters from ff99sb., 2015, 11. DOI: https://doi. org/10. /tt//0..021/ac/ jetc. 5b00255, pp.
Montavon, G., Hansen, K., Fazli, S., Rupp. M., Biegler, F., Ziehe, A., Tkatchenko, A., Lilienfeld, A., and Müller, K.-R. Learning invariant representations of molecules for atomization energy prediction. Advances in neural information processing systems, 25, 2012.
Jaeger, S., Fulle, S., and Turk, S. Mol2vec: unsupervised machine learning approach with chemical intuition. Jour- nal of chemical information and modeling, 58(1):27-35, 2018.
Joshi, C. K., Bodnar, C., Mathis, S. V., Cohen, T., and Lio, On the expressive power of geometric graph neural networks. In International conference on machine learning, pp. 15330-153555. PMLR, 2023.
Musaclian, A., Batzner, S., Johansson, A., Sun, L., Owen, C. J., Kornbluth, M., and Kozinsky, B. Learning local equivariant representations for large-scale atomistic dy- namics. Nature Communications, 14(1):579, 2023.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Židek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-589 2021.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., Vito, Z.,, Rai- son, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, I.. Bai, J., and Chintala, Pytorch: An imperative
10

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
style, high-performance deep learning library. In Wal- lach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnttt, R. (eds.), Advances in Neural In- formation Processing Systems 32, pp. 8024-8033 8035. Curran Associates, Inc., 2019.
on Artificial Intelligence and the 17th Pacific Rim In- ternational Conference Artificial Intelligence (IJCAI- PRICA12020). International Joint Conferences Artifi- cial Intelligence Organization, 2020.
Thölke, P. and De Fabritiis, G. Torchmd-net equivariant transformers for neural network based molecular poten- tials. arXiv preprini orXiv:2202.02544 2022.
Perdew, J. P., Burke, K., and Ernzerhof, M. Generalized gradient approximation made simple. Physical review letters, 77(18)38865 1996.
Thölke, P. and Fabritúi, G. D. Equivariant transform- ers for neural network based molecular potentials. In International Conference on Learning Representations, 2022. URL https: http:///ppeeeeeeeeeeeeeeeeeeeeee net/ net//orunt i id=zNHzqZ79wrRB
Rao, J., Zheng, S., Lu, Y., and Yang, Y. Quantitative evalua- tion of explainable graph neural networks for molecular property prediction. Patterns, 3(12), 2022a.
Rao, J., Zheng, S., Mai, S., and Yang. Y. Communica- tive subgraph representation learning for multi-relational inductive drug-gene interaction prediction. In Proceed- ings of the Thirty-First International Joint Conference Artificial Intelligence, IJCAI-22, pp. 3919-3925, 2022b.
Tkatchenko, A., DiStasio Jr, R. A., Car, R., and Scheffler, M. Accurate and efficient method for many-body van der waals interactions. Physical review letters, 108(23): 236402, 2012.
Unke, O. T. and Meuwly, M. Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. Journal of chemical theory and computation, 15 (6):3678-3693, 2019.
Rao, J., Xie, J., Lin, H., Zheng. S., Wang, 7.,, and Yang, Y. Incorporating retrieval-based causal learning with information bottlenecks for interpretable graph neural networks. arXiv preprini arXiv:2402.04710, 2024a.
Unke, O. T., Chmiela, S., Sauceda, H. E., Gastegger, M., Poltavsky, I., Schütt, K. T., Tkatchenko, A., and Müller, K.-R. Machine learning force fields. Chemical Reviews, 121(16):01122-00888 2021.
Rao, J., Xie, J., Yuan, Q., Liu, D., Wang, Z., Lu, Y., Zheng, S., and Yang, Y. variational expectation-Ne contiioonmmimizztiion framework for balanced multi-scale learning of protein and drug interactions. Nature Communications, 15(1): 4476, 2024b.
Vaswani, A. Attention all you need. Advances Neural Information Processing Systems, 2017.
Rogers, D. and Hahn, M. Extended Extended-connecciitty finger- prints. Journal of chemical information and modeling, 50 (5):742 754, 2010.
Wang. Li, S., Wang. T. Shao, B., Zheng, N., and Liu,
T.-Y. Geometric transformer with interatomic positional encoding. Advances in Neural Information Processing Systems, 46:55981-559999 2023a.
Satorras, V. G., Hoogeboom, E., and Welling, M. E (n) equivariant graph neural networks. In International con- ference on machine learning, pp. 9323-9332. PMLR, 2021.
Wang, Y., Wang. T., Li, S., He, X., Li, M., Wang. Z.,, Zheng, N., Shao, B., and Liu, T.-Y. Enhancing geometric repre- sentations for molecules with equivariant vector-scalar interactive message passing. Nature Communications, 15 (1):313, 2024.
Schütt, K., Unke, O., and Gastegger, M. Equivariant mes- sage passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pp. 9377-9388. PMLR, 2021.
Wang, Z., Liu, G., Zhou, Y., Wang, T., and Shao, B. Quin- net: efficiently incorporating quintuple interactions into geometric deep learning force fields. In Proceedings of the 37th International Conference Neural Information Processing Systems, pp. 777043-77055. 2023b.
Schütt, K. T., Sauceda, H. E., Kindermans, P.-.., Tkatchenko, A., and Müller, K.-R. Schnet-a deep learn- ing architecture for molecules and materials. The Journal of Chemical Physics, 148(24), 2018.
Wu, H., Wu, L., Liu, G., Liu, Z., Shao, B., and Wang, Z. Se3set: Harnessing equivariant hypergraph neural networks for molecular representation learning. arXiv preprint arXiv:2405. 16511, 2024.
Simeon, G. and De Fabritiis, G. Tensornet: Cartesian ten- sor representations for efficient learning of molecular potentials. Advances in Neural Information Processing Systems, 36, 2024.
Yang, Y. and Zhou, Y. Specific interactions for ab initio folding of protein terminal regions with secondary struc- tures. Proteins: Structure, Function, and Bioinformatics, 72(2):793 803, 2008
Song. Y., Zheng, S., Niu, Z., Fu, Z.HH,, Lu, Y., and Yang, Y. Communicative representation learning on attributed molecular graphs. In 29th International Joint Conference

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
A. Additional Details
A.1. Further Details on Feature Embedding
Radial Basis Expansion Interatomic distances are projected into a higher-dimensional space using exponential normal radial basis functions (RBFs), coupled with a cosine cutoff function to ensure smooth behavior near the cutoff distance.
The distance dıy between atoms and j is first expanded into a higher-dimennionall space using set of exponential normal radial basis functions (RBF) (Unke & Meuwly, 2019; Thölke & De Fabritiis, 2022). Additionally, cosine cutoff function is applied to di to ensure smoothness. The vector d consists of the values of v. radial basis functions defined as:
d.j o(dig) exp (-8u (exp (a (-dig)) -mn)))
(21)
1 cos (T 0,
di dout
+ 1, if dij < deuts
((d;))
(22)
if dij 2 deut-
where Sn and μn represent the width and mean parameters of the radial basis function n. respectively. The parameter on controls the sensitivity of the radial basis function, while fln defines its center. a is a scaling factor that adjusts the rate of exponential decay. The width parameter 3n is set to (2n (2n= exp(-deut)) -2, following the approach described Unke & Meuwly (2019).
Scalar and Vector Node Feature Embedding The scalar feature z; node i derived from the initial embedding which encodes the atom type 2i, and the aggregated features from its neighboring nodes. First, node i maps its atom type 2; to high-dimensional space using the embedding function feme resulting in the initial node feature h; f.mmb (2mm(2z).
Next, node i gathers information from all its neighboring nodes j. The distance dij between nodes i and j is first truncated using the cutoff function p(di)), as described in Equation (22), to limit the influence of long-range interactions. A linear transformation is then applied to the edge features d' defined in Equation (21), followed by multiplying the result with the initial feature hy of the neighboring node j. The aggregated neighbor features knah for node i are obtained by summing the contributions from all its neighbors:
Inbbi
Σ (oldes) (Wnd S + bn) o hT), JEN())
(23)
where @ denotes element-wise multiplication. W, and b, are learnable parameters, and (i) denotes the set of neighboring nodes of node i. Finally, the scalar feature x; of node is computed by concatenating its initial feature h; with the aggregated neighbor features Inbh, followed by linear transformation:
Ws(h: Frbbi) bs:
(24)
where the denotes the concatenation operation. Wa and b2 are learnable parameters.
The vector feature oū of node is initialized as Ö.
Edge Feature Embedding The edge features Ciy computed by combining the scalar features of nodes i and j. obtained from Equation (24), with the projected edge attributes drbs derived from Equation (21). First, arb5 transformed into a higher-dimensional space using linear projection. Then, the edge features are calculated as:
Esj = (x; + 23) @ (W,dt br).
(25)
where W, and b, are learnable parameters.
A.2. Quadruple Attention for Dihedral Angles
Our Mony-body Interaction method employs Quadruple Attention to directly capture four-body interactions, specifically the relationships among nodes i, k These four atoms form a dihedral angle, as illustrated Figure 2. In chemistry, a dihedral angle defined as the angle between two planes, each formed by three atoms, involving total of four atoms. The
12

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
common edge of these planes corresponds chemical bond (atoms k and while the planes themselves determined by the additional atoms i and interacting with this bond.
Dihedral torsional potential serves as crucial descriptor of energy variations in non-collinear four-atom molecular system. Compared to bond stretching and bond angle bending potentials. the dihedral torsional potential is relatively weaker but plays a decisive role in molecular conformational changes. This is because variations in dihedral angles can lead to large-scale rearrangements of molecular chains, ultimately influencing the overall shape and function of a molecule. For instance, in biomacromolecules such as proteins and DNA, dihedral angle torsion dictates their drre-dimmessioal conformations, which are essential for their biological functions. Our approach enables comprehensive dihedral angle information capture through direct four-body interactions, making it highly relevant for molecular dynamics simulations.
1
Quad.Attn. v
Quad.Attn.
k
k
i
Figure 2. Quadruple Attention-Base Visualization of the Dihedral Angle Formation by Atoms i, k.l.
A.3. Datasets
A.3.1. MD22
MD22 is a newly developed molecular dynamics benchmark dataset introduced by (Chmiela et al., 2023), designed to provide novel challenges for the research and development of Machine Learning Force Fields (MLFFs). The dataset includes variety of molecular systems ranging from tens to hundreds of atoms, encompassing biomolecules such simple peptide chains and complex carbohydrates. These systems provide diverse training and testing scenarios for MLFF models. The primary goal of MD22 is to enhance the accuracy and scalability of MLFF models, particularly in dealing with large molecular systems that exhibit long-range interactions and complex molecular dynamics hehaviors. The molecular dynamics trajectories the dataset are sampled within a temperature range of 400 K to 500 K, with time resolution of femtosecond (fs). The potential energy and atomic forces for each system are computed using the PBE+MBD (Perdew et al., 1996; Tkatchenko al., 2012) theory level. Compared to the earlier MD17 dataset, the molecular systems in MD22 are larger and more flexible, making them better suited for simulating long-range interactions and complex molecular dynamics behaviors. Furthermore, MD22 offers greater molecular flexibility, allowing for the simulation of complex dynamic processes such as molecular vibrations, rotations, and conformational changes. Table 6 provides an overview of the atomic counts, molecular formulas, number of conformations, and the ranges of energy and force values for the systems MD22.
Table Properties of the MD22 datasets. Energies are in kcal/mol, forces in kcal/mol/A.
Energies
Forces
Dataset
Atoms
Formula
Conformations
Range
Variance
Range
Variance
Ac-Ala3-NHMo
42
C,H222,04
85,109
102.19
67.30
437.99
678.02
DHA
CzHHg:22
69,753
75.71
93.31
420.07
673.99
Stachyose
27,272
106.15
189.33
426.08
655.55
AT-AT
60
20,001
139.29
120.09
444.01
779.81
AT-AT-CG-CG
118
Ca
10,153
243.50
246.64
407.55
768.13
13

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
A.3.2. MD17
MD17, proposed by (Chmiela et al., 2017), a molecular dynamics dataset specifically designed support the development of MLFFs. It includes ab initio molecular dynamics (AIMD) trajectories for variety of small organic molecules across multiple conformations. These molecules include, but not limited to, benzene, toluene, naphthalene, ethanol, uracil, and aspirin, offering range of sizes and complex potential energy surfaces. Each molecular dataset contains between 150,000 and nearly I million conformational geometries, sampled at a time resolution of 0.5 femtoseconds (fs). The energy and force labels for all systems were computed using the PBE+vdW-TT electronic structure method. The MD17 dataset was designed to facilitate the development and validation of novel machine learning algorithms aimed at constructing accurate and -enerriinng molecular force fields. It has been widely used demonstrate the efficiency and accuracy of machine learning models in predicting forces and energies, partieularly when achieving high predictive accuracy with limited number of training samples. Table 7 summarizes the atomic counts, molecular formulas, number of conformations, as well as the energy and force ranges for the molecules included in MD17.
Table 7. Properties of the MD17 datasets, Energies are in kcal/mol, forces in kcal/mol/A.
Energies
Forces
Dataset
Atoms
Formula
Conformations
Range
Variance
Range
Variance
Ethanol
9
C2H,O
555,092
36.61
17.57
432.00
689.43
Malonaldehyde
9
C₂H,O2
993,237
43.83
17.14
570.65
820.10
Uracil
12
C,H,N:02
133,770
39.92
24.24
476.63
901,43
Toluene
15
C-Hl 442,790
52.08
26.01
425.60
746.91
Salicylic acid
16
CıIL,,;
320,231
47.47
29.34
453.77
817.37
Aspirin
21
CyH&O,,
211,762
55.29
35.36
423.87
779.99
A.3.3. SPICE
SPICE is a quantum chemistry dataset specifically designed for training machine learning potentials, introduced by (Eastman et al., 2023). It focuses on modeling interactions between drug molecules and proteins. The dataset includes diverse conformations of drug molecules, dimers, dipeptides, and solvated amino acids, with tens of thousands to hundreds of thousands of conformations for each category, as shown in Table 8. It spans more than a dozen chemical elements and incorporates both charged and neutral molecules, providing a comprehensive sampling of covalent and non-covalent interactions. All energies and forces are calculated using the additional useful information such as multipole moments and bond orders. Additionally, SPICE includes other valuable quantum chemical properties, such as partial charges and multipole moments, which can be used to train different types of models or enhance potential models through multitask learning.
level of theory, alongside
SPICE comprises multiple subsets, each designed to provide specific types of information. To validate the efficacy of the proposed many-body interaction approach in capturing the intricate details of large molecular systems, subsets with a higher number of atoms were selected from the SPICE dataset for training and testing. Specifically, these subsets include Dipeptides, Solvated Amino Acids, and PubChem Molecules. These subsets encompass variety of chemical structures ranging from dipeptides to complex drug-like molecules, thereby providing comprehensive evaluation of the model's performance in handling large molecular systems.
Dipeptides The Dipeptides subset provides extensive coverage of covalent interactions in proteins. includes 676 dipeptides composed of all possible combinations of 20 natural amino acids and their common protonation states, as well as one pair of cysteine residues linked by a disulfide bond, resulting in 677 dipeptides. Each dipeptide is capped with ACE and NME groups and has 50 conformations: half representing low-energy states and the other half high-energy states. These conformations were generated using RDKit to create initial structures, followed by molecular dynamics simulations with OpenMM 7.6 (Eastman et al., 2017) and the Amber14 14 (Maier et al.) force field at varying temperatures. This subset is sufficient to sample all covalent interaction types found in naturally occurring proteins, excluding special cases such as post-translational modifications.
Solvated Amino Acids The Solvated Amino Acids subset focuses on sampling non-covalent interactions between proteins and water, as well water-water interactions, which are crucial for protein simulations. This subset includes the 26 amino
14

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
acid variants mentioned earlier, also capped with ACE and NME groups, surrounded by 20 TIP3P-FB water molecules. For each amino acid, 50 conformations 2.2 nanometers, and a 1-nanosecond molecular dynamics simulation was performed at 300 K. saving conformations every 20 picoseconds. The 20 water molecules closest to the amino acid were retained for each conformation, while the rest were discarded. This allows researchers to study how the local water environment around amino acids influences protein behavior effectively.
generated. Each amino acid was placed in cubic water box with side length of
PubChem Molecules The PubChem Molecules subset comprises a large and diverse collection of drug-like small molecules. Approximately 1.5 million entries were downloaded from the PubChem database, specifically from Bind- ingDB (Liu et al., 2007) ChemIDplus (cited in the U.S. National Library of Medicine databases). A filtering process was applied to select as diverse sample possible based Tanimoto similarity using ECFP4 fingerprints. For each molecule, 50 conformations were generated using the same procedure as the dipeptides to obtain both low- and high-energy states. This subset is designed to provide broad chemical diversity, enabling models to better understand and predict molecular behavior in complex and varied chemical environments.
Table 8. Properties the SPICE datasets.
Subset
Atoms (avg.)
Molecules
Conformations
Elements
Solvated Amino Acids
12
26
1,300
H,C.N,O,,
Dipeptides
44.23
677
33,850
PubChem Set
33,78
2372
114,359
PubChem Set
37.05
2431
115,997
PuhChem Set
37.46
2446
118,931
H.C.N,O.F.
PubChem Sct
37.70
2455
118,486
P,S.CIBB,I
PubChem Set 5
37.02
2463
117,841
PubChem Set
38.49
2476
123,786
A.4. Baselines
1. BASELINES OF MD22
ViSNet (Wang et al., 2024) is an equivariant geometry-cnhanced graph neural network designed to efficiently extract and utilize geometric features for molecular modeling. By leveraging equivariance principles, ViSNet effectively integrates geometric information into its architecture, enabling precise modeling of molecular structures with reduced computational costs. This approach enhances the representation of molecular conformations and provides interpretability by mapping geometric representations to molecular structures, making it powerful tool for addressing challenges in molecular dynamics and related fields.
Equiformer (Liao& & Smidt, 2023) is graph neural network that integrates the strengths of Transformer architectures with 3E(3)E(3)-qquvarrinn features based on irreducible representations (irreps). By replacing standard Transformer operations with their equivariant counterparts and incorporating tensor products, Equiformer effectively encodes equivariant information within irreps feature channels without adding complexity to graph structures. Furthermore, it introduces novel equivariant graph attention mechanism, which replaces traditional dot product attention with multi-layer perceptron attention and incorporates non- non-linear message passing, enhancing its ability to capture geometric and relational information in 3D atomistic graphs. With these innovations, Equiformer provides simple yet powerful architecture for modeling 3D molecular systems.
MACE (Batatia et al.) is state-of-the-art machine learning force field architecture designed for a wide range of in-domain, extrapolation, and low-data regime tasks. By employing strictly local atom-centered model, MACE achieves high data efficiency and is capable of accurately modeling diverse systems, including amorphous carbon, universal materials, small organic molecules, large molecular systems, and weakly interacting molecular assemblies. Its architecture excels in tasks such as constrained geometry optimization and molecular dynamics simulations, showcasing its versatility and ability to handle complex physical and chemical phenomena.
LSR-MP (Li et al., 2023) is a novel framework that generalizes existing equivariant graph neural networks (EGNNs) by efficiently incorporating long-range interactions while maintaining an effective description of many-body interactions.
15

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Inspired by fragmentation-basee methods, LSR-MP aims to address the limitations of traditional machine learning and fragmentation approaches in modeling chemical and biological systems. By extending the message-passing paradigm to capture both short- and long-range interactions, this framework enhances the modeling of complex molecular interactions and can be seamlessly integrated into existing EGNN architectures. Its general applicability and robustness are demonstrated through consistent performance improvements across various EGNNs, including its application to ViSNer(ViSNer--LSMM and EguiformerrgquifommeeL.....
Allegro (Musaelian et al., 2023) is a strictly local equivariant deep neural network interatomic potential architecture designed to achieve both high accuracy and scalability. Unlike traditional atom-centered message passing neural networks (MPNNs), which are limited by the range of their information propagation, Allegro avoids message passing altogether. Instead, it uses iterated tensor products of learned equívariant representations model many-body interactions. This approach allows Allegro to retain the accuracy of many-body potentials while maintaining the scalability of local methods, making it a powerful tool for modeling the potential energy surfaces of molecules and materials.
So3krates (Frank et al., 2022) is a self-attention-bast message passing neural network designed to capture non-local quantum mechanical effects in molecules and materials. It introduces spherical harmonic coordinates (SPHCs) to encode higher-order geometric information, enabling a non-local attention mechanism in SPHC space. By decoupling geometric information from atomic features, So3krates constructs spherical filters that extend continuous filters to SPHC space, forming the foundation for its spherical self-attention mechanism. This allows So3krates to model non-local effects over arbitrary length scales with high data efficiency and generalization. It achieves state-of-the performance while being significantly faster and more marameter-efficiee than other models.
DOrchMD-NET((T) (Thölke & De Fabritiis, 2022) is a versatile framework for molecular dynamics simulations that integrates classical and machine learning potentials. By expressing all force computations- as bond, angle, dihedral, Lennard-Jones, and Coulomb interactions-as as Py operations, TorchMI enables seamless compatibility with machine learning techniques. TorchMD's s capabilities are validated through diverse applications, including standard Amber all-atom simulations, learning ab initio potentials, and coarse- saarseegriinddd protein folding models.
PaiNN (Schütt et al., 2021) is a message passing neural network that extends traditional formulations by incorporating rotationally equivariant representations, addressing limitations of invariant approaches in data efficiency. By leveraging equivariant atomwise representations, PaiNN improves the prediction of chemical properties and tensorial quantities while achieving state-of-the- making it both accurate and efficient. PaiNN demonstrates its capability by simulating molecular spectra with speedups of 4-5 orders of magnitude compared to electronic structure methods, showcasing its potential for accelerating molecular dynamics and quantum chemistry studies.
performance on molecular benchmarks. Its architecture reduces model size and inference time,
sGDML (Chmiela et al., 2023) is global machine learning force field designed to capture collective many-atom interac- tions in molecular systems without introducing locality assumptions or uncontrolled approximations. Unlike traditional approaches, sGDML preserves full correlation among all atomic degrees of freedom, allowing it to accurately model complex molecules and materials with far-reaching interaction lengths. Using an exact, iterative, and parameter- free training method, sGDML scales to systems with up several hundred atoms while retaining the accuracy of global force fields. Its performance is demonstrated on the MD22 benchmark dataset (42-370 atoms) and in robust nanosecond-scale path-integral molecular dynamics simulations for supramolecular complexes.
A.4.2. BASELINES OF SPICE
TensorNet (Simeon& & De Fabritiis, 2024) is an -quuiiauraa message-passing neural network designed for efficient and accurate representation of molecular systems. It leverages Cartesian tensor atomic embeddings, simplifying feature mixing through matrix product operations. By decomposing tensors into rotation group irreducible representations, TensorNet processes scalars, vectors, and tensors separately when needed, making it highly efficient. Compared to spherical tensor models, TensorNet achieves state-of-the-art performance with significantly fewer parameters, even with a single interaction layer for small molecular potential energies. Its framework also enables accurate predictions of vector and tensor molecular properties alongside potential energies and forces, all while reducing computational cost. TensorNet thus provides a flexible and powerful foundation for designing advanced equivariant models.
DimeNet++ (Gasteiger et al., 2020a) is a machine learning model for molecular property prediction that builds upon directional message-passing mechanism to capture intricate angular dependencies in molecular systems. It improves
16

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
Table 9. Hyperparameters used for MD22, SPICE and MD17.
Parameter
MD22
SPICE
MD17
initial LR
le-4
le-4
3e-4
min LR
le-7
le-7
le-7
LR warm up steps
1000
1000
1000
LR decay factor
0.8
0.8
0.8
LR patience (epochs)
30
30
30
optimizer
Adam
Adam
Adam
energy loss weight forces loss weight
0.05
1.0
0.05
0.95
10.0
0.95
embedding dimension
256
256
256
attention heads batch size number of layers number of RBFs cutoff (Á)
8
8
8
2,4
2,4
9
32
32
32
5.0
5.0
5.0
Table 10. Mean absolute errors of energy (meV) and force (me) for 7 small molecules MD17.
Molecular
SchNet
DimeNet++
PaiNN
SpookyNet
GemNet
ET
NequlP
SO3KRATES
ViSNet
Equiformer
MABNet
Energy Forces
0.37
0.204
0.167
0.151
0.123
0.131
0.139
0.116
0.122
0.101
Aspirin
1.35
0.499
0.338
0,258
0.217
0.253
0.184
0.236
0.135
0.152
0.166
Energy
0.08
0.064
0.064
0.052
0.052
0.051
0.061
0.051
0.051
0.039
Ethanol
Forcs
0.39
0,230
0.224
0.094
0.085
0.109
0.071
0.096
0,060
0.067
Malonaldehyde
Energy Forces
0.13
0.104
0.091
0,079
0.077
0.076
0.077
0.075
0.074
0.051
0.66
0.383
0.319
0.167
0.155
0.169
0.129
0.147
0.100
0.125
0.122
Naphthalene
Energy Foness
0.16
0.122
0.116
0.116
0.085
0.113
0.115
0.085
0.085
0.053
0.58
0.215
0.077
0.089
0.051
0.061
0.039
0.074
0.039
0.046
Salicylic Acid
Energy Fonces
0.20
0.134
0.116
0.114
0.093
0.106
0.106
0.092
0.099
0.059
0.85
0.374
0.195
0,180
0.125
0.129
0.090
0.145
0.084
0.090
0.087
Toluene
Energy Fonces
0.12
0.102
0.095
0.094
0.074
0.092
0.095
0.074
0.085
0.047
0.57
0.216
0.094
0.087
0.060
0.067
0.039
Energy 0.14
0.115
0.106
0.105
0.095
0.104
0.103
0.095
0.099
0.057
Uracil
Forces
0.56
0,301
0.139
0.119
0.097
0.095
0.076
0.111
0.062
0.076
0.067
computational efficiency and simplifies the architecture compared to its predecessor, DimeNet. Additionally, DimeNet++ incorporates ensembling and mean-variane estimation techniques to enable uncertainty quantification, making it suitable for exploring complex molecular configurations, including non-equilibrium structures.
SchNet (Schüttt et al., 2018) is a deep learning architecture designed for modeling quantum interactions in molecules, lever- aging continuous-fill convolutional layers capture local correlations without relying on grid-structure data. By directly utilizing the precise locations of atoms, SchNet preserves essential physical information and ensures rotationally invariant energy predictions. The model jointly predicts total energies and interatomic forces, yielding smooth, differentiable potential energy surface according to quantum-chemical principles.
A.5. Alyper-paramedd
To account for the differences in dataset sizes and data distribution, distinct hyperparameter configurations were selected for the MD22, SPICE, and MD17 7 datasets, as summarized in Table 9. The Adam optimizer was chosen as the unified optimization strategy, and the energy loss weight and forces loss weight were adjusted based on the specific characteristics of each dataset. In particular, for the SPICE dataset, which involves more intricate intermolecular interactions, the force loss weight was increased to enhance the model's sensitivity to subtle variations in the force field. This adjustment aims to improve the model's 's performance in handling complex molecular systems.
17

---

Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
B. Additional Experiments
B.1. Results of MD17
The performance of MABNet on the MD17 dataset was presented in Table 10. It achieved higher accuracy than models like So3krates and comparable to ViSNet and Equiformer. MABNet showed no significant improvements compared to other state-of-the-art (SOTA) models, higher-order many-body interactions were not pronounced in small molecular systems.
B.2. Computational Cost and Analysis
To further validate the efficiency of our approach, a tripeptide Ac-Ala3-NHMe was used to quantify computational efficiency. Table 11 summarized a comparative study across different configurations: 3-body attention without equivariance (denoted as "3-body Attn. (w/o Equi.), akin to TGT), standard 3-body attention, and several variants of 4-body attention with varying edge counts. All experiments were conducted on NVIDIA GeForce RTX 4090 GPU with batch size of 4. The results, presented in Table 4, showed that 4-body attention with 6N edges significantly reduces the mean absolute error (MAH) for both energy and force predictions, compared to the 3-body Attn. (w/o Equi.). This improvement in accuracy was achieved with only a 6.8% increase in memory consumption and no notable degradation in training speed. Furthermore, we conducted a comparative analysis of training and inference costs with two representative baselines, Geoformer (Wang et al., 2023a) and ViSNet (Wang et al., 2024). Our method exhibited a favorable trade-off between accuracy and computational overhead, demonstrating that the proposed 4-body attention mechanism offered improved predictive performance without compromising training or inference efficiency.
Table 11. Comparison of memory consumption, training and inference speeds for different attention mechanisms on the Ac-Ala3-NHHM dataset.
Type of Many-body Attn.
Memory (MiB)
Training (it/s)
Inference (it/s)
Methods
Geoformer
14362
5.09
13.58
ViSNet
15962
4.44
11.66
3-body Attn.
16542
3.93
10.65
4-body Attn. (1.5N Edges)
17264
2.78
6.77
MABNet
4-body Attn. (3N Edges)
17400
2.66
6.62
4-body Attn. (6N Edges)
17758
2.35
5.91
B.3. Additional Ablation Atudy Comparing ViSNet and MABNet on Ac-Ala3-NHMe.
To further investigate the impact of the body attention mechanism in MABNet, we conducted ablation studies comparing its performance with ViSNet on Ao-Ala3-NHMe ViSNet used a runtime geometry calculation (RGC) mechanism to encode angular and torsional information via pairwise message passing, indirectly modeling higher-order interactions. MABNet, the other hand, introduced direct many-body attention mechanism, enabling joint updates among multiple atoms (e-g., four atoms involved in dihedral angle). This direct interaction allowed our model to more effectively capture complex molecular geometries. Table 12 presented the comparison between the two models. MABNet achieved a significant reduction in energy prediction error, improving over ViSNet by 32.9%.
Table 12, Comparison of energy and force prediction errors between ViSNet and MABNet with different many- body attention mechanisms on Ac-Ala3- MaH- (energy in kcal/mol and force in kcal/mol/AA.
Ac-Ala3-NHMe
Methods
Energy
Forces
ViSNet
0.080
0.097
MABNet (2-body)
0.072
0.092
MABNet (3-body)
0.061
0.082
MABNet (4-body)
0.053
0.077
18


Reviews:
[{"review_id": "71X3CqIkfQ", "reviewer": "Reviewer_7EDj", "summary": "This paper introduces MABNet (MAny-Body interaction Network), a novel geometric attention framework designed to explicitly model four-body interactions for accurate molecular property predictions. MABNet aims to address the limitations of existing models that often implicitly approximate these complex interactions or handle at most triplets of nodes. The paper demonstrates that MABNet achieves sota performance on challenging benchmarks such as MD22 and SPICE.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s careful review of our paper, positive feedback, and recognition of our work, particularly the novelty of our approach and the meaningfulness of our experiments. Below, we address the concerns point by point:\n\n> Comparison with Geometric Transformers\n\nR: We have included a discussion of the geometric transformer literature [1, 2, 3, 4, 5] in the revised manuscript. Additionally, we conducted benchmarking experiments on the QM9 dataset to provide a more direct comparison. Due to time constraints, we focused on two properties. Our method achieves an MAE of 0.038, compared to Transformer-M (0.041) and Geoformer (0.040). These results demonstrate that our approach achieves state-of-the-art performance, even on smaller molecular systems where higher-order multi-body interactions are less prominent. This highlights the strength and versatility of our method.\n\n|QM9 | $\\alpha$ | $\\epsilon_{HOMO}$ | \n| --- | ----------- | ----------- |\n| Transformer-M |  0.041 |  17.5 |\n| Geoformer| 0.040 | 18.4|\n| MABNet| 0.038 | 14.4 |\n\n> Training and Inference Costs\n\nR: In Section B.2 of the supplementary material, we have included a detailed discussion regarding the training and inference costs of our method. Additionally, we performed a comparative analysis of inference costs with Geoformer [1] in the MD22 dataset. Our results indicate that the improvement in prediction accuracy comes at a modest cost, with no significant degradation in training speed. This demonstrates the efficiency of our approach compared to similar models.\n\n|Methods\t|\tMemory Used |  Inference Time (mins) |\n| --- | ----------- | ----------- |\n| Geoformer |\t14362 MiB |\t33.43 |\n| MABNet (3-body) |\t16542 MiB |\t42.65|\n| MABNet (4-body) | 17400 MiB\t|  \t49.79 |\n\nWe hope these revisions address your concerns and further strengthen our manuscript. Thank you again for your valuable feedback.", "claims_and_evidence": "- Explicit modeling of 4-body interactions - there's convincing evidence\n- sota performance on MD22 and SPICE - results demonstrate this\n- Efficient handling of higher-order interactions -While the complexity is stated as O(|N|^2 · |E|), the ablation study in Section 5.4 and the computational cost analysis in Appendix B.2 (Table 10) indicate that the increase in computational cost compared to methods considering fewer-body interactions is manageable", "methods_and_evaluation_criteria": "- MD22 and SPICE datasets were chosen as they have larger graphs and are known benchmarking datasets in this field - which makes sense.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "- [Table 2] Compares results with previous sota approaches on MD22 benchmark\n- [Table 3] Compares results with previous sota approaches on SPICE benchmark\n- [Table 4] Ablation study demonstrating the impact of equivariance, 4-body attn w/ different cutoffs\n- [Table 10] Computational cost comparison across the ablations\n\nAll of these experiments make sense.", "relation_to_broader_scientific_literature": "- Prior work have encoded triplets for attentions explicitly [TGT, Graphormer], however this work extends it to 4-body attn and make it scalable with cutoffs", "essential_references_not_discussed": "- This work does not discuss a vast literature of geometric transformers and doesn't compare with them as well [1, 2, 3, 4, 5]. Even though none of these explicitly model 4-body attn in the way this work does. If this work would have have benchmarked their results on QM9 datasets, it would make it easy to compare against the vast literature in this field.\n\n1. Wang, Yusong, et al. \"Geometric transformer with interatomic positional encoding.\" Advances in Neural Information Processing Systems 36 (2023): 55981-55994.\n2. Kwak, Bumju, et al. \"Geometry-aware transformer for molecular property prediction.\" arXiv preprint arXiv:2106.15516 (2021).\n3. Shi, Yu, et al. \"Benchmarking graphormer on large-scale molecular modeling datasets.\" arXiv preprint arXiv:2203.04810 (2022).\n4. Luo, Shengjie, et al. \"One transformer can understand both 2d & 3d molecular data.\" arXiv preprint arXiv:2210.01765 (2022).\n5. Choukroun, Yoni, and Lior Wolf. \"Geometric transformer for end-to-end molecule properties prediction.\" arXiv preprint arXiv:2110.13721 (2021).", "other_strengths_and_weaknesses": "- A stronger comparison against prior literature on geometric transformers would solidify the importance of this work.", "other_comments_or_suggestions": "- There appears to be a typographical error in Equation (3) on line 19, where \"Sotfmax\" should likely be \"Softmax\".", "questions_for_authors": "- How does the training and inference cost compare against other geometric transformers or models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XKza7taPvf", "reviewer": "Reviewer_xpVq", "summary": "This paper introduces MABNet, an attention-based framework for molecular property prediction that explicitly models four-body interactions. The authors argue that current Graph Neural Networks (GNNs) and Transformers, while promising, struggle to directly capture complex many-body interactions, often relying on approximations. MABNet addresses this by enabling direct communication between atomic quartets through a new \"quadruple attention\" mechanism. This mechanism incorporates geometric features and E(3) equivariance, and employs spatial sparsity and quantum-inspired pruning to manage computational cost. The key algorithmic innovation is the many-body attention module within equivariant message-passing blocks. The paper claims state-of-the-art performance on challenging molecular property prediction benchmarks, MD22 and SPICE.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the detailed feedback and constructive suggestions to improve our manuscript. Briefly, the reviewer has two main concerns: (1) the contribution of MABNet compared to VisNet, and (2) missing details for baselines, computational efficiency, and visualization of attention patterns. Below, we summarize the reviewer’s questions and provide point-by-point responses.\n\n> The contribution of MABNet compared to VisNet\n\nWe respectfully disagree with the reviewer’s assertion and would like to clarify the difference between MABNet and VisNet. \n\nVisNet’s primary contribution is the **runtime geometry calculation (RGC) mechanism**, which extracts angular and dihedral torsion information using vectors that connect the target node to its neighbors. This mechanism relies on pairwise interactions (message passing) to simulate higher-order (three-body, four-body) interactions.\n\nIn contrast, MABNet introduces a novel **many-body attention mechanism**, which directly enables communication among multiple atoms (e.g., four atoms involved in a dihedral angle). For a dihedral angle, MABNet computes attention scores directly among the four atoms involved. This allows their features to interact directly, which results in a single joint update for the features of all four atoms. \n\nAdditionally, we have included ablation studies in the revised manuscript to validate the impact of these modules. MABNet shows significant improvement (32.9% of energy) over VisNet on the MD22 dataset.\n\n|MD22 (Ac-Ala3-NHMe) | MAE|\n|---|--------|\n|VisNet|0.079|\n|MABNet (2-body)|0.072|\n|MABNet (3-body)|0.061|\n|MABNet|**0.053**|\n\n\n> Q1. The broader quantum many-body effects\n\nWhile our current implementation of MABNet focuses on explicitly modeling four-body interactions (e.g., dihedral/torsional interactions), the underlying many-body attention mechanism is designed to be flexible and can be extended to capture broader many-body effects, including higher-order interactions (e.g., five-body or beyond).\n\n> Q2. The performance of the MABNet scale with molecular size\n\nThe computational complexity of MABNet is O(N × N × E), where N is the number of atoms and E is the number of edges in the molecular graph. By controlling the number of combinations during many-body attention computation, MABNet can be efficiently scaled to larger molecular systems.\nWe would include additional experiments on larger molecular systems to demonstrate the scalability of MABNet.\n\n> Q3. Module Comparison between MABNet and VisNet\n\nWhile MABNet shares conceptual similarities with VisNet in *graph construction, feature embedding, and message-passing techniques*, the core interaction mechanism is fundamentally different:\n\n1.  VisNet relies on pairwise interactions and **runtime geometry calculation (RGC)** to indirectly simulate higher-order interactions. MABNet introduces a novel **many-body attention mechanism** that directly models four-body interactions by enabling simultaneous communication among multiple atoms (e.g., four atoms in a dihedral angle).\n\n2. VisNet considers pairwise interactions between **all atom pairs**. MABNet, by contrast, requires considering combinations of atomic groups. To manage computational cost, MABNet restricts **the number of many-body combinations** (e.g., N × N × E, where N is the number of atoms) while maintaining high accuracy and efficient computations.\n\n> Q4. The generalizability of MABNet\n\nMABNet is explicitly designed to model four-body interactions, but it can also capture two/three-order interactions effectively. Results on MD17 and QM9 show that MABNet achieves SOTA performance on small molecular systems, while its ability to handle high-order interactions gives it a significant advantage on larger molecular systems.\nThus, MABNet is a general framework that is effective across molecular sizes.\n\n> S1. The visualization of the attention patterns\n\nWe agree that visualizing the attention patterns enhances interpretability. We will include visualizations of the learned attention patterns, showing how MABNet captures meaningful four-body interactions.\n\n> S2. The performance on the rMD17 dataset\n\nWe acknowledge the reviewer’s concern regarding the use of random splits in our original MD17 evaluation. To address this, we have re-evaluated MABNet on the rMD17 dataset. The updated results, included in the revised manuscript, show that MABNet maintains competitive performance and further validate its robustness.\n\n| rMD17| Azobenzene | Paracetamol |\n| --- | -------| -------|\n|VisNet | 0.0156 | 0.0258 |\n|MABNet | **0.0153** | **0.0252** |\n\n> S3. Efficiency analysis\n\nWe have included timing information for key baselines in the supplementary material.\n\n|Methods|Training(mins)|Inference(mins) |\n| --- |----------- | ----------- |\n|ViSNet|7.12 |38.95|\n|MABNet 3-body|8.05|42.65|\n|MABNet 4-body|9.39|49.79|", "claims_and_evidence": "The claims of state-of-the-art performance on MD22 and SPICE benchmarks are presented, but the evidence is undermined by the lack of transparency regarding the codebase's origin. If the performance is primarily achieved by leveraging and slightly modifying existing VisNet code without proper attribution, then the claim of \"state-of-the-art contribution\" is significantly weakened. Most of the codebase is adopted from VisNet without any proper citation.\n\nThe central issue is not whether the numbers are good, but whether the claimed algorithmic contribution is genuine and properly attributed. If the performance gains are largely due to the underlying VisNet framework, which is not acknowledged or properly cited in the methodology, then the claims are misleading and unsupported in terms of originality. A convincing demonstration of independent algorithmic contribution, beyond incremental modification of VisNet, is severely lacking.", "methods_and_evaluation_criteria": "The evaluation criteria (MD22, SPICE benchmarks, MAE metrics) are standard and appropriate if the presented method were genuinely novel. However, the methodology section is deeply flawed by its complete omission of any mention or citation of VisNet or any related works, despite strong indications that the MABNet codebase is substantially derived from it. This omission is a critical ethical and scientific failing. The evaluation, therefore, becomes questionable because it's unclear what is truly being evaluated: a novel method, or a minor modification of VisNet presented as a major breakthrough.\n\nIn addition, the authors didn't compare with any baselines regarding the efficiency analysis, which is crucial in many cases.\n\nLastly, it is generally recommended to use rMD17 with standard splits compared with MD17 with random splits. Since the codebase provided by the author has the rMD17 dataset file, its questionable about the dataset choice, and why the authors chose to use MD17.", "theoretical_claims": "The authors claim the network to be E(3) equivariant message-passing, but didn't provide any theoretical proof on this.", "experimental_designs_or_analyses": "The ablation studies in Table 4, while showing some performance changes, are insufficient to demonstrate genuine algorithmic novelty independent of the underlying VisNet architecture or different modules discussed in the methodology section. There's no ablation study with the removal of certain modules discussed in the methodology section. The computational cost analysis is also questionable when it lacks of baselines.", "relation_to_broader_scientific_literature": "The paper discusses its relation to prior work in molecular property prediction, Graph Neural Networks, Transformers, and methods for modeling many-body interactions. However, the literature review could be more comprehensive. The authors should provide a broader overview of molecular property prediction works, even if they aren't used for direct comparison in the experiments. This would give readers a clearer understanding of the current landscape of the domain.", "essential_references_not_discussed": "The paper discusses its relation to prior work in molecular property prediction, Graph Neural Networks, Transformers, and methods for modeling many-body interactions. However, the literature review could be more comprehensive. The authors should provide a broader overview of molecular property prediction or equivariant GNN or Geometric Graph Transformer works, even if they aren't used for direct comparison in the experiments. This would give readers a clearer understanding of the current landscape of the domain and where this article fit in the big picture.  Review [1] gives a great overview of the geometric GNNs, several references are missing in this article, especially the Equivariant GNNs and Geometric Graph Transformers listed in Fig.4\n\n[1] Han, Jiaqi, et al. \"A survey of geometric graph neural networks: Data structures, models and applications.\" arXiv preprint arXiv:2403.00485 (2024).", "other_strengths_and_weaknesses": "- The computational cost analysis in Table 10 shows that their 4-body attention approach is significantly slower than 3-body attention (2.35 it/s vs. 3.93 it/s for training), which could limit its applicability to very large molecular systems.\n- The paper could benefit from a more detailed discussion of how their approach might scale to even higher-order interactions (e.g., 5-body or beyond).\n- The presentation of the method could be improved, with more consistent notation and clearer symbolic systems throughout the paper.\n- The paper narrows the use of the network to many-body systems, while VisNet offers a more general framework that could potentially be adapted for many-body systems if needed.", "other_comments_or_suggestions": "- A visualization of the attention patterns learned by their model could provide additional insight into how it captures four-body interactions.\n- The presentation can be further improved. It is very hard to read and understand this article because of the presentation. There are notation inconsistencies throughout the manuscript (e.g., \"Sotfmax\" vs. \"Softmax\", inconsistent bold/non-bold for learnable matrices).\n- For MD17 evaluations, the authors should use the standard predefined splits on rMD17 dataset rather than random splits on MD17 dataset to ensure fair comparison with existing methods. The rMD17 dataset, a revised version of MD17, addresses the issue of numerical noise present in the original data, thus is preferred over the original MD17 dataset.", "questions_for_authors": "1. The term \"many-body\" is broad. Can authors clarify if MABNet only focuses on dihedral/torsional interactions or is it intended to capture broader quantum many-body effects?\n2. How does the performance of MABNet scale with molecular size, particularly for systems larger than those in the MD22 and SPICE datasets? \n3. Could you clarify the relationship between your implementation and VisNet? Specifically, which modules/sections in methodology are completely novel and which modules/sections are the same as previous work\n4. Your method appears to be specifically designed for many-body systems, whereas VisNet is a more general framework. Could you discuss the trade-offs of your more specialized approach versus a more general framework that could potentially be adapted for many-body interactions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TRW5XHBUNM", "reviewer": "Reviewer_xeLN", "summary": "The paper introduces MABNet, a machine-learning model designed to improve molecular property predictions by explicitly modeling four-body interactions. The model is designed to be computationally efficient and maintains E(3)-equivariance, ensuring consistency with physical symmetries. Experiments on MD22 and SPICE datasets suggest that MABNet outperforms existing methods in predicting molecular energies and forces.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the detailed review and insightful comments. Below, we provide point-by-point responses.\n\n> Limited discussion of computational cost.\n\nR: We have added a detailed discussion of computational cost in the revised manuscript, along with timing comparisons against baselines.\nWe have included timing information for key baselines to provide a better understanding of the computational cost of our method relative to others. The updated results will be included in the supplementary material.\n\n|Methods\t|\tMemory Used  |  Training Time (mins)\t|  Inference Time (mins) |\n| --- | ----------- | ----------- | ----------- |\n| ViSNet\t |\t15962 MiB |\t7.12 |\t38.95 |\n| MABNet (3-body) |\t16542 MiB |\t8.05 |\t42.65|\n| MABNet (4-body) | 17400 MiB\t| 9.39 | \t49.79 |\n\n> Limited discussion of baseline hyper-parameters.\n\nR: We have added a detailed description of the hyperparameters used for baselines in the revised manuscript and supplementary materials. Additionally, we have ensured that the comparisons involve models with similar embedding dimensions or comparable parameter counts.\n\n| Parameter\t| MABNet\t| ViSNet | Equiformer |\n| --- | ----------- | ----------- | ----------- |\n| emb. dim. |\t256\t|  256 |  128 | \n| layers |\t9 |\t9 | 6 |\n| Param.\t| 12.3M\t| 9.8M |\n\n> Missing error bars to assess the statistical significance of MABNet’s performance.\n\nR: We have conducted multiple independent runs of MABNet with different initializations and computed error variances on the test set. These error bars are now included in the revised manuscript to demonstrate the statistical significance of MABNet’s performance.\n\n|Ac-Ala3-NHMe| |\tAT-AT| |\n| --- | ----------- | ----------- | ----------- |\n|Energy|\tForces|\tEnergy|\tForces|\n|0.05336$\\pm$0.00117|\t0.07725$\\pm$0.00126|\t0.06375$\\pm$0.00183|\t0.07323$\\pm$0.00043|\n\n> Equations in Section 3 are difficult to follow.\n\nR: We apologize for the lack of clarity in our equations. In the revised manuscript, we have provided clear definitions of all variables, including \"s,\" \"a,\" and \"v,\" along with detailed explanations of their roles in the four-body attention mechanism. This should make Section 3 more accessible to readers.\n\nWe hope these revisions address your concerns and further strengthen our manuscript. Thank you again for your valuable feedback.", "claims_and_evidence": "The claims are clear:\n1. Introduction of an attention layer to model four-body terms in molecular conformations\n2. Competitive performance on two relevant benchmarks. \n\nGiven the importance of many-body interactions in determining molecular properties, the proposed architecture is sensible, and the reported experimental results are reasonable.", "methods_and_evaluation_criteria": "The evaluation is based on well-established molecular property benchmarks (MD22 and SPICE), using metrics like Mean Absolute Error (MAE) for energy and force predictions. The comparison against multiple baselines ensures a reasonable assessment.", "theoretical_claims": "The paper is methodological and does not present any new theoretical claim", "experimental_designs_or_analyses": "The experimental design follows a standard energy+forces regression paradigm. However, I didn't find any information concerning:\n\n1. The model hyperparameters for the baselines. It would be good to compare models that are \"close\" either in terms of the number of weights or in terms of the embedding dimension. (Table 8 discusses the embedding dimension of MABNet only, right?)\n2. Some timings for the other baselines.\n3. Error bars are missing. Either from multiple independent initializations of the networks or it is too costly, it is also sufficient to report the error variance on the test set. This would help in understanding whether the superior performance of MABNet is statistically significant.", "relation_to_broader_scientific_literature": "The work builds on advances in geometric deep learning, molecular property prediction, and many-body physics. It references prior methods, such as SE3Set, VisNet, or QuinNet. The \"Related Work\" Section provides enough context to the understanding of the paper.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Strengths:**\n\n- Novel direct modeling of four-body interactions.\n    \n- Strong empirical performance on molecular benchmarks.\n    \n- Theoretical grounding in many-body physics and equivariance.\n    \n\n**Weaknesses:**\n\n- Limited discussion on computational cost.\n\n- Limited discussion of the baseline hyperparameters.\n\n- While the paper is well written overall, I've found the equations in section 3 difficult to understand: the authors only explain what are the q and k variables without mentioning what are the \"s\", \"a\" or \"v\" variables.", "other_comments_or_suggestions": "Throughout the paper, the architectures based on message-passing and attention are presented as complementary options. However, they are not, and it is not clear where MABNet stands. As far as I understood, MABNet can be classified as a specific form of graph attention, and I think that giving a clearer context on the fundamental paradigms used to model molecules can improve the paper.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors present a novel machine-learned force field based on a four-body formulation of self-attention, which goes beyond existing approaches like the triangle attention used by AlphaFold. This enables the inclusion of dihedral angles and other rich four-body geometry in these force fields. They demonstrate state of the art results on MD22. Many reviewers had concerns about the additional computational overhead of the 4-body attention compared to existing methods, and it would be good to show if comparable accuracy could be achieved simply by increasing model size. One reviewer felt that the paper did not give VisNet sufficient credit, and claimed that the method in the paper was only an incremental advance, however the authors provided ablation experiments showing a noticeable increase in performance on MD22 relative to VisNet. As long as VisNet is properly credited and compared against in the paper, I am happy to accept.
Decision: Accept (poster)