Learning Long-Range Representations with Equivariant Messages
Authors: Egor Rumiantsev
Marcel F. Langer
Tulga-Erdene Sodjargal
Michele Ceriotti
Philip Loche
Venue: Accepted by TMLR ()

Abstract: Machine learning interatomic potentials trained on first-principles reference data are becoming valuable tools for computational physics, biology, and chemistry. Equivariant message-passing neural networks, including transformers, achieve state-of-the-art accuracy but rely on cutoff-based graphs, limiting their ability to capture long-range effects such as electrostatics or dispersion, as well as electron delocalization. While long-range correction schemes based on inverse power laws of interatomic distances have been proposed, they are unable to communicate higher-order geometric information and are thus limited in applicability. To address this shortcoming, we propose the use of equivariant, rather than scalar, charges for long-range interactions, and design a graph neural network architecture, Lorem, around this long-range message passing mechanism. We consider several datasets specifically designed to highlight non-local physical effects, and compare short-range message passing with different receptive fields to invariant and equivariant long-range message passing. Even though most approaches work for careful dataset-specific choices of their model hyperparameters, Lorem works consistently without such changes, with excellent benchmark performance.

Published in Transactions on Machine Learning Research (04/2026)
Learning Lange Representations with Equivariant Messages
Egor Rumiantsev"
eont.rumiontkeeddeppf.t
Laboratory of Computational Science and Modeling, Ecole Polytechnique Fédérale de Lausonne
Marcel F. Langer*t
marcel marrelllanger@eepf...c
Laboratory of Computational Science and Modeling, École Pokylechnique Fédérale de Lausanne
Tulga-Erdent Sodjargal
Vulga-erdene.s@kaise
Laboratory of Computational Science and Modeling, École Polytechnique Fédérale de Lausanne Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST)
Michele Ceriotti
michele.ceriotti@eef...cc
Laboratory of Compulational Science and Modeling, École Polylechnique Fédérale de Lausanne
Philip Lochet
philip. Saslip.loche@tumm..
Laboratory of Computational Science and Modeling, École Polytechnique Fédérale de Lausanne Department of Physics, Technicol University of Munich Alomistic Modelling Center, Munich Data Science Institute, Technical Universily of Munich Munich Center for Machine Learning (MCML)
Contributed equally
Corresponding authors
Reviewed on OpenReview: https: o/ porum? id=p219e4SW9P
Abstract
Machine learning interatomic potentials trained on firs--priccples reference data are becoming valuable tools for computational physics, biology, and chemistry. Equivariant pessage-passing neural networks, including transformers, achieve state-of-tho-art accuracy but rely on cutoff- based graphs, limiting their ability capture long-range effects such as pectrostatics or dispersion, as well as electron delocalization. While long-range correction schemes based on inverse power laws of interatomie distances have been proposed, they are unable to communicate higher-order geometric information and are thus limited in applicability. To address this shortcoming, we propose the use of equivariant, rather than scalar, charges for long-range interactions, and design a graph neural network architecture, LOREM, around this long-range message passing mechanism. We consider several datasets specifically designed to highlight non-local physical effects, and compare short-range message passing with different receptive ficlds invariant and equivariant long-range message passing. Even though most approaches work for careful dataset-specific choices of their model hyperparameters, LOREM works consistently without such changes, with excellent benchmark performance.

---

Published in Transactions on Machine Learning Research (04/2026)
1 Introduction
Machine learning interatomic potentials (MLIPs) for atomistic simulations are trained on quantum-mechanical simulations to predict energies and forces of new atomic structures. Most MLIPs assume locality: The energy of each atom depends only on neighbors within cutoff radius. This leads to linear scaling with respect to the number of atoms but fails for systems with long-range interactions, such as electrostatics, dispersion, or
electron delocalization. [Behler & Parrinello 2007 Ko et al.) 2021: Tkatchenko & Scheffler 2009 Grisafi
& Ceriotti 2019; Unke et al. 2021b Huguemin- Duniittnn et al. 2023] Several approaches aim to overcome
this limitation. Messago-passing graph neural networks (MPNNs, Gilmer al. (2017)) overcome locality by iteratively exchanging information belween neighboring aloms, but are still constrained by the number of message-passing steps, graph connectivity, and reduced information flow with increasing number of iterations
CCai & Wang 2020
Alon & Yahav 2020 Nigam et al. 2022).
An alternative approach is to use physics-inspired corrections to the total energy, written as an inverse power law of interatomic distances, 1/r" with = 1 for charge-charge p 3 for dipole-dipole, and p= 6 for dispersion. Some models predict partial atomic charges, either using explicit charge labels and equilibration schemes, or learming them implicitly from energies and forces include electrostatic terms (Unke al.)
2021a Ko et al. 2021 Fedik al. 2022 Pellegrini al. 2023 Maruf et al.] 2025 Cheng 2025 Physical
long-range interactions can also serve as building block: For instance, Kosmala et al. 2023 propose Ewald
message passing, i.e., the use of electrostatic interactions for message passing, and Grisafi & Ceriotti (2019)
propose the
equivariant (LODE) framework that uses inverse power-law interactions to compute equivariant features from the scalar potential around each atom.
In this work, we combine the strengths of physical inverse power law 1/r2 interactions with the ability of equivariant message passing communicate higher-order geometric information. Inverse power-law interactions are well-defined for periodic systems, have physically meaningful asymptotic behavior, and can be computed efficiently using established techniques from computational physics. Extending the idea of Ewald message passing, we treat charges equivariant objects and use inverse power-law potentials as a mechanism for long-range communication. Based on this mechanism, we design LOREM, an MLIP architecture that combines short- and long-range message passing. LOREM offers consistently high accuracy across a scries of long-range benchmark lasks, outperforming other short- and long-range message passing models.
Our contributions are:
We introduce an equivariant, global, long-range message passing mechanism that can leverage efficient methods from computational physics for asymptotic O(N log N) scaling in periodic systems, and potentially O(N) scaling in non-periodic ones,
We design novel MLIP architecture, LOREM, around this mechanism,
We conduct series of experiments probe the limits equivariant. short-range message passing to model long-range physics.
2 Background
Machine learning interatomic potentials Under the Born & Oppenheimer 1927 decouples the nuclear and the electronic degrees of freedom, the atoms in a molecule or material move on potential energy surface (PES) E = E (f (r,, Z = 1..NN} where r; and Z; are positions and atomic numbers for the atoms; in a periodic system, i.e., materials or liquids, the arrangement of atoms is repeated periodically in space, described by three cell vectors { c,, aa = 2,2,3) } and corresponding integer offsets [ne aa = 1, 2,3 so that each position r; is associated with the replicas {r; + Ea= C, n E Z }. The energy is then computed for the positions in the unit cell (n 0) considering their interactions with the infinite 'crystal' system. The forces, which drive the dynamics of the atoms, are defined as derivatives of the energy f. = -r, Traditionally, this PES has been approximated by physics-insping analytical expressions, called force fields, that are parametrized manually or through global optimization. In the past.
approximation, which
2

---

Published in Transactions on Machine Learning Research (04/2026)
decades, in tandem with the increasing availability of large datasets of quantum mechanical reference data, MLIPPs have emerged as less computationally efficient, but more accurate, data-driven alternative.
Atomistic graph neural networks Most MLIPss can be seen as graph neural networks (GNNs, Battaglia let al. (2018)) acting on a description of an arrangement of atoms in space as a geometric graph 9 - (E,V) with edges & corresponding to interatomic relative-position vectors ray and nodes (or vertices) V corresponding to atoms. Edges connect all nodes that lie within a cutoff radius Te of each other. In periodic systems, rij are constructed to respect periodic boundary conditions, i.e., if a replica lies closer than an original atom, Tij points to the replica. If more than one replica lies within the cutoff, multiple edges with different labels are drawn between nodes. By restricting the range of interactions to neighbors on this graph, linear scaling with the number of atoms N (at constant density) can be achieved; efficient scaling with system size is required make MLIPs practical for large-scale simulations. MLIPs typically predict E a sum over atomic contributions, E Σ> == E;, predicted from node features.
Invariance and equivariance The potential energy E is invariant under permutations, i.e., reordering,
of atomic positions, as well as global translations and rotations of the coordinate system E In other words, E is invariant under actions of the Euclidean symmetry group E(3) applicd to all positions (including replicas). MLIPs must respect these symmetries, at. least approximately. Translation invariance is respected by construction in atomistic GNNs through the use of relative-posiien position vectors edge labels. Permutation invariance is typically ensured through commutative aggregation functions, for instance sums. Finally, rotation invariance can either be learned through data augmentation (Pozdnyakov & Ceriotti (0233 Langer et al. (2024): ; Domina et al. (0266), or ensured by requiring that internal features remain aware of their geometric meaning, i.e., that they are equivariant to rotations. While in principle, MLIPs could be constructed from invariant features only, equivariant internal features have been found to improve accuracy and data efficiency by allowing the model to access orientation information (Thoas et al. (2018); Batzner et al. [2022); Batatia
ct al. 2022)). A thorough discussion of oquivariance for MLIPs can be found in other works (Smidt,) 2021
Cnke & Maennel 2024); essentially, we consider the transformation of internal features of the models under rotations E S3) applied equally to all geometric inputs (positions and cell vectors), leading to joint rotation of all bulk positions. Consider the MLIP up to some hidden layer as a function f X + Y, where X = (N+3) for periodic systems and = Ra*N for molecules. Equivariance is defined by f o = go for all g E SO(3): rotations can be equivalently applied before or after f- To ensure that this is the case, internal features must be constructed as direct sum of different irreducible representations of SO(3), indexed by 4, combined with feature (channel) dimension Y (0/ g2(+1 & RS Rotations act as linear transformations on these irreducible representations. We write spherical features as lensor S with the last three indices corresponding to the representation order 7= 0, 0ma., the component m = -I,..,,, and the channel index c. Such tensors are therefore ragged: Different / are associated with different nunbers of components m. The = 0 components are called scalars and are invariant under rotations. Collections of purely scalar features are also denoted as matrices P. In Appendix A we describe a number of operations that can be applied to spherical features without disrupting equivariance.
Long-range interactions The potential energy E arises from the many-body Schrödinger equation, which involves only Coulomb interactions between electrons and muclei without any range separation. Efficient approximations, such as empirical force fields or MLIPs, typically restrict interactions to local environments, motivated by the nearsightedness principle of electronic matter (Prodan & Kohn (2005) ), which states that electronic properties are insensitive to distant perturbations. In the long-range regime, interactions reduce to inverse power laws 1/r0 of the interatomic distance. Since in nature no fixed nearsightedness length scale exists, MLIPs must capture both local many-body quantum effects, possibly extending beyond the model's cutoff re, and formally infinite-range electrostatic interactions. Additional complexity arises from charge distributions that depend on distant atoms and from electron wavefunctions that may delocalize over large distances; see Section 0
In fact, the energy invariant under global inversions well, ic., under the action of the full group 0(3). To simplify notation in this manuscript, we focus on proper scalars and tensors, i.e., irreducihle representations of SO(3) (which does not contain reflections). All arguments and operations can readily generalized to 0(3).
3

---

Published in Transactions on Machine Learning Research (04/2026)
Ewald summation The evaluation of inverse power-law potentials 1/r" has been the object of much study in computational physics and chemistry. The general task is to compute the potential V, = (rr)) at a given atomic position ri, induced by (generalized) point charge: 2 placed at the position of other atoms j:
N V
Σ J= RE
Σ
4 r; (ry + n1 0202 + 13C3)
(1)
Where |-| denotes the vector norm. In periodic systems, Eq. implics an infinite sum over replicas, denoted by Eneza and defined by the cell vectors Ca; for non-periodic systems (e.g,, molecules), this sum and the extra term the denominator are omitted. For p E 3, the periodic, infinite, sum converges conditionally, i.c., convergence or divergence depends on the summation order. Ewald (1921) summation was developed lackle this problem for electrostatics (p- = 1) and later extended to other exponents; its basic concept is to split 1/r into short-ranged part, which converges fast. in real space, and long-range part, which is smooth, and therefore converges well, in reciprocal space. A naive implementation of Ewald summation scales O(N2), which can be brought down to ((N3/2) by choosing the cutoffs to be proportional to the size of the simulation cell. This is, however, undesirable for ML.PPs which are typically constructed and trained for a fixed cutoff radius. Particle mesh Ewald (PME, P3M) algorithms reduce this to O(N log N) by interpolating charges onto a grid and employing the fast Fourier transform for the reciprocal-space part (Darden et al., 1993 Hockney & Eastwood] 2021p. While such methods are standard in force fields, implementations in popular machine learning frameworks, PyTorch (Paszke et al. 2019) and JAX (Bradbury et al. 2018), have only become available recently (Lcche et al., 2025). In non-periodic systems,, the naive evaluation of Eq. (I) scales O(N2). However, methods like the fast multipole expansion can reduce this cost. to O(N) [Greengard & Rokhlin 1987). Another approach is multi-level summation (Hardy et al.) 2015). which scales O(N) for non-periodic and periodic systems and has recently been implemented in JAX (Buchner et al. 2025). For small systems on the order hundred atoms, naive implementations Ewald summation for periodic systems and direct. summation for non-periodic systems- are typically faster than more complex, but better-scaling, methods; this is benchmarked, for instance, in Appendix and Loche et al. (0255 As we largely focus on small systems in this work, naive implementations throughout.
3 Related work
Equivariant message passing Bond-order potentials first introduced the idea of repcatedly updating atomic environments to extend interactions beyond the cutoff radius re (Tersofi, 1988 Brenner 1990).
principle now central to modern MLIPs. In MPNNs (Gilmer et al. 2017], atoms exchange messages over M steps, so features and energies depend on neighbors within M Te- Letting & P. denote the features at atom and message-
message-passing passing step k:
1++1 M, = Σ mm(( PP,)))
(2)
3 P. = === x+1 M,)
(3)
with Iearned message and update functions and Early models used invariant updates (Schütt al.
2017
Xie & Grossman 2018), later extended to equivariant ones (Gasteiger et al. 2019 Schütt al. 2021
Batatia et al. 2022 Batzner et al. 2022: Frank et al.) 2024b). Equivariant MPNNs are restricted in the
choice of operations within the network, as they must retain equivariance throughout (see Appendix A
Recent universal MLIPs trained on big datasets (Batatia et al. 2025 Wood et al.) 2025 Mazitov et al. 2025: Rhodes et al. 2025) sometimes replace message passing with local sell-attention. While effective
capturing semi-local interactions, MPNNs cannot model true long-range effects, since distant atoms without
intermediates never interact, and many steps reduce expressivity (Cai & Wang 2020 Alon & Yahav 2020
Nigam al. 2022).
For = I, Le., electrostatics, common speak charges. For p > 1, for example dispersion (p= 6), 'coefficients' is more common.
4

---

Published in Transactions on Machine Learning Research (04/2026)
Physics-based long-range models Many long-range models explicitly add physics-inspired terms to E that capture interactions decaying more slowly with distance. A common example of such a form is given by Eq. I glong = E: 9: V where are latent per-atom descriptors (e.g., partial charges or polarizabilities predicted by the model and V; is computed as an inverse power law of interatomic distances. Usually, these 9 are predicted directly as scalar functions of local atomic Teatures (Unke el. al. 2021a Staacke et al. 2021
Loche et al.) 2025 Kim et al., 2024] Ji et al., 2025; Kabylda, et al.; 2025; Cheng 2025). A modification of this approsch is to include the qi in an equilibration scheme, thus allowing these descriptors to capture information otherwise missed due to their initial dependence on local environments (Ko et al. 2021 Pellegrini
et al. 2023 Maruf et al. 2025). Grisafi & Ceriotti 2019); Sogguenin-uumitan et al. (2023
propose to use
physics-inspired kernels to compute long-range features, mathematically equivalent to a multipole expansion, but find that higher-order features contain little additional information.
Other long-range models Some models avoid handcrafted corrections and instead learn long-range interactions directly. Ewald message passing augments GNNs with Fourier-space invariants (Kosmala el al. 2023), while fully connected approaches use all-to-all distances (Chmiela et al. 2018) or global attention (Unke) ct al. 2021a). though these lose efficiency distance information. Lincar-scaling attention with geometric embeddings (Frank et al., 2024a) enables global orientation exchange but needs symmetrization and has not yet been adapted to periodic systems. Alternatives include virtual nodes for global aggregation (Caruso) et al. 2025) or message passing in spherical harmonics space (Frank et al. [022]. Despite approximations, most methods still struggle to bridge periodic and non-periodic systems. Some methods outside atomistic modeling also aim to capture long-range effects (Dwivedi al., 2022: Bamberger et al., 2025 Moskalev et al.
2025
Zhdanov ct al. 2025 but require further adaptation to include geometric information or periodicity.
4
Equivariant long-range message passing
Ewald summation, as introduced in Section 2 and Eq. ( allows the efficient, and convergent in periodic systems, evaluation of a potential Vú at each atomic position, based on coefficients qj associated with all other atoms j and a power of the inverse distances 1/r./. This computation can be seen as a physics-inspired form of message passing (Eqs. (2) and (3)), with radial filter 1/r9y, neighbor feature qt, and resulting message V;; this correspondence was used by Grisafi & Ceriotti 2019) to compule general long-range features and cast. in the language of MPNNs by Kosmala et al. [2023].
To bring it more in line with standard message passing, the operation can be carried out in parallel across an extra feature dimension, promoting d to an array q: This allows to communicate more information, but in this form, this information is restricted to geometric invariants. We propose to promote q to an equivariant tensor instead: Qt..m. This results in equivariant messages, or potentials:
N Σ
Qy..m
Vid.m =
(4)
T - (ry + nic + m262 + n2ca)
TEZA
carrying out the Ewald summation over each spherical order 1 and component m in parallel.
To see that the result is equivariant, recall (see Appendix A that adding two equivariant objects yields another equivariant object, and that multiplication With a prefactor, provided that the factor is shared across all entries for a given 1, also retains equivariance. The prefactor 1/r?, depends on the pair (i.j) and therefore varies across pairs; however, for any given pair, it is a single scalar that multiplics all m components equally within each L. Since the action of rotations on spherical features scalar multiplication commutes with it. It is then easy to see that multiplying Q, with a factor 1/r², yields an equivariant quantity, and that the sum in Eq. (4) also yields an equivariant, since all summands are equivariant objects. We argue and numerically confirm in Appendix B that compensating background charge correction preserves equivariance.
linear map along the m index, this
Overall, this approach allows the global aggregation of oquivariant messages in a way that amenable to efficient implementation, scaling O(N log N) in periodic systems (see Appendix E and-at least in principle, see Section 2 O(N) in finite systems. Even more advanced computational methods can bring O(N) scaling
5

---

Published in Transactions on Machine Learning Research (04/2026)
to both. The method also naturally accommodates the long-range effects in physical systems, which decay asymptotically following a power law.
5 LOREM
P
X
Rudial coeffcieats
A P
MIP
LaverNorm
rú Ca
Av
Regandim
Tnd
Buneediing
K
Update
P
LayerSorm
Short Range MP
Y!
P
X
S
M
Radial coefficients
Linear
r;; C
Mg
M
Long- Rauge
AS!
Linear
Linnar
Lincar
Update
Short- Mhort-Rangeena MP
Long Range MP
2
Q
Update
1++1 Ps
Update
X.M
M
S
M P.
Vi
MLP
Sphorieal norm
MLP
Multi-layer perceptzon
Emnbediing
Embedding lnokup
Tensor product Concatenation
Multiplication
Eelge pode aggregation
layer
cad
Cutoff function
Ropandon
expansion
Figure Sketch of the LOREM architecture. See main text for a dctailed description.
LOREM follows the blueprint of equivariant MPNNs, making a few modifications: Scalar and spherical features are handled separately, following Frank et. al. (2022 2024b), and mixed using variant of the power spectrum
BBrtóók et al. 2013) and residual update block He et al. 2015).
Overview LOREM, illustrated in Fig. maps a point cloud of N atomic positions (ra), (r,), potentially contained in periodic unit cell C1, C2 C3, and labeled with chemical species { 24 J. to atomic energies { { } which, when summed, yicld the total potential energy E. Internally, the point cloud processed as a graph, with edges between nodes, the atoms, defined by a cutoff radius re- Accordingly, edges correspond to vectors Tiy = T; connecting atoms and j. Node features are updated through either short-range message passing long-range message passing. Spherical information is used update scalar features by computing their spherical norm and passing it through update block. Updates of scalar features are followed by residual prediction of atomic energy contributions. We discuss the main components of LOREM below; specialized equivariant operations are described in more detail in Appendix A
Short-range message passing Initial scalar features PP are a learned embedding of chemical species. At each short-range message passing step k edge features K,, are obtained from distances riş and scalar features *P; and & *P, through radial expansion. These edge features are then linearly transformed twice, with different learned weight matrices: Once to yield scalar messages that aggregated into scalar updates, and

---

Published in Transactions on Machine Learning Research (04/2026)
again to yield pre-factors for the spherical harmonics Y ig (angular expansion), which are combined with neighboring spherical features Sj in a tensor product 3 The results are aggregated into a spherical message, which updates the spherical features via another tensor product, resulting in S+ The spherical norm of the updated spherical node features is then used to further update the scalar features, finally yiclding k+ P, This process repeated M times (the total number of short-range message passing steps).
Long-range message passing We use the method discussed in Section communicate equivariant information beyond the effective interaction cutoff of short-range message passing. To minimize computational cost, which is proportional to the number of channels over which Ewald summation is carried out, node features are transformed into low-dimensional charges: The scalar features M P, are transformed into single per-atom charge 9:, and spherical features SS; are likewise, using a linear transformation followed by a self tensor product, transformed into Q.d.m with a low mmax, LR and a singular feature dimension. Unless otherwise noted, we use mmxx, LR = 2. The scalar charge is concatenated with the = spherical charge, which yields a total of 10 charge channels for mmax, LR = =2 _ After Ewald summattonn, which is carried out, in parallel across and m, the potentials are split back into the purely scalar V₁ and the spherical V,. The spherical potentials are combined with spherical node features through tensor product; the spherical norm of the result is concatenated with the scalar polential to updale scalar representations We find that using only p=1, i.e., the Coulomb interaction, rather than set of different exponents, is sufficient in practice. Experiments for mmxx 0, 1, can be found in Appendix
Update block Updates Y to scalar features X use an update block consisting of multi-layer perceptrons (MLP) and layer normalization (LayerNorm)
(Ba et al, 2016), following a residual structure (He et al. 2015
X + X + MLP(Y)
+ LayerNorm (X)
X x MLP(X)
X LayerNorm(X))
Radial expansion LOREM processes information about the distance between two aloms Tay through learned coefficients for linear transformations of an initial radial basis expansion of ray. Distances Túj are first expanded in Bernstein polynomials multiplied with feut, the cosine cutoff function (extending from 0 to re), yielding initial radial features Pij.e- These features are multiplied with a weight matrix A, which is in turn obtained through MLP (and subsequent reshaping operation) applied to the concatenated scalar atom features P. and P,. This allows the model to learn a radial basis based on the features of both atoms and j. The resulting edge features are called K.j.
Y(fu), which are polynomials of vector components that produce outputs in different irreducible representations of SO(3) (Unke & Maennel 2024). During message passing, they are multiplied with per-! coefficients: Yi fj..mm. A.j.t ijd.m. where Aij are obtained as linear transformation of Kij, followed by reshaping operation. Since the factors are shared across m, this preserves equivariance.
Angular expansion The vectors Ti are expanded in spherical harmonics Yus
6 Experiments
We perform experiments on a number of existing benchmark tasks designed to probe the ability of MLIPs to model long-range interactions, comparing LOREM to short and long-range MLIPs. The experiments are divided into two parts: In Section 6.1. we compare LOREM with other models using standardized settings. We find that LOREM performs well, but also observe that most tasks can be solved by models that do not. consider long-range interactions at all the effective interaction range of typical message passing models is
"In the initial message passing step, no spherical node features available, and therefore the tensor product with S, is omitted. Initial spherical node features are obtained via self tensor product (dotted lines) rather than tensor product of the updates with the previous features. "Two entries with I=0: One from the scalar features and one from the scalar component of the spherical features.
7

---

Published in Transactions on Machine Learning Research (04/2026)
Table 1: Root mean squared errors (RMSE) for energy E and forces f for datasets and models used in Section 6.1 Where available, a held-out test set was used; otherwise, the validation set was used instead and indicated in the table. The lowest error is indicated in bold, the second lowest is underlined. The second line after each model name indicates the number of short-range (SR) meessage-passingg steps and whether form of long-range (LR) interactions are included in the model A mean absolute error (MAE) version of this table is provided in Appendix
LOREM
CACE-LES 1xSR+LR
MACE 2xSR
PET 2x SR
4G-NN 1xSR+LA
SPOOKYNET 6XSR+LR
Dataset
1XSR+LR
MgO surface (Validation)
E (meV/au) /at)
0.064
0.071
0.376
0.210
0.219
0.107
(meV/A)
4.076
7.913
5.971
6.261
66.000
5.337
NaCI cluster
E (mcV/ade /at)
0.112
0.210
1.681
1.517
0.481
0.135
(Validatiou)
f (meV/As
1.155
9.784
40.219
42.438
32.780
1.052
Biodimers
E (meV /at) (meV/A)
0.222
2.259
7.793
6.758
1.646
3.163
16.150
16,470
Cumulene E (meV/at) /at)
3.309
17.803
12.592
3.205
f (meV/A)
50.084
147.616
104.318
46.905
wc probe this limit of short-range message passing. We find that while careful consideration of the number of message--assinn passing steps and the cutoff radius required to resolve long-range interactions with short-range models, LOREM can solve these benchmarks without adaptation. Additional experiments can be found in the appendix: Runtime benchmarks comparing Ewald and particle-mesh Ewald implementations of LOREM, demonstrating near-linear scaling up to 30 atoms in Appendix F ablations of mmax, Ln and the long-range block in Appendix F well competitive performance the larger ADAPT dataset (Dramko et al. 2025 in Appendix J
sufficient. However, predictions break down beyond this interaction range. In Scction 6.2
Models We compare LOREM with a number of purcly short-ranged MLIPs, MACE and PET, as well as the recently introduced CACE-LES model that combines short-range message passing with a scalar long-range part, and 4G-NN, which includes a physics-based based long- range energy contribution and charge equilibration. Additionally we include metrics for SpookyNet (Unke et al. 2021a), which predicts scalar partial charges and nuclcar dipoles for long-range corrections, together with global attention. Full details on models can be found in Appendix D approximate parameter counts are: LOREM ~ M, CACE-LES 700 k for MgO and ~ k for others, MACE 800 k, PET ~,,, 4G-NN ~5 (estimated), SPOOKYNET ~ M (from (Blücher et al. 2023 ).
The datasets and associated benchmark tasks used in our experiments are briefly introduced below; additional details are given in Appendix C
An overview of validation test set (where available) metrics for all datasets is given in Table [ LOREM performs very well across datasets, and is competitive with, or more accurate than, other models.
MgO surface This benchmark task is the first in a series designed by Ko et al. (2021) to highlight the need for long-range information in MLIPs. Illustrated in Fig. 2 it consists of magnesium oxide (MgO) surface on which gold (Aug) dimer is placed. Depending the presence of aluminum (Al) dopant deep inside the surface, the lowest-energy position of the gold dimer is cither in the 'wetting' (flat), or 'non-wetting' (standing up), position. The benchmark consists of two parts: Correctly identifying the ordering between upright and flat geometries in the doped and undoped case, and reproducing the energy-distance curve for the non-wetting gcometry, in particular the local minimum corresponding to the equilibrium distance.
NaCl cluster Similar to the presence of a dopant modifying the potential-energy surface for the gold dimer in the MgO surface task, this benchmark by Ko et al. (2021) relies on the presence or absence of a sodium atom at one end of a charged sodium chloride (NaCI) cluster changing the behavior of a sodium atom at the
SAG-NN training requires DFT-computed charges, which are not available for the biodimers and cumulene datasets.
8

---

Published in Transactions on Machine Learning Research (04/2026)
opposite end: Since atom is removed while the charge remains constant, the charge must redistribute over the remaining atoms. The benchmark task consists of reproducing the location of the local minimum and the energy profile when moving the sodium atom farthest from the removed one, indicated in Fig, 2C
Cumulene The cumulenee benchmark task was proposed by Unke et al. (2021b) as an example of a long-range problem that is due to a non-local effect of the electronic structure of a molecule. This molecule consists of a chain of nine carbon atoms, with pair of hydrogen atoms, the rotors, at the opposite ends. The orientation of the rotors determincs the shape of the atomic orbitals for each carbon, which propagates along the chain to the other end; in the absence of bending or stretching, the energy is therefore fully determined by the relative orientation of the rotors. The task is to recover the energy profile of this rotation for idcalized fully extended geometry, illustrated in Fig. 3
This benchmark dataset by Huguenin- Daminntt-DDmitann et al. 2023) consists of pairs of relaxed organic molccules placed at distances of 4 A to 15 A from cach other, illustrated in Fig. 4A. Depending on the chemical nature of the molecules, interactions between the molecules take different asymptotic power-law forms, ranging from charge-charge 1/r interactions to apolar-apolar 1/r interactions. Here, we simply compare energy and force prediction errors on test sets stratified by the dominating power-law interaction.
Biodimers
SN2 reactions Finally, the Sx2 reactions benchmark was introduced by Frank et al. (2024a to probe the ability of MLIPs to model the long- range interactions required to mediate gas- gas-phase chemical reactions. It consists of the nucleophilic substitution of methyl halides by another halide ion: X + HHC-Y X-CH,+ where X,Y F, CI, Br, I. The benchmark task in this case is predicting the energy along the reaction coordinate, i.e., correctly modeling the potential energy profile the two reactants approach one another, react, and separate again.
6.1 Standardized settings
We compare LOREM to other models using standardized settings: Two message-passing steps for short-range models, one message-paseee passing step for long-range models, and re = 5 Å for most models (see Appendix Dj.
MgO surface The results for this experiment can be seen in Fig. 2 Despite being designed to require long-range interactions, this benchmark task can be solved by both purely short-range and long-range presage-passingg models. In this case, the success of short-range message passing is due to the small size of this benchmark system: With effective cutoff radius above A,, centrally located atoms 'see' the full system and hence, MACE and PET can solve this task. All models also resolve the orientation preference for the Au dimer between the doped and undoped surfaces.
NaCl cluster The results of this experiment, seen in Fig. 2D, are drastically different from the previous one: Here, only models with a long-range component are able to resolve the difference between Ca,Clg* and Clg. All such models show excellent agreement with reference values. The failure of short-range message-passinn is due to the larger system size compared to the MgO surface: Here, effective cutoff radii exceeding 10.5 Á are required to solve the benchmark task.
Cumulene Resolving the cunulene energy profile, seen in Fig. 3 requires simultaneous knowledge of the orientation of both rotors at opposite ends of the molecule. This be achieved in two ways: Through equivariant short-range message passing (MACE, PET), provided that the chain is not too long (see Section 6.2 or through equivariant long-range message passing (LOREM). For this reason, CACE-LES cannot solve this benchmark: Scalar charges are not sufficiently expressive to communicate relative orientation. All equivariant models that are able to access this information can solve this benchmark, achieving good agreement with the reference data. PET resolves the dihodral angle as well, but requires long training and an adaptation in the number of transformer layers to succeed at this benchmark (see Appendix D). We note that the sharp cusp 180 is an artifact of the underlying reference method; it is desirable behavior of MLIPs to smooth it out.

---

Published in Transactions on Machine Learning Research (04/2026)
A
B
Doped
Undoped
non- won-wetting wetting
Reference
4G-NN IXSR
LOREM IxSR +1R
MACE 2
CACE-LES
Emin (meV)
~ 12 A
dopant
2.2
2.3
2.1
Au-O Distance (A)
Au-O Distance (A)
NaşCIg
NaaCl
21
B Emin (mey)
10
3.4
3.5
3.6
3.3
3.4
3.5
Na-Na Distance d (A)
Na-Na Distance d (A)
Figure 2: (A) Atly dimer MgO surface, showing both wetting and non-wetting geometries, well as the Al dopant. (B) Energy over distance d for the non-wetting geometry for the doped and undoped surface. The minima are indicated with a diamond symbol; the reference energy curve is drawn in grey. Offsets are added to distinguish the curves and the value at the minimum is subtracted. (C) Na (top) and NagClgt (bottom) cluster, the moving atom marked with transparent copies of itself, and the distance of interest labeled with d. (D) Energy over distance for both clusters.
A
B
3.3
Reference
MACE 285R
LOREM 1xSR LR
PET 2x SR
CACE-LES
5.8 A
Energy (eV)
0.4
1.3 Á
0.2
1.1
2.1 A
0.0
90
120
150
180
Dibedral angle 0 (Degrees)
Figure 3: (A) Mustraatioo of cunulene laid flat, indicating relevant distances between atoms. (B) Energy profile over a 90 rotation of one rotor. The minimum value of each curve is subtracted before plotting, The inset shows 3D representation of cumulene, defining the dihedral angle 0.
10

---

Published in Transactions on Machine Learning Research (04/2026)
A
B
LOREM
20
CACE-LES
Forces MAE MAE (IeViA)
15
MACE
10
PET
15A
AA
CA
CC
CP
PA
PP
Dimer class
Figure 4: (A) Charge-charge pair from the biodimers dataset, (B) Mean absolute error on forces for different models on the different dimer classes: Apolar-apolar (AA), charge-apolar (CA), chargo-charge (CC), charge- polar (CP), polar-apolar (PA), polar-polar (PP). Note that the vertical axis has been split at 2.75 meV/A.
Biodimers Since the pairs of molecules in the biodimers benchmark are placed at separations up to 15 A, much beyond the cutoff used for graph construction, high accuracy requires a long-range component. This is confirmed by the results of this experiment, seen Fig. 4 The long range LOREM and CACE-LES models yield lower error than MACE and PET. Predictive error varies between dimer classes, i.e., the expected type of inverse power-law interaction, for all models, with the exception for LOREM, which yields consistent, and in all but one class the highest, accuracy. It is important to note that LOREM long-range message passing while different exponents describe the underlying physics of the dimer classes: The nonlinearity after the long-range message passing block may allow the model to correct this mismatch between exponents; see also the supplement of ((uguenin-Dumiitan Dumittan et al. 2023
p (Coulomb) for
6.2 Limits of short-range message passing
In the previous section, observed that the performance of models with short-range message passing strongly depends problem-specific choice of the effective interaction cutoff. The biodimers experiment also demonstrated that message passing fails when no intermediate atoms are present. To study these failure modes, we perform additional experiments using LOREM, varying cutoff and long-range message passing.
SN2 reactions Solving the benchmark task for the Sx2 reactions dataset re- quires long-range interactions, since it involves modeling intra-molecular inter- actions over distances exceeding the typ- ical cutoffs used for MLIPs. This con- firmed by Fig. which compares the performance of LorEM with and with- long-range message passing. The former reproduce the energy over the course of the reaction with excellent accuracy, including the tails as the reac- lants approach and separate. The latter, which does not include long-range inter- actions, cannot account for the tails and consequently predicts constant once the molecules separate more than 5 Á.
Reference IxSR LR 2xSR
-200
Energy (meV)
4400
-600
.000
Reaction coordinate
Figure 5: Energy over the reaction coordinate for the mucle- ophilic substitution reaction CI CIT HHCC-BB Br C1-CHg Br;; snapshots are shown as insets.
11

---

Published in Transactions on Machine Learning Research (04/2026)
Table 2: Ability of different LOREM models, with and without long-range message passing and with different numbers of short-range message passing steps, to solve the cumulene benchmark task. A tick (V) indicates yes, a cross (x) indicates no. Figure shows the full curves.
1 1xSR LR
2xSR + LR
Cutoff
1xSR
2xSR
3xSR
4XSR
2.5 A
V
X
x
x
v
3.0 Å
V
X
X
V
v
3.5
X
V
V
V
Cumulene with different cutoffs We probe the dependence of the performance of message-passing models on their hyperparameters by training a set LOREM models with and without long-range message passing at different cutoffs (2.5 A, 3 Á and 3.5 A) and numbers of short-range message passing steps (1, 2, and 4). Small cutoffs are chosen simulaat the longer chain lengths of real biomolecular systems. The results are shown in Table 2 with additional plots in Appendix G In all cases, models that include long-range message passing are able to resolve the angle. Short-range models, on the other hand, can only resolve the angle in certain combinations of hyperparameters: One message passing step is never sufficient. For two, a minimum cutoff of 3.5 is required. For three message passing steps, 3.0 is required. Therefore, an effective cutoff substantially larger than the 5.8 Å distance between the central carbon atom in the chain and the hydrogen rotors is needed. This is because MPNNs rely on the input graph structure (sce Fig. 3A) for information flow: Below re 2.6 A, connections in the graph only extend to nearest neighbors, with the exception of the rotors and the second-to-last carbon atoms. Consequently, at least four message passing steps are required. This example illustrates that short-range message passing is difficult to apply to this class of problems: Parameters have to be adapted to the chain length, with careful consideration of graph structure. In contrast, long-range message passing is robust, requiring no change in model hyper-parameters to solve this task.
7 Discussion
We introduced a simple yet effective equivariant rang-aange message passing scheme: Latent equivariant charges are predicted from local node features, and well-ostablished techniques for ovaluating inverse power-law potentials are used to efficiently compute long-range messages in way that is convergent and well-defined for periodic systems. Building on this message message-passing mechanism, we developed LOREM, an MLIP architecture that achieves consistently strong performance across benchmarks that require accurate long-range modeling.
By construction, our model assumes that interactions decay asymplotically with distance. While it is therefore not suited for learning truly global representations that have no notion of locality, this limitation largely theoretical for physical systems: Electrostatics dominates most long-range behavior in realistic systems, and even other effects typically do not extend over arbitrary distances.
A more practical limitation arises in non-periodic systems, where the cost of naive long-range message evaluation scales as O(N2). Although this is acceptable for small molecules and unit cells, this is a bottleneck for larger systems. However, lincar-scaling methods for the evaluation of inverse power-law potentials are available: Fast multipole methods (Greengard & Rokhlin 1987 Andy T. Jones 2020) or multi-level summation (Hardy et al., 2015 Buchner et al., (025)). Being able to leverage such methods is a key advantage of our proposed physics-inspired prressgg-passinng scheme.
We also investigated the capabilities of purely short-range message passing and found that, in many cases, it performs very well-even on datasets explicitly designed to require long-range interactions or charge equilibration. However, its success depends on matching the cutoff radius and number of message passing steps to the specific task, a process that can be both tedious and error-prone, More fundamentally, short-range methods cannot resolve interactions between distant atoms without intermodiarics. Our augmentation with long-range message passing overcomes these limitations, providing robust results across different tasks without requiring changes to model architecture or its hyperparameters (cutoff radius, number of message-passing
12

---

Published in Transactions on Machine Learning Research (04/2026)
steps, representation order). Training hyperparameters such as learning rate, optimizer, and loss weights are tuned per dataset, as is standard practice for ML.Ps.
Finally, our results underscore a broader issue: the lack of challenging long-range benchmarks. Many current datasets are based on simplified, small-scale systems and fail to capture the complexity of real-world applications where long-range interactions are essential. Our deliberate focus on these benchmarks reflects the proof-of-concept nature of this work: On large, heterogeneous datasets, improvements in aggregate loss metrics can often be achieved by increasing the capacity of the short-range model, making it difficult to isolate and attribute gains to improved modeling of long- rangrange physics The controlled benchmarks we consider allow clearer mechanistic validation of the proposed equivariant Ewald message passing. Nevertheless, as a step towards larger-scale evaluation, we present results on the ADAPT silicon point-defect dataset (Dramko et al.) 2025) in Appendix J where LOREM achieves competitive force accuracy and substantially better energy predictions compared to all basclines, using default model hyperparameters. Addressing the gap of challenging large long-range datasots is a key direction for future work, which requires the development of applicalion-orientee benchmarks.
-umtennin-Dmmttann et al.) 2023
Reproducibility statement
Data, code, configuration files, scripts, and trained models are available at doi:10.528] zenodo..100 1942194499999 Scripts include data pre-processing, model training, model evaluation, and the creation of figures and tables in this work. Hyperparameters are also described in Appendix D The reference implementation of LOREM in JAX is available at httts:://iitub.... httpttubbcccommaaaaooooo//rraaaa jax a PyTorch implementation as part of
metatrain (Bigi et al., 2026) at httpp://ggtthhubccccm////attttes
An overview of code and
data repositories can also be found at htttps//naareal.cceene///oorr
Acknowledgements
MFL acknowledges funding from the German Research Foundation (DFG) under project number 5449947822. MFL,, MC, and PL acknowledge funding from the European Research Council (ERC) under the rescarch and innovation programme (Grant Agreement No. 11001890-FIAMMAN ER, MC, and PL ackuowledge funding from the NCCR MARVEL, funded by the Swiss National Science Foundation (SNSF, grant mumber 182892). The authors would like to thank Matthias Kellner, Filippo Bigi, J. Thorben Frank, Bingqing Cheng, and Niklas Schmitz for insightful discussions, and also thank Romain Greub, Luis Falke, and Harsh Talathi for contributing initial experiments on the ADAPT dataset.
References
Joseph W. Abbott, Carlos Mera Acosta, Alaa Akkoush, Alberto Ambrosetti, Viktor Atalla, Alexej Bagrets, Jörg Behler, Daniel Berger, Björn Bieniek, Jonas Björk, Volker Blum, Saeed Bohloul, Connor L.. Box, Nicholas Boyer, Danilo Simoes Brambila, Gabriel A. Bramley, Kyle R. Bryenton, María Gómara Christian Carbogno, Fabio Caruso, Sucismita Chutia, Michele Ceriotti, Gábor Csányi, William Dawson, Francisco A. Delesma, Fabio Della Sala, Bernard Delley, Robert A. DiStasio Jr, Maria Dragoumi, Sander Driessen, Mare Dvorak, Simon Erker, Ferdinand Evers, Eduardoo Fabiano, Matthew R. Farrow, Florian Fiebig, Jakob Filser, Lucas Foppa, Lukas Gallandi, Alberto Garcia, Ralf Gehrke, Simiam Ghan, Luca M. Ghiringhelli, Mark Glass, Stefan Goedecker, Dorothea Golze, Matthias Gramzow, James A. Green, Andrea Grisafi, Andreas Grüncis, Jan Günzl, Stefan Gutzeit, Samuel J. Hall, Felix Hanke, Ville Havu, Xingtao He, Joscha Hekele, Olle Hellman, Uthpala Herath, Jan Hermann, Daniel Hernangómez-Pére,, Oliver T. Hofmann, Johannes Hoja, Simon Hollweger, Lukas Hörmann, Ben Hourahine, Wei Bin How, William P. Huhn, Marcel Hülsberg, Timo Jacob, Sara Panahian Jand, Hong Jiang, Erin R. Johnson, Werner Jürgens, J. Matthias Kahk, Yosuke Kanai, Kisung Kang, Petr Karpov, Elisabeth Keller, Roman Kempt, Danish Khan, Matthias Kick, Benediki P. Klein, Jan Kloppenburg, Alexander Knoll, Florian Knoop, Franz Knuth, Simone S. Köcher, Jannis Kockläuner, Sebastian Kokott, Thomas Körzdörfer, Hagen- Henrik Kowalski, Peter Kratzer, Pavel Kús, Raul Laasner, Bruno Lang, Björn Lange, Marcel F. Langer, Ask Hjorth Larsen, Hermann Lederer, Susi Lehtola, Maja-Olivis Lenz-Himmer, Moritz Leucke, Sergey Levchenko, Alan Lewis, O. Anatole von Lilienfeld, Konstantin Lion, Werner Lipsunen, Johannes Lischner, Yair Litman, Chi
13

---

Published in Transactions on Machine Learning Research (04/2026)
Liu, Qing-Long Liu, Andrew J. Logsdail, Michael Lorke, Zekun Lou, Iuliia Mandzhieva, Andreas Marek, Johannes T. Margral, Reinhard J. Maurer, Tobias Melson, Florian Merz, Jörg Meyer, Georg S. Michelitsch, Teruyasu Mizoguchi, Evgeny Moerman, Dylan Morgan, Jack Morgenstein, Jonathan Moussa, Akhil S. Nair, Lydia Nemoc, Harald Oberhofer, Alberto Otcro-de-la-Roza, Ramón L. Panadés-Bar Patlolla, Mariia Pogodaeva, Alexander Pöppl, Alastair A. Price, Thomas A. R. Purcell, Jingkai Quan, Nathanicl Raimbault, Markus Rampp, Karsten Rasim, Ronald Redmer, Xinguo Ren, Karsten Reuter, Norina A. Richter, Stefan Ringe, Patrick Rinke, Simon P. Rittmeyer, Herzain I. Rivera-Arrieta, Matti Ropo, Mariana Rossi, Victor Ruiz, Nikita Rybin, Andrea Sanfilippo, Matthias Scheffter, Christoph Scheurer, Christoph Schober, Franziska Schubert, Tonghao Shen, Christopher Shepard, Honghui Shang, Kiyou Shibata, Andrei Sobolev, Ruyi Song, Aloysius Soon, Daniel T. Speckhard, Pavel V. Stishenko, Muhammad Tahir, Izumi Takahara, Jun Tang, Zechen Tang, Thomas Theis, Franziska Theiss, Alexandre Tkatchenko, Milica Todorović, George Trenins, Oliver T. Unke, Álvaro Waldschmidt, Han Wang, Yanyong Wang, Jürgen Wicferink, Jan Wilhelm, Scott Woodley, Jianhang Xu, Yong Xu, Yi Yao, Yingyu Yao, Mina Yoon, Victor Wen-zhe Yu, Zhenkun Yuan, Marios Zacharias, Igor Ying Zhang, Min- Ye Zhang, Wentao Zhang, Rundong Zhao, Shuo Zhao, Ruiyi Zhou, Yuanyuan Zhou, and Tong Zhu. Roadmap on Advancements of the FHl-aims Software Package, June 2025.
Thamss
Oscar van Vuren, Daniel
Uri Alon and Fran Yahav. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations, October 2020. URL https://ppenneevlee..nttt
Andy L Jones. pybbfim, 2020. URL https://www.github. http:///adddllooo/////bbbaaa
Jimmy Lei Ba, Jamic Ryan Kiros, and Geoffrey E. Hinton. Layer Normalization, July 2016.
Jacob Bamberger, Benjamin Gutteridge, Scott le Roux, Michael M. Bronstein, and Xiaowen Dong. On Measuring Long-Range Interactions in Graph Neural Networks. In Forty-Second International Conference on Mochine Learning, June 2025. URL http:p/s//pppprereeeeeeeeeereeee wttf:/oorrrrrii11a2/o000001
Albert P. Bartók, Risi Kondor, and Gábor Csányi. On representing chemical environments. Physical Review B, 1111)11184 May 2013. doi: 1...00//PyysRReBB88..111...
llyes Batatia, David P. Kovacs, Gregor Simm, Christoph Ortner, and Gabor Csanyi MACE: Higher Order Equivariant Mossage Passing Neural Networks for Fast and Accurate Force Fields. Advances in Neural In- formation Processing Systems, 35:11423 11436, December 2022. URL http:ttp://pproceenng.eeeeee/ .tml
/c/
llyes Batatia, Philipp Benner, Yuan Chiang, Alin M. Elena, Dávid P. Kovács, Janosh Riebesell, Xavier R. Advincula, Mark Asta, Matthew Avaylon, William J. Baldwin, Fabian Berger, Noam Bernstein, Arghya Bhowmik, Filippo Bigi, Samuel M. Blau, Vlad Cärare, Michele Ceriotti, Sanggyu Chong, James P. Darby, Sandip De, Flaviano Della Pia, Volker L. Deringer, Rokas Elijošius, Zakariya El-Machachi, Edvin Fako, Fabio Falcioni, Andrea C. Ferrari, John L. A. Gardner, Mikolaj J. Gawkowski, Annalena
Janine George, Rhys E. A. Goodall, Jonas Grandel, Clare P. Grey, Petr Grigorev, Shuang Han, Will Handley, Hendrik H. Heenen, Kersti Hermansson, Cheuk Hin Ho, Stephan Hofmann, Christian Holm, Jad Jaafar, Konstantin S. Jakob, Hyunwook Jung, Venkat Kapil, Aaron D. Kaplan, Nima Karimitari, James R. Kermode, Panagiotis Kourtis, Namu Kroupa, Jolla Kullgren, Matthew C. Kuner, Domantas Kuryla, Guoda Liepuoniute, Chen Lin, Johannes T. Margraf, loan-Bogdan Magdäu, Angelos Michaelides, J. Harry Moore, Aakash A. Naik, Samuel P. Niblett, Sam Walton Norwood, Niamh O'Neill, Christoph Ortner, Kristin A. Persson, Karsten Reuter, Andrew S. Rosen, Louise A. M. Rosset, Lars L. Schaaf, Christoph Schran, Benjamin X. Shi, Eric Sivonxay, Tamás K. Stenezel, Christopher Sutton, Viktor Svahn, Thomas D. Swinburne, Jules Tilly, Cas van der Oord, Santiago Vargas, Eszter Varga- Umbrich, Tejs Vegge, Martin Vondrák, Yangshuai Wang, William C. Witt, Thomas Wolf, Fabian Zills, and Gábor Csányi. A foundation model for atomistic materials chemistry. The Journal of Chemical Physics, 163(18):184110, November 2025. ISSN 0021-9606. doi: 0111066/500970000
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Gonzzlzz,,,zzezzz Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis
14

---

Published in Transactions on Machine Learning Research (04/2026)
Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks, October 2018.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E(3)-cquivariant graph neural networks for data-cfficient and accurate interatomic polentials. Nature Communicationss, 13(1), May 2022. ISSN 2041-1723. doi:
Jörg Behler and Michele Parrinello. Generalized Neural-Network Representation of High-Dimensional Potential Totential-Energy Surfaces. Physical Review Letters, 9814))144640,, April 2007. ISSN 0031-9007, 1079-7114. doi: 10. 100.//pbysollllt.....44444
Filippo Bigi, Joseph W. Abbott, Philip Loche, Arslan Mazitov, Davide Tisi, Marcel F. Langer, Alexander Goscinski, Paolo Pegolo, Sanggyu Chong, Rohit Goswami, Pol Febrer, Sofiia Chorna, Matthias Kellner, Michele Ceriotti, and Guillamme Fraux. Metatensor and metatomic: Foundational libraries for interoperable atomistic machine learning. The Journal of Chemical Physics, 164(6):064113, February 2026. ISSN 0021-9606. doi: 10.1063/5.030499
Stefan Blücher, Klaus-Robert Müller, and Stefan Chmiela. Reconstructing Kernel-Based Machine Learning Force Fields with Superlinear Convergence. Journal of Chemical Theory and Computation, 19(14):4619 4630, doi: 1.002//acs.jett220013...
July 2023. ISSN 1549-9618.
M. Born and R. Oppenhcimer. Zur Quantentheoric der Molekeln. Annalen der Physik, 389 (0): January 1927. ISSN 0003-3804, 1521-3889. doi: 1..1002/and.... 1927389200..
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Mille and Qiao Zhang. JAX: Composable transformations of Python+NNumPy programs, 2018. URL htttp://giihub.coo cotp://mm/oooj//am//ma/a-aa
Donald W. Brenner. Empirical potential for hydrocarbons for use in simulating the chemical vapor deposition of diamond films. Physical Review B, 12(15):9455) 9471, November 1990. doi: 10. 10.1103/PP hhth/hyyyyBBB2..... B.4.9945.
Florian Buchner, Johannes Schörghuber, Nico Unglert, Jesús Carrete, and Georg K. H. Madsen. msmJAX: Fast and Differentiable Electrostatics on the GPU in Python, October 2025.
Lori A Burns, John C Faver, Zheng Zheng, Michael S Marshall, Daniel GA Smith, Kenno Vanommeslaeghe, Alexander D MacKerell, Kenneth M Merz, and C David Sherrill. The biofragment database (bfdb): An open-data platform for computational chemistry analysis of noncovalent interactions. The Journal of chemical physics, 147(16), 2017.
Chen Cai and Yusu Wang. A Note on Over-Smoothing for Graph Neural Networks, June 2020.
Alessandro Caruso, Jacopo Venturin, Lorenzo Giambagli, Edoardo Rolando, Frank Noé, and Cecilia Clementi. Extending the RANGE of Graph Neural Networks: Relaying Attention Nodes for Global Encoding, February 2025.
Bingqing Cheng. Cartesian atomic cluster expansion for machine learning interatomic potentials. nyj Computational Materials, 10(1):157, July 2024. ISSN 2057-3960. doi: 10.108//115224024-1133--
Bingqing Cheng. Latent Ewald summation for machine learning of long-range interactions. npj Computational Materials, 11(1):80, March 2025. ISSN 2057-3960. doi: (0.10088/$11224-05---1777711
Stefan Chmiela, Huziel F. Sauceda, Klaus- Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned force fields. Nature Communications, 9(1):8887, September 2018. ISSN 2041-1723. doi: 10.1038/s11166--016-0001669
Tom Darden, Darrin York, and Lee Pedersen. Particle mesh Ewald: An N log(N) method for Ewald sums large systems. The Journal of Chemical Physics, 180129200000-10092, June 1993. ISSN 0021-9606. doi: 10. 10.1063/1.46444 1.449977
15

---

Published in Transactions on Machine Learning Research (04/2026)
Michelangelo Domina, Joseph William Abbott, Paolo Pegolo, Filippo Bigi, and Michele Ceriotti. How unconstrained machine-learning models learn physical symmetries, March 2026.
Evan Dramko, Yihuang Xiong, Yizhi Zhu, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, and Anastasios Kyrillidis. ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs, September 2025.
Geneviève Dusson, Markus Bachmayr, Gábor Csányi, Ralf Drautz, Simon Etter, Cas van der Oord, and Christoph Ortner. Atomic cluster expansion: Completeness, efficiency and stability. Journal of Computa- tional Physics, 454:110946, April 2022. ISSN 0021-9991. doi: 10. 10.1016/1.jp.22222 10946.
Vijay Prakash Dwivedi, Ladislav Rampášek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long Range Graph Benchmark. In Thirty-Sirth Conference on Newral Information Processing Systems Dalasets and Benchmarks Track, June 2022. URL Sttpp:://ppeneeiieenete//ee
id=in7XC5RcjBn
P. P. Ewald. Die Berechmung optischer und elektrostatisshhe Gitterpotentiale. Annalen der Physik, 369(3): 253 253-287, 287, 1921. ISSN 1521-3889. doi: 191002/aandp.11211600000
Nikita Fedik, Roman yubatyuk, Maksim Kulichenko, Nicholas Lubbers, Justin S. Smith, Benjamin Nebgen, Richard Messerly, Ying Wai Li, Alexander 1. Boldyrev, Kipton Barros, Olexandr Isayev, and Sergei Tretiak. Extending machine learning beyond interatomic potentials for predicting molecular properties. Nature Reviews Chemistry, 6(9):653 662, September 2022. ISSN 2397-3358. doi: 10.1038/s41570-00 022-00416-3.
J. Thorben Frank, Stefan Chmiela, Klaus-Robert Muller, and Oliver T. Unke. Euclidean Fast Attention: Machine Learning Global Atomic Representations at Linear Cost, December 2024a.
J. Thorben Frank, Oliver T. Unke, Klaus-Robert Müller, and Stefan Chmiela. A Fuclidean transformer for fast and stable machine Icarned force fields. Nature Communications, 15(1):6539, August 2024b. ISSN 2041-1723. doi: 10.1038 010003///114477-22--5566202
Thorben Frank, Oliver Unke, and Klaus-Robert Müller. So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems. Advances in Neurol Information Processing Systems, 35:29400-29411 December 2022. URL htttps:/pproceedings.eeee neeredddiininnn.n.....e.. wwtppppp.ccpppeppppppppppppp (aappper_filleesppe/////000
Awtree--bbtrttt-eeeeeee-e-ae--ee Conference html
Johannes Gasteiger, Janck Groß, and Stephan Günnemann. Directional Message Passing for Molccular Graphs. In International Conference on Learning Represenlations, September 2019. URL httpa:
net ner/foorum?idd-BeeWbssss
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantun Chemistry. In Proceedings of the 34th International Conference on Machine Learning, pp. 1263-1272. PMLR, July 2017. URL hateps://proceedingss mlr ptwss//7o//iimerr1/11
L Greengard and V Rokhlin. A fast algorithm for particle simulations. Journal of Computational Physics, 73 (2):325-30 348, December 1987. ISSN 0021-9991. doi: 10. 10.1016/0000 (016//0021-99911/7//)009
Andrea Grisafi and Michele Ceriotti. Incorporating long-range physics in atomic-scale machine learning. The Journal of Chemical Physics, 151 (20):00444, November 2019. ISSN 0021-9606. doi: 10.1063/1.512588
David J. Hardy, Zhe Wu, James C. Phillips, John E. Stone, Robert D. Skeel, and Klaus Schulten. Multilevel Summation Method for Electrostatic Force Evaluation. Journal of Chemical Theory and Computation, 11 (2):766-779, February 2015. ISSN 1549-9618. doi: 10.1021 (0.1021/ct5009072 /c5500075
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition, December 2015.
R. W. Hockney and W. Eastwood. Computer Simulation Using Particles. CRC Press, Boca Raton, March 2021. ISBN 978-0-367-80693-4 doi: 1.1200/9780367706333.
16

---

Published in Transactions on Machine Learning Research (04/2026)
Kevin K. Huguenin-Dumiee Philipp Loche, Ni Haoran, and Michele Ceriotti. Physics-Inspired Equivariant Descriptors of Nonbonded Interactions. The Journal of Physicul Chemistry Letters, 14(43))9612-90) 9618, November 2023. doi:
Yajie Ji, Jiuyang Liang, and Zhenli Xu. Maarning Interatomic Potentials for LonggRange Systems, February 2025.
Keller Jordan, Yuchen Jin, Vlado Boza, You Jiacheng, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL. https://kellerrooddan .iit.ub io/posta/muon/
Adil Kabylda, J. Thorben Frank, Sergio Suarez Dou, Almaz Khabibrakhmanov, Leonardo Medrano Sandonas, Oliver T. Unke, Stefan Chmicla, Klaus-Robert Müller,, and Alexandre Tkatchenko. Molccular Simulations with a Pretrained Neural Network and Universal Pairwise Force Fields, June 2025.
Dongjin Kim, Daniel S. King, Peichen Zhong, and Bingqing Cheng. Learning charges and long-range interactions from energies and forces, December 2024.
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, January 2017.
Tsz Wai Ko, Jonas A. Finkler, Stefan Goedecker, and Jörg Behler. A Nourth-generation high-dimensional neural network potential with accurate electrostatics including non-local charge transfer. Nature Communications, 12(1):398, January 2021. ISSN 2041-1723. doi: 10.1038/ s4 467-020-20427-2.
Arthur Kosmala, Johannes Gasteiger, Nicholas Gao, and Stephan Gümnemann. Ewald-based Long- Range Message Passing for Molecular Graphs. In Proceedings of the 40th International Conference on Machine Learning, pp. 17544 17563. PMLR, July 2023. https: ptottp::/prroceeingg mlr www.mmrrr..ss..... httpppss///222//osaaaala//// html
Marcel F Langer, Sergey N Pozdnyakov, and Michele Ceriotti. Probing the effects of broken symmetries in machine learning. Machine Learning: Science and Technology, 5(4):04LTO1, October 2024. ISSN 2632-2153. doi:
Philip Loche, Kevin K. Huguenin-Dumiin Melika Honarmand, Qianjun Xu, Egor Rumiantsev, Wei Bin How, Marcel F. Langer, and Michele Ceriotti. Fast and flexible long-range models for atomistic machine learning. The Journal of Chemical Physics, 162(14):142501, April 2025. USSN 0021-9606. doi: 041063/5.0251713.
Moin Uddin Maruf, Sungmin Kim, and Zeeshan Ahmad. Equivariant Machine Learning Interatomic Potentials with Global Charge Redistribution, March 2025.
Arslan Mazitov, Filippo Bigi, Matthias Kellner, Paolo Pegolo, Davide Tisi, Guillaume Fraux, Sergey Pozdnyakov, Philip Loche, and Michele Ceriotti. PET-MAD, a lightweight universal interatomic potential for advanced materials modeling, August 2025.
Artem Moskalev, Mangal Prakash, Junjie Xu, Tianyu Cui, Rui Liao, and Tommaso Mansi. Geometric Hyena Networks for Large-scale Equivariant Learning. In Forty-Second International Conference on Machine Learning, June 2025. URL htttts:://opprreeee httppp///pppaeeeeeaaaeee////aaap
Jigyasa Nigam, Sergey Pozdnyakov, Guillauree Fraux, and Michele Ceriotti. Unified theory of atom-centered representations and message-passing machine-learning schemes. The Journal of Chemical Physics, 156(20): 204115, May 2022. ISSN 0021-9606. doi:
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Porformance Deep Learning Library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL htttps://papotts nips. http:///w...a..aoo/////////aaaaa
html
17

---

Published in Transactions on Machine Learning Research (04/2026)
Franco Pellegrini, Ruggero Lot, Yusuf Shaidu, and Emine Küggükbennl.. PANNA 2.0: Efficient neural network interatomic potentials and new architectures. The Journal of Chemicall Physics, 159(8):084117, August. 2023. ISSN 0021-9606. doi: 0..1063/5.015800
John P. Perdew, Kieron Burke, and Matthias Ernzerhof. Generalized Gradient Approximation Made Simple. Physical Review Letters, (7(18):3865-388 October 1996. doi: 10.1103 Bets:/yysseeeeet777.....
Sergey Pozdnyakov and Michele Ceriotti. Smooth, exact rotational symmetrization for deep learning on point clouds. Advonces in Newral Information Processing Systems, 36:79469
79501, December 2023. URL httpp://prooceddngs.eeee neurips. http:/ss....rrr.....ppp....eee wwwps///pppee//2////eaeeeeee Tla476622255772228ccbbbb22 httpr//eeceeeeeeeeeaeaaaraaaa-rr
html
E. Prodan and W. Kohn. Nearsightedness of electronic matter. Proceedings of the National Academy of Sciences, 102(33):11633-116633 August 2005. doi: 211073/pnas.00044555
Benjamin Rhodes, Sander Vandenhaute, Vaidotas Simkus, James Gin, Jonathan Godwin, Tim Duignan, and Mark Neumann. Orb-v3: Atomistic simulation at scale, April 2025.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: A contimuous-filee convolutional neural network for modeling quantum interactions. In Advances in Newral Information Processing Systems, vol- ume 30. Curran Associates, Inc., 2017. URL https://papers: wnis..nii............ www.:.iwpppppppppee cc/paper 3B0e44699950332200555333336066-3 Abstaaat.... html
Kristof Schütt,, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In Proceedings of the 38th International Conference on Machine Learning, pp. 9377 9388. PMLR, July 2021. URL htttps://proceedinggg mlr prress/v139/schutt222 html
Tess E. Smidt. Euclidean Symmetry and Equivariance in Machine Learning. Trends in Chemistry, 3(2):82 85, February 2021. ISSN 2589-5974. doi: 10.1016/j.trechm.22222 2tt.hmm.222.0000000
Carsten G. Staacke, Hendrik H. Heenen, Christoph Scheurer, Gábor Csányi, Karsten Reuter, and Johannes T. Margraf. On the Role of Long- Range Electrostatics in Machine-Learee Interatomic Potentials for Complex Battery Materials. ACS Applied Energy Materials,
November 2021. doi:
J. Tersoff. New empirical approach for the structure and energy of covalent systems. Physical Review B, 37 (12):6991-7000, April 1988. doi: 10.1103 Btt////ysSeev.3..6600
Nathaniel Thomas, Tess Smidt,, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and transsatioon-equivvriand neural networks for 3D point clouds, February 2018. /btp/abb.11022.21199 URL https://arxiv.org/aaa
Alexandre Tkatchenko and Matthias Scheffler. Accurate Molecular Van Der Waals Interactions from Ground- State Electron Density and Free-Atom Reference Data. Physical Review Letters, 102(():073005, February 2009. ISSN 0031-9007, 1079-7114. doi: http:/phhreeieet112......
Oliver T. Unke and Hartmut Maennel. E3x: ((3ivvqqivariant Deep Learning Made Easy, November 2024.
Oliver T. Unke and Markus Meuwly. PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. Journal of Chemical Theory and Computation, 15(6):3678 3668-363,, June 2019. ISSN 1549-9618. doi: /a1021accs.ettc.bb00
Oliver T. Unke, Stefan Chmiela, Michael Gastegger, Kristof Schütt, Huziel Sauceda, and Klaus-Robert. Müller. SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nature Communications, 12(1), December 2021a. ISSN 2041-1723. doi: 10.0008//4146670001--77404-
Oliver T. Unke, Stefan Chmicla, Huziel E. Saucoda, Michael Gastegger, Igor Poltavsky, Kristof T. Schütt, Alexandre Tkatchenko, and Klaus-Robert Müller. Machine Learning Force Fields. Chemical Reviews, 121 (16):10142-100 101666, August 2021b. ISSN 0009-2665. doi: 10.102) (/001/acs.chee hwss.chemrev.0001111
18

---

Published in Transactions on Machine Learning Research (04/2026)
Brandon M. Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso-Luque, Kareem Abdelmagsoud, Vahe Gharakhanyan, John R. Kitchin, Daniel S. Levine, Kyle Michel, Anuroop Sriram, Taco Cohen, Abhishek Das, Ammar Rizvi, Sushree Jagriti Sahoo, Zachary W. Ulissi, and C. Lawrence Zitnick. UMA: A Family Universal Models for Atoms, June 2025.
Tian Xie and Jeffrey C. Grossman. Crystal Graph Convolutional Neural Networks for Accurate and Interpretable Prediction of Material Properties. Plyssccal Review Lellers, 120(14):145301, April 2018. doi: Paa.oo//hyyseeLeet..... 145301.
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsich. Large Batch Optimization for Deep Learning: Training BERT in 76 minutes. In International Conference on Learning Represenlations, September 2019. URL
https://pennrrreww.nn wttp://www.....oooooooooooooaaaa
Maksim Zhdanov, Max Welling, and Jan- Willem de Meent. Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems. In Forty-Second International Conference on Machine Learning, June 2025. URL http:://oooaaeeiieeeeeee//rrrrrr
19

---

Published in Transactions on Machine Learning Research (04/2026)
A Equivariant modules in Lorem
As discussed in Section 2 only certain operations can be applied to equivariant features 5 without disrupting equivariance. We briefly discuss the operations used in LOREM; a more thorough discussion can be found
in Smidt 2021]: ; Unke & Maennel (2024). All operations are implemented using the e3x library (Unke &
Maennel 2024). We work with spherical features 5, dropping the atom index i, as operations are typically broadcast across or pairs of atoms.
S is a three-dimensional tensor Stmmc in which 1 enumerates the order of the irreducible representation of SO(3), the 2/ + 1 components of that irreducible representation, and the channel (feature) dimension. For fixed and St..c can be thought of as generalised Cartesian vector. Indeed, for I = 1 it is vector in three-dimensional space.
Prerequisites A rotation E SO(3) applied to all inputs causes a corresponding rotation of the spherical features expressed in terms of irreducible representations. For a set of features with order 1, this rotation is represented by matrix R(g) E R2++x22++ and acts on these features through matrix multiplication along the m index,
S Lmm.
Σ m' (g)
(5)
g
In other word, the action of rotations on spherical features is linear map.
Addition, Multiplication, Linear layers Since the action of rotations is linear, adding two spherical features of equal together does not disrupt equivariance. By the same logic, any multiplication features that is broadcast along the m index, i.e., that scales spherical features of order equally, is permissible. Therefore, a linear layer can applied lo spherical features, provided it is only applied to the chaunel index, and broadcast along m. We therefore define a learned linear transformation as
spherical
(tnear(SS...)) = Σ Wi S
(6)
with per-/ learned weights W A bias term could be applied 0, i.e., the scalar part of the spherical features, but choose not to.
Tensor products Two spherical features of order 4 and la can be combined into new spherical feature with (3 using a specialized tensor product
Tensor(SSi.mp.. Qig.ma.) Ulg.mai 12.ss
Σ m21m
Cladaus mi Magm Sty.mi Q1zma.
Chidails are the Clebsch- Clebsch-Gordan Gordan coefficients, and d/id2.. E IR is a per-channel weight vector for given combination of 1 12, 13- This tensor product can be carried out for every valid combination 1| 12 S la S 14 + 12)- In LOREM, unless specified otherwise, lensor products are only carried out to fixed maximum dimension (max, which all spherical features have in common, not the maximum possible one. Some operations, for example the preparation of the cquivariant message-passing block, perform a tensor product only to a specified target order. If both inputs of the tensor product are the same feature, we call the operation a self-tensor product. This operation increases the body-order of the representation (Dusson et al. 2022 Batatia et al.] 2022).
To predict invariant quantities, we require a way to extract invariant information from spherical, equivariant, features. One way to do achieve this is a tensor product to target order / = 0. Another, which is used in LOREM, is to take the norm of each irreducible representation
Spherical Norm
Norm(S) V
(21 + 1) Σ Stmm.
(21
(8)
m
20

---

Published in Transactions on Machine Learning Research (04/2026)
We found empirically that the prefactor (21 + 1) (2//1) helps reduce variance across L. As opposed to tensor product to 1= 0, this operation, inspired by the power spectrum (Bartók et al. 2013), keeps the norms per separate; a tensor product would linearly combine all into one.
B Invariance of Lorem
As explained in Section 4 and Appendix A the potential at each atom, V; is equivariant because the long-range message passing step consists of equivariant operations: Addition and multiplication with a factor shared per 1. In practical implementations of Ewald summation periodic systems, there is one extra step: To prevent divergences, the total charge must be zero, which can be done either by simply subtracting the sum of the total charge from each charge, equivalently, by subtracting an analytical correction from the potentials. Both approaches are equivalent, and since summation is equivariant, also do not break invariance. For this reason, the total procedure retains equivariance.
To numerically confirm these considerations, we compute the mean absolute error of energy predictions over a degree L = Lebedev grid of rotations (Domina et al. 2026), plus inversions, with respect to the unrotated casc, for the first structures of the MgO surface validation set. The errors are line with precision expectations: For single precision, 1.031 10-6 meV and for double precision 5.913 10-15 meV.
C Dataset construction
Mg0 surface The MgO surface dataset from Ko el. al. (2021) is used without modification. A representative snapshot of the unit cell is shown in Fig. 2 The dataset is built from four configuration types, each derived from distinct initial structure:
1. A pure MgO surface with the Au dimer oriented perpendicular to the surface,
pure MgO surface with the Au dimer oriented parallel to the surface,
An Al-doped MgO surface with the Au, dimer perpendicular to the surface, and
4. An Al-doped MgO surface with the Au, dimer parallel to the surface.
Each of these initial configurations was first geometry optimized. For the two perpendicular, 'non-wetting', cases (1 and 3), the distance between the lower Au atom and the O atom directly beneath it was systematically varied, displacing the Au, dimer as a whole. From these distance-dependdet samples, a subset was randomly selected. To introduce structural diversity, Gaussian noise was applied to each configuration: a standard deviation of 0.02 À for atoms in the MgO substrate and 0.1 Á for the gold cluster. After perturbation, 1250 structures were selected from each of the four configuration types, yielding a total of 5000 samples. For these, energies and forces Perdew Burke Ernzerhof (PBE) functional (Perdew et al. 1996). A random 90/10 train validation split was used, as reported in Ko et al. (2021). Additional energy distance curves with equal spacing were constructed for the perpendicular cases and are shown in Fig. 3
computed using the FHI-aims code (Abbott et al., 2025) and the
energy-distance
NaCI cluster The NaCl cluster dataset from Ko et al. 2021] is used without modification. First, the Ca,Clg cluster was optimized vacuum. snapshot is shown in the top panel of Fig. 2 From this geometry, the Na atom farthest from all other Na atoms was removed, yielding the NagC1ş* cluster shown in the bottom panel of Fig. 2C. From these two structures, additional configurations were created by varying the distance between a selected pair of Na atoms along the line connecting them. In Fig. 2C, this moving atom is illustrated by transparent copics along its trajectory. Training datascts for both Nag and Nag were constructed by randomly sampling configurations along these trajectories. Gaussian noise with a standard deviation 0.05 was applied to the atomic coordinates, resulting in 2500 perturbed structures for each molecule and a total of 5000 instances. Energies and forces were computed using the FHI-aims code and the PBE functional. random 90/10 train-validation split was used, as reported by Ko al. (022)) Additional energy-distance curves with equal spacing were constructed and shown in Fig.
21

---

Published in Transactions on Machine Learning Research (04/2026)
The cumulenee dataset from Unke et al. (0221b) is used without modification. The geometry of the linear molecule is shown in Fig. 3 The dataset contains 4500 randomly sampled cumulene structures with nine carbon atoms, divided into training, validation, and test sets with 0000/500/2000 instances, respectively. The energy curves shown in Fig. BB are based on a controlled subset in which only one terminal carbon atom is rotated, while the rest. of the molecule remains fixed. visualization of this rotational motion is provided in the inset of Fig. 3.
Cumulenes
Biodimers The Biodimers dataset from Duguenin--umittan al. (2023) Burns al. (2017) is used without modification. It consists of 2291 relaxed organic sidechain sidechain fragments, including small molecules such as ethanol, acetamide, and others. Based molecular properties, the dataset is divided into six categories: each molecule is classified as either polar, apolar, or charged, resulting in the following dimer types-polaat pess--olar-ppaa (PP), polar-apolar (PA), charged-polar (CP), apolar- applarr (AA), charged-apolar (CA), and charged charged (CC). A representative CC dimer is shown in Fig. DA. The initial separation between molecules reflects their posit in protein structures. From this configuration, the intermolecular distance is incrementally increased up to 15 A, resulting in a total of 29 783 dimer instances. Instances where the final separation exceeds the initial distance by more than 4 Á (13 743 samples) are designated as the test set. The remaining 16 040 instances form the training set. Energies and forces were computed using the FHIlaims code and the HSE06 hybrid functional. For each of the six dimer types, a random 80/20 train- validation split was applied. The resulting subsets were then merged into a single training set and a single validation set.
Sx2 reactions The Sy2 reactions dataset from Unke & Menwly 2019 |; Frank et al. 2024a] is used without modification. It contains molccular structures in vacuum that model nuclcophilic substitution (Sx2) reactions. The dataset includes molecules of the types XCH,Y, CH3X, HX, CHX, CH₂X, XY, X, and Y, for all possible combinations of X, Y (F, CI, Br, I]. A representative reaction coordinate with corresponding snapshots is shown in Fig. 5 Additional species such as H2, CH2y, and CH3z are also included in the dataset. Training configurations were generated using ab initio molecular dynamics simulations at 5000 with time step of 0.1 fs. Full computations ional details, including the level of theory, are provided in the original publications. The dataset is randomly split. into 405 000 training, 5000 validation, and 42 708 test samples.
D Model description and training setup
For all models, training paramcters, and in some cases, model paramcters, vary slightly between datascts. Where we were unable to use models from previous work, we tuned parameters to minimize the error on the validation set aaming to perform a similar number of experiments, on the order of ten, per model and experiment. With this, we aim to report results that reflect a practical degree of reper-paramette tuning.
For all experiments, except the ones where variations were tested, the same LOREM model architecture (Section was used: A cutoff radius of 5 Å, 4max = 6 for spherical features and maxx LR = 2 for the long-range message passing, 128 scalar features, 8 channels for spherical features, and 32 radial basis functions. We perform only the initial short-range message passing step; performing additional steps typically increases accuracy but hinders comparison of long-range expressivity. This model has 021 198 learnable paramoters. It was implemented using the e3x library (Unke & Maennel| 2024). Training is performed entirely in float32 precision; while we find that reduced precision has only a minor effect on training dynamics, it can significantly alter validation and test results.
Lorem
Training parameters vary slightly between datasets; the exact parameters can be found in Table 3 We report errors for the better out of two runs with different seeds; the presented conclusions hold for both models.
In all cases, except N the learning rate was docayed lincarly after 10 epochs, from the indicated starting value to 1. 10;6; the ADAM (Kingmaa & Ba, 2017) and LAMB (You et al. 2019) optimizers were used. The loss function was a simple mean squared error, with the energy residuals normalized by number of atoms.
We note that minimizing the validation error does not always yicld the best qualitative agrecment: In the cumulene datasct, it does not readily correlate with ability to prodict the dihcdral curve Fig. 3 while all models below a certain error threshold are able resolve the task, the qualitative agreement varies. Here, exponential learning rate decay was employed.
22

---

Published in Transactions on Machine Learning Research (04/2026)
Dataset
Optimizer
Initial LR
Epochs
Batch size
E weight.
f weight
Time
MgO surface
ADAM
4. 10-4
4000
32
1000
1.0
17h
Biodimers
ADAM
1. 10-4
4000
32
0.5
0.5
42h h
Cumulene
LAMB
1. 10-3
2000
32
0.5
0.5
2h
NaCl cluster
ADAM
1- 10-3
8000
64
0.5
0.5
7h
Sy2 reactions
ADAM
1. 10-
500
32
0.5
0.5
17h
Table 3: Training settings for LOREM for different datasets. Training times are given for single Nvidia H100
SXM5 GPU.
The squared residuals were averaged over the whole batch, including over aloms and components in the case of forces, and then summed and weighted with a factor. The checkpoint with the lowest summed R² of energy and forces, evaluated on the validation set, was used for experiments. Training times are given for the entire run, not the time until the best checkpoint.
For NaCl and AuMgO, prior work (Kim et al. 2024 Ko al. 2021p reports error directly on the validation set, as the benchmark was originally intended Appendix H that, due to the statistical uniformity of the dataset, there is no significant difference to tuning hyper- prper-parameters systematically on an inner train/vall /validation split, keeping the data used for Table held out.
overfitting exercise. We follow this practice, but verify in
Mace The MACE MLIP (Batatia et (2022) is an equivariant message- messago-passing neural network as described in Section [ It is used with the standard setting of two message- passeppgsinnggggg steps, i.e., an effective cutoff of 10 A. Training generally used default hyperparamcters, with some adjustments made on hyperparameters related to training dynamics. Specifically, the cumulene system was trained using the default energy-to-forees loss weight ratio of 1:100 and '128x0e + 128x1o + 128x2e' hidden irreps. All other systems were trained using a 1:10 ratio and '128x0e + 128x1o' hidden irreps. The MACE model has 848 528 parameters.
All experiments, except those involving cumulenes and the NaCI cluster, used the SWA protocol, which swaps the loss weights between energy and forces at specified epoch. This epoch was set where the energy RMSE plateaued, with the subsequent training continued until loss saturation. All models were trained with Imax = 2, batch size of 32, and utilized cuEquivariance acceleration.
Pet PET (Pozdnyakov & Ceriotti 2023) is an unconstrained transformer model, consisting of multiple edge-to-edge transformer layers within local neighborhoods followed by message passing, Similar to MACE, it is also used with two message- mossage-passing steps and an effective cutoff of 10 A. To ensure approximate rotational invariance, trained with data augmentation. No inference-time symmetrizationn is used our experiments.
Unless otherwise specified, we used default hyperparameters (cutoff radius of 5 with cosine smoothing over the outermost 0.2 Å, dPET 128, dhead 128, dieedforward = 512, with 8 heads per attention layer, and 2 attention layers per GNN layer), with some adjustments made related to training dynamics. Models were trained using an basedd scheduler, which halved the learning rate after 250 epochs. This applies to all datasets except biodimers. For biodimers, à ReduceeLOOPPlateau scheduler was used instead, reducing the learning rate by 20% if the loss did not improve for 100 consecutive epochs. To improve training stability, biodimers training also employed gradient clipping with a maximum gradient norm of 5. Every training run included 10 warmup epochs, during which the learning rate was linearly increased from zero the preset learning rate of 1 x 14-
For the biodimers and NaCl cluster datascts, the targets were normalized by their standard deviation in the training set. The energy-to-forees loss weight ratio was set to 1:1 for biodimers and MgO surface datasets, while the NaCl cluster dataset used a ratio of 1:10.
For cumulene, PET was trained for 20 100 epochs with batch size of 16, equal energy and forces weight, and a maximum learning rate of 10-4, The learning rate was increased linearly from over 100 epochs, and then reduced by 10% every 500 epochs over the entire training run. Architecturally, 4 attention layers
23

---

Published in Transactions on Machine Learning Research (04/2026)
and cosine cutoff function was employed to resolve the cumulenes. We found that increasing the number of attention layers, and adapting the cutoff function, were critical to resolve this benchmark task. This PEr model consequently has 1 1836 804 parameters, while other models have 1 1043716.
Cace~Les The CACE-LES model was originally presented in Cheng 2024). with an additional modification introducing a long-range component in Cheng (025). Similar to MACE, CACE-LES is an equivariant message- passing neural network. After short-range message passing, invariant scalar pseudo-charges are predicted and passed the long-range part of Ewald summation; the resulting energy contribution is added to the energy predictions of the short-range message-passing model. We use this model with the recommended setting of the system, the following corresponding effective cutoff radii: 5.5 for the MgO surface, 5.29 A for the NaCl cluster-thest are the settings used in the original paper Kim et al. 2024) -add we chose cutoff of 5 Å for biodimers and cumulene.
one message-passing step and, depending
For the MgO surface and NaCI cluster datasets, models were taken from Kim et al. (2024). For the cumulenes and biodimers datasets, the hyperparameters were as follows: Bessel radial functions, c= 8, mmax 3, Vmaxx Nembedding = 2, one message-passingg layer, one-dimensional hidden variable, followed the example in the original paper: the first 200 epochs used an energy loss weight of 0.1 and forces loss weight of 1000, after which the energy loss weight was changed to 1, 10, and 1000 every 100 epochs, yielding total of 500 epochs. The learning rate was 5 x 10-3, with a step learning rate schedule decreasing it by a factor of 2 every 20 stops. The model for the MgO surface has 72340 paramcters, all others 12 808.
and dl 2. Training
4G-NN The 4G-NN model was designed to tackle the benchmarks introduced by Ko et al. (2021) and introduced in that work. It consists of a shallow neural network acting on rotationally invariant features, predicting both local energy contribution and an electronegalivity, which is then used in charge equilibration procedure that globally redistributes charges to minimize energy expression. This process can be as physics-inspired long-range message passing scheme iterated until fixed point is reached. As the model requires charge labels to train, which are not available for all datasets, we did not train this model for our experiments but instead include results from Ko et al. (2021), which are only available for the NaCI cluster and MgO surface datasets. The cutoffs used in the original paper are as follows: 4.23 Á for the MgO surface and 5.29 for the NaCl cluster. We estimale around 5k parameters for these models.
SpookyNet SpookyNeet (Unke et al. 2021ah an equivariant MLIP that predicts scalar partial charges and nuclear dipoles, which are used to compute long-range electrostatic and dispersion corrections via Ewald summation. Additionally, the model includes global attention without geometric information. The model has approximately M parameters (Blücher et al. 2023). Results for the MgO surface and NaCl cluster datasets are taken from Ko et al. (2021)
24

---

Published in Transactions on Machine Learning Research (04/2026)
10a
NaC1 (Ecdd
+
(i.dlE (PM8
Noti (PA2)
Cid (Bwak))
Car (PMI)
Inference time (ms)
10a
10
109
10
10
10a
10
Number of atoms N
Figure 6: Runtime scaling of LOREM with Ewald (dashed) and PME (solid) long-range implementations for
NaCI, CsCI, and CuzO supercells. Reference scaling lines (ox N2 and x N) are shown for comparison.
Table 4: Runtime of LOREM with Ewald and PME long-range implementations for NaCl supercells of varying
size N. LR denotes the time spent in the long-range block alone.
Ewald
PME
N
SR+LR (ms)
LR (ms)
LR (%)
SR+LR (ms)
LR (ms)
LR (%)
8
0.7
0.2
32.4
0.8
0.3
41.9
64
0.9
0.4
45.4
1.0
0.5
50.0
512
6.4
5.3
82.6
2.1
1.0
46.0
1728
61.2
58.4
95.5
4.8
2.1
42.6
4096
475.8
469.8
98.7
12.1
6.1
50.5
8000
20.2
8.9
43.8
17576
41.5
16.9
40.7
32768
101.2
51.6
51.0
E Runtime benchmark
To estimate the runtime overhead of the long-range block and compare the Ewald and particle-mesh Ewald (PME) implenentations, we benchmarked LOREM on supercells of three physical crystal struttuees: NaCI, CaCI, and Cu,O. For each crystal, we constructed supercells of increasing size and ran predictions (energy and forces) for LOREM models with and without the long-range block, using both Ewald and PME for the long-range evaluation. All benchmarks were repeated ten times and averaged, and executed on a single NVIDIA H100 GPU.
The scaling behavior is shown in Fig. 6 Ewald summation scales quadratically with system size and runs out of memory beyond approximately 8000 atoms; PME scales roughly linearly and extends to over 000 atoms. At 4096 atoms, PME is approximately 40x faster than Ewald. Detailed timings, including the fraction of total runtime spent the long-range block, are reported in Tables 4 to G With Ewald, the LR fraction grows from around % at small sizes to over 95 % at large sizes. With PME, the LR fraction remains roughly constant and depends on the density of the crystal: Cu.O (the densest, 35 neighbors per atom) shows around 25 %, while CsCl (the least dense, 14 neighbors) shows around 60 %. Overall, performance with PME on the order of single-digit ps/atom is in line with expectations for modern MLIPs.
25

---

Published in Transactions on Machine Learning Research (04/2026)
Table Runtime of LOREM with Ewald and PME long-range implementations for CsCl supercells of varying
size N.
Ewald
PME
N
SR+LR (ms)
LR (ms)
LR (%)
SR+LR (ms)
LR (ms)
LR (%)
0.7
0.3
42.7
0.8
50.2
54
0.8
0.4
48.1
0.9
0.5
55.4
432
5.1
4.3
84.5
1.7
0.9
52.4
1458
46.0
44.3
96.2
4.2
2.5
58.8
4394
710.1
705.7
99.4
11.0
6.7
60.6
8192
19.0
11.2
59.2
18522
42.5
25.5
60.0
31250
72.9
44.2
60.7
Table Runtime of LOREM with Ewald and PME long-range implementations for CuzO supercells of varying
size N.
Ewald
PME
N
SR+LR (ms)
LR (ms)
LR (%)
SR+LR (ms)
LR (ms)
LR (%)
0.7
0.3
42.2
0.9
0.5
55.6
48
0.8
0.4
42.8
1.0
0.5
52.5
384
2.5
1.5
58.6
1.7
0.6
38.4
1296
43.0
40.3
93.9
3.8
1.1
30.4
4374
177.1
169.1
95.4
11.0
2.9
26.5
7986
773.1
759.0
98.2
19.9
5.7
28.7
16464
40.5
11.3
27.8
29478
78.1
19.6
25.1
26

---

Published in Transactions on Machine Learning Research (04/2026)
Table 7: Root mean squared errors for energy and forces f for datasets used in Sec. 6.1 of the main text, for different LOREM variants. See Tab. of the main text for details.
LOREM
LOREM
LOREM
LOREM
Dataset
No LR
LRI=0
LR 1=
LRR 1 = 2
MgO surface
(mcV
2.234
0.063
0.062
0.064
(Validation set)
f (meV/AA
61.487
5.870
5.284
4.076
NaCl cluster
E (meV /at)
1.582
0.113
0.111
0.112
(Validation set)
f (m)V/A)
50.110
1.243
1.062
1.155
Biodimers
(meV /at)
8.259
0.302
0.329
0.222
f (meV/A)
16.452
1.677
1.725
1.646
Cumulene
E (mcV/at
16.203
15.793
5.576
3.309
f (meV/A)
126.000
133.092
85.956
50.084
F Ablations of LR block and
We repeated the experiments on the MgO surface, NaCl cluster, cumulene, and biodimers with LOREM models with = and 1, as well as without the long-range message-passing block. The parameter counts for no LR and 0, 1,2 2 (in that order) are 839129, 1020300, 1020749, and 1021198.
Results be seen in Table and Figs. to Generally, higher I improve error metrics. For experiments where only scalar long-range interactions are required (MgO surface, NaCI cluster), higher 1 do not improve qualitative agreement. The cumulene example, which requires access to relative orientation between rotors, requires equivariant long-range interactions with = 2 to be resolved. Removing the long-range message- passing block significantly increases error and renders the model unable solve all the benchmark tasks.
27

---

Published in Transactions on Machine Learning Research (04/2026)
A
B
Doped
Undoped
non-wetting wetting
Reference
LOREM, LR =
LOREM. LR
LOREM, No
LOREM, LR
Emin (me V)
122
dopant
20
2.2
2.3
2.4
2.1
2.2 Au-O Distance (A)
Au-O Distance (A)
C
D
NagCCg
NagClg
21A
E Emna (mevV)
3.4
3.5
3.6
3.3
3.4
3.5
Na-Na Distance d (A)
Na-Na Distance d (A)
Figure 7: (A) Aug dimer MgO surface, showing both wetting and non-wetting geometries, well as the Al dopant. (B) Energy over distance d for the non-wetting geometry for the doped and undoped surface. The minima are indicated with a diamond symbol; the reference energy curve is drawn in grey. Offsets are added to distinguish the curves and the value at the minimum is subtracted. (C) NagC1ş* (top) and NagClg (boltom) cluster, the moving alom marked with transparent copies of itself, and the distance of interest labeled with d. (D) Energy over distance for both clusters.
Reference
LOREM. LR
LOREM, No
LOREM, /R/= =
LOREM, LR
Energy
0.4
0.2
0.0
90
120
150
180
Dihedral angle 0 (Degrees)
Figure Energy profile over a 90 rotation of one rotor. The minimum value of each curve subtracted before plotting.
28

---

Published in Transactions on Machine Learning Research (04/2026)
LOREM,
LOREM, I.R
Forces MAE (me))A)
15
LOREM,
10
LOREM, I.R =
AA
CA
CC
CP
PA
PP
Dimer cluss
Figure Mean absolute error on forces for different models on the different dimer classes: Apolar-apolar
(AA), charge-apolar (CA), charge-charge (CC), charge-polar (CP), polar-apolar (PA), polar-polar (PP).
29

---

Published in Transactions on Machine Learning Research (04/2026)
Cutoff 2.5 A
Reference
1xLR
0.6
2 + 1xLR
Enery (eV)
SR
2x
3R
Cutoff 3.0 Á
Reference
1 1xLR
0.6
2xSR 1xLR
Energ (eV)
3xSR
4xSR
0.4
2xSR
0.2
0.0
Cutoff 3.5 Á
Reference
IxSR + 1xLR
2xSR + 1xLR
2xSR
Energy (eV)
3xSR
0.4
4XSR
X
90
120
150
180
Dihedral angle (Degrees)
Figure 10: Rotational profile of cumulene for variations of the LOREM model with different Te, different numbers of short-range message passing steps, and with and without long-range mossage passing. All curves that are identical to zero have been collapsed into a single one for readability.
G Additional results for cumulene
In Fig. 10 we present the curves related to Table 2 All models where the dihedral information accessible can resolve the benchmark, but vary match to the ground truth. In particular, we note that models with two short-range message passing steps perform better than single one. This may be due the high degree of degeneracy in atomic environments short cutoffs, which is alleviated by message passing.
30

---

Published in Transactions on Machine Learning Research (04/2026)
H Hlyper-prameet sweep for the NaCl cluster and Mg0 surface datasets
We report the results of hyper-paramcter sweep for the NaCl cluster and MgO surface datasets on a 'inner' taainvvalidatin split of the training set used for the experiments in Section aiming to gauge the impact of hyper-parameter tuning and early stopping on the validation set used for Table
We usod random train main/validation split (4000 samples 500 samples) of the NaCl cluster and MgO surface train- ing datasets, sweeping the following combinations of parameters: Initial learning rate (0.001, 0.0001), learning rate decay (linear, exponential): batch size (32, 64), and energy /force loss weights (0.5/0.5, 100/1, 1000/1) over 4000 epochs of training with the ADAM optimizer. From the resulting models, selected those with the best 'inner' validation RMSE on energy and forces and evaluated the models on the 'outer' validation split used for Table ! The results are presented in Table g and Table 9 The model selected by the best energy error on the validation set performs best in both cases, similar to the model used in the main text. The model with the best force error ranks second for the MgO surface task. We also verified that both models are able to reproduce the curves in Fig. In summary, explicitly tuning the model on an inner vaaiivalidattion split, including performing early stopping this validation set, does not significantly impact benchmark results reported in Table
Table 8: Errors the validation set for the hyperparameter set with the best inner validation metrics for the MgO task.
Model
Test RMSE E (mcV/atom)
Test RMSE f (mcV/A)
LOREM
0.064
4.076
Nearest other
0.071 (CACE-LES)
5.971 (MACE)
Best E model
0.064
5.229
Best F model
0.078
3.630
Table 9: Errors on the validation set for the hyperparameter set with the best inner validation metrics for the NaCl task.
Model
Test RMSE E (mcV /ato/m
Test RMSE f (mcV/A)
LOREM
0.112
1.155
Nearest other
0.210 (CACE- LES)
9.784 (CACE-LES)
Best E model
0.076
2.613
Best F model
0.101
1.473
31

---

Published in Transactions on Machine Learning Research (04/2026)
Result variants: MAE metrics, RMSE biodimers forces, different seeds
Table in the main text reports root mcan squared errors (RMSE). For completeness, we provide the corresponding mean absolute errors (MAE) in Table 10 Additionally, Fig, 11 shows the biodimers force
errors using RMSE (the main text uses MAE). The conclusions are unchanged: LOREM achieves the lowest. or competitive errors across datasets and dimer classes.
To assess sensitivity random initialization and dataset shuffling, we also report results for an alternative seed for LOREM (f) in Table 11 (RMSE) and Table 12 (MAF). This represents the worse of two seeds trained per task. The results consistent with the main table, confirming that LOREM's performance is robust to the choice of random seed. We also confirmed that qualitative performance on the benchmark tasks is similar.
Table 10: Mean absolute errors for energy E and forces f. Models for which MAE metrics are not available (4G-NN, SPOOKYNET) are omitted. Best in bold, second best underlined. See Table for details.
LOREM
CACE-LES
MACE 2xSR
PET
Dataset
IXSR+LR
1xSR+LR
2x SR
MgO surface (Validation set)
E (meV /at)
0.041
0.036
0.353
0.188
(meV /À)
2.869
5.584
3.620
4.191
NaCI cluster
E (meV /at)
0.090
0.161
1.345
1.249
(Validation set)
f (meV/A)
0.778
6.388
23.598
20.011
Biodimers
E (meV /at) (meV/A) /À)
0.155
0.566
3.220
2.657
1.023
1.787
5.278
5.352
Cumulene
E (meV /at)
1.096
14.567
8.961
1.363
f (mcV/A)
22.078
107.946
70.467
13.610
Table 11: RMSE for energy E and forces with LOREM results for alternative seed (t), representing the worse of two seeds. Best in bold, second best underlined. See Table T for details.
LOREMt
CACE-LES IxSR+LR
MACE 2xSR
PET 2x SR
4G-NN IxSR+LR
SPOOKYNET 6xSR+LR
Datasct
1XSR+LR
MgO surface (Validation)
E (meV/au /at) (meV /A))
0.065
0.071
0.376
0.210
0.219
0.107
4.381
7.913
5.971
6.261
66.000
5.337
NaCI cluster
E' (meV /at)
0.112
0.210
1.681
1.517
0.481
0.135
(Validation)
f (meV /A)
1.275
9.784
40.219
42.438
32.780
1.052
Biodimers
E. (meV/ att) (mcV/A)
0.370
2.259
7.793
6.758
1.985
3.163
16.150
16.470
Cumulene
E IneV/at)
3.307
17.803
12.592
3.205
f (moV/A)
55.412
147.616
104.318
46.905
32

---

Published in Transactions on Machine Learning Research (04/2026)
Table 12: MAE for energy E and forces f, with LOREM results for alternative seed (t), representing the worse of two seeds. Models for which MAE metrics are not available (4G-NN, SPOOKYNET) are omitted. Best in bold, second best underlined. See Table for details.
LOREM 1XSR+LR
CACE-LEI LES 1xSR+LR
MACE
PET
Dataset
2xSR
2x SR
MgO surface (Validation set)
E (meV /at)
0.040
0.036
0.353
0.188
(meV/A3
3.133
5,584
3.620
4.191
NaCI cluster
E (meV /at)
0.090
0.161
1.345
1.249
(Validation set)
f (meV/A3
0.882
6.388
23.598
20.011
Biodimers
E (meV /at) (meV /À)
0.322
0.566
3.220
2.637
1.110
1.787
5.278
5.352
Cumulene
E (meV /at) (meV/A)
1.212
14.567
8.961
1.363
28.448
107.946
70.467
13.610
LOREM
40
CACE-LES
Forces RMSE (meViA)
30
MACE
20
PET
10
AA
CA
CC
CP
PA
PP
Dimer class
Figure 11: Root mean squared error on forces for different models on the different dimer classes. Compare with Fig. in the main text, which shows MAE.
33

---

Published in Transactions on Machine Learning Research (04/2026)
J ADAPT benchmark
To evaluate LOREM on larger-scale dataset, we trained it on the ADAPT silicon point-defect dataset Dramko et al. 2025 b. This dalaset contains 206 973 training, 51 743 validation, and 4425 test structures, each consisting of 217 atoms (215 Si atoms plus dopant atoms drawn from 24 elements, forming 52 unique dopant pairs). The test set consists of the first frames from 100 held-out relaxation trajectories, following the evaluation protocol of the original work.
We used LOREM with default model hyperparameters (re 5 Imax 6, mmxx Ln 2, 128 scalar features, 8 spherical channels, 32 radial basis functions). Training hyperparameters were selected via sweep of 24 combinations on 9500-structure debug subset (100 epochs): optimizer (Muon Jordan et al. (2024), ADAM), initial learning rate (1 10-3, 1 10-1), learming rate decay (linear, exponential), and energy Sarrgy/force loss weights (0.5/0.5, 100/1, 1000/1). Four configurations from the Parcto frontier were selected for production training on the full dataset, all using the Muon optimizer with initial learning rate 10- euergy /force loss weights 0.5/0.5 with linear and exponential decay, and 100/1 with linear and exponential decay, These were trained for 300 epochs (65 h single H100 GPU).
Results are shown in Table 13 reporting the best force MAE and best energy MAE across the four production runs, each evaluated at the checkpoint with the highest summed R of energy and forces on the validation set. LOREM nearly matches the force accuracy of the ADAPT model (which uses an all-to-all transformer architecture with M parameters) while achieving substantially better energy accuracy deauracc-despite using a single model for both properties, compared to ADAPT's separate energy-only model. LOREM also significantly outperforms all other baselines, including MACE (both retrained and foundation models) and MatterSim.
Table 13: Mean absolute errors on the 100-structure ADAPT benchmark lest set. Baseline results are from Dramko et al. (2025). The ADAPT model uses a separate energy-only model for energies; all other models predict both properties jointly. Best in bold, sccond best underlined. Parameter counts: ADAPT Small 4 M,, ADAPT Large ~88 M.
Model
MAE (eV/A)
MAE (eV)
LOREM (best f) LOREM (bost E)
0.0128
0.097
0.0136
0.076
ADAPT Small
0.0126
0.578
ADAPT Large
0.0136
MACE Retrained
0.0217
1.313
MACE MP0a Large
0.0439
6.101
MACE MPA-0 Medium
0.0349
2.048
MACE OMAT-0 Medium
0.0283
3.223
MatterSim 1M
0.0323
1.743
MatterSim 5M
0.0335
0.829
34


Reviews:
[{"review_id": "nzHAuI4rQc", "reviewer": "Reviewer_mtDJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose Lorem, a machine learning interatomic potential (MLIP) architecture that incorporates equivariant long-range message passing via Ewald summation. The key technical contribution is extending the traditional scalar-charge Ewald summation to equivariant tensors (Q_{j,l,m}), allowing long-range communication of orientational information beyond what scalar charges can provide. The authors design an architecture combining short-range equivariant message passing with this long-range mechanism and evaluate it on several benchmark datasets designed to test long-range interaction modeling.\n\nStrengths:\n1. The mathematical formulation of equivariant Ewald summation is clean and the equivariance argument is sound\n2. A reasonable experimental section covering multiple datasets with different long-range physics, and an honest discussion of when short-range methods suffice\n\nWeaknesses:\n1. The contribution is relatively limited. \n2. There are some ablation/baseline issues, see below. Similarly, I believe the example systems are potentially too toy-like.\n3. The unique advantage of equivariant (vs scalar) long-range interactions is only clearly demonstrated on one benchmark (cumulene)", "claims_and_evidence_yn": "Yes", "claims_explanation": "The core mechanistic claims (equivariance of the operator; how it’s integrated into Lorem) are reasonably supported. However, the 'works consistently without adjustments' appears not supported. For example, Table 3 shows different optimizers/learning rates/epochs/loss weights, showing significant per-dataset training tuning occurs. The scaling claim is argued by compatibility rather than demonstrated at regime where asymptotics matter. See below for requested changes.", "audience_yn": "Yes", "audience_explanation": "The problem of modeling long-range interactions in MLIPs is important for computational chemistry. The paper addresses a limitation of current models and proposes a physically motivated solution. The systematic comparison of short-range vs. long-range methods at different cutoffs is valuable for practitioners.", "requested_changes": "1. Add missing baselines: Include comparisons with LODE, SpookyNet, and ideally Ewald MP on applicable datasets. At minimum, provide a detailed technical discussion explaining why these comparisons are omitted.\n2. Please try to include additional datasets where the interplay between short and long range is critical, such as protein-ligand binding or electrolyte simulations. You could even do some more complex materials from CO20.\n3. Related to 2 (could be one experiment), demonstrate at least one experiment using an actual fast method (e.g., PME/P3M in periodic; FMM/multi-level summation in non-periodic) at sizes where naive O(N2) becomes clearly suboptimal.\n4. Either remove claims about working \"without adjustments\" or standardize training protocols across datasets. The current phrasing is misleading.\n\n\nLess critical changes:\n5. Please try to replace 'best of two seeds' with mean +- std over multiple seeds.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ddDxkMdnEK", "reviewer": "Reviewer_MnVa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes LOREM, a graph neural network architecture designed to capture long-range interactions in atomistic systems without relying solely on large cutoff radii or deep message passing.The core contribution is a long-range equivariant message passing mechanism. Unlike prior methods that treat long-range interactions as purely scalar, e.g., charge-charge Coulomb interactions, LOREM promotes atomic charges to equivariant tensors $Q_{i,l,m}$. It utilizes a physics-inspired potential formulation similar to Ewald summation to broadcast these equivariant \"charges\" globally via an inverse power law $1/r^p$. The authors demonstrate that this approach outperforms short-range baselines and scalar long-range models on benchmarks designed to probe non-local geometric correlations, such as the Cumulene chain.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The submission provides accurate and convincing empirical evidence to support its primary claim: that strictly scalar long-range interactions as used in prior works like CACE-LES or 4G-NN are insufficient for capturing non-local geometric dependencies, and that LOREM’s equivariant formulation effectively bridges this gap. The evidence is particularly strong in three key points:\n\n### 1. Validation of Equivariant Necessity\nThe paper’s most convincing evidence comes from the Cumulene chain experiments. The authors demonstrate that scalar long-range models completely fail to resolve the energy profile of the rotating terminal groups. This provides clear, negative evidence that scalar charges cannot transmit the necessary orientational information over distance. In contrast, LOREM resolves this profile accurately, directly supporting the claim that propagating higher-order tensors ($l > 0$) via the Ewald mechanism is functionally necessary for this class of physical problems.\n\n### 2. Robustness Compared to Short-Range Baselines \nThe authors conduct a thorough \"stress test\" in Section 6.2 to verify that their performance isn't just due to hyperparameter luck. They show that while short-range models (MACE, PET) can solve these tasks, they are brittle, requiring specific tuning of interaction cutoffs and depth to artificially \"reach\" the interaction range. The evidence shows LOREM succeeds consistently without such task-specific tuning, supporting the claim that the architecture learns genuine long-range representations rather than just extending a receptive field.\n\n### 3. Comprehensive Ablation Studies\nThe claims are further supported by clear ablation studies in Appendix G that isolate the specific contributions of the architecture. The results show a significant degradation in performance when the long-range block is removed or when the representation order is reduced to $l=0$. This confirms that the performance gains are causally linked to the proposed equivariant mechanism.", "audience_yn": "Yes", "audience_explanation": "The findings of this paper are likely to be of significant interest to a substantial subset of the TMLR audience, particularly those working at the intersection of machine learning and physical sciences, e.g., AI4Science, as well as researchers focused on geometric deep learning.", "requested_changes": "The paper introduces a mathematically elegant and physically motivated mechanism (Equivariant Ewald) that addresses a clear deficiency in the field (scalar-only long-range models). The logic is sound, and the proofs of equivariance are correct. However, the experimental validation is restricted to small-scale, synthetic benchmarks. The authors would need to demonstrate that this method provides tangible benefits on large-scale, realistic materials datasets where long-range effects are subtle but impactful, rather than just on systems designed to be impossible for local GNNs.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "v0pxYDe5l7", "reviewer": "Reviewer_sGoj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper concerns learning interatomic potentials, which is valuable in physics, biology, and chemistry.\nThe paper proposed LOREM, which is an equivariant neural network and can handle long-range dependencies (message passing).\nAuthors consider several datasets specifically designed to highlight non-local physical effects, and compare short-range message passing with different receptive fields to invariant and equivariant long-range message passing. This allows using of higher-order geometric information across arbitrary distances via Ewald summation, preserving SO(3) equivariance. \nEven though most approaches work for careful dataset-specific choices of their hyperparameters, LOREM works consistently without adjustments, with good performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The main claim of the paper: equivariant long-range message passing in LOREM provides accurate modeling of non-local physical effects without hyperparameter tuning is supported. \nFrom theoretical perspective, multiplying equivariant features by distance-dependent scalars (1/r^p) and summing preserves equivariance (Section 4). \nAppendix B numerically verifies rotational invariance of energy predictions to machine precision (10^{-15} meV in double precision), confirming no symmetry breaking occurs during Ewald summation with background charge correction.\nExperiments are convincing. NaCl cluster demonstrates scalar long-range models succeed where short-range message-passing fails.\nCumulene benchmark proves scalar long-range models (CACE-LES) cannot transfer dihedral orientation information, while equivariant long-range (LOREM) succeeds.\nAppendix E provides hyperparameter details for reproducibility. Runtime benchmarks in Table 4 show that computational overhead is modest (<10% for most systems),", "audience_yn": "Yes", "audience_explanation": "Applications of deep learning to natural sciences is a promising topic.", "requested_changes": "1) Clarify equivariance proof in Section 4: The statement \"multiplication with a prefactor, provided that the factor is shared across all entries for a given l, also retains equivariance\" is imprecise. The prefactor depends on r_i, r_j and it is not constant.\n2) What is the purpose of Appendix B with experimental validation of equvariance? The rigorous mathematical proof must be enough.\n3) Why LOREM is not best in \"Cumulene\" dataset in Table 1?\n4) Does methods in Table 1 have the same number of parameters? The comparison isn't correct otherwise.\n5) Why you switched from p=1 vs p=6 for apolar-apolar (AA) dimers in biodimers dataset. \nThe claim that \"p=1 is sufficient in practice\" (Section 5) is not justified.\n6) Table 1 reports RMSE but Fig. 4 reports MAE for biodimers forces. \n7) Do you have a comparison with standard GNN, GAT, Graph Transformers?\n8) How your paper is related to:\nBatzner, S., Musaelian, A., Sun, L., Geiger, M., Mailoa, J. P., Kornbluth, M., ... & Kozinsky, B. (2022). E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1), 2453.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)