Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing
Authors: Viktor Zaverkin
Francesco Alesiani
Takashi Maruyama
Federico Errica
Henrik Christiansen
Makoto Takamoto
Nicolas Weber
Mathias Niepert
Venue: NeurIPS 2024 poster (2024)
Keywords: equivariance
graph neural networks
interatomic potentials
irreducible Cartesian tensors
many-body interactions
molecules
materials
Primary area: machine_learning_for_physical_sciences
TL;DR: We introduce higher-rank irreducible Cartesian tensors and their products for equivariant message passing.

Abstract: The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as equivariance to group actions on an atomic system, e.g., equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message passing. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. Cartesian tensors offer a promising alternative, though state-of-the-art methods lack flexibility in message-passing mechanisms, restricting their architectures and expressive power. This work explores higher-rank irreducible Cartesian tensors to address these limitations. We integrate irreducible Cartesian tensor products into message-passing neural networks and prove the equivariance and traceless property of the resulting layers. Through empirical evaluations on various benchmark data sets, we consistently observe on-par or better performance than that of state-of-the-art spherical and Cartesian models.

Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing
Viktor Zaverkin
Francesco Alesiani
Takashi Maruyama lit
Federico Errica² 2
Henrik Christiansen' Makoto Takamoto'
Nicolas Weber
Mathias Niepert
NEC Laboratories Europe
NEC Italy
University of Stuttgart
Abstract
The ability to perform fast and accurate atomistic simulations is crucial for advanc- ing the chemical sciences. By learning from high-quality data, muchine-leamme interatomic potentials achieve accuracy par with ab initio and first firt-pprncisles methods at fraction of their computational cost. The success machine-learned interatomic potentials arises from integrating inductive biases such equivariance to group actions on an atomic system, e.g. equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message passing. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. Cartesian tensors offer a promising alternative, though state-of- the-ahr art methods lack flexibility in message- peessage-psssigg passing mechanisms, restricting their architectures and expressive power. This work explores higher-rank irreducible Cartesian tensors address these limitations. We integrate irreducible Cartesian tensor products into sassage-passing neural networks and prove the equivariance and traceless property the resulting layers. Through empirical evaluations various benchmark data sets, we consistently observe on-par better performance than that state-of-the-art spherical and Cartesian models.
Introduction
The ability to perform sufficiently fast and accurate atomistic simulations for large molecular and mate- rial systems holds the potential to revolutionize molecular and materials science [1-8]. Conventionally, computational chemistry and materials science rely on ab initio or first-principles approaches-e., e.g. coupled cluster (РH or density functional theory (DFT) 13] respectively-tih are accurate but computationally demanding, thus limiting the accessible simulation time and system sizes. However, the ability to generate high-quality, prst-principles-eee based data sets has prompted the development of machine-leane learned interatomic potentials (MLIPs). These potentials enable atomistic simulations with accuracy that is on with the reference first pesst-prncciles method but at fraction of the computational cost. Message-passing neural networks (MPNNs) [14 have been employed chemical and materials sciences, including the development of MIIPs, due to their efficient, processing of the graph representation of the atomic system (6)-66) Achieving the desired MLIPs performance, however, requires the inclusion of inductive biases, like the invariance of total energy to translations, reflections, and rotations in the three-dimensional space, and an effective encoding of the atomic system into a learnable representation [27]. Designing equivariant MPNNs [5] 26 28.36), which preserve the directional information of the local atomic environment, has been one the most active research
directions of the last years. Previous work often achieves equivariance by representing an atomic
"ooresponding author: viktor. wwwerkin@neclab.ccmm eu 'hese authors contributed equally.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

system as a graph, with node features expressed in spherical harmonics basis or using lower-rank
Cartesian tensors and designing ad-hoc message-passenn layers [25 28 37|.
which are the basis for irreducible representations of the three-dimensional rotation group, often demonstrate better performance compared those that use lower-rank Cartesian representations (scalars and vectors) 28 37.. Spherical tensors, however, require the definition of a particular rotational axis, often chosen the z-axis, resulting in inherent bias 38). Furthermore, coupling spherical tensors via tensor products, involved in designing equivariant convolutions and constructing many-body features [3
MLIPs based on spherical harmonics 22 33
requires the definition of complicated numerical coefficients, e.g., Wigner 3-j symbols defined in terms of the coefficients [41). and can be computationally demanding. In contrast, irreducible Cartesian tensors have no preferential directions; their tensor products are simpler and have better computational complexity- up to certain tensor rank-thar the tensor products of spherical tensors 33 42 45
39 40
Recent work improved results of Cartesian MPNNs by using rank-two tensors and decomposing them into irreducible representations of the three-dimensional rotation group 30. i.e., into representations
of dimension I (trace), 3 (anti-symmetric part), and 5 (traceless symmetric part) [46). Higher-rank
reducible Cartesian tensors have also been integrated into many-body MPNNs 31.. These state-
of-the-art Cartesian approaches, however, lack the flexibility of their message-passee passing mechanisms.
They rely exclusively on convolutions with invariant filters and restrict the construction of many-body features, limiting the range of possible architectures and their expressive power.
Contributions. In this work, we address the limitations of state-of-the Cartesian models and demonstrate that operating with irreducible Cartesian tensors leads to on-par or, sometimes, even better performance than that of spherical counterparts. Particularly, this work goes beyond scalars, vectors, and rank-two tensors and explores the integration of higher-rank irreducible Cartesian tensors and their products into equivariant MPNNs: i) We demonstrate how irreducible Cartesian tensors that are symmetric and traceless can be constructed from unit vector and a product of two irreducible Cartesian tensors, essential for equivariant convolutions and constructing many-body features; ii) We prove that the resulting tensors are traceless and equivariant under the action of the three-dimensional rotation and reflection group: iii) We demonstrate that higher-rank irreducible Cartesian tensors can be used to design cost-efficient-upp up to certain tensor rank- andd accurate equivariant models of many-body interactions; iv) We conduct different experiments to assess the effectiveness of equivariant message passing based irreducible Cartesian tensors, and achieve state-of -ftthe--rr- performance on benchmark data sets such as rMD17 (47), MD22 [88] 3BPA [4),, acetylacetone (3), and Va-V-Cr-W [50]. We hope our contributions will offer new insights into the use of Cartesian tensors for constructing accurate and cost-efficient MLIPs and beyond.
Background
Group representations and equivariance. A group .) is defined by set of elements G and group product :. A representation D of group is a function from to square matrices such that D[g] = Dig- g'), Vg.g' E G. This representation defines action of G to any vector space X' (of the same dimension as the dimension of square matrices) through the matrix-vector multiplication, i.., (g.x) + Dx[g] x for Vg G and Vx E X. For vector spaces X' and ) function f X y is called equivariant to the action of a group G to X' and y iff
f(Dxlg)x) = D
Invariance can be seen as special type of equivariance with Dy[g] being the identity for all g. An important class of equivariant models focuses on equivariance to the action of the Euclidean group E(3), comprising translations and the orthogonal group O(3), i.e., rotation group SO(3) and reflections, in IR3 MLIPs based on equivariant models usually focus on equivariance to the action of the orthogonal group (33) and are invariant to translations.
Cartesian tensors. vector x E 3 transforms under the action of the rotation group (O(3) as x' Rx, i.e., each component of it transforms as (x), = -E Rıg (x),, Here, R Dx [g] 3333 denotes the rotation matrix representation of g E SO(3). Cartesian tensors of rank n are described by 3" numbers and generalize the concept of vectors. A rank-n Cartesian tensor T can be viewed as a multidimensional array with n indices, i.e., (T) ig...... with i E (1,2,3) for V E (1.....7)) Furthermore, each index of (T), (2g transforms independently as a vector under rotation. For
example, rank-two tensor transforms under rotation as T' RTRT. i.c., each component of

---

it transforms as (T),3a = Enrse Ruch Rigj2 (T) J1d2 For Cartesian tensors, one defines an r- fold tensor contraction and an outer product, i.c., (T) = Eigenng (U) (U) (S),
and ( (U), ( (8))- respectively. Cartesian tensors are generally reducible and can, thus, be decomposed into smaller representations that transform independently within their linear subspaces under rotation. For example, a rank-two reducible Cartesian tensor contains representations of dimension (trace), 3 (anti-symmetric part), and 5 (traceless symmetric part): L,, (T),, i, (T) (T) and (T) + (T)),s, 2/3 Σ;, (T) respectively. Under rotation, the traceless symmetric part remains within its irreducible subspace, with similar behavior for other irreducible components of (T) Furthermore, for a reducible Cartesian tensor of rank n, an action of the rotation group (3(3) can be represented with 3 37-dimensional rotation matrix. This rotation matrix is also reducible, meaning that an appropriate change of basis can block diagonalize it into smaller subsets, the irreducible representations of the rotation group SO(3).
Spherical tensors. Spherical harmonics (or spherical tensors) Y are functions from the points on a sphere to complex or real numbers, with degree / 2 and components -1 m S Collecting all components for a given we obtain (2l + -)-dimensional object Y = (Y- YY 7)) (D) ym Yn (x), where (D) mm is
Spherical tensors transform under rotation Y (Rx) the irreducible representation of 50(3) the Wigner D-matrix- and R denotes the rotation matrix. Spherical tensors are irreducible and can be combined using the Alessch-Gordal tensor product
(Y 8cG
/ m3
=E
12
Clam damidams
Y-1 y
ma )
mI==II
m2==12
with Clebsch-Gordan coefficients Clems 1ami, and 1s {14 t2l. ... .11 + 12]. Furthermore, any reducible Cartesian tensor of rank n can be written in spherical harmonics as (x),, (x);2 ... (x), = XXmm X!m iig Y with coefficients X!m iig..... defined elsewhere [5T]
Ei E
3 Related work
Equivariant message- passing potentials. Eguivariant MPNNs (25] 27 39 52 54] often outper- form more traditional invariant models (55+60]. While many invariant and equivariant MPNNs rely on two-body features within a single message-passing layer, there is a growing interest in constructing higher-body-order features, such as angles and dihedrals, to model many-body interactions in atomic systems [57 59 60). Note that equivariant models with two-body features in single pressage-passing layer build many- body ones through repeated pesssage-passingg iterations, increasing the receptive field and, thus, computational cost. Recent work recognized the importance of higher- body-order fea- tures but faced challenges due to explicit summation over triplets or quadruplets (77 59 In contrast, MACE advances the current -fate-of-the-art by combining ACE and equivariant message passing, introducing cost-efficient many-body message- pasgee-ppssnggggae potentials [R] With just two message-passing layers, MACE yields accurate potentials for interacting many-body systems (61.
Beyond MebschhGordde tensor product. Despite the success of equivariant MPNNs based on spherical tensors, the high computational cost of the Clebsch-Gordan tensor product limits their computational efficiency 28 30
Thus, current research focuses on alternatives to the Clebsch-Gorde Gordan tensor product, which has a O(I3) complexity for tensors up to degree L. Recently, the relation of Clebsch- Gohhhan coefficients the integral of products of three spherical harmonics, known as the Gaunt coefficients [11] has been exploited to reduce the computational cost of the tensor product of spherical tensors to O (.3). compared to (I.)) of the full Clebsch-Gordee tensor product including all (2,,2) - la (3 However, this approach excludes odd tensor products and, thus, restricts the expressive power and the range of possible architcctures.
Cartesian tensors and their products offer another promising alternative to the Clebsch-Gordan tensor product, enabling the efficient construction of message-passing layers with two- and many- body features. Recent work has explored decomposing rank-two tensors into their irreducible representations and using higher-rank reducible tensors 30 BI] These approaches, however, are limited to convolutions with invariant filters, restricting the range of possible architectures and, thus, the expressive power of resulting Cartesian models. Furthermore, they provide limited mechanisms for constructing higher-body-order features, restricted to three-body features obtained through matrix- matrix multiplication and invariant many-body features obtained through full tensor contractions, similar to moment tensor potentials and Gaussian moments (65|67]. Finally, using reducible Cartesian
3

---

(a)
Ti
F
(b)
Ixllaa
+ -1
even
odd
even
Bun
T
T
T1-1
T--2
T2-
Figure I: Schematic illustration of (a) the construction of an irreducible Cartesian tensor for local atomic environment and (b) the tensor product of two irreducible Cartesian tensors of rank 4 and l2. The construction of an irreducible Cartesian tensor from unit vector i is defined in tensors with the same rank and weight 1, i.e., nl I, avoiding the need for embedding tensors with / < in a sigher-dimeesionall tensor space. Therefore, we use to identify the rank and the weight of irreducible Cartesian tensor. The tensor product defined in Eqs. 2 and (3). resulting in new tensor Ti, = (Ti, @Cart Tia)ss of rank l = {14 12). ... .1 + 22). Transparent boxes denote the linearly dependent elements of symmetric and traceless tensors. The tensor product can be even or odd, defined by I + l2 lz.
Eq. (T). In this work,
tensors during message-passia passing leads to mixing different irreducible representations, which can hinder the performance of resulting models compared state-of the-t-rrrt spherical models (31
4 Methods
We define an atomic configuration S (rw: ) where ra E R.3 denotes Cartesian coordinates and Zu E N represents the atomic number of atom u, with total of Nac atoms. Our focus lies on pessage-passsng MLIPs, parameterized by 0, that learn mapping from configuration S to the total energy E, i.., fe S in three-dimennsional Euclidean space. An edge {u, v} exists if atoms and distance T.. i.e., [r, r. ||2 S r.. For more details on MPNNs, see Appendix [A] The total energy of atomic configuration is defined by the sum of individual atomic energy contributions [68), i.., E Nna E,, Atomie forces computed negative gradients of the total energy with respect to atomic coordinates, i.c., F. Vr.. E.
E R. Thus, we represent molecular and material systems as graphs within a cutoff
4.1 Irreducible Cartesian tensor product
In the following, we explore irreducible Cartesian tensors and their products based on the three- dimensional vector space and the three-dimensional orthogonal group O(3), which comprises ro- tations and reflections. The respective tensors and tensor products are schematically illustrated in Fig. (a) and (b), respectively, and are further employed in constructing MPNNs for atomic systems equivariant under actions of the orthogonal group. An irreducible Cartesian tensor T, (RR3) @n of rank n and weight < n (related to the degree of spherical tensors) can be represented by tensor with 3" components in dirre-dimensional vector space. These 3" components form basis for a (2l- + 1)-dimensional irreducible representation of the three-dimensional rotation group SO(3) 69 70 Thus, only 2/ of the 3" components are independent.
The rotation the space of tensors of rank n is induced through the n-fold outer product of rotation matrix R. E p333 i.c., 8m R 8 @ R. The obtained representation of the

---

rotation group (3(3) is reducible for all n except {0, 1). Reducing a tensor of rank yields unique irreducible tensor with the same weight and rank (n 1), which is characterized by being symmetric, i.e., (T.)......... = (Tn)........... i *
{ir {ir... (n): J. and traceless, i.e., (.TT.) TT.............................. = 0 Vi E (i,,..., J. An irreducible tensor of rank n and weight / with < n can be viewed as a I-rank tensor embedded in the n-rank tensor space, e.g., by computing an outer product with the identity matrix. However, the embedding is often not unique. Thus, we construct tensors with the same weight and rank (n I) in the following.
Irreducible Cartesian tensors from unit vectors. Hereafter, we denote the rank and the weight of irreducible Cartesian tensors by to distinguish them from reducible counterparts. An irreducible Cartesian tensor of an arbitrary rank can constructed from a unit vector i in the form [45]
11/21 E (-1) (21 2m 1)!!
(+( - C m=0
(p0(1-2m)
@ [
(1)
(21 1)!!
resulting in a symmetric and traceless tensor of rank Here, I denotes the 3 x identity matrix, (.(1--2m @ .... & r and I&m = - & I are the corresponding (L 2m)- and m-fold outer products. The curly brackets indicate the summation over all permutations of the / unsymmetrized indices [4], i.e., (Tija mutations. The expression in Eq. (1) involves three distinct outer products (f @ f) = A, fsz- (I & D Disisis = ó1 and (r 69 iniii = fss Sizi, where 8:2is denotes the Kronecker delta. The normalization constant = (21 - 1)!l/!! is chosen such that an I-fold contraction of T2 with the unit vector i yields unity. An example of an irreducible Cartesian tensor with rank 1 = 3 is (Ta=a)inaas = (f,, fi Pig { Sizis + Sish + +1)))
En (Ti) with St being the corresponding set of per-
Ireeduibble Cartesian tensor product. The following defines the product of two irreducible Cartesian tensors X1, E (1R))8 and yı, (IR)) yielding an irreducible Cartesian tensor of rank la i... Z1, (x), &Cart yta)t,
1B3)8/a7l, E {14 1/1-4/1 12), .1 + (2), that is symmetric and traceless. The irreducible Cartesian tensor product is crucial for designing equivariant MPNNs and is used for equivariant convolutions and constructing many-body features in Section 4.2 For an even
2 + 12- - 1/- = 2k: the general form of an irreducible Cartesian tensor of rank /3 reads 45
(xi, @Cart Viz)ls
-Ci,ials Emirtt (-1)m2m (21g 2m 1]!! (213 1)!!
{ (xx,, (k + m) y1,) Immm
where (xi, (k + mt) =En.... ii-m (xi, 1.mm (yi, )i denotes an (k + m)-fold ten- contraction, which results in tensor of rank + 12 2(k + m). For simplicity, we skip the uncontracted indices in the above definition. For example, for 4 = 4 and L2 = 3 and a three-fold tensor contraction we obtain (x,, 3 Viz)ia i2. (xx, ii (V1,),,22t i.c., the correspond- ing tensors contracted along i1, i2. and iz. Note that the final result is independent of the index permutation, as the contracted tensors are symmetric. For an odd 4 + 12 - 13 = 2k + 1, we define 45
(X1, &Cart Yia)ta
minth (-1) (213 2m - 1)!! (2lg 1))!! { (E: X1, : (Ik + m) yr,) X I@m]
(3)
=Diitata
m=0
with e denoting the Levi-Civita symbol (Sigigis = -igiii and Gapasia = 0). The double contraction
with the Levi-Civita symbol reads (e: Xi, (k (0++m))--- y1,)i, = EE Eipigia (x1, (k + m)) Yt, yi,)
and yields tensor of rank 1 + 12 2(k + m) 1. Details on the normalization constants Ch
and Di,td, are provided in Appendix B.I
4.2 Equivariant message-passing based on irreducible Cartesian tensors
The following section introduces the basic operations for constructing equivariant MPNNs based on irreducible Cartesian tensors. Using their irreducible tensor products, we demonstrate how to build equivariant two- and many-body features, crucial for modeling many-body interactions in molecular and materials systems. Particularly, we focus on MLIPs based on equivariant MPNNs and extend the state-of-the-ee MACE architecture [33] to the Cartesian basis. Following the MACE architecture, we use only even tensor products. We split vectors Twe = ra F, E R3 from atom u to atom schematically shown in Fig. [ (a). into their radial and angular components (unit vectors), i.c.,
5

---

Tuv = =||rullla E R and Fuo = Tuve/rae E R respectively. In the f-th crassage-passingg layer, edges {u, 5} are embedded using fully connected neural network Rift) kkzz22 R - R. with & output feature channels. The radial function R Rii, takes as an input radial distances Tue. which are embedded through Bessel functions and multiplied by smooth polynomial cutoff function [60]. Finally, we use irreducible Cartesian tensors T (f), similar to spherical tensors (+). to embed unit vectors into the tensor space of maximal rank maxx.
Equivariant convolutions and two-body features. Rotation equivariance in MPNNS is typically achieved by constraining convolution filters to be the products between learnable radial functions and spherical tensors, i.e., R t2g, ((uw))yt mi (fuv). The two-body features A A uklyms are further obtained through the tensor product the point-wise convolution (3) between the respective filters and
neighbors' equivariant features h (t) mukigmy' The permutational invariance is enforced by pooling over the neighbors (u)).. Here, use irreducible Cartesian tensors, with the rotation-equivarian filters given by (2(t) 124g (Tww) (Ti, (Fww)) staar...." Thus, two-body features (A (kktg) i 6262 are obtained using the irreducible Cartesian tensor product and are represented by rank-lz irreducible Cartesian tensors. The Cartesian two-body features are defined by
(A (t) (Aukly) (1/2---1g EvEN(N) (Belzlatg (Fuo) Ti,, (Far) &Cart
1 dd
Σ
wilt (1) Rkll Bok i16g
(4)
where de represents the number of feature channels in the node embeddings h of the t-th message- passing layer. In the first message- paaaigg layer, node embeddings are initialized as learnable weights Waz, that are invariant to actions of the orthogonal group, i.e., are scalars or tensors of rank l2 = 0. and embed the atom type Z.. Thus, constructing equivariant two-body features simplifies to
(A ukts in
E.EN((
Ril (raw) (Tr, (fuv)) syggr-e (f WkZ.-
(5)
Equivariant mony-body features. The importance of many-body terms arises from the fact that the interaction between atoms changes when additional atoms are present; see Appendix AA] Furthermore, many-body terms are often required to ensure the generalization of interatomic potentials, i.e., their ability to accurately predict energies and forces for temperatures and stoichiometries on which they
were not trained 771. Here, we construct (v + 1)-body equivariant features from (A (t)
ukle/ ing
obtained using Eqs. 44 or (5). The v-fold Cartesian tensor product, which yields (v + -)-b fcatures represented by irreducible Cartesian tensor of rank L, reads
B (t) an, iiigeii
-A
(t) ukl, &Cart ... BCart A (t) akl, izig":ie
inta..iz
i-fold
where Th counts all possible f-ld products of {41, ... J-rnkk tensors, yielding rank-L L irreducible Cartesian tensors, and (+) ukle VJ. Σx
(((t) A (t) with d, feature channels.
Aikll uklle
The irreducible Cartesian tensor product does not allow pre-computing the coefficients of the v-fold tensor product, differing from spherical tensors that use the generalized Cebsch-GGrrdae coeffi- cients, contracted with weights from Eq. A and summed over the possible paths Ien (en()).), ), for this purpose [33 40, 51. Thus, we obtain the result of Eq. 6 by iteratively applying the irreducible Cartesian tensor product (v 1) times and refer to the respective models as irreducible Cartesian tensor potentials (ICTPs) with the full product basis or ICTP However, two-fold tensor products in Eq. 66) are symmetric to permutations of involved tensors. Thus, the number of the y-fold tensor products, len(n>), can be significantly reduced; we refer to the corresponding models as ICTP sym- Furthermore, we can reduce the computational cost of the Cartesian product basis by performing the calculations in latent feature space. We use learnable weights W Wasegg to reduce the number of feature channels for the product basis calculation and then increase it again for subsequent steps; we refer the corresponding models as ICTP,, For more details on the model architecture, such the construction of updated many-body node embeddings, readout functions, and different options for the Cartesian product basis, see Appendix B.2
Runtime considerations. When choosing an architecture to implement MIIPs, the runtime per energy and force evaluation is crucial. The computational complexity as a function of rank L is

---

(b)
(c)
Inference time in ms
GPU memory GB
Inference time in
10.0
10.0
1.00
1.0
GPU
0.1
0.1
1.0
MACE
MACE MACE
MAC KCP
MACE
(C
RCTP ( (C
ICTP
ICTP 3)
Figure 2: Inference times and memory consumption as a function of the tensor rank (a)-(b) and the correlation order y (c) All results are obtained for the 3BPA data set and mmx = L. We used eight feature channels to allow experiments with larger v values. MACE models use intermediate tensors with 1 > mmx for their product basis, which we fixed to = mmax- Otherwise, pre-computing generalized Clebsch-Gordae Gordan coefficients for v > 4 would require more than 2 TB of RAM. For ICTP, we used the full product basis to compute the same number of v-fold tensor products as in MACE.
O (9/ (2//2 for the irreducible Cartesian tensor product and O (L for the Clebsch-
Gordan one: see Appendix [B.3 for more details. Thus, equivariant convolutions based on spherical lensors are more computationally efficient for L + oo than those based on irreducible Cartesian tensors. However, Mate-of-the-ar models and physical properties typically require L S 4 [33 722, making sub-leading terms and implementation-depeel
amplitudes crucial. Based on our analysis, for L S 4, we can expect equivariant convolutions based on irreducible Cartesian tensors to more computationally and memory efficient than their spherical counterparts. For L S 4, the v-fold tensor product in the Cartesian basis can also offer computational advantages. Its cost, as a function of and KL2=(v+33 for ICTP and MACE, respectively. The pre-factor K,, which counts all possible r-fold tensor products, is removed in MACE through generalized Clebsch-Gorde spherical models use. Therefore, it is essential to consider the inference times for fair comparison, which we provide in Section
rank L and correlation order vs is KC(9 (1/((2/2LL/221))
coefficients, though these coefficients increase the memory
Equivariance and tracelss property of geessag-ppassing layers. We conclude this section by giving a theoretical result that ensures the equivariance of pessage-passing layers based on irreducible Cartesian tensors and their irreducible tensor products to actions of the orthogonal group. The proof provided in Appendix(C C We also prove in Appendix that these message-pasee pasggng layers preserve the traceless property of irreducible Cartesian tensors.
Proposition 4.1. The wressage-passingg layers based on irreducible Cartesian tensors and their irreducible tensor products equivariant to actions of the orthogonal group.
Proposition 4.2. The message paassagepaassngg layers based on irreducible Cartesian tensors and their irreducible tensor products preserve the traceless property of irreducible Cartesian tensors.
5 Experiments and results
This section presents the results for the five benchmark data sets: rMD17. MD22, 3BPA, acetylacetone, and Ta-V-Cr-W. We describe data sets and training details in Appendices E.T and F.2 respectively.
Scaling and computational cost. The expressive power and computational efficiency of equivariant many-body message-passingg potentials depend on the tensor ranks employed in equivariant message passing and embedding local atomic environments, i.e., L and ( respectively, as well as the correlation order v. Recent work has shown that for identifying environments with L-fold symmetries, least rank- tensors are required
(73). These symmetries are typically lifted in atomistic simulations, motivating the use of L < 4. Higher body-order correlations v, in turn, are required if atomic environments are degenerate to lower body-order correlation v 773 74). Figure demonstrtesss inference times and memory consumption of models based on irreducible Cartesian and spherical tensors, i.e., ICTP and MACE, respectively, as a function of the tensor rank and the correlation order. Table Al presents the corresponding numerical results. We find that irreducible Cartesian tensors
7

---

Table 1: Energy (E) and force (F) mean absolute errors (MAEs) for the rMD17 data set. E- and F-MAE are given in meV and meV/Å, respectively. Results are shown for models trained using Nuain (950, 50) configurations randomly drawn from the data set, with further 50 used for early stopping. All values obtained by averaging over five independent runs, with the standard deviation provided available. Best performances, considering the standard deviation, are highlighted in bold.
Nyai = MACE
950
Neur MACE
50
ICTF
TensorNet 30
83
Allegro
XX
NequlP
AS
Nequll
[33
ICTP
2.27
19.5
17.0
14.84
Aspirin
F
6.67 1 0.19
8.9 a 0.1
6.6
7.3
8.2
52.0
43,9
40.19
0.95
1.20 + 0.01
0.7
5.47
0.63
Azohenzene
2.92 = 0.03
3.1
2.6
20.0
17.7
17.25
0.53
E
0.26 + 0.00
0.02
0.4
0.3
0.04
0.6
0.7
0.38
0.02
Benzene
0.3
0.13 2.45
Ethanol
B
0.43 + 0.02
0.5
0.4
0.4
0.4
8.7
6.7
6.15
0.26
2.63 +: 0.10
3.5
2.1
2.1
2.8
40.2
32.6
29.53
1.14
Malonaldehyde
0,82
0.8
12.7
10.0
4,96 + 0.21
5.4
4.1
3.6
5.1
52.5
43.3
42.88
3.08
Naphthalene
0,56 0:00
0.2
2,06
0.10
1.45 +: 0.05
1.6
1.6
0.9
10.0
9.2
0.46 9.43
Paracetamol
R
1.44 + 0:03
1.3
1.3
1.5
1.4
14.3
9.7
8.94
0.66
4,89
5.9
39.7
31.5
Salicylic acid
B
0.97 = 0.01
0.8
0.9
0.9
0.7
8.D
6.5
5.95
0.43
3,66 1 0.06
4.6 0.1
3.1
2,9
35.9
28.4
27.78
1,93
Toluene
0,46 +
0.3
2.45
0.13
F
1.61 = 0.02
1.7
1.5
1.8
1.6
15.1
12.1
11.24
0.55
Unacil
0.57 0.01
0.4
4,66
0.16
2.64 1: 0.08
3.1
2.1
1.8
40.1
25.9
25.97
0.78
outperform spherical ones for most parameter values. In particular, irreducible Cartesian tensors allow spanning the v-space more efficiently, in line with our theoretical results in Section 4.2
Molecular dynamics trajectories. We assess the performance of ICTP models using the revised MD17 (rMD17) data set, which includes structures, total energies, and atomic forces for ten small organic molecules obtained from ab initio molecular dynamics simulations [47]. [able[]shows that ICTP sym achieves accuracy on par with sfate-of-the-ard spherical and Cartesian models. Notably, several methods exhibit similar accuracy when trained with 950 configurations. However, the achieved accuracy is much lower than the desired accuracy of 43.37 meV x 1 kcal/mol, making model comparison less meaningful. Therefore, we also compare ICTP, sym with MACE and NequlP, trained using 50 configurations, making learning accurate MLIPs more challenging. From Table we that ICTPsym sym outperforms MACE and NequlP for most molecules in this scenario.
We further evaluate the performance of ICTP using the MD22 data set, which contains seven molccular systems with 42-370 atoms (48). This data set spans four major classes of biomolecules and supramolecules and was designed to challenge short-range models. Table A2 shows that ICTP achieves an accuracy on par with or better than oatte-of-the-ar models, including long-range ones.
Extrapolation to out-of-domain data. We continue to assess the performance of ICTP models using the 3BPA data set (49. The training data set comprises 500 configurations, total energies, and atomic forces acquired from molecular dynamics at 300 K. The test data set is obtained from simulations at 300 K, 600 K, and 1200 K. We also test our models using energies and forces along dihedral rotations of the molecule. Table shoows that ICTP models trained using 450 configurations perform on par with state-of-the-ar spherical models, similar to the results for rMD17. However, we were not able to reproduce the original results using the current MACE source code and the described training setup [33]. Therefore, for a fair comparison, we unified the training setup for ICTP and MACE (see Appendix E.2) and Table 2 reports the corresponding results for 450 training configurations. In Table A3 we present the results obtained for ICTP and MACE trained using 50 configurations.
we observe that ICTP slightly outperforms MACE in total energy and atomic force RMSES but is ~ 1.4 times less computationally efficient. This difference arises from MACE using the generalized Flebsch-Gordas coefficients and pre-computing their product with the weights in the linear expansion in Eq. (AT) (33), which reduces the effective number of evaluated tensor products. Thus, we may attribute the lower computational efficiency of ICTP full to the use of the
From Table 2
8

---

Table 2: Energy (E) and force (F) root-mean-square errors (RMSEs) for the 3BPA data set. E- and F-RMSE are given in meV and meV/Å, respectively. Results are shown for models trained using 450 configurations randomly drawn from the training data set collected 300 K, with further 50 used for early stopping. All ICTP results are obtained by averaging over five independent runs. For MACE and NequlP, the results are reported for three runs. The standard deviation is provided if it is available. Best performances, considering the standard deviation, are highlighted in bold. Inference time and memory consumption are measured for a batch size 100. Inference time is reported per structure in ms, while memory consumption is provided for the entire batch in GB.
ICTPba
ICTF.gm
ICTP 1C
MACE
CACE
MACE
53
Nequll'
300
E
2.70 + 0.22
2.70
0.08
2,98
0.34
2.81 + 0.18
6.3
3.0
0.2
3.28 + 0.10
F
9.45
0.29
9.39
0.31
9.57
0.20
9.47
0.42
21.4
0.3
10.77
0.19
E 10.74
0.31
10.38
0.80
10.29
0.90
11.11
1,41
18.0
11,16
0.14
600 K
F 22.99
1: 0.64
0.91 22.87
23.03 = 0.76
23.27
1: 1.45
45.2
21.8
0.6
26.37 = 0.09
29.8
38.52
1200 K
F 62.82
1.23
64.54
3,88
65,36 1. 3.47
65.22 3.52
113.8
62.0
0.7
76.18 1. 1.11
Dihedral slices
10.64
1.00
13:03
23.2
F
17.52
0.54
17.18 0.81
19.31 = 0.83
17.69 1,29
16.5
1.7
23.1
Inference time
6.45 # 0.50
5.31 = 0.02
3,51 + 0.22
4.66 : 0.05
24.3*
103.5
Memory consumptiin
0.00
0.00
During inference time measurements with the MACE source code, we were not able to reproduce the original results [33] Thus, we re-run MACE experiments using training setup similar to that of ICTP; see Appendix E.2 à The original publication did not report the batch size used to measure inference time [33]. Therefore, the values provided used solely demonstrate the relative computational of MACE and NequlP.
a - 71.0", 1200"
< 07.0", 150JPP
- 151.0, 18000
DFT
Rel. in mev
ICTPsa
60-
ICTP
40-
MACE
100
Figure 3: Potential energy profiles for three cuts through the 3BPA molecule's potential energy surface. All models trained using 50 configurations, and additional 50 used for early stopping. The 3BPA molecule, including the three dihedral angles (a, B, and 7), provided shown as inset. The color code of the inset molecule is C grey, red, N blue, and H white. The reference potential energy profile (DFT) shown in black. Each profile is shifted such that each model's lowest energy is zero. Shaded areas denote standard deviations across five independent
degrees 9, is
MACE architecture, which results a larger pre-factor I. for Cartesian models but facilitates fair comparison between irreducible Cartesian and spherical tensors. Using the symmetric Cartesian product basis and that in the latent space, for example, we further improve the runtime of our models while maintaining accuracy on par with MACE.
Table A4 presents additional results obtained for = 1, i.c., focusing on models that rely exclusively on two-body interactions. We observe that Cartesian models exhibit shorter inference times than spherical ones, with MACE and ICTP full achieving 2.96 0.06 ms and 2.63 0.02 ms, respectively. Regarding memory consumption, MACE and ICTP perform similarly despite the larger number of tensor products required for the Cartesian product basis in Eq. ( This observation can be attributed to, for example, the Beebsch-Gordan tensor product requiring the computation of intermediate tensors with (2L + 1)² elements, whereas irreducible Cartesian tensors contain 3 elements.
Figure 3 compares potential energy profiles obtained with ICTP and MACE trained using 50 configu- rations. For the potential energy cut = 120" (left panel), ICTP and MACE perform similarly, except for the energy barrier at 7 2 o which ICTP tends underestimate stronger than MACE.

---

Table 3: Energy (E) and force (F) root-mean-square errors (RMSEs) for the acetylacetone data
set. E- and F-RMSE are given in meV and meV/Å, respectively. Results are shown for models trained using 450 configurations randomly drawn from the training data set collected at 300 K, with further 50 used for early stopping. All ICTP results are obtained by averaging over five independent runs, For MACE and NequlP, the results are reported for three runs, The standard deviation is provided it is available. Best performances, considering the standard deviation, are highlighted in bold.
ICTPPil
ICTP
ICTP ICTP ym
MACE
MACE X
NequlP 32
E
0.75 + 0.04
0.76 + 0.03
0.77 0.04
0.75 + 0.05
0.9 + 0.03
0.81 0.05
300 K
F
5.08
0.11
5.17 0.10
5.18 0.16
5.00 + 0.17
5.1 +
5.90
0.46
E
5.39
1.22
4,43
0.34
5.12
4,96 :: 0.64
4.6 +:
6.04 1.54
600 K
F
23.21 + 1.96
22.90 + 1.62
24.05 + 1.71
23.25 + 1.82
22.4 + 0.9
27.80 + 4.03
Number of parameters
2,774,800
2,736,400
2,648,080
2,803,984
2,803,984
3,190,488
Similar to Table we re-run MACE experiments using the similar training setup as for ICTP; see Appendix R.2
For B = 150 (middle panel), however, ICTP and ICTP outperform MACE across nearly the entire range of the dihedral angle y. For / = 180 (right panel), all models perform similarly. Fig- ure A1 shows the corresponding potential energy profiles for models trained with 450 configurations. All models perform similarly in this scenario, with energy profiles close to the reference (DFT).
Flexibilityy and reactivity. We further use the acetylacetone data set to assess the ICTP models' extrapolation capabilities to higher temperatures (similar to 3BPA), bond breaking, and bond tor- sions [3]. (ableB]show that ICTP models achieve state-of-the-are art results while employing fewer parameters than spherical counterparts. Appendix E.3 includes additional results for the acetylacetone data set, such as total energy and atomic force RMSEs for models trained with 50 configurations and details on the potential energy profiles for hydrogen transfer and C-C bond rotation. Overall, ICTP and MACE perform similarly, demonstrating excellent generalization capability. However, when trained using 50 configurations, ICTP is the only MLIP consistently producing the potential energy profile for hydrogen transfer close to the reference (DFT).
Multicomponent alloys. We finally evaluate the ICTP and MACE models using the Ta-V-Cr- W data set, designed to assess the performance of state-of-the-ee MLIPs in modeling chemically complex multicomponent systems. In this evaluation, we attempt to predict energies and forces for Ta-V-Cr-W subsystems under two scenarios: The K energies and forces in binary, ternary. and quaternary systems and near-melting temperature energies and forces in 4-component disordered alloys. Yable(A6) shows that ICTPsym outperforms MACE in nearly all subsystems, particularly in energy prediction. ICTP achieves an overall accuracy of 1.38 + 0.09 me for energies and 0.028 + 0.001 ev/À for forces, compared to 2.19 + 0.31 me V/atom and 0.029 + 0.001 eV/À by MACE. However, the K. pre-factor from the v-fold tensor product results in longer inference times for ICTP than MACE, in line with the discussion for 3BPA.
6 Conclusions and limitations
This work introduces many-body equivariant MPNNs based on higher-rank irreducible Cartesian tensors, offering an alternative to spherical models and addressing the limitations of state-of-the-art Cartesian models. We assess the performance of resulting MPNNs using five benchmark data sets, such as rMD17, MD22, 3BPA, acetylacetone, and Ta-V-Cr-W In these experiments, MPNNs based on irreducible Cartesian tensors show a lower computational cost of individual operations compared to spherical counterparts. Furthermore, we demonstrate that these Cartesian models achieve accuracy and generalization capability on par with or better than state-of-the oftth--rrt spherical models while memory consumption is comparable. Our results hold across the typical range of tensor ranks used in modeling many-body interactions and relevant physical properties, i.e., L < 4.
Limitations. We emphasize our focus on introducing MPNNs based on irreducible Cartesian tensors and prove their equivariance and traceless property. We adapted the MACE architecture, which uses only even tensor products, to enable fair comparison with state-of-the thee--r-tte-rttttte spherical models. Further modifications to the architecture are possible and necessary, e.g., to reduce the pre-factor arising from the Cartesian product basis, before can fully exploit the potential of irreducible Cartesian tensors.
10

---

Data availability
All data sets used in this study are publicly available: rMD17 /xtpp://@dl.org/10.66860
http://do..rgg/100o
m9. 1o.figsto fi.fiighrre... . v3). MD22
org). 3BPA (https://githu)
http////kooacce/Beeee-daaa-aaa-a BOTNett-datasets) and Ta-V-Cr-W (https://doi.cm orw.oi..orr11..0.. 18t../11////rruuu3311111
acetylacetone (https://githubb.com heeel/eeeooeeeeess/////aaaoocooa
Code availability
The source code is available on GitHub and can be accessed via this link: https: https://githbb com/ nec- rec-researhhhictp
Acknowledgements
MN acknowledges support from the Deutsche Forschungsgemeiinccaff (DFG, German Research Foundation) under Germany's Excellence Strategy EXC 2075 - -390740016 and the Stuttgart Center for Simulation Science (SimTTech).
References
[1] K. T. Butler, D. W. Davies, Cartwright, O. Isayev, and A. Walsh: Machine learning for molecular and materials science. Nature 559, 547-555 (2018)
[2] Vamathevan, D. Clark, Czodrowski, Dunham, E. Ferran et al.: Applications machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463-477 (2019)
[3] J. A. Keith, V. Vassilev-Galindo, B. Cheng, S. Chmiela, M. Gastegger et al.. Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems. Chem. Rev. 121, 9816-9872 (2021)
[4] O. T. Unke, S. Chmiela, H. E. Sauceda, M. Gastegger, I. Poltavsky et al:: Machine Learning Force Fields. Chem. Rev. 121, 10142-10186 (2021)
[5] N. Fedik, Zubatyuk, M. Kulichenko, N. Lubbers, J. S. Smith al.: Extending machine learning beyond interatomic potentials for predicting molecular properties. Nat. Rev. Chem. 6, 653-672 (2022)
[6] A. Merchant, S. Batzner, S. S. Schoenholz, M. Aykol, G. Cheon et al.: Scaling deep learning for materials discovery. Nature 624, 80 85 (2023)
[7] D. P. Kováes, Transferable Machine Learning Force Fields for Organic Molecules. https://arxiv.cr org/ abs/ 23122332222222 15211 (2023)
Moore, N. J. Browning, Batatia, T. Horton et al.: MACE-OFF23;
[8] I. Batatia, P. Benner, Y. Chiang, A. M. Elena, D. P. Kovács et al.: A foundation model for atomistic materials chemistry. Intpss://arxii org/ abs /bs/ 2401 00096 (2023)
[9] G. D. Purvis and R. J. Bartlett: A full coupled-cluster singles and doubles model: The inclusion of disconnected triples. J. Chem. Phys. 76, 1910-191 (1982)
[10] D. Crawford and H. F. Schaefer III: An Introduction to Coupled Cluster Theory for Computa- tional Chemists, pp. 33-136. John Wiley & Sons, Ltd (2000)
[11] R. J. Bartlett and M. Musial: Coupled-cluster theory in quantum chemistry. Rev. Mod. Phys. 79, 291-352 (2007)
[12] Hohenberg and W. Kohn: Inhomogeneous Electron Gas. Phys. Rev. 136, B864 B871 (1964)
[13] W. Kohn and I.. Sham: Self-Consistent Equations Including Exchange and Correlation Effects.. Phys. Rev. 140, A1133-A1138 (1965)

---

[14] A. Micheli: Neural network for graphs: A contextual constructive approach. IEEE Trans. Neural Netw. 20, 498-511 (2009)
[15] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini: The graph neural network model. IEEE Trans. Neural Netw. 20, 61-80 (2009)
[16] Gilmer, S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl: Neural Message Passing for Quantum Chemistry. Int. Conf. Mach. Learn. 70, 1263-1272 (2017)
[17] W. L. Hamilton, R. Ying. and J. Leskovec: Representation learning on graphs: Methods and applications. IEEE Data Eng. Bull. 40, 52-74 (2017)
[18] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst: Geometric deep learning: going beyond Euclidean data. IEEE Signal Process. Mag. 34, 18-42 (2017)
[19] Z. Zhang, P. Cui, and W. Zhu: Deep Learning on Graphs: A Survey. IEEE Trans. Knowl. Data Eng. 34, 249-270 (2022)
[20] S. Zhang, H. Tong, J. Xu, and R. Maciejewski: Graph convolutional networks: a comprehensive review. Comput. Soc. Netw. 6, 11 (2019)
[21] D. Bacciu, F, Errica, A. Micheli, and M. Podda: A Gentle Introduction to Deep Learning for Graphs. Neural Netw. 129, 203-221 (2020)
[22] C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo et al.: An Introduction Electrocatalyst Design using Machine Learning for Renewable Energy Storage. https / //arxiv. oareiv.org/ao www.///////////eeee// abaaabaa///00// 09435 (2020)
[23] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov et al.: Highly accurate protein structure prediction with AlphaFold. Nature 596, 583-589 (2021)
[24] J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte et al.: Robust deep learning-based protein sequence design using ProteinMPNN. Science 378, 49-56 (2022)
[25] Batzner, A. Musaelian, L. Sun, M. Geiger, Mailoa al.: -(3)-evuvvariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022)
[26] A. Duval, S. V. Mathis, C. K. Joshi, V. Schmidt, S. Miret et al.: A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems. https:///axxi: org/ haagg/ab//// abs//3/2..222// 07511 (2024)
[27] M. F. Langer, A. Goeßmann, and M. Rupp: Representations of molecules and materials for
interpolation of quanttum-mechanich simulations via machine leurning. npj Comput. Mater. 41 (2022)
[28] K. T. Schüu, O. T. Unke, and M. Gastegger: Equivariant message passing for the prediction of tensorial properties and molecular spectra. Int. Conf. Mach. Learn. 139, 9377-9388 (2021)
[29] M. Haghighatlari, J. Li, X. Guan, O. Zhang, A. Das et al.: NewtonNet: a Newtonian message passing network for deep learning of interatomic potentials and forces. Digital Discovery 333-343 (2022)
[30] G. Simeon and G. D. Fabritiis: TensorNet: Cartesian Tensor Representations for Efficient Learning of Molecular Potentials. Adv. Neural Inf. Process. Syst. 36, 37334-37353 (2023)
[31] B. Cheng: Cartesian atomic cluster expansion for machine learning interatomic potentials. npj Comput. Mater. 10, 157 (2024)
[32] I. Batatia, S. Batzner, D. P. Kovács, A. Musaelian, G. N. C. Simm et al.: The Design Space of E(3) Equivariant Atom-Centered Interatomic Potentials, https://arxiv.oo 06643 (2022)
[33] I. Batatia, D. P. Kovacs, G. N. C. Simm, C. Ortner, and G. Csanyi: MACE: Higher Order Equivariant Message Pussing Neural Networks for Fast and Accurate Force Fields. Ady. Neural Inf. Process. Syst. 35, 11 11423-1 1436 (2022)
12

---

[34] A. Musaelian, S. Batzner, A. Johansson, L. Sun, C. J. Owen et al.: Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun, 14. 579 (2023)
[35] S. Passaro and C. L. Zitnick: Reducing SO(3) Convolutions to S0(2) Efficient Equivariant GNNs. Int. Conf. Mach. Learn 202,
(2023)
[36] S. Luo, T. Chen, and A. S. Krishnapriyan: Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products. Int. Conf. Learn. Represent, https://arxiv..
abs/2401 10216 (2024)
[37] Thölke and G. D. Fabritiis: Equivariant Transformers for Newral Network based Molecular Potentials. Int. Conf. Learn. Represent. https:///ariiv/ org/ ab22 02541 (2022)
[38] R. F. Snider: Irreducible Cartesian Tensors. De Gruyter, Berlin, Boston (2018)
[39] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li et al: Tensor field networks: Rotation- and translation-equuvarri neural networks for 3D point clouds. https://arxiv org/ abs/ 1802. 08219 (2018)
[40] R. Drautz: Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B 99, 014104 (2019)
[41] E. Wigner: Group Theory and Application to the Quantum Mechanics of Atomic Spectra, volume 5. Elsevier (2012)
[42] A. R. Coope, R. Snider, and F. R. McCourt: Irreducible Cartesian Tensors. J. Chem. Phys. 43, 2269-2275 (1965)
[43] J. A. R. Coope and R. F. Snider: Irreducible Cartesian Tensors. II. General Formulation. J. Math. Phys. 11, 1003-1017 (1970)
[44] J. A. R. Coope: Irreducible Cartesian Tensors. III. Clebsch-Gordan Reduction. J. Math. Phys.
11, 1591-1612 (1970)
[45] D. R. Lehman and W. C. Parke: Angular reduction in multiparticle matrix elements. Math. Phys. 30, 2797-2806 (1989)
[46] M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. S. Cohen: 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. Adv. Neural Inf. Process. Syst. 31 (2018)
[47] A. Christensen and O. A. von Lilienfeld: On the role of grudients for machine learning of molecular energies and forces. Mach. Learn.: Sci. Technol. 1, 045018 (2020)
[48] S. Chmiela, V. Passilev-Galindoo O. T. Unke, A. Kabylda, H. Sauceda al.: Accurate global machine learning force fields for molecules with hundreds of atoms. Sei. Adv. 9, eadf0873 (2023)
[49] D. Kováes, C. v. Oord, Kucera, A. E. A. Allen, D. Cole et al.: Linear Atomic Cluster Expansion Force Fields for Organic Molecules; Beyond RMSE. J. Chem. Theory Comput. 17, 7696-7711 (2021)
[50] K. Gubaev, V. Zaverkin, P. Srinivasan, A. I. Duff, J. Kästner et al.: Performance of two complementary machine-learned potentials in modelling chemically complex systems. npj Comput. Mater. 9, 129 (2023)
[51] R. Drautz: Atomic cluster expansion of scalar, vectorial, and tensorial properties including magnetism and charge transfer. Phys. Rev. B 102, 024104 (2020)
[52] B. Anderson, T. S. Hy, and R. Kondor: Cormorant: Covariant Molecular Neural Networks. Adv. Neural Inf. Process. Syst. 32, 14537-14546 (2019)
[53] V. G. Satorras, F. Hoogehoom, and M. Welling: F(n) Equivariant Graph Neural Networks. Int. Conf. Mach. Learn. 139, 9323-9332 (2021)
13

---

[54] J. Brandstetter, R. Hesselink, E. van der Pol, E. J. Bekkers, and M. Welling: Geometric and Physical Quantities improve E(3) Equivariant Message Passing. Int. Conf. Learn. Represent. https: http://aaxvvv... wttp///v.orr//oooooo... abs/ 02905 (2022)
[55] K. Schütt, P.-J. Kindermans, H. E. Sauceda Felix, S. Chmiela, A. Tkatchenko al.: SchNet: A contimuous-fiie convolutional neural network for modeling quantum interactions. Adv. Neural Inf. Process. Syst. 30, 991-1001 (2017)
[56] O. T. Unke and M. Meuwly: PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. J. Chem. Theory Comput. 15 (6), 3678-366 3693 (2019)
[57] Y. Liu, L. Wang, M. Liu, Y. Lin, X. Zhang et al.: Spherical Message Passing for 3D Molecular Graphs. Int. Conf. Learn. Represent,
org/
05013 (2022)
[58] J. Gasteiger, S. Giri, J. T. Margraf, and S. Günneeann: Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules. Machine Learning for Molecules Workshop, Adv. Neural Inf. Process. Syst. http::::///xxii.orraabbbb//////s
14115 (2020)
[59] J. Gasteiger, F. Becker, and S. Günnemann: GemNet: Universal Directional Graph Neural Networks for Molecules. Adv. Neural Inf. Process. Syst. 34, 6790-6802 (2021)
[60] J. Gasteiger, J. GroB, and S. Günnemann: Directional Message Passing for Molecular Graphs. Int. Conf. Learn. Represent. attps://arxiv wrg/abs//////ooo.oooooo.oo..ssra absaaasssa03 00000000123333 (2020)
[61] D. P. Kováes, I. Batatia, S. Arany, and G. Csányi: Evaluation of the MACE force field architecture: From medicinal chemistry to materials science. J. Chem. Phys. 159, 044118 (2023)
[62] Fuchs, D. Worrall, V. Fischer, and M. Welling: SEn3)-Transformmrs:: 3D Roto-Translatin Equivariant Atlention Networks. Adv. Neural Inf. Process. Syst. 33, 1970-1981 (2020)
[63] T. Frank, O. Unke, and K.-R. Müller: So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems. Adv. Neural Inf. Process. Syst. 35, 29400-29413 (2022)
[64] Y.-L. Liao, Wood, Das, and T. Smidt: EquiformerV22 Improved Equivariant Transformer for Scaling to Higher-Degree Representations. Int. Conf. Learn. Represent. attpss//aarx org/ arg/abs/2n aa//ab//230000 2306. 12059 (2023)
[65] A. V. Shapeev: Moment Tensor Potentials: A Class of Systematically Improvable Interatomic Potentials. Multiscale Model. Simul. 14, 1153-1173 (2016)
[66] V. Zaverkin and J. Kästner: Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials. J. Chem. Theory Comput. 16, 5410-5421 (2020)
[67] V. Zaverkin, D. Holzmüller, Steinwart, and Kästner: Fast and Sample-Esficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments. J. Chem. Theory Comput. 17, 6658-6670 (2021)
[68] J. Behler and M. Parrinello: Generalized Neurul-Network Representation of Highh-Dienssionll Potential-Enent -nnegy Surfaces. Phys. Rev. Lett. 98, 146401 (2007)
[69] U. Fano and G. Racah: Irreducible Tensorial Sets. Academic Press Inc., New York (1959)
[70] I. M. Gel' fand, R. A. Minlos, and Z. Y. Shapiro: Representations of the Rotation and Lorentz Groups and Their Applications. Pergamon Press, Inc. (1963)
[71] S. V. S. Martin H. Müser and L. Pastewka: Interatomic potentials: achievements and challenges. Adv. Phys. X 8, 2093129 (2023)
[72] I. Grega, I. Batatia, G. Csanyi, S. Karlapati, and V. Deshpande: Energy-conserving equivariant GNN for elasticity of lattice architected metamaterials. Int, Conf. Learn. Represent, httpps //arxiv. org/abg///ooooooooo.oe abs/ 2401 16914 (2024)
14

---

[73] C. Joshi, C. Bodnar, S. V. Mathis, T. Cohen, and P. Lio: On Expressive Power Geometric Graph Neural Networks. Int. Conf. Learn. Represent, https::/axxtiii wttp:b/////.....aae./e/ hetppe///arrvvvorrr/////2...... (2023)
[74] S. N. Pozdnyakov, M. J. Willatt, A. P. Bartók, C. Ortner, G. Csányi et al.: Incompleteness of Atomic Structure Representations. Phys. Rev. Lett. 125, 166001 (2020)
[75] J. Behler: Atom-centered symmetry functions for constructing high-dimennional neural network potentials. J. Chem. Phys. 134, 074106 (2011)
[76] A. P. Bartók, M. C. Payne, R. Kondor, and G. Csányi: Gaussian Approximation Potentials: The Accuracy of Quantum Mechanics, without the Electrons. Phys. Rev. Lett. 104, 136403 (2010)
[77] A. P. Bartók, R. Kondor, and G. Csányi: On representing chemical environments. Phys. Rev. B 87, 1841 15 (2013)
[78] M. S. Daw and M. I. Baskes: Embedded--atom method: Derivation and application to impurities, surfaces, and other defects in metals. Phys. Rev. B 29,
(1984)
[79] Y.-M. Kim, B.-J. Lee, and M. I. Baskes: Modified embedded method interatomic potentials for Ti and Zr. Phys. Rev. 74, 014101-014112 (2006)
[80] A. Stone: Transformation between cartesian and spherical tensors. Mol. Phys. 29, 1461-1471 (1975)
[81] A. Stone: Properties of contesian-spheriiee transformation coefficients. J. Phys. A 9, 485 (1976)
[82] J. M. Normand and J. Raynal: Relations between Carlesian and spherical components of irreducible Cartesian tensors. J. Phys. A 15, 1437 (1982)
[83] K. He, X. Zhang, S. Ren, and J. Sun: Deep Residual Learning for Image Recognition. IEEE Conf. Comput. Vis. Pattern Recognit. hattps://doi www.
http:::://00..or01101110000 http://oo/./....////....... 2016 (00.1000000P (2016)
[84] S. Chmiela, A. Tkatchenko, H. Sauceda, I. Poltavsky, K. T. Schütt al.: Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 3, e1603015 (2017)
[85] K. T. Schütt, F. Arbabzadah, S. Chmicla, K. R. Müller, and A. Tkatchenko: Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 13890 (2017)
[86] Chmiela, H. Sauceda, K.-R. Müller, and A. Tkatchenko: Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun. 9, 3887 (2018)
[87] S. Elfwing, E. Uchibe, and K. Doya: Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning. Neural Netw. 107, 3-11 (2018)
[88] P. Ramachandran, B, Zoph, and Q V. Le: Searching for activation functions. Int. Conf. Learn. Represent. https://arxis: wwwpexxiiii.ireeeeeeeee abs/
05941 (2018)
[89] J. Reddi, S. Kale, and S. Kumar: On the Convergence of Adam and Beyond. Int. Conf. Learn. Represent. https://arxiii org/ abs/ 1904. 09237 (2018)
[90] Y. Li, Y. Wang, L. Huang, H. Yang, X. Wei et al.: Long-Short Sae---hort--gggg Message-Passing: A Physics-Informen Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics Simulation. Int. Conf. Learn. Represent, https: / arwiv org/ 2000// 1010..1344... 13542 (2024)
[91] J. Frank, O.T. T. Unke, K.-R. Müller, and S. Chmiela: A Euclidean transformer for fast and stable machine learned force fields. Nat. Commun. 15, 6539 (2024)
15

---

Appendices
Appendix Contents
Background
17
Methods
18
B.
Normalization constants
18
B.2
Further details on the equivariant message passing
18
B. B.3 Computational complexity and runtime analysis
19
Proof of Cartesian cesssage-passing equivariance actions of the orthogonal group
20
D Proof of the traceless property for irreducible Carfesian tensors
23
Experiments and results
29
Description of the data sets
29
Training details
30
Additional results
31
Broader social impact
36
16

---

A Background
Message-passee neural networks. Message-passing neural networks (MPNNs) learn node repre- sentations graph by iteratively processing local information sent by the nodes' neighbors. The initial features of node u are represented as the vector X,. and undirected edges {u, v} connect pairs of nodes u, v. node v belongs to the neighborhood of node u, denoted as N(u). if there exists an edge {u, v in the graph. Typically, the (f + 1)-th peessage-passing layer computes new node u's representation h. 11++) by applying a permutation invariant aggregation function over the neighbors [6 21
h (++1) = 0l (n..
D (0() (b!) (b!...)))
where (f(),, v((() are often implemented as learnable fully fully-connected neural networks (NNs), and (0) To learn mapping from learned representation (t) to the atoms' energies, we can couple pressage-passingg layers with corresponding readout functions R2 E {l....,7) such that
E. = - (b."....
Many-body interatomic potentials. Interatomic potentials approximate the potential energy of atoms-the electronic ground state energy-as function of their coordinates [71]. Many-body potentials naturally arise because the interaction between two atoms is influenced by the presence of additional atoms, changing their electronic structure. This concept is formalized by expanding the atomic energy E, of many-atom system into series of two-body, three-body, and nigher-body-ordee contributions
= E (T.) + E
Σ 42
E (rui Fe, Fo,) +...
where represents the position of atom and the superscript v in E(r) indicates the order of the many-body interaction. the absence external fields, E contributions to the atomic energy are invariant to rotations, and the two-body potential depends only distances Tav ||r, r.l12
Ey
Σ E(2) U2
(22) (Twe,)
+
Σ E (Tee, Tuvgy Terez)
1/1 <22
Expansions of this form have found broad application in constructing machine- learned interatomic potentials (MLIPs), significantly advancing the field [40 65].
Higher-body-orde local descriptors. Recent advances in MLIPs have been influenced by ment tensor potentials (MTPs) [65] and atomic cluster expansion (ACE) [40] These approaches enable systematic construction of higher-body-order polynomial basis functions, encompassing representations like atom-centered symmetry functions (ACSFs) [68] (5), smooth overlap of atomic
positions (SOAP) (7..77). Gaussian moments [66] 677. and embedded atom//ulti-scale embedded atom method (EAM/MEAM) potentials 77 79. Furthermore, a reducible Cartesian tensor can be
represented as a linear combination of irreducible spherical counterparts, and vice versa [40] 5I]. More general expressions, including tensor contractions, have also been provided, hinting at the relationship between Cartesian and spherical models [2]. Despite the success of MTP and ACE, defining smaller cutoff radii and rigid architecture can result in limited accuracy compared to MPNNs.
Many-body message passing. Designing accurate and computationally efficient interatomic potential models for interacting many-body systems requires including gigher-body-oreee energy contributions and, thus, higher-body-order learnable features. Recently, a new message construction mechanism has been proposed by expanding the messages ma to include many-body contributions
B
m (e)
Σ 2
+
Σ
(1)
(
Σ
(e) Pot1
(h(()
h (0
where ( 1) denotes the order of many-body interactions, defining the number of contracted tensors. Using E.,.. instead of Ev...eo, <....,, circumvents the exponential increase in computational cost with v. It allows exploiting the product structure of many-body features, which differs from other approaches (58-60).
17

---

B Methods
B.1
Normalization constants
The following provides the normalization constants Oi142 and Disl for even and odd irreducible Cartesian tensor products in Eqs. 2 and (3). respectively. The respective normalization constants
read 45
Chitala = (212 I)I!((L) (1) + 1)/2)
21, 2?!!2 Is![g!![,!![,,(((((3)))
Ditlls (13 - 1)1(41 + 1)!!( + 1)!!(43 + (0L((++1//)))
with T, = +1 + and T4 = I. 21, -1 Here, Ciasst is defined such that an I3-fold contraction
of the tensor Zı., obtained through the irreducible Cartesian tensor product between Xi, and yı, (x,, and yi, are obtained using Eq. (I). with the unit vector i yields unity. refer to the original publication for the motivation behind SthindDDi12 Dist2 45
B.2 Further details the equivariant message passing
The following provides additional details on the employed equivariant message-passinng architecture. Leamable weights employed in Eqs. (4) 5 and (6 (i.e., RK'T w A.. and W kikt with k k' run- ning over feature channels) are initialized by picking the respective entries from normal distribution with zero mean and unit variance.
Many-body message-passing The many-body equivariant features, represented by an irreducible Cartesian tensor of rank and obtained through the v-fold tensor product in Eq. (6). are combined using the linear expansion
(m ukt) (kkk)) inga..i
( ( (1) KL WnokL) in
(A1)
where ((()) Z.. denotes learnable weight matrix which depends on the chemical element Z, and rank L and which elements are initialized by picking the respective entries from normal distribution with zero mean and a standard deviation of 1/len(M))) The updated node embeddings are further
obtained as linear function of (m
(k ig ( and the residual connection
(f)
(h (1+1)) ukL ig
dd
Σ w(O
KKLL
(m (6)
ok iiigoii +
VdiNZ
Σ
(1)) Z.kk'L (h i
ig
(A2)
where Nz denotes the number of atom types and learnable weights W-(t) RE'L and W-(t) k. are initialized by picking the respective entries from normal distribution with zero mean and unit variance.
Full and symmetric product basis. In the MACE architecture, the authors pre-compute products between the generalized Clebsch-Gordal coefficients, which define the interactions of (7, ... 1-)-
rank spherical tensors, and the learnable weight W-(t) Z.n.kL of the linear expansion in Eq. (A1) 33
This approach reduces the computational cost of constructing the product basis with spherical tensors
the effective number of evaluated tensor products is smaller by A. len (n,). For irreducible Cartesian tensors, operations like matrix-vector and matrix-matrix products define the interactions
between {4,... I, -rank tensors. Thus, equivalent operation to those proposed for spherical tensors may be generally impossible for irreducible Cartesian tensors or would lead to an architecture different from MACE.
Note that one of our main goals is to demonstrate that a many-body equivariant cressag-ppassing architecture defined using higher-rank irreducible Cartesian tensors can be as expressive as the one using spherical tensors. Therefore, we compute mukr. by iteratively evaluating all possible u-fold tensor products, which effectively leads to a larger number of operations than for MACE. We refer to the models based on this architecture as irreducible Cartesian tensor potentials (ICTPs) with the full product basis, ICTP We also noticed that the number of tensor products len(n,) leading to
(t)
18

---

the tensor of rank L, can be reduced by the symmetry of the two-fold tensor product in Eq. ( i.e., Ar, @ A Ar & A,, if A,, and A1, are symmetric. We refer to this design choice as ICTPss with the symmetric product basis or ICTPPsym sym-
Coupled feature channels. Using coupled feature channels instead of uncoupled ones in Eq. {AT) can improve the performance of the final model. Additionally, the number of feature channels and, thus, the overall computational cost can be reduced when constructing the product basis. However,
the number of parameters and, thus, the expressive power of the model can be preserved. Specifically, we can define linear expansion for combining many-body equivariant features as
m (1) wkL 1112-001
(ΣΣΣ WlO W
((() Zkr Pankk'k) igig--in."
(A3)
where wr(t) 7,,,, K denotes learnable weight matrix which depends on the chemical element Z, and rank L and which elements are initialized by picking the respective entries from a normal distribution with zero mean and standard deviation of 1 1/(V x len (1)))) The construction of the product basis in the latent feature space requires encoding the two- woo-boyy features A ukig using the learnable weight matrices in Eq. (6). The linear expansion is then constructed in the latent feature space and decoded using the Iearnable weight matrices of the linear function in Eq. (A2). We refer to this design choice as ICTPs with the product basis constructed in the latent feature space or ICTP Combining it with the symmetric product basis, we obtain ICTP, +l-
(t)
Readout, Atomic energies expanded into a series of many-body contributions, E,, (0( + (1)
...+ EETT are obtained by applying readout functions R2 to node features with L 0, which are invariant to rotations
vol En W(O A (t) (L=0)))) NN (1 he)
ift < T, ift ift= T,
E = Re (
(A4)
with learnable weights (((0) or those of NN(") initialized by picking the respective entries from a normal distribution with zero mean and unit variance. Linear readout functions for t < T preserve the
many-body orders in (2) =u) while a one-layer fully-connected NN is used for the last message- passing layer and accounts for the residual higher-order terms in the expansion
32
B.3 Computational complexity and runtime analysis
We analyze the computational complexity of the irreducible Cartesian tensor product with respect to the maximum rank L used. For each tuple of (i1,-.. .iz) of a rank-L tensor in Eq. (2). the single contraction term (x (A: + m) yr) has a cost of 34+m. The total computational cost is 3 for even tensor products and 3 for odd ones due to the additional double contraction with the Levi- Civita symbol. The computation of the set of permutations over the L unsymmetrized indices scales as L!/ (24/2 (L/2)!) for even tensor products and L!/ (2(1-1)/2 ((L - 1) /2)!) for odd ones [45). Because each final rank-L tensor has 3 elements, the complexity for computing irreducible Cartesian tensor of rank L is O (9* L!/ (2 (L/2)!)). This expression is used throughout this work as it captures contributions from tensor contractions and index permutations, though it simplifies asymptotically to O (T.*) using Stirling's approximation for factorials.
The number of calculations required obtain rank- 7. spherical tensor through the Clebsch-Gordal tensor product is (2L 1)S. Thus, the computational complexity of the Clebsch-Gordan tensor product is O (I.3). While the Clebsch-Gordan tensor product is more computationally efficient than the irreducible Cartesian tensor product for T, - oo, the latter is expected to be advantageous for smaller tensor ranks, i.e., L < 4, assuming similar multiplicative factors and negligible sub-leading terms. Tensors of rank T. A 4 are particularly relevant for equivariant messagg-passsing architectures and representing physical properties [32 33 72). Compared to the Gaunt tensor product with the computational complexity of O (L²), including all (41,42 + la (36), the irreducible Cartesian tensor product may be less advantageous for L > 3. However, the Gaunt-coefffcienns---aeee -basedd approach excludes odd tensor products, limiting the expressive power and the range of possible architectures.
As regards the cost of performing message passing each layer, the general neighbors' aggregation of Eq. (4) has cost of EN (24/2 (L/2)!), where & is the number of edges in the atomic
19

---

system and Ne. is the number of feature channels. Computing many-body features via Eq. (6) requires to iteratively perform 1 Cartesian tensor products for all K = len(n,) possible v--fold tensor products, resulting in a computational cost M TN (X/(2///222((//)))/)>> with M denoting the number of nodes. For the Crebsch-Gordaas tensor product, the corresponding number of calculations is &N L and MN Spherical models, however, can use generalized coefficients, resulting in MN (v(e++3) for the product basis. We can remove the factor from the above expression by restricting the parameterization to uncoupled feature channels as Eq. AA and obtain MN 2r(0+33, Thus, MACE with this specific choice of product basis can be more computationally efficient than ICTP only for large Neh and small given L 4. this work, we have shown that leveraging the symmetry the irreducible Cartesian tensor product and coupled feature channels improve the computational cost of ICTP. The former. in particular, reduces the effective number of y-foodd tensor products K. Further optimization of the architecture, however, is possible and is expected to fully exploit the advantage of Cartesian tensors.
Finally, memory consumption for Cartesian tensors is often believed be less advantageous than that of spherical tensors. However, for contracting two rank-L spherical tensors, the memory requirement about (2L + 1)², resulting from the definition the Clebsch- Gordan tensor product in Section 2 For Cartesian tensor, the memory requirement is about 3 Thus, the irreducible Cartesian tensor product can expected to require the same or less memory for S than the Clebsch-Gordan counterpart. The memory consumption of models based on spherical tensors is further increased by the use of generalized Brebsch-Gordie coefficients, as we demonstrate in Section 5
C Proof of Cartesian message-passing equivariance to actions of the orthogonal group
We first recap the standard action of the 0(3) group onto (R3) 81 orthogonal group ((3) (also in line with Section 2 i.e.,
Let Dx be representation of the
Dx 0(3) + R 3x3 g Dxíg] = R.
We define the action of the ((3) group onto (R3) @l
as follows
0(3) x (R3) 81
8t
(g. T Til
Σ....... Bus.
Ray (Ti)
where T2 E (123) @! We hereinafter denote & (g. Tí) also by RT,, The outer product @ is equivariant this action, and acts trivially on the 3-identity matrix.
Lemma C.1. Let R be a representation of element of the orthogonal group O(3). Then,
(i) A YT E (R") 80 and V Tr, E (I23) 681z
R (T @ T(,) (RT;,) @ (RT,)))
(ii) For the 3 x 3-identity matrix I, we have
RI R I.
Proof. (i) We first show that R RTr, & Ti,) = (RT;,) (.) (RT,,)
((RT,,,) @ (RTT (RTT,)) Da ( 4ir
(RT, da (RTT))ear
Σ
Rash ... Ruch (Tt) (Tu)semsi Pe4tammdiyti
Reperd Ru, Ra (Ths ( bn
Σ
Σ
Bay Brggh Buytrdien"" Begeeghhe (Ti) (T Tr
Jn
(R,,,, 2 Ty,)) doa
20

---

(ii) With Σ; Rsk =
E5k, or equivalently R² R = I, we get
(RT) = Σ
Rin Rig 8ji0 = Σ
Raih
Riis Righ 8/162.
51:30
Using Lemma C.1 we can show that the inreducible Cartesian tensor operator ; R + (R3) is
cquivariant to actions of the O(3) group.
Proposition C.2. Let 2 O be positive integer. Then, the irreducible Cartesian tensor operator (R is equivariant actions of the 0(3) group.
Ty: R
Proof. It suffices to show that the map f + (m0(1-2m) Im is equivariant actions of the 0
group 44 2 1 and V m S [1/2] (m 2 0).
(Rir) 01 11-12mm) 2m)
@m Lemma
(RT)) ((1-2m)mmm)))) 2m) @ (RI) 8m
R (p01l-2m) I pm
Our next claim that for 13 E [41 - 121. 14 - /2 + 1.......
.4 + 12) the irreducible Cartesian tensor
product &Cart defined in Eqs. (2 and (3) diagram commutes
is equivariant to actions of the O(3) group, i.., the following
(
64
(2
S12
CCat
(R3) B4
v
R 24
812
S/a
Cart
The proof for this claim is very similar to Proposition equivariance for the (k + m) ffld tensor contraction.
C.2
The only difference to show the
Proposition C.3. The irreducible Cartesian tensor product @Cart makes the above diagram commute.
(R3)
84
(R3))
(R3) 0/s
Proof. It suffices to show that the (A: + m)-fold tensor contraction equivariant to actions of the 0(3) group. For X1, E (R3))1, orthogonal group ((3), we can write
188,,, Vi, (IR3) and being a representation of an element of the
((Rx), (s) (Ryi,)) p1 -
Σ
(Rxi, ) (1 aa
01
Σ
Σ
Rarn ... Rayy Remn ... R (Xi, ( --
a1, as
Moe Mi1-
x Σ
Ro ... Ra.7, Ro, ... Ro - (Viz (i m.
Σ
Rgmm R Ran-
Ráym
Râ -
mi
(Xts " -m
= -R (Xi, (s) y1,) -
21

---

A similar derivation applies to the odd case 3 which completes the proof.
We finally give a proof for Proposition 4.1 i.e., the equivariance of message-passing layers based on irreducible Cartesian tensors to actions of the O(3) group.
Proof of Proposition 4.1 Since message-passinng layer in Section 4.2 and Appendix B.2 is defined
by stacking layers corresponding to Eqs. 4 5 (AT) (A2), and (A3 we show the equivariance for each of these equations. We show the equivariance by induction.
Case t =
Equivariance of Eq. D Since learnable weights Waz, and Iearnable (invariant) radial ba- sis kli (Fuv) are defined for each feature channel k, the weights are multiplied with scalars. Therefore,
(Tr, (Puo)) i1 11
ukly 1162-1111 = =E RR (Fwv) ( (Tr, (fav)) i, i1 Wkz.
is equivariant to actions of the 0(3) group.
Equivariance of Eq. [ The proof is similar to the case of Eq. (), since we again have learnable
parameter W kk't, for each A (1) uuki. Therefore, noting that the Cartesian tensor product is equivariant to actions of the O(3) group, the following function
B Bn.kkL fnip.k)iiieg..
iség...ir. (A ukl, &Can
(1)
@Cart A ukl.) isig.mir.
also equivariant actions of the 0(3) group. Equivariance Eq. (AT): The equation defined by
(m
(1) ukt.) S122
[ΣΣ
w
(1)) Zun.k. k.. issguiis
T/e
equivariant actions of the 0(3) group, since for each set of indices u, yuu and we have a scalar learnable parameter [=(1)) kL
Equivariance of Eq. A2)) : The respective parameters w-(1) LkkLL and (
Z in Eq. A2
(2) (h
(hu 6112--11 ddd
Σ
W RRrI,
(m (m.) 1n Vd,22
Σ w
(1) Z. (h inte-42"
are multiplied as scalar with h and m ak 4. which are shown to be equivariant to actions of (1)
(1) the (3) group. Thus, h (2) is equivariant to actions of the 0(3) group as a function h h,
and moke
Equivariance of Eq. A3): The cquivariance of Eq. A3) actions of the 0(3) group is also
immediate since learnable parameters apply to each tuple of (11,22, .iz) as scalar.
General case f 1: The equivariance of Eqs. the O(3) group for arbitrary > follows in similar way to those obtained for the f = 1 case.
AI
and (A3) to actions of
However, we need show the equivariance of Eq. (4). Suppose that Eqs. (5). (6) AI (2 and
(3)) are equivariant to actions of the O(3) group for 1. Then, for the (t+ 3-th meesaage-pssiipg layer,
(A()) (8) wkii)) ( inigerin = -
(Rilsstt (Riliss (rwv) Tr, (fav) &Cart
(((t) h dele kkll, 1(
1 Σ
is equivariant to actions of the 0(3) group. Here, the irreducible Cartesian tensor product &Cat and Tr, are equivariant to actions the O(3) group, b is equivariant by the assumption, and (() w sRU, applies to pkit as a scalar.
22

---

D Proof of the traceless property for irreducible Cartesian tensors
This section provides a proof for Proposition 4.2 in the main text. The corresponding proposition states the following
Proposition D.1. The message-passing layers based on irreducible Cartesian tensors and their irreducible lensor products preserve the traceless property of irreducible Cartesian lensors.
Our proof of Proposition 4.2 comprises main parts. First, show that given two irreducible Cartesian tensors, the irreducible Cartesian tensor product yields again an irreducible Cartesian tensor. We then can this result the second part, where show that expressions defined Eqs. 4. (
(6) [AI. (A2). and (A3) preserve the traceless property of irreducible Cartesian tensors. The proof of the second part is straightforward since the message-passingg layers defined by multiplications with scalars, summations, and the irreducible Cartesian tensor product of tensors defined in Eqs. ①
( and (3.. Therefore, we provide the proof only for the first part in the rest of this section.
Irreducible Cartesian tensors from unit vectors. Recall that an irreducible Cartesian tensor of arbitrary rank for unit vector i TR33 is defined
[1/2] m== (-1)m (21 2m 1)!!
Ts (f) = C
(1112mm) @ pmm 1mm
(21 - 1)!!
The trace this tensor reads
Tr (T, (f))
Σ 11
(Tr (f))iigs
Σ 11
C >
11/21
(-1) (21 2m 1)1!
((+0))2m) I0m) in
m=0
(21 1)!!
[1/2]
(-1)
(21 - 2m 1)!!
Σ
(100l-2m)) (00m)
m=0
(21 1)!!
inmis
5182
1} with 2m distinct elements, and subset 1m-2m. with 2m distinct elements in (i,,..., it] is written as I12m {iks i.2...., dkg 2m We also let Ji-2m be set of m disjoint subsets with 2 elements in {ir,..., it I-2m i.e., J1-2m = Hik--mm11 t1 ik ............................ {ik 1, }.. While we introduce 11-2m and Jr Jm as sets, we assume the order of elements in those sets if the order is necessary and there is no confusion. Furthermore,
Let (k1, k2.................... Kj 2m) be subset of
we introduce two notations used throughout this section
(1-22m)
fizz Pikg Fim
11 2m
pm
dieg 2m+1 4k1-2m42
din Axy
Then, we can write
(+s(1- 2m) [mm]
Σ, Jt 2mm
Σ Ji J:- am
9m1-2m)
&m
11 2m
2m
= Jm- Σ Emmmom
Pins
Fins dia, ami Shy 2mc1
where I1-2m runs all subsets of {di....,úi) it} with I 2m elements and J1-2m comprises all (II (2))
/ml combinations of subsets with 2 elements of ( i V1-2m- Depending on whether i and/or ig belong to Im the above expression may be further reduced to
(a)m Σ
(m)
(c)m
(4)
(c)m
Σ
Σ
Σ
+
Σ
Pizz
P
hm
i 2m fr 2m
$1.82 f.
izEJ 2m
61112
bett Segtt
Dieg 2m 2 2m12
23

---

We get the following traces for each pair of 11-2m and J1-2m
Tr (()))
-1, (1- 2m))
11-2m (
X
mm J1-2m²
Tr (()) Ein=io Pirs fir Fin- 2m
- 2m 2h 22122
Diky Disa
Ping ".Pi fim dike 201
Dike tke
2m
2m
(1122m) liv, Ju Mit
(mm-1)
& Jm Mil,
Tr (())mm) En,, in= Pin ... Pig Pim- 2m
di
Diny tay Dina
2m+1 Phi 2m+2
Ping ..ii fm
dins 2 11
din: te1
des 2m12
2m
(112mm
(mm-1) Je-am M6
(ie, Ju Mit
Tr Trd())m)) = E ii= Pin,
Ping-a 2m Din
Ding- iig Sigés fig 2m
(+0(1-mm)
(m-1)
3 11-2m
Tr Tr)))m))
= Σ i1
Pigg
Pang am
dimg thy 6igir Sizis Ding
=1/ 2m1-mm)
((m-1) J/-2m 1111.10)
Lemma D.2. Lett 2 2 and & m S 11/21-1. For each pair (It-2m: J JL-2m), there exist 21 2m pairs of (Ti-2(m+1): J 2mm whose trace equals to the trace for the (J--m 2m i.e.,
(2l - 2m - 1) (rr
(1-2m) 2m
Σ
(0112((++1)
8(m+1) 2(-)(--1)
J1 2m
Tr
1-2(++1)
(1:-2(m4 21-2()+1))
Proof. Without loss of generality, may assume I1-2m (11, i2, ......... Pk 2m)) and J1-2m
(2k--mm4 (k-2m42)..... (
{ik 1ika-:ihh ik For V m. Tr ((a)m) has the following expression
Tr Tr((a)m) = Σ E,,
iú fig Ping ... Piey 2m dim fiz 2m
ding 1ig
=1
= Pirg Pin--am Si 2mt1 42-222
For Tr ((b)m+1) and m++1, 1,, we have set of (112(m+1)) 21 with the cardinality of o 1-2mm3 and the corresponding trace equals to Tr (())m) For 3 S Vosi 2m, let
((5) 1-2(m+1) = {ikss
in 2m }.
a-th index
(/00) 1m2((m+1) {(ie
2m 1iy 12 J. ..... o fir 1. } (
Here, note that assume the order of elements in (s) 2(m+1)- Then, obtain
Tr (()m+1) = Σ in=iz
fine
f Pikz-am din 22422
din dx Sia,
3=3
s-th vector
-''
1-2m "'22m
3 3
Pina
Pin.
Fin-
8i 2m 2m
(1 2m Tr ((a)m)
The same derivation as above holds for Tr ((c)m+1). For Tr ((d)m+1), let
1-2(m++1) fikgs...y 2m
J, J1-2(m+1) (fik) dig Aha-2maah )maa). ..... 1 fir, (2)
24

---

and we get
Tr (r((m)++1)= = E 11 Pina
Six Pay Sirba Fin 2m - 2k1-2m+2
=3 (())m)
For Tr ((e)m+1) and 1 S m, we let
(8) .
(kk }:
1m=((m+1)
1100) J1-2(m+1)
{(i 2m11: dag amir)..... (ike 3 fiz, dey 2m12 J.... (ike 1 1
and obtain
Tr ((e)m=1) =
m
Fing ... Pinz- 2m
ding 2m fag 2m
Diny am +2 Seate, 2m42.
11 =1
==1
Pips
Ok 2k Ram
2-1
2m Tr ((a),.)
The additional factor of two originates from the permutation of indices is and ig.
Using all the above results, we obtain
Tr(())++1) TT((()m++)) + (r((()m+1))+ (T(0mmt))(((())mtt))) (r((())+1))
= =(1 - 2m - 2) Tr ((a)m) + (1 2m - 2) Tr ((a)))))) + 3 ((a)m) + 2m Tr ((a))))))
(2) 11-2m 2m 1) Tr ((a),))
which completes the proof of the Lemma.
) Tr ((d)m) Tr ((e) Then, the trace of T2 (f) may be written as a summation of (m) and B(m) over m
For each m, define A(m) Tr ((a)m) ) and B(m) Tr ((6) Tr ((c)
Tr (f)) C Σ
m=u (-1)
[1/2]
(21 2m 1)!!
Σ
(f:80-2m) & 1 11122
(21 -1]!
11
C
(A(0) + -(0))
+ (-1) (21 - 1)!!
(A(1) + B()))
(21 1)!!
+ (-1) (21 2m 1)!!
(A(m) + B(m))
(21 - 1)!!
+ (-1) i+1
(21- - 2 + 1) 1)!!
(A(m 1) B(m + 1))
(2l 1)!!
By Lemma D.2, each pair of the diagonal terms (A (m) B (m + +)) is cancelled, and B()) and A (L1/2]) are left, which by definition are trivial. This result completes the proof of the traceless property for Eq. (
Even irreducible Cartesian tensor product. The trace of an even irreducible Cartesian tensor product may be written
Tr ((xI, @Cart Yia)
Σ
Chista Emint minnk (-1)) (21g- - 1)!!
mm (21g 2m 1)!!
(f (xi, (k + m) yts) 2 I I 1)
Σm ) (-1)mmm (21g 2m 1)!!
Σ (f (xi, (k + m) yea) I pm 1i
(21g - 1)!!
(4)
25

---

In the following, we let
IA (1e4m) (ike..... iki -(++m) / C fi1....,dss
J1 (k+mm
{iii -14mm(++))-1-- Thig 2m fir..... , //. (k+-()+m))
Kig-2m {iiis 2m 111 Mag 2m 2m12) p.... ibng 1 fhigg }
For simplicity, we omit the subscripts 1 (k + m)) 12 (k + m)) and 1a 2m for I, J, and K, if there is no confusion. We also introduce new notation " = " where x is an irreducible Cartesian tensor and Q {q1....,90 Ja which is used throughout this section. Then, non-trivial terms in even may be written as
(a)m
())
(c)m
(d)
(c)m
Σ
Σ
Σ
Σ
Σ
Σ
(x12) (y jn
Sm
11=12
QE
HeruJ
LEIUJ
f1
faundkem
12EJ
FIEJ
KgE
We note that the trace is trivial when both of i and i2 belong to either of I or J, because Xt, and V12 are traceless by assumption.
Lemma D.3. Let m min((1, - k - 1. For each triplet ([1--(++m)) J1s (k++m) Kig (l-22m)) there exist 213 2m triplets of (It.((++++1)- Jtg (k+m-1)) Kig- ((m=2((+1)) such that the trace for each triplet equals to the trace for (It--(++m) J1g (k (k) Kig-2m).
Proof. Without loss of generality, may assume
[1--(m-m) {ir (k1, dkk (ttm) C (ii....,iss)
Jln-(k-m) fiz, iki, (( ikig Migg-mm 2m } C fir..... .....,,,, its Vi, -k+m))
Klg-2m (ikkg-2m 2m 0kg 2m12 iig ikig }
Then, Tr ((a)m) has the following expression
Tr( Tr((a).. m
Σ Σ
(x1, j (k-m)
(yis) fn dait, -(++m)+2 tia 2m
Sm
(1=12 Janrndketm
Σ Je Jk4m+1
(xh)) da (eim)
(y (k+m)42 Mag -mm
60m
The same derivation applies Tr ((b)..) and it has the same result as above.
For Tr((c)m-1) and m have 03 2m of (I1,- (4,,((+m++)) - (++m+1) Kis 115-2(ktme m triplets whose trace equals Tr ((a),,.. Indeed, like in the proof of Lemma D.2 we let
I (k=(k+++1) J1g - -1+-((+++1) iikg..... ....... dkg (k (m)) ddi, de (2)))22--... ikey 1kgg-2m }
Kt, 2mm=((+)))) = { (00g-2244 iki 1kg-22+42 J..., (ikig ikig F. fee, }2)
Then, by letting j (j..... jutm Jk+m+1) } we obtain
Tr((c)m+1)
Σ
Σ
Xx Jiny (ii ()+m) (ylz) Jin, (x+m))+ wiii 2m
11=12
dinig 2mt1 thig 2m
8 dinig iiis di in
ΣΣ (xi, J. J.ing wri widd, m)+m) (ylz J.in),inn (2+m)++ wiig 2m
Sisig 2m Miig 2m42
dinig Sii
= (l3 2m - -2) ((a)m)
26

---

The same derivation applies to mr((d)m-1), and we get Tr((d)++1) = (43 - 2m 2) Tr ((a)m)
For Tr(())m+1), the same derivation for (r((d)++1) and (r(())++1) in the proof of Lemma D.2 applies. Therefore, we have
Tr((e)m+1) = (2m + 3) Tr ((a)
Hence, we obtain
Tr((c)++1) + (((((d)))))))))) + Tr((e)m+1)
(la 2m - 2) (r((()))))) (l3 - 2m 2) Tr((amm) + (2m + 3) 3)((())m
(2l3- 2mm 1) Tr((a)m) = (21g 2m - 1) Tr (b)m),
which completes the proof of the Lemma.
For each m, define A(m) 'r ((a) + II ((6),)) and B(m) Tr (()) ((e)) Then, the trace for (X1, CCart Yt, )g may be written as a summation of A(m) and B(m) over m
((d))
Tr ((xi, &Carr v(a)1a)
=
Ch,it
Σ Dmo
min((s.da)
(- (213 2m 1)!!
(21, 1)!!
Σ
-12 (1 - (I: + m) m),) @ 10m) in
Chilsls
(A(0) + B(O))
(21s
1)!!
(-1)2
(A(1) + B(1))
(213 1)!!
...
+ (- (21g 2m 1)!!
(A(m) + B(m))
(21a 1)!!
(-1)++120++
(21g-
2(m 1) 1)!!
(A(m + 1) B(m+ 1))
(21g 1)!!
By Lemma D.3
each pair of the diagonal terms (A(m), Bm + 1)) is cancelled taking into account an additional factor of 2 in front of +(++1):: B(0 and (min(4, (2) (4 + + - (22) are left,
which by definition are trivial. This result completes the proof of the traceless property for Eq.
Odd irredueible Cartesian tensor product. The trace of an odd irreducible Cartesian tensor product may be written as
Tr ((xi, &Cart Nislis)
min((1562)-k- mm (21s-
1)!!
Σ Dislats Em=o
(((e X1, (k ()) yi.)) @ jm D)s
(2ls- 1)!!
Sminltate m=0
miin(f1.d))
mm (21g- 2m 1)!!
= -Diials
(((e: x1 (k + m)) v..) @ pm "i
(21g 1)!!
(*)ae
In the following, we let
Fag = {iro} C (ii,..., [ta}
41 (k=+m)
fiter.... ike, (m-m)) C fir..., it MEss
J12((km))
{ike ((A 2hi amd fir...., .,, V (41, ((++m) U Eig)
Kig-2m 2m
{fiti 2m lik am 7.................... (ing 1: .. }}.
For simplicity, omit the subscripts la, 1 (k m), l2 (k m), and L3 2m for E, I, J. and K, if there is no confusion. Then, non-trivin terms inside (*) can be written as
(o)m
Wi
(c)m
+
Σ
Σ
+
Σ
Σ Juundeim 02
Σ
EBoues (Xt)) Ssame.. (Yi)) Jesnns Pifos) 18
HEAUIJM K
J
GERJIU HEK
hilskR
27

---

Note that the trace is trivial when i or is belongs to E and the remaining index belongs to J.J. Indeed, let J = {J......K*to ji+m] and suppose i E E and i2 E I. Then, and we have
(Tr ((x1, @Crt v(s)ig) (1ud))(1a)
Σ
Σ ΣΣ
Σ
eiozas (Xi,, (, (yrz) Joss
02:03
Eipogas (xi1)
(112) J (M1a)
J
11 mig 02
Σ J
Σ
Eijaza (og (y12)J,aa.J - (xn)) ( (
(11.02.05)
(1,2,3)
(2,1,3)
(3,1,2
= 0.
For the non-trivial terms inside (*)odd we have the lemma below, whose proof is identical to that of Lemma D.3
Lemma D.4. Let m min((1, (211,,2)-44 For each tuple (Fit -k++mmm)) -t-())))t---ae (k+-()) Kig (l5-2m)) there exist 213- 2m tuples of (Eig -1 Jig (k+m+1) Kl, 2(m++ 13) such that the trace for each of the tuples equals to the trace for (Etg: Lt, - (k+m)) J1g (k+m)) Kmg-2m).
Proof. Without loss of generality, we may assume
Eig {iko {d1,....úts) iis)
(k+((mm))
{úe, Ukgs . dhe (mr))) c (ir,..., ;i,.. |Bts
J/2 1k=(kmm)
{dz die, -(4+m)+) iki am } C fer..... in P (4, (k+m) ((+))(++) U Eis))
Kig-2m 2m {{ Lin
ihig 2m422 .... like Skeg }}.
2m 151g-22-11:
Then, Tr ((a)m) has the following expression
Tr ((a)m)
Σ
Eiug (Xis (xis)semneemm aalithy -atm:
(yis) fumlemmmemm hempdeemaaaaatt asdatti, -ct "id
mi u2yd0 hunddemm
Σ
uiaguzus (Xi, Sarteeaa maiks "ii ()
(Y (y (4-m0)4 Aag -mm
06.0g Peyeegooo Jesm+3
Here, we assume that E 4t, (k+m) and 0g E J12- (k+t)) The same derivation holds for Ir ((b)..)
For Tr((c)m-1) and m + 1, we have l3 - 2m 2 of (Fig. Jt (k+(mm++1) Jiz- (k+m+ 1): K -is-2(m+11) triplets whose trace equals to Tr ((a),,) Indeed, like in the proof Lemma D.2 we let
Ei, U (k+m+1) U Jig (k+m+1) = {ike ikys dn Rks (m)t dig (tt ikg K(--((++1) (iing 2m+1 ik 2m ...., fi ikz }. {ix, 22)
28

---

Then, by letting J (fi.... Jo+m- jk+m+1) J, we obtain (r((c)m+1)
Eizg azag (XL,, Jazingg "og ind (k1)) (Vlz W (k (kim)) "hing
11 02,03 J
doiig am+1 faig am++ Össig Kheg
dix,
Σ ΣEΣ
azes 0283 (Xi, Jazinz win mhy (k)m)
(vla) Jaster, (kim)) "iii
02,93
dsii am+ fRig am++ Ossig-1 tkgg
= (13 2m - 2) Tr ((a)m))
The same derivation applies to mr( and we get Tr((d)m+1) (1y - 2m 2) Tr ((a)
For Tr((e)m+1), the same derivation as for Tr((d)m+1) and Tr((e)m+1) the proof of Lemma D.2 applies. Therefore, we obtain
Tr((e)m+1) (2m + 3) Tr ((a)
Finally, we can write
Tr((c)++1) (r((() m+1) + Tr((e)m+1)
= (13 - 2m 2) Tr((a)m) + (13 2m -2) T)) + (2m + Tr((a)m)
(243 2m 1) (r(())m) (2ls 2m 11) (r())))
which completes the proof of the Lemma.
Noting that Tr ((a)m) ) is doubled for every due the factor 2mh the proof is completed similarly to the proof of Lemma D.3. This result completes the proof of the traceless property for Eq. (3).
E Experiments and results
E.1 Description the data sets
rMD17 data set. The revised MD17 (rMD17) data set is a collection of structures, energies, and atomic forces of ten small organic molecules obtained from initio molecular dynamics (AIMD) [47. These molecules are derived from the original MD17 data set [47 86), with 100,000 structures sampled for each. Our models are trained using 950 and 50 configurations for each molecule randomly sampled from the original data set using five random seeds, with 50 additional configurations randomly sampled for early stopping. We use the remaining configurations to test the final models. Table T the mean absolute errors (MAEs) in total energies and atomic forces averaged over five independent runs, including the standard deviation between them.
MD22 data set. The MD22 data set includes structures, energies, and atomic forces of seven molecular systems derived from AIMD simulations at elevated temperatures, spanning four major classes of biomolecules and supramolecules [48). These molecular systems range from small peptide containing 42 atoms to a double-walled nanotube comprising 370 atoms. Characterized by complex intermolecular interactions, this data set was designed to challenge short-range models. The training set sizes used in this study are consistent with those the original publication [48), selected to ensure that the sGDML model stays within target accuracy of 1 kcal/mol (* 43.37 V). We randomly selected an additional set of 500 structures for each molecule the data set for early stopping, while the remaining configurations were reserved for testing the final models. The corresponding training and validation data sets were randomly selected using three random seeds. Table repoorts MAEs in total energies per atom and atomic forces averaged over three independent runs, including the standard deviation between them.
3BPA data set. The 3BPA data set comprises structures, energies, and atomic forces of a flexible drug-like like organic molecule obtained from AIMD various temperatures [49]. The training data set consists of 500 configurations sampled at 300 K, while three separate test data sets are obtained from
29

---

AIMD simulations at 300 K, 600 K, and 1 1200 K. An additional test data set provides energy values along dihedral rotations of the molecule. This test directly assesses the smoothness and accuracy of the potential energy surface, influencing properties such as binding free energies to protein targets. Our models are trained using 450 and 50 configurations randomly sampled from the training data set using five random seeds, with further 50 configurations reserved for early stopping. Table 2 reports the uquaae errors (RMSEs) in total energies and atomic forces averaged over five independent runs for a training set size of 450 structures, including the standard deviation between them. Table A3 presents the corresponding results obtained with a training set size of 50 structures.
A Acetylacetone data set. The acetylacetone data set includes structures, energies, and atomic forces of a small reactive molecule obtained from AIMD at various temperatures [32]. The training data set comprises configurations sampled at 300 K. while the test data sets are sampled at 300 K and 600 K. The generalization ability of final models evaluated using an elevated temperature of 600 K and along two internal coordinates of the molecule: The hydrogen transfer path and a partially conjugated double bond rotation featuring a high rotation barrier. Our models are trained using 450 and 50 configurations randomly sampled from the training dataset using five random seeds, with further 50 configurations reserved for early stopping. Table B reports RMSEs in total energies and atomic forces averaged over five independent runs for training set size of 450 structures, including the standard deviation between them. Table A3 presents the corresponding results obtained with a training set size of 50 structures.
Ta-V-Cr-W data set. The Ta-V-Cr-W data set includes 0 K energies, atomic forces, and stresses for binaries, ternaries, and quaternary and near-melting temperature properties in four-component disordered high-entropy alloys (50). In total, this benchmark data set comprises 6711 configurations, with energies, atomic forces, and stresses computed at the density functional theory (DFT) level. More precisely, there are 5680 0 K structures: 4491 binary. 595 ternary, and 594 quaternary structures, along with 1031 structures sampled from MD at 2500 K. Structure sizes range from 2 to 432 atoms in the periodic cell. All models are trained using 5373 configurations, with 4873 used for training and 500 for early stopping. The remaining configurations are reserved for testing the models' performance. The performance is evaluated separately using 0 K binaries, ternaries, quaternaries, and near-melting temperature four-component disordered alloys. The original data set provides ten different training-test splits. In our experiments, each training set is further split into training and validation subsets using random seed. Furthermore,
use additional binary structures strained along the [100] direction as a part of the test data set. Note that the final models, in this case, were obtained by training using the whole data set of 6711 configurations (training test), 500 of which were reserved as a validation data set. Table reports RMSES in total energies per atom and atomic forces averaged over ten independent runs, including the standard deviation between them.
E.2 Training details
All ICTP and MACE models employed in this work were trained on a single NVIDIA A100 GPU with 80 GB of RAM. Training times for ICTP and MACE models typically ranged from 30 minutes to a few days, depending on the data set, the data set size, and the employed precision. We used double precision for 3BPA and acetylacetone data sets and single precision for rMD17, in line with the original experiments (3). Double precision was also used for MD22, while single precision was employed in our C--V experiments. Unless stated otherwise, we used two message- layers and irreducible Cartesian tensors or spherical tensors of a maximal rank of (max = 3 to embed the directional information of atomic distance vectors.
For ICTP models with the full (ICTPrat) and symmetric (CCTPsym) product basis and MACE, we employ 256 uncoupled feature channels. Exceptions include our experiments with the 3BPA data set, aimed investigating scaling and computational cost, and the Ta-V-Cr-W experiments. where we used eight and 32 feature channels, respectively. For ICTP models with the symmetric product basis evaluated in the latent feature space sym))s we use 64 coupled feature channels for the Cartesian product basis and 256 for two-body features. Radial features are derived from eight Bessel basis functions with polynomial envelope for the cutoff with p = 5 (60) These features are fed into a fully connected NN of size [64, 64, 64). We apply SiLU non-linearities to the outputs of the hidden layers [87 [8). The readout function of the first message-passing layer is implemented as a linear layer. The readout function of the second layer is single-layer fully connected NN with 16 hidden
30

---

neurons, A cutoff radius of 5.0 Å is used across all data sets except MD22, where we used a cutoff radius of 5.5 À for the doublc-walled nanotube and 6.0 Á for the other molecules in the data set. All parameters of ICTP and MACE models were optimized by minimizing the combined squared loss on training data Durii (Airain Verain), where Xuain {S ) and Verain
k==
{E [Fref Na. Narain
a,k Ja- 0% kk=
v( + 4=1
Narain Σ
(0, Dirain) =
Co Exet E(S(A),
Σ
Fu. - F. (S((1) l1:
(A5)
+ C.
Vioke - Viao (stk), 00
(
Here, 0 is the stress tensor defined strain deformation with symmetric tensor E R and V is the volume of the periodic box.
V c= where E denotes total energy after
When training ICTP models on rMD17, 3BPA, and acetylacetone data sets, we neglected the stress loss and set C. 1//vat (R) and atomic forces, respectively. For MACE and 3BPA/accctlaacetone, C. = 1/(B x ((() and Ả G 10000 Nv1h) )
at and C 10 A to balance the relative contributions of total energies
were used with B denoting the batch size. For ICTP models trained on (R) and G = 1 Å using energies and forces in eV and ev/,, respectively. For the Ta C-Cr-W dataset, the stress loss was incorporated into the combined loss in Eq. (A5), along with the energy and force losses. For ICTP, we used Ce 1/Na& Cr = 0.01 À and C² = 0.001 0.001/N(6) to balance the relative contributions of total energies, atomic forces, and stresses, respectively. For MACE, chose C. L/(B x N:(h) _V(()) G L/(B 3 x Nx(k)) A?, and Cg = OB05/(B x 9 x (*(k) x N-(k)). Here, ((S(k), 0). F,, (S(R), 0). and o (S(K), 0) are total energies, atomic forces, and stresses predicted by ICTP or MACE.
the MD22 data set, we set C. = 10/N~
at
All models for rMD17, 3BPA, and acetylacetone were trained for 2000 epochs using the AMSGrad variant of Adam [89]. with default parameters of 8, 0.9, 833 0.999, and E = 10-8, For MD22 and Ta-V-Cr-W,
models were trained for 1000 epochs. For rMD17, 3BPA, and acetylacetone data sets, used learning rate of 0.01 and batch size of 5. For MD22 and Ta-V-Cr-W, we again chose a learning rate of 0.01 but a mini-batch of 2 and 32, respectively. For evaluations on the validation and test data sets, we used batch size of 10 for rMD17, 3BPA, and acetylacetone, while mini-batcher of 2 and 32 were used for MD22 and Ta-V-Cr W, respectively.
The learning rate was reduced using an on-plateau scheduler based on the validation loss with patience of 50 and a decay factor of 0.8 for all data sets except for MD22, for which we used a patience of 10. We utilize exponential moving average with weight of 0.99 for evaluation on the validation set and for the final model. Additionally, in line with MACE 3R we apply exponential weight decay of 5 x 10-7 on the weights of Eqs. (6. (A] and (A3). Furthermore, we incorporate
per-atom shift of total energies via the average per-atom energy over all the training configurations, including the energies of individual atoms for 3BPA and acetylacetone datasets. If no atomic energies are provided, as for rMD17, MD22, and Ta-V-CrW the per-atom shift is obtained by solving a linear regression problem (67). Additionally, a per-atom scale is determined as the pot--man--sqann of the components of the forces over the training configurations.
E.3 Additional results
Scaling and computational cost. Table Al provides the numerical results complementing Fig. in the main text. All results in Table AT and Fig. were obtained using irreducible Cartesian tensors with maximal rank of Imax = L to represent local atomic environments. Here, L denotes the tensor rank of employed equivariant messages. We facilitated the exploration of larger y values by setting the number of uncoupled feature channels to eight. In the MACE model, intermediate spherical tensors with a rank of / Imax are used to construct the product basis. However, the pre-computation of generalized Clebsch-Gordas coefficients for > 4, in some cases, would require more than TB of RAM. Therefore, in our experiments, we fixed the maximum rank of intermediate tensors to I = Imax. We also used the full product basis for ICTP to calculate the same number of y-fold tensor
31

---

Table A1: Inference times and memory consumption as function of the tensor rank T. and the correlation order y for the 3BPA data set. All values for ICTP and MACE models are obtained by averaging over five independent runs. The standard deviation is provided if it is available. Best performances are highlighted in bold. Inference time and memory consumption are measured for batch size of 10, Inference time is reported per structure in ms; memory consumption is provided for entire batch GB.
1=1
=
L=
ICTP full
MACE
ICTP full
MACE
ICTP fall
MACE
Inference times
V 0.76 + 0.17
1.02 0.03
0.87
0.18
1.38 + 0.04
0.98
0.26
1.88 - 0.03
V = 2
0.59 0.20
0.20
1.12:000
1.03 = 0.21
1.52 0.05
1.34 0.08
2.0 = 0.10
V 3
0.79 0.22
1.23 +
1.15 0.08
1.67 0.03
2.23 = 0.03
V = 4
0.17
+.11
1.31 0.21
1.83
2.07 + 0.20
2.53 + 0.01
= 5
1.02
0.08
1.72 0.07
2.26 1.26:0000
3.61 0.02
0OM
V = 6
1.0 0.07
1.77 - 0.05
1.83 0.16
27.85 + 0.01
16.76 = 0.35
0OM
Memory consumption
V = 1
0.05 + 0.00
0.04 + 0.00
0.08 0.00
0.06 + 0.00
0,21 + 0.00
0.13 + 0.00
2
0.05 0.00
0.04 0.00
0.08
0.00
0.07 + 0.00
0.28 0.09
0.13 + 0.00
V = 3
0.05
0.04 0.00
0.10 = 0.00
0.08 + 0.00
0.00
0.23 = 0.00
4
0.05 0.00
0.18 0.08
0.30
1.07
4.16 = 0.00
V = 5
+
0.07 0.00
0.35 0.07
0.00
5.07
OOM
V = 6
0.11 + 0.09
0.22 + 0.00
0.93 0.00
50.49 + 0.00
28.48 + 0.03
0OM
Table A2: Energy (E) and force (F) mean absolute errors (MAEs) for the MD22 data set." E- and F-MAE are given in me V/atom and meV/Å, respectively. Results are shown for models trained using training set sizes defined in the original publication [48.. All values for ICTP models
obtained by averaging over three independent runs. We also use an additional subset of 500 configurations drawn randomly from the original data set for early stopping. The standard deviation is provided if available. Best performances, considering the standard deviation, are highlighted bold.
ICTP
VISNe--ISRM
Equiformer
Seikmnes 01
MACE 61
Allegno Xn
TorchMDD-Net
.GDML,
Nc-A
0.02
3.91
3.49
10.58
4.63
8.15
34.26
0.01
3.17
Succhyos
052
4.13
Buckyboll catcher
14.39
minotibe
12.64 0.23
14.51
11.91
12.0
14.87
43.5
22.55
ICTP. MACE, Equiformer. Allegro, and TorchMD-Na are short-range models (i.e., they use local or semi-local atomic representations), ViSNet-LSRM and SO3krates include long-range information, and sGDML a global model.
* For consistency, these results obtained for relative energy and force weights of C. 10/N(A> and experiments with the Ac Ac-Ala3-NHiie molecule. For comparison, also trained
C = A A based
ICTP for 800 epochs with C. 100/N((k) and C 1A A? achieving MAEs of 0.209 + 0.007 V/atmm for energy and 13.98 + 0.30 meV/À for force.
products, i.e., K. = len (nw), used in MACE with 1 1max- Finally, we note that ICTP and MACE use different approaches to optimize their runtimes; however, the scaling with respect to the tensor rank and the correlation order is independent of these optimization methods.
32

---

Table A3: Energy (E) and force (F) root-mean-square errors (RMSEs) for the 3BPA data set (results for Nerain = 50). E- and F-RMSE are given in meV and meV/Ä, respectively. Results are shown for models trained using 50 molecules randomly drawn from the training data set collected at 300 K, with further 50 used for early stopping. All ICTP and MACE results are obtained by averaging over five independent runs, with the standard deviation provided if available. Best performances, considering the standard deviation, are highlighted in bold.
ICTP full
ICTP sym
ICTP sym++I
MACE
E
14.98 = 1.62
13.43 1.00
16.03 + 1.26
14.54 = 1.02
300 K
F
37.21
2.14
37.27 + 1.63
38.38 + 1.55
37.68 = 1.33
E
31.68
3,56
31.63 + 3.94
30.74 + 1.82
30.71 + 3.43
600 K
F
69.87 = 3.04
68.87 + 2.94
69.62 + 2.88
69.88 + 3.88
E
92.16 = 9.33
86.0 + 12.03
78.51 + 8.88
83.99 = 8.20
1200 K
F
157.72 = 6.53
153.16 9.62
151.37 + 9.59
154.46 10.84
33.69 = 8.03
31.06 6.27
33.44 + 5.56
27.79 + 7.41
Dihedral slices
F
47.12 5.27
47.68 = 2.21
49.08 + 4.05
48.91 = 4.71
Molecular dynamics trajectories. Table A2 presents the energy and force mean absolute errors (MAEs) for the ICTP models trained on the MD22 data set. This data set was designed to challenge short-range potential models-i.e. those based on local or semi-local atomic representations and includes large molecular systems with complex intermolecular interactions. We compare errors obtained for ICTP models with those of state-of-the- long-range information. From Table we see that ICTP achieves accuracy in predicted energies and atomic forces par with better than state-of-the-an methods. The ICTP model uses cutoff of 6.0 À (5.5 À for the double-walled nanotube), resulting in receptive field of 12.0 Á (11.0 Å), considering the two pesssage-passingg layers. This receptive field is, in most cases, smaller than the diameter of molecular systems in the data set. Thus, ICTP is, most, semi-local potential model-similar to MACE, which used 5.0 A cutoff and two message-passing layers.
approaches that incorporate global, short- and
We chose a larger cutoff for ICTP than the one used by MACE motivated by the results in the recent work 77. In particular, the authors compared MACE to VisNet-LSRM- the best model reported to date for MD22, which employs mixed short- and long-range information in their message passing-and and reported that MACE can achieve lower force errors. However, VisNet-LSRM typically had lower energy errors, attributed the improvement from considering atomic interactions beyond 10.0 A. Using larger cutoff radius for ICTP compared MACE, we expected results closer to those of ViSNet-LSRM.. Table A2 shows that using receptive field of 12.0 À is, in most cases, sufficient to achieve accuracy in predicted energies close to the one obtained by VisNet-LSRM.
Extrapolation to out-of-domain data. Table A3 demonstrates total energy and atomic force RMSEs obtained for ICTP and MACE models trained using 50 configurations randomly drawn from the original data set. ICTP and MACE models perform similarly, considering the standard deviation obtained across five independent runs. However, ICTP models often have lower mean RMSE values compared to MACE. Furthermore, Table Table|A4|e presents [Areseentsss the total energy and atomic foree RMSEs for ICTP and MACE models that use v = to compute the product basis. Thus, we provide results for models which rely exclusively two-body interactions. We note that for ICTP sym are identical; though, we include both results for completeness.
ICTP full and
Figure Al compares potential energy profiles obtained with ICTP and MACE models trained using 450 configurations. Potential energy cuts at B 120 and 3 = 180" are casier tasks for MLIPs, as there are training points in the data set with similar combinations of dihedral angles (32 In contrast, the potential energy cut at / = 150% is more challenging, with no training points close to it. Notably, Fig. Al shows that all models produce smooth potential energy profiles close to the reference ones (DFT) for all values of 3. These results again demonstrate excellent extrapolation capabilities of irreducible Cartesian models that are on par with the spherical MACE model.
Flexibility and reactivity. Table A5 demonstrates total energy and atomic force RMSEs obtained for ICTP and MACE models trained using 50 configurations randomly drawn from the original data set.
33

---

Table A4: Energy (E) and force (F) root-mean-square errors (RMSES) for the 3BPA data set (results for v 1). E- and F-RMSE are given in meV and meV/Á, respectively. Results are shown for models trained using 450 configurations randomly drawn from the training data set collected at 300 K, with further 50 used for early stopping. For all models, v 1 is used. Thus, the employed models rely exclusively on two-body interactions. All ICTP and MACE results are obtained by averaging over five independent runs. Best performances, considering the standard deviation, are highlighted in bold. Inference time and memory consumption are measured for batch size of 100. Inference time is reported per structure in ms, while memory consumption is provided for the entire batch GB.
ICTP
ICTP sym
ICTP sym+
MACE
E
12.90 + 1.06
12.90 = 1.06
14.97 + 1.64
13.50 + 1.71
300 K
F
29.90
0.25
29.90 0.25
30.93 + 0.47
30.18 + 0.38
E
29.97
0.94
29.97 0.94
31.64 + 1.83
31.32 + 2.16
600 K
F
62.80 + 0.45
62.80 = 0.45
64.54 + 0.95
63.04 + 0.73
E
81.03 + 1.64
81.03 + 1.64
79.26 +: 4.66
81.54 + 2.02
1200
F
146.96 1.30
146.96 1.30
151.45 + 5.21
149.44 + 1.94
E
22.84 2.96
22.84 + 2.96
29.16 + 7.96
28.08 + 4.04
Dihedral slices
F
48.82 + 5.25
48.82 + 5.25
52.53 + 5.27
49.62 + 2.92
Inference time
2.63 + 0.02
2.62 = 0.02
2.53 + 0.01
2.96 + 0.06
Memory consumption
32.57 + 0.00
32.57 = 0.00
32.34 + 0.09
23.32 + 0.00
- TLIP. 2 120.1"
67.0" 15000
161.P, 180.1
DFT
Rel. evergy meV
ICTPμa
ICTP
MACE
20-
in
7 in
Figure A1: Potential energy profiles for three cuts through the 3BPA molecule's potential energy surface (results for Nirain = 450). All models are trained using 450 configurations, the remaining 50 are used for early stopping. The 3BPA molecule, including the three dihedral angles (a, 3, and 7), provided in degrees is shown as an inset. The color code of the inset molecule is C grey, O red, N blue, and H white. The reference potential energy profile (DFT) is shown in black. Each profile is shifted such that each model's 's lowest energy is zero. Shaded areas denote standard deviations across five independent runs.
Similar to the 3BPA data set, ICTP and MACE models demonstrate comparable accuracy in predicted energies and forces, considering the standard deviation obtained across five independent runs.
Figure A2 further investigates the generalization capabilities of ICTP models trained using 450 configurations, demonstrating potential energy profiles for the rotation around the C-C bond, i.c.. the C-C-C-O dihedral angle (a), and for the hydrogen transfer (i.e., the O-H distance doh in Fig. A2). The training data set encompasses dihedral angles less than 30". Furthermore, the energy barrier of 1 eV is outside the energy range of the training data set obtained at 300 K. As for the hydrogen transfer. the training data does not contain transition geometries, but the reaction still occurs region that is not too far from the training data. Overall, ICTP models perform on with MACE for predicting potential energy profiles for the rotation around the corresponding C-C bond and for the hydrogen transfer.
34

---

Table A5: Energy (E) and force (F) root-mean-square
(RMSEs) for the acetylacetone data set (results for Ntrain 50). E- and F-RMSE are given in meV and meV/Å, respectively. Results are shown for models trained using 50 configurations randomly drawn from the training data set collected at 300 K. with further 50 used for early stopping. All ICTP and MACE results are obtained by averaging over five independent runs. Best performances, considering the standard deviation, are highlighted in bold.
ICTP full
ICTP sym
ICTP sym+I
MACE
4.42 = 0.39
4.45 + 0.36
4.38 0.24
4.22 = 0.52
300 K
F
28.85 = 3.00
28.28 + 1.45
29.17 = 1.65
28.38 + 2.74
E
17.61
3.14
16.13 + 1.37
17.3 2.05
17.53 + 3.58
600 K
F
75.09 = 8.70
69.99 + 5.12
74.96 + 4.68
74.93 + 9.91
(a)
(b)
1250
15
Rel. energy in meV
1000
750
10-
ICTP
ICTP Pmill iua
5-
MACE
250
dou
Count
1.3 Distance Á
1.2
Dihedral angle in
Figure A2: Potential energy profiles of (a) the dihedral angle describing the rotation around the C-C bond and (b) hydrogen transfer between two oxygen atoms (results for Nurain 450). All models are trained using 450 molecules, and the remaining 50 are used for early stopping. The acetylacetone molecule, including the dihedral angle in degrees describing the rotation around the C-C bond (a), is shown as an inset in (a). The color code of the inset molecule is C grey, O red, and H white. The reference potential energy profile (DFT) is shown in black. Each profile is shifted such that each model's s lowest energy is zero. The histograms demonstrate the distribution of dihedral angles and O-H distances in the training data. Shaded areas denote standard deviations across five independent runs.
Figure A3 shows potential energy profiles for MLIPs trained with 50 configurations, similar to Fig. A2 Notably, Fig. A2 (b) demonstrates that ICTP is the only MLIP consistently producing the potential energy profie for the hydrogen transfer close to the reference (DFT). This task is particularly challenging, as most data splits do not include configurations sufficiently close to the transition structure as in Fig. A2
Multicomponent alloys. The Ta Ta-V-Cr-W data set is designed to evaluate the performance of MLIPs across atomic systems with varying numbers atom types/components comprising both relaxed (0 and high-temperature structures [50]. particular, this data set includes 0 K energies, forces. and stresses for 2- 3-, and 4-component systems and 2500 K properties in 4-component disordered alloys. It contains 6711 configurations with sizes ranging from two to 432 atoms the periodic cell. Table A6 demonstrates the energy and force RMSEs for the ICTP and MACE models, evaluated separately on K binaries, ternaries, quaternaries, and eear-melting temperature comp-cnmpn disordered alloys. ICTP consistently outperforms state-of- of-te-of-the-arr tte-thh-rrt art models for the Ta-V-Cr-W W data set, i.e., MTP and GM-NN. contrast, for MACE, we were not able to identify a set of relative weights for energy, forces, and virial losses that consistently yield results better than those of MTP and GM-NN in both energies and forces. Table A6 shows that MACE often matches the accuracy of ICTP on forces but is typically outperformed by factor of < 2.0 on energies.
35

---

(a)
(b)
Rel energy in meV
1250
DFT
1000
750
10-
ICTP
ICTP gon-ix
500
MACE
250
don
Count
1.3 Distance in Á
Dihedral angle in
Figure A3: Potential energy profiles of (a) the dihedral angle describing the rotation around the C-C bond and (b) hydrogen transfer between two oxygen atoms (results for Ntrain = 50). All models trained using 50 molecules, and additional 50 used for early stopping. The acetylacetone molecule, including the dihedral angle in degrees describing the rotation around the C-C bond (a), shown as inset in (a). The color code of the inset molecule is C grey, O red, and H white. The reference potential energy profile (DFT) is shown in black. Each profile is shifted such that each model's lowest energy is zero. The histograms demonstrate the distribution of dihedral angles and O-H distances in the training data set for one random seed used to split the original data. Shaded areas denote standard deviations across five independent runs.
We further compare the ICTP, MACE, MTP, and GM-NN models using the separate test data set containing binary structures strained along the [100] direction. We find that neither ICTP MACE consistently outperforms MTP and GM-NN in this case. However, because this test data set contains only a single configuration of 432 atoms per binary, it may not serve valuable benchmark in this study and is included merely for completeness. Additionally, we trained Cartesian model with and mmax 2, which resembles TensorNet-like architecture [30], though it incorporates equivariant convolution filters. The corresponding results are provided in Table A6 We found that model configurations with y 3 and
(and often with L 1) outperform the 2 and L mmax 2 configuration by factors of < 1.4 and < 1.2 in energy and force RMSEs, respectively.
results for the 3BPA data set, observe that MACE can be computa- tionally efficient than ICTP. We attribute longer inference times of ICTP to the pre-factor K. arising from the Cartesian product basis Eq. (6) Thus, attribute ICTP's lower computational efficiency to the use of the MACE architecture, which is optimized for spherical tensors. We expect that further modifications the architecture can facilitate more efficient use of the advantages operations based on irreducible Cartesian tensors.
Finally, similar
F Broader social impact
This section discusses the broader social impact of the presented work. Our work has important implications for the chemical sciences and engineering, many problems in these fields require atomistic simulations; we also discuss in Section I Although this work focuses on standard benchmark data sets, our experiments demonstrate the scalability of our method to larger atomic systems. Beyond constructing machine-learne interatomic potentials, equivariant models based on irreducible Cartesian tensors be applied for molecular property prediction, protein structure prediction, protein generation, ribonucleic acid structure ranking, and many more.
Our work has no obvious negative social impact. As long as it is applied to the chemical sciences and engineering way that benefits society, it will have positive effects.
36

---

Table A6: Energy (E) and force (F) root-mean-square errors (RMSEs) for the Ta-V-Cr-W data set. E- and F-RMSEs are given in me V/atomom and eV/Å. Results are obtained by averaging over ten splits of the original data set, except for the deformed structures. For the latter, the results obtained using the whole data set (training + test). For the ICTP, MACE, and GM-NN models, we randomly selected validation data set of 500 structures from the corresponding training data sets. Best performances, considering the standard deviation, are highlighted in bold. Inference time and memory consumption measured for batch size of 50. Inference time" is reported per atom in pss; memory consumption is provided for the entire batch in GB.
Sangrum
ICTP
MAITO 73 0:8
MA
MTF
GM-NNN
BAM
0.37
1-66- 0-0-3
2.2 1:34
01500
1.51
181 1.25
1.0 0.23
2.15 2..0
3.36 042
331
6.55
0.33
3.26
2.88
43.00
TUW
0.11 = 0/912
1775 2.11
0101
2,5 1131
0200
2755 1631
0/992
3:1 5.3
6/92
W629
3.5 1.4.55
0,818
2.19 1454
0,992
2.22
2.9
0.348
Ver
0.816 6/892
1.74 1.2
0.00 1000
2:52:: 2.43
0.09
113 1.34
11301
0.06 0.901
2.9 0.76
IL81T B/81
252.5:13 1.25
5.1155122
01907
0.099
1,00
= 1.31
0/501
0116
1.29
2.22
44.00
0.700
VW
0.114 : 0/902
1,52
0315 2 1088
1.46
0008
130
1000
0.04 5: 0.992
134) 11.9
LL815 BJ82
154 0:21
WB17 1 2-01812
2.22
0106 2 00002
1.M1 10.5
03877
1.50
0322
2001
0.252
21.30
Crw
0L0N ( 6.001
0000 0.50
126
0.0h 6/009
1L11 BAM
0053 10010
0,00
6/006
0011
4.3
0109
0.76
23.40
TAYO:
1003
0.2
120
0.35
1,0
1.00
1030
601
0856 1:000
0331
1LLS
122
12% 1129
11:3
1.4%
2.t 1.6.7
2315
211
2150
4 1000
VEW
110 0.16
100 = 2014
187 2017
1.7 0.5
221 0:42
110 0114
1,31
1.6
10.00
TWYDWW (m))
0801 : 0/902
1.22 8 1087
0 : 1980
1300 1.80
0105
2 113
0102 L 6.991
2.20 1455
M603
246 1.00
W823 1 1101
5.051.0.54
0104 à
1.3 1.37
01999
2,09
01677
2.56
5080
0.488
IAVEWW (2011))
6116 0.902
160 0.87
1304 013
0001
144
300 2000
0.30
2.2
0445
233-
600
0.90
366 5.,20
2.00 0.00
0/00
2,00
0.179
2.07
50.00
0.511
Oseil
BLOSK L 6/001
0.22 L 1088
1:56 (421
0100
100.1.000 1000
1391
2.19 11:31
11000 0000
242 1.6
U434 1 01911
N.5 0.255
0102 & 00003
1.87 1131
0484
3,43
03445
3.52
0.453
50.14
TA
5.7 006
4.0 =
536
3.57 1.61
341 1.97
1,34
550 1.55
4.43
3,63
365
CWW
0:90
2.19
081
1.35
231
1.69
1.35
1.8
17.1
0.1 1173
4.76 0.6
833
2.7 1.41
357 3.11
1.0
1.57-18 1.85
1.29
0.99
155
VW
0.49 e 8443
0.40
648 642
200 0091
238 038
361 8:30
0.8 = 0.38
6.11
0.52
06.1
0.3
532 1.79
8.5
$.00
3.34
3.11
1615
VCY
6.30 e 2:65
1.49 333
1:0 178
7.30 = 2.95
43142.1
2.50
4.0 5338
4.29
2.70
283
110
Intereee time
51.78 1.18
2530 = M422
14.00 2 1001
26.88 1431
1837 DU9
6.4
1497
37.57
7.25
030
Inference times for MTP and EAM were measured on two Intel Xeon E6252 Gold (Cascade Lake) CPUs. For GM-NN, an NVIDIA GeForee RTX 3090 Ti 12GB GPU was used, while all other models were evaluated using NVIDIA A100 GPU with 80 GB of RAM.
37

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction include the presented work's main contributions, assumptions, and results. Each of these claims are backed up through theoretical and empirical results in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the presented work in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
38

---

Answer: [Yes]
Justification: We provide the proof of the equivariance and traceless property of MPNNS
based on irreducible Cartesian tensors in Appendices C Cnd D
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe the entire experimental setting and all details on the performed experimens in Section 3 and Appendix E. We also provide detailed description of MPNNs
employed in this work, including the weight initialization; see Section 4.2 and Appendix (B.2 Furthermore, we describe the construction of irreducible Cartesian tensor products and introduce the irreducible Cartesian tensor products in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
39

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The source code is available on GitHub and can be accessed via
this link: https: /ttpas://ghhub.b. com//ec--reeeerrch/iccee
All data used in this
study are publicly available: rMD17 (https://doi.org wtt.://r//...../////.. 6084/00..00mm 60.4//9...ggaaaa 12672038. v3), MD22 (http://www www.sggmml..ggg.. 3BPA (bttps://github. com/ davkovacs//BOTNet http://cssBBoeeee-tttttttttt acetylacetone
com dawkovacs htt.://aoo.........14/////aa-3a0
BOTNet-datasets, and C--
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.cor
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ptapss.cc/publicguuidees http://ggggieeeeeeeeuuuuuiiiiiii CCt///oooeeeeeseeeeeeeeeeee ssinnolicc for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The details the experimental setting provided in Section 5 and Ap- pendix
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We run experiments using several random seeds and provide the corre- sponding standard deviations and error bars in all tables and figures.
40

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details on the computer resources in Section and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: ttatpps:/neeuriee nwwrips....../. htcpppblic/tthicccuueeeeell.eeed
Answer: [Yes]
Justification: The presented work follows the NeurlPS Code of Ethics. We also include Appendix [F] which discusses the broader social impact.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
41

---

Answer: [Yes]
Justification:
We include Appendix which discusses the broader social impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper does not release any data or models with a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets g.g code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We reference all employed data sets and source codes in the main text.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
42

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pataaets..aperesttt paperswi thcode. com/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: We provide detailed description of the construction of irreducible Cartesian tensors, computation their irreducible tensor products, and the resulting equivariant paasaggg layers; see Section 4 and Section B We also provide training details in Appendix E.2 and describe the employed data sets in greater detail in Appendix E.1 The source code includes comprehensive documentation as well.
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: We performed neither crowdsourcing nor research with human subjects.
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We performed neither crowdsourcing nor research with human subjects.
Guidelines:
43

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
44


Reviews:
[{"review_id": "pRQE0qr0C1", "reviewer": "Reviewer_69Cm", "summary": "In this work, the authors proposed higher-rank irreducible Cartesian Tensor Product, and explored its usage in equivariant neural networks design in scientific applications such as molecular modeling. The authors firstly prove that irreducible Cartesian Tensor Product is equivariant to O(3) group, and further show that higher-rank (e.g., > 2) operations can be used in an efficient way for models using many-body interactions. Experiments are conducted to demonstrate the effectiveness of the proposed approach.", "questions": "1. Could you detailedly explain the difference in computational complexity between iCTP and spherical Tensor Products on both two-body and many-body interactions?\n\n2. Could you comprehensively compare this work with TensorNet and compare the strengths and weaknesses of them?", "limitations": "The authors have discussed the limitations.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The problem this work aims to tackle is of great significance in real-world scientific applications. \n2. The proposed approach is interesting and can potentially improve a new class of equivariant neural networks for crucial tasks.\n3. The paper is easy to follow.", "weaknesses": "1. **The motivation of this work needs to be better explained and presented**. As stated in the Introduction and Related Works, the major disadvantage of spherical tensors is computationally demanding, which motivates the development of Cartesian-Tensor-Product-based approaches. However, the authors do not well discuss the disadvantages of existing Cartesian-Tensor-Product-based approaches (e.g., inefficiency in scaling tensor ranks up), which is necessary as a solid support for the motivation of this work. Besides, the lack of comprehensive discussion and comparisons between this work and existing Cartesian-Tensor-Product-based approaches makes the actual value of this work doubtful for readers who are not familiar with the context.\n\n2. **Experimental results are weak in demonstrating the superiority of the proposed approach**:\n    - Lack of detailed efficiency comparisons: in this work, the authors demonstrate that the proposed irreducible Cartesian Tensor Product can be used for both two-body and many-body feature interaction or equivariant convolutions with better theoretically-proved efficiency. However, there are no comprehensive comparisons covering different operations and also different rank L.\n    - Lack of large-scale experiments: all datasets used in this work (rMD17, 3BPA and Acetylacetone) have limited scales of molecular systems and number of samples. Since the proposed approach is claimed to bring benefits in efficiency, it would be necessary to verify it on larger-scale datasets such as OC20/22.\n    - Lack of experiments on applying iCTP for two-body operations only: MACE is mainly used to compare iCTP and spherical tensor products. However, two-body operations like equivariant feature interaction/equivariant convolution are also widely used in equivariant networks. It would be better to further verify the effectiveness of iCTP on these operations only to demonstrate its generality.\n\nOverall, it is of great significance to design more powerful and efficient equivariant networks for real-world applications. However, several issues exist in the current submission. My recommendation is Borderline Accept, and I will carefully read the rebuttal and other reviews to decide whether to decrease or increase my scores.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback and comments. We have addressed each of their points below and included all new results in Tabs. 1–3 and Fig. 1 of the attached PDF.\n\n**W1:** We agree that discussing these points would improve our work. Indeed, our approach includes TensorNet and CACE as special cases. A TensorNet-like architecture could be defined with $\\nu=2$ and $L = l_\\text{max} = 2$, though with equivariant convolution filters; see Tab. 3. For the computational complexity, see W2.1 and Q1.\n\nUnlike TensorNet and CACE, which use invariant radial (both) and many-body (CACE) filters, we propose equivariant convolution filters based on irreducible Cartesian tensors. Additionally, while TensorNet and CACE embed atomic environments using reducible tensors, we use exclusively irreducible ones and ensure that irreducible representations are not mixed during message-passing. TensorNet decomposes reducible rank-2 tensors before computing messages and builds explicit 3-body features. In contrast, our approach introduces the product basis using irreducible Cartesian tensor products and systematically constructs equivariant many-body messages. The limited message-passing mechanisms in TensorNet and CACE restrict their architectures and expressive power. Our approach enables the systematic construction of O(3) equivariant models and enhances their expressive power.\n\n**W2.1:** We agree that an ablation study on $L$ and $\\nu$ would improve our work. Tab. 1 and Fig. 1 show the inference time and memory consumption for varying ranks $L$ (messages) and $l_\\text{max}$ (tensors embedding environments); we also vary the number of contracted tensors $\\nu$. Indeed, ICTP outperforms MACE for most parameter values. Particularly, ICTP allows spanning the $\\nu$-space more efficiently and, thus, improves models' expressive power for tasks requiring correlations of higher body orders. These correlations become more important when, e.g., environments are degenerate with respect to lower body orders, and higher accuracy is required [B, G].\n\nICTP is also more computationally efficient if $\\nu = 3$ and $L \\leq 4$. Note that $L \\leq 4$ is sufficient for most applications in physics; see, e.g., [A]. For environments with $L$-fold symmetries, however, rank-$L$ tensors may be required [B]. These symmetries are typically lifted in atomistic simulations. Fig. 1 shows that for $L > 4$, Cartesian models will also be advantageous if $\\nu > 4$. These results agree with our complexity analysis in Q1.\n\n**W2.2:** We added a large-scale data set that aims to assess the model's performance on a varying number of atom types/components and relaxed (0 K) as well as high-temperature structures. The Ta-V-Cr-W data set is diverse and includes 0 K energies, forces, and stresses for 2-, 3-, and 4-component systems and 2500 K properties in 4-component disordered alloys [C]. It contains 6711 configurations with sizes ranging from 2 to 432 atoms in the periodic cell. We run a hyperparameter search for both models to obtain suitable relative weights for energy, force, and virial losses. No configuration for MACE provides competitive accuracy for energies and forces simultaneously. Tab. 3 shows that MACE at most matches the accuracy of ICTP on forces but is typically outperformed by a factor of $\\leq$ 2.0 on energies. \n\nWe decided not to use OC20/22 as it does not allow systematic evaluation of models' performance across different crystal structures, temperatures, and stoichiometries, which can facilitate further method development.\n\n**W2.3:** Tab. 2 shows ICTP/MACE results with $\\nu=1$, i.e., using only two-body interactions. ICTP has accuracy comparable to or better than that of MACE and inference times smaller by 1.13.\n\n**Q1:** For irreducible Cartesian tensors, the computational complexity of a tensor product is $\\mathcal{O}(9^{L} L!/(2^{L/2}(L/2)!))$; see Section B3. Thus, for two-body interactions, we get $\\mathcal{O}(E N_\\text{ch} 9^{L} L!/(2^{L/2}(L/2)!))$ ($E$: number of edges; $L$: tensor rank; $N_\\text{ch}$: number of features). The Clebsch-Gordan (CG) tensor product in spherical models has the complexity of $\\mathcal{O}(L^5)$. Thus, for an equivariant convolution, we get $\\mathcal{O}(EN_\\text{ch}L^5)$. For many-body interactions, we obtain $\\mathcal{O}(M N_\\text{ch} K (9^{L} L!/(2^{L/2}(L/2)!))^{\\nu-1})$ and $\\mathcal{O}(M N_\\text{ch} K L^{5(\\nu-1)})$ for ICTP and MACE, respectively ($M$: number of nodes; $\\nu$: number of contracted tensors; $K = \\text{len}(\\eta_\\nu)$: all possible $(\\nu-1)$-fold tensor contractions). Spherical models can use generalized CG coefficients, resulting in $\\mathcal{O}(M N_\\text{ch} K L^{\\frac{1}{2}\\nu(\\nu + 3)})$. The factor $K$ is removed in MACE by restricting the parameterization to uncoupled features, i.e., we have $\\mathcal{O}(M N_\\text{ch} L^{\\frac{1}{2}\\nu(\\nu + 3)})$. However, this choice of the product basis makes MACE more computationally efficient than ICTP only for large $N_\\text{ch}$ and small $\\nu$; see also W2.1. Tab. 2 in the manuscript also shows that leveraging the symmetry of tensor products and coupled features improves the computational efficiency of ICTP.\n\n**Q2:** ICTP includes TensorNet as a special case with $\\nu=2$ and $L = l_\\text{max} = 2$, though with equivariant convolution filters; see also W1 and Tab. 3. TensorNet uses reducible Cartesian tensors to embed atomic environments and decomposes them into irreducible ones before computing products with invariant radial filters. It includes explicit three-body features since it computes a matrix-matrix product between node features and messages. Our approach uses exclusively irreducible Cartesian tensors for embedding environments, equivariant convolutions, and product basis. Thus, we do not mix irreducible representations during message passing. We use irreducible tensor products for the equivariant convolution and go beyond invariant filters. Finally, we systematically construct equivariant many-body messages.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EkhZhdlyBl", "reviewer": "Reviewer_qF29", "summary": "This paper introduces the use of higher-rank irreducible Cartesian tensors as an alternative to spherical tensors for equivariant message passing in machine learning interatomic potentials. The authors illustrate clearly on how to construct these tensors and their products, prove equivariance properties, and evaluate the approach empirically on several molecular datasets.", "questions": "* What are the core differences between this method and TensorNet? A clearer comparison would help position this work in the context of existing literature.\n* Is the proposed model compatible with Hamiltonian prediction? This could be an interesting avenue for future work.\n* Can the authors provide plots showing how their model scales with increasing L? \n* The paper mentions \"transferability\" in line 225. Could the authors clarify what they mean by this term in this context?\n* The claim that Cartesian tensors are advantageous to spherical tensors requires further explanation. From a representation power perspective, aren't they equivalent? Is it possible that the observed performance gains are due to hyperparameter tuning rather than fundamental differences in representation power?", "limitations": "See above.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The mathematical foundations are clearly illustrated, with detailed explanations of how to construct irreducible Cartesian tensors and compute their products.\n* The experiments on out-of-domain extrapolation, particularly on the 3BPA and acetylacetone datasets, provide valuable insights into the generalization capabilities of the proposed method.\n* The paper demonstrates that irreducible Cartesian tensor-based models can achieve comparable or sometimes better performance than state-of-the-art spherical tensor models.", "weaknesses": "* The empirical evaluation is limited to relatively simple molecular datasets. The paper would be strengthened by including experiments on more challenging datasets such as MD22 or heterogeneous datasets like QM9.\n\n*  The efficiency gain and the performance gain is not that appealing to my eye. It seems little more than “instead of using that math, you can use this math!” without a very strong theoretical justification for why to do so. The author could have done a better job of explaining what is the fundamental difference/advantage of the proposed cartesian tensors when compared with the sphereical tensors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive assessment of the manuscript. We have addressed each point they raised below. All numerical results for the performed experiments are presented in Tabs. 1–3 and Fig. 1 of the attached PDF.\n\n**W1:** We added a large-scale data set that aims to assess the model's performance on a varying number of atom types/components and relaxed (0 K) as well as high-temperature structures. The Ta-V-Cr-W data set is diverse and includes 0 K energies, forces, and stresses for 2-, 3-, and 4-component systems and 2500 K properties in 4-component disordered alloys [C]. It contains 6711 configurations with sizes ranging from 2 to 432 atoms in the periodic cell. We were experimenting with this data set before the rebuttal and performed a hyperparameter search for both models to obtain suitable relative weights for energy, force, and virial losses. No configuration for MACE provides competitive accuracy for energies and forces simultaneously. Tab. 3 shows that MACE at most matches the accuracy of ICTP on forces but is typically outperformed by a factor of $\\leq$ 2.0 on energies.\n\nWe decided not to use MD22 and QM9 since they do not include variations in atom types or MD trajectories, respectively.\n\n**W2:** We agree that we could better motivate our approach regarding the computational advantages compared to spherical tensors. Considering results from Q3 and computational complexities (Q1 by the Reviewer 69Cm) for equivariant convolutions ($\\mathcal{O}(E N_\\text{ch} 9^{L} L!/(2^{L/2}(L/2)!))$ and $\\mathcal{O}(EN_\\text{ch}L^5)$ for ICTP and MACE, respectively) and the product basis ($\\mathcal{O}(M N_\\text{ch} K (9^{L} L!/(2^{L/2}(L/2)!))^{\\nu-1})$ and $\\mathcal{O}(M N_\\text{ch} L^{\\frac{1}{2}\\nu(\\nu + 3)})$, respectively), employing irreducible Cartesian tensors offers more than simply replacing mathematical expressions. Indeed, ICTP is more efficient in covering larger $\\nu$ (and larger $L$ if $\\nu$ is large) than MACE; see also Q3. \n\nWe expect our approach to inspire the development of computationally efficient models and frameworks using strategies other than those employed for spherical ones. We also demonstrate improved efficiency by leveraging the symmetries of tensor products and coupled product features; see Tab. 2 in the manuscript. Apart from the above, symmetric tensors allow for more efficient implementations and algorithms for the general matrix-matrix multiplications (GEMM), which PyTorch has not yet provided. Finally, our approach can exploit the symmetry of tensors when computing forces and stresses, omitting the transpose calculation.\n\n**Q1:** Our approach includes TensorNet as a special case with $\\nu=2$ and $L = l_\\text{max} = 2$, though with equivariant convolution filters; for results, see Tab. 3. Please see also W1 and Q2 by the Reviewer 69Cm. TensorNet uses reducible Cartesian tensors to embed atomic environments and decomposes them into irreducible ones before computing products with invariant radial filters. It includes explicit three-body features since it computes a matrix-matrix product between node features and messages. Our approach uses exclusively irreducible Cartesian tensors for embedding atomic environments, equivariant convolutions, and product basis. Thus, we do not mix irreducible representations during message passing. We use irreducible tensor products for the equivariant convolution and go beyond invariant filters. Finally, we systematically construct equivariant many-body messages.\n\n**Q2:** We see no hurdle to applying our approach to $N$-center properties such as single-particle Hamiltonian matrices; see [D]. Our approach also includes all operations required for [E] and [F].\n\n**Q3:** We agree that an ablation study on $L$ and $\\nu$ would improve our work; see also W2.1 and Q1 by the Reviewer 69Cm. Tab. 1 and Fig. 1 show the inference time and memory consumption for varying ranks $L$ (messages) and $l_\\text{max}$ (tensors embedding environments); we also vary the number of contracted tensors $\\nu$. Indeed, ICTP outperforms MACE for most parameter values. Particularly, ICTP allows spanning the $\\nu$-space more efficiently and, thus, improves models' expressive power for tasks requiring correlations of higher body orders. These correlations become more important when, e.g., environments are degenerate with respect to lower body orders, and higher accuracy is required [B, G].\n\nICTP is also more computationally efficient if $\\nu = 3$ and $L \\leq 4$. Note that $L \\leq 4$ is sufficient for most applications in physics; see, e.g., [A]. For neighborhood orientations with $L$-fold symmetries, however, at least rank-$L$ tensors may be required [B]. These symmetries are typically lifted in atomistic simulations. Fig. 1 shows that for $L > 4$, Cartesian models will also be advantageous if $\\nu > 4$. These results agree with our complexity analysis in Q1 by the Reviewer 69Cm.\n\n**Q4:** When we mention an interatomic potential's transferability, we refer to its ability to accurately predict energies, forces, and stresses for crystal structures, temperatures, and stoichiometries on which it was not trained; see [77] in the manuscript.\n\n**Q5:** We agree that spherical and irreducible Cartesian tensors (also reducible ones) should have comparable expressive power for a fixed $L$ since both tensors are related through a linear transformation; see [40, 49, 59-61] in the manuscript. However, our results in Q3 demonstrate that MACE scales worse than ICTP when increasing $\\nu$, i.e., when increasing the expressive power of the model. Particularly, for MACE with $L=3$, we could raise $\\nu$ to a maximal value of 4 since, for larger values, we obtained an OOM error on an NVIDIA A100 GPU with 80GB. We also conducted a careful hyperparameter tuning for MACE and ICTP to ensure a fair comparison, expecting similar energy and force errors due to the comparable expressive power of spherical and Cartesian tensors.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9vPw0RxI7D", "reviewer": "Reviewer_Prcd", "summary": "This work builds upon advances on equivariant and many-body architectures for the construction of neural network potentials. It lays out the formalism to substitute the conventionally-used spherical tensors in higher-rank models for Cartesian tensors. Taking as a reference the MACE architecture, the authors intend to show that this is a competitive approach to SOTA models in terms of accuracy and computational efficiency.", "questions": "1) It seems that the first model to explore the idea of higher-rank Cartesian tensors was TensorNet [30], even though it is not flexible to incorporate arbitrary ranks, and it does not explicitly account for many-body interactions. I miss some more thorough discussion on their differences, even more taking into account that [30] seems to display competitive performance to ICTP without making use of those more sophisticated approaches. I would encourage the authors to include some discussion in this regard. Is ICTP a combination of CACE and TensorNet?\n\n2) I do not intend the authors to address the following question with more experiments, I acknowledge the limited time frame, but: experiments have been conducted on datasets consisting of single systems. Do the authors have any reference of how the model performs on datasets with varying chemical composition?", "limitations": "The authors have adequately addressed the limitations", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The exploration of new methods to more efficiently learn machine learning force fields is an active area of research, and the use of higher-rank Cartesian approaches is quite novel, in contrast to the use of the spherical basis. The authors demonstrate that they can obtain results on benchmarks datasets that compete with SOTA models (both spherical and higher-rank Cartesian). The exposition of mathematical concepts is quite clear for a reader familiar with the literature. In terms of accuracy, the model is very satisfactory.", "weaknesses": "Although the achievement of competitive performance when compared to SOTA models is relevant enough, and the mathematical machinery is novel for the neural network potential field, I am not sure whether the authors have been able to demonstrate in some way why their method should be chosen in contrast to MACE, for example. The comparison of inference times seems to not favor the use of ICTP. However, I am aware of the fact that the formalism laid out in the paper allows the construction of other architectures, and that the design space of these models could be further investigated to find even more efficient models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive assessment of the manuscript and have addressed each point they raised below. All numerical results for the experiments conducted for this review are presented in Tabs. 1, 2, and 3 of the attached PDF. We also include Fig. 1, illustrating inference time and memory consumption of ICTP and MACE as a function of $L$ and $\\nu$.\n\n**W1:** To demonstrate the advantage of a Cartesian approach we can consider computational complexities for equivariant convolutions ($\\mathcal{O}(E N_\\text{ch} 9^{L} L!/(2^{L/2}(L/2)!))$ and $\\mathcal{O}(EN_\\text{ch}L^5)$ for ICTP and MACE, respectively) and the product basis ($\\mathcal{O}(M N_\\text{ch} K (9^{L} L!/(2^{L/2}(L/2)!))^{\\nu-1})$ and $\\mathcal{O}(M N_\\text{ch} L^{\\frac{1}{2}\\nu(\\nu + 3)})$, respectively). For more details on the asymptotic computational complexity, see W2.1 and Q1 by the Reviewer 69Cm and Q3 by the Reviewer qF29. Furthermore, Tab. 1 and Fig. 1 assess the inference time and memory consumption varying rank of messages $L$ and tensors embedding atomic environments $l_\\text{max}$; we also vary the number of contracted tensors $\\nu$. Our results in W2.1 by the Reviewer 69Cm and Q3 by the Reviewer qF29 demonstrate that MACE scales worse than ICTP when increasing $\\nu$. Particularly, for MACE with $L=3$, we could raise $\\nu$ to a maximal value of 4 since, for larger values, we obtained an OOM error on an NVIDIA A100 GPU with 80GB. Spanning the $\\nu$-space more efficiently may be important to improve the model's expressive power for tasks requiring correlations of higher body orders. These correlations become more important when, e.g., environments are degenerate with respect to lower body orders, and higher accuracy is required [B, G].\n\nApart from the theoretical complexity and obtained inference times, as seen from an implementation perspective, symmetric tensors allow for more efficient implementations and algorithms for the general matrix-matrix multiplications (GEMM), which PyTorch has not yet provided. Finally, our approach can exploit the symmetry of tensors when computing forces and stresses, omitting the transpose calculation.\n\n**Q1:** Our approach includes TensorNet and CACE as special cases. A TensorNet-like architecture could be defined with $\\nu=2$ and $L = l_\\text{max} = 2$, though with equivariant convolution filters. We provide the corresponding results in Tab. 3. We found model configurations with $\\nu=3$ and $L=2$ (and $L=1$) outperform the model configuration with $\\nu=2$ and $L = l_\\text{max} = 2$ by factors of $\\leq$ 1.4 and $\\leq$ 1.2 in energy and force RMSEs, respectively. Tab. 2 in the manuscript demonstrates a better accuracy for ICTP by $\\leq$ 2.3 compared to CACE. We will add this discussion to the revised manuscript.\n\nMore specifically, TensorNet uses rank-2 reducible Cartesian tensors to embed atomic environments and decomposes them into irreducible ones before computing products with invariant radial filters, i.e., before computing messages. It includes explicit 3-body features in a message-passing layer since it computes a matrix-matrix product between node features and messages. CACE uses reducible higher-rank Cartesian tensors to embed local atomic environments and their full tensor contractions (see also MTP or GM-NN) to build invariant many-body filters. Our approach uses exclusively irreducible Cartesian tensors for embedding environments, equivariant convolutions, and product basis. Thus, we do not mix irreducible representations during our many-body message passing. We use irreducible tensor products for the equivariant convolution and go beyond invariant filters. Finally, we systematically construct equivariant many-body messages.\n\n**Q2:** We added results for ICTP and MACE for the large-scale Ta-V-Cr-W data set [C], which is diverse and includes 0 K energies, forces, and stresses for 2-, 3-, and 4-component systems and 2500 K properties in 4-component disordered alloys [C]. It contains 6711 configurations with sizes ranging from 2 to 432 atoms in the periodic cell. We were experimenting with this data set before the rebuttal and performed a hyperparameter search for both models to obtain suitable relative weights for energy, force, and virial losses. No configuration for MACE provides competitive accuracy for energies and forces simultaneously. Tab. 3 shows that MACE at most matches the accuracy of ICTP on forces but is typically outperformed by a factor of $\\leq$ 2.0 on energies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We thank you for your time and effort in evaluating the manuscript and providing positive feedback and constructive suggestions. Below, we provide a general response to your comments, with more details available in the individual discussions. We will revise the manuscript accordingly, incorporating all results produced during this review process.

All reviewers shared similar concerns:

* We improved the motivation for our work by discussing its differences from and advantages over recent Cartesian models, like TensorNet and CACE. We clarified that we are using irreducible Cartesian tensors for all parts of our message-passing layers, ensuring that irreducible representations are not mixed. Additionally, we define equivariant convolution filters that extend beyond the invariant ones used by recent work. We construct equivariant many-body messages using the product basis. Thus, our work enables a systematic construction of message-passing architectures that are equivariant under the action of $O(3)$ using irreducible Cartesian tensors, enhances the expressivity of resulting models, and captures TensorNet/CACE as special cases. We support our discussion by evaluating the TensorNet-like model on the Ta-V-Cr-W data set, which we included for this review.

* We extended our complexity analysis to demonstrate further the advantages of models based on irreducible Cartesian tensors over their spherical counterparts. For equivariant convolutions (two-body interactions), we obtain $\mathcal{O}(E N_\text{ch} 9^{L} L!/(2^{L/2}(L/2)!))$ and $\mathcal{O}(EN_\text{ch}L^5)$ for ICTP and MACE, respectively. For the product basis (many-body interactions), we obtain $\mathcal{O}(M N_\text{ch} K (9^{L} L!/(2^{L/2}(L/2)!))^{\nu-1})$ and $\mathcal{O}(M N_\text{ch} L^{\frac{1}{2}\nu(\nu + 3)})$ for ICTP and MACE, respectively. Here, $M$ is the number of nodes, $E$ is the number of edges, $N_\text{ch}$ is the number of feature channels, $L$ is the maximal tensor rank, $\nu$ is the number of contracted tensors, and $K = \text{len}(\eta_\nu)$ counts all possible $(\nu-1)$-fold tensor contractions. The obtained complexities show that ICTP gets more computationally efficient than MACE as $\nu$ increases. Larger values of $\nu$ become more important when, e.g., local atomic environments are degenerate with respect to lower body orders, and higher accuracy is required [B, G]. We support our complexity analysis by performing an ablation study on the hyper-parameter $L$ and $\nu$ using the 3BPA data set.

* We included the large-scale Ta-V-Cr-W data set in our analysis of the models' performance [C]. This data set includes 0 K energies, atomic forces, and stresses for binaries (i.e., two different atom types), ternaries (i.e., three different atom types), and quaternary (i.e., four different atom types) and near-melting temperature properties in four-component (i.e., four different atom types) disordered alloys. In total, this benchmark data set contains 6711 configurations that are computed with density functional theory (DFT). More precisely, there are 5680 0 K structures: 4491 binary, 595 ternary, and 594 quaternary structures, along with 1031 structures sampled from molecular dynamics (MD) at 2500 K. Structure sizes range from 2 to 432 atoms in the periodic cell. All models are trained using 5373 configurations (4873 are used for training and 500—for early stopping), while the remaining configurations are reserved for testing the models' performance. The performance is tested separately using 0 K binaries, ternaries, quaternaries, and near-melting temperature four-component disordered alloys. ICTP systematically provides energies and forces better than the current state-of-the-art (MTP/GM-NN) for the Ta-V-Cr-W data set. In contrast, for MACE, we were not able to identify a set of relative weights for energy, forces, and virial losses that consistently yield competitive results for both energies and forces.

All numerical results for the experiments conducted for this review are presented in Tabs. 1, 2, and 3 of the attached PDF. We also include Fig. 1, illustrating inference time and memory consumption of ICTP and MACE as a function of $L$ and $\nu$.

We look forward to your feedback and await your opinions on our responses.

Yours Sincerely,

The authors.

&nbsp; &nbsp; &nbsp;

**References**

[A] I. Grega, I. Batatia, G. Csányi *et al.*: Energy-conserving equivariant GNN for elasticity of lattice architected metamaterials. *Int. Conf. Learn. Represent.* [https://arxiv.org/abs/2401.16914](https://arxiv.org/abs/2401.16914) (2024)

[B] C. K. Joshi, C. Bodnar, S. V. Mathis *et al.*: On the Expressive Power of Geometric Graph Neural Networks. *Int. Conf. Learn. Represent.* [https://arxiv.org/abs/2301.09308](https://arxiv.org/abs/2301.09308) (2023)

[C] K. Gubaev, V. Zaverkin, P. Srinivasan *et al.*: Performance of two complementary machine-learned potentials in modelling chemically complex systems. *npj Comput. Mater.* **9**, 129 (2023)

[D] J. Nigam, M. J. Willatt, and M. Ceriotti: Equivariant representations for molecular Hamiltonians and $N$-center atomic-scale properties. *J. Chem. Phys.* **156**, 014115 (2022)

[E] O. T. Unke, M. Bogojeski, M. Gastegger *et al.*: SE(3)-equivariant prediction of molecular wavefunctions and electronic densities. *Adv. Neural Inf. Process. Syst.* [https://arxiv.org/abs/2106.02347](https://arxiv.org/abs/2106.02347) (2021)

[F] I. Batatia, L. L. Schaaf, G. Csányi *et al.*: Equivariant Matrix Function Neural Networks. *Int. Conf. Learn. Represent.* [https://arxiv.org/abs/2310.10434](https://arxiv.org/abs/2310.10434) (2024)

[G] S. N. Pozdnyakov, M. J. Willatt, A. P. Bartók *et al.*: Incompleteness of Atomic Structure Representations. *Phys. Rev. Lett.* **125**, 166001 (2020)

Meta-review: This paper suggests SO(3) equivariant networks which are based on irreducible representations of tensor representations, rather than the more standard spherical representations. This is a novel and refreshing approach, though ultimately it is equivalent, in a sense, to the spherical approach. Initially, the reviewers were not convinced that this approach yielded significant performance advantage over spherical approaches such as MACE. These concerns were some what alleviated by later efficiency comparison in the rebuttal, both theoretical and empirical, that show the proposed approach has better complexity when several tensor products are computed simultaneously. 

Ultimately, this is a refreshing new perspective to an important problem, and I recommend acceptance.
Decision: Accept (poster)