Title: Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing
Venue: NeurIPS 2024
Accepted: yes
Keywords: equivariance

graph neural networks

interatomic potentials

irreducible Cartesian tensors

many-body interactions

molecules

materials
Abstract: The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as equivariance to group actions on an atomic system, e.g., equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message passing. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. Cartesian tensors offer a promising alternative, though state-of-the-art methods lack flexibility in message-passing mechanisms, restricting their architectures and expressive power. This work explores higher-rank irreducible Cartesian tensors to address these limitations. We integrate irreducible Cartesian tensor products into message-passing neural networks and prove the equivariance and traceless property of the resulting layers. Through empirical evaluations on various benchmark data sets, we consistently observe on-par or better performance than that of state-of-the-art spherical and Cartesian models.

Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing

Abstract

The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as equivariance to group actions on an atomic system, e.g., equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message passing. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. Cartesian tensors offer a promising alternative, though state-of-the-art methods lack flexibility in message-passing mechanisms, restricting their architectures and expressive power. This work explores higher-rank irreducible Cartesian tensors to address these limitations. We integrate irreducible Cartesian tensor products into message-passing neural networks and prove the equivariance and traceless property of the resulting layers. Through empirical evaluations on various benchmark data sets, we consistently observe on-par or better performance than that of state-of-the-art spherical and Cartesian models.

1 Introduction

The ability to perform sufficiently fast and accurate atomistic simulations for large molecular and material systems holds the potential to revolutionize molecular and materials science [1-8]. Conventionally, computational chemistry and materials science rely on ab initio or first-principles approaches-e.g., coupled cluster [9-11] or density functional theory (DFT) [12, 13], respectively-that are accurate but computationally demanding, thus limiting the accessible simulation time and system sizes. However, the ability to generate high-quality, first-principles-based data sets has prompted the development of machine-learned interatomic potentials (MLIPs). These potentials enable atomistic simulations with accuracy that is on par with the reference first-principles method but at a fraction of the computational cost. Message-passing neural networks (MPNNs) [14-21] have been employed in chemical and materials sciences, including the development of MLIPs, due to their efficient processing of the graph representation of the atomic system [22-26]. Achieving the desired MLIPs' performance, however, requires the inclusion of inductive biases, like the invariance of total energy to translations, reflections, and rotations in the three-dimensional space, and an effective encoding of the atomic system into a learnable representation [27]. Designing equivariant MPNNs [25, 26, 28-36], which preserve the directional information of the local atomic environment, has been one of the most active research directions of the last years. Previous work often achieves equivariance by representing an atomic

system as a graph, with node features expressed in spherical harmonics basis or using lower-rank Cartesian tensors and designing ad-hoc message-passing layers [25, 28-37].

MLIPs based on spherical harmonics [25, 33, 34], which are the basis for irreducible representations of the three-dimensional rotation group, often demonstrate a better performance compared to those that use lower-rank Cartesian representations (scalars and vectors) [28, 37]. Spherical tensors, however, require the definition of a particular rotational axis, often chosen as the z -axis, resulting in inherent bias [38]. Furthermore, coupling spherical tensors via tensor products, involved in designing equivariant convolutions and constructing many-body features [33, 39, 40], requires the definition of complicated numerical coefficients, e.g., Wigner 3 -j symbols defined in terms of the Clebsch-Gordan coefficients [41], and can be computationally demanding. In contrast, irreducible Cartesian tensors have no preferential directions; their tensor products are simpler and have a better computational complexity-up to a certain tensor rank-than the tensor products of spherical tensors [38, 42-45]. Recent work improved results of Cartesian MPNNs by using rank-two tensors and decomposing them into irreducible representations of the three-dimensional rotation group [30], i.e., into representations of dimension 1 (trace), 3 (anti-symmetric part), and 5 (traceless symmetric part) [46]. Higher-rank reducible Cartesian tensors have also been integrated into many-body MPNNs [31]. These stateof-the-art Cartesian approaches, however, lack the flexibility of their message-passing mechanisms. They rely exclusively on convolutions with invariant filters and restrict the construction of many-body features, limiting the range of possible architectures and their expressive power.

Contributions. In this work, we address the limitations of state-of-the-art Cartesian models and demonstrate that operating with irreducible Cartesian tensors leads to on-par or, sometimes, even better performance than that of spherical counterparts. Particularly, this work goes beyond scalars, vectors, and rank-two tensors and explores the integration of higher-rank irreducible Cartesian tensors and their products into equivariant MPNNs: i) We demonstrate how irreducible Cartesian tensors that are symmetric and traceless can be constructed from a unit vector and a product of two irreducible Cartesian tensors, essential for equivariant convolutions and constructing many-body features; ii) We prove that the resulting tensors are traceless and equivariant under the action of the three-dimensional rotation and reflection group; iii) We demonstrate that higher-rank irreducible Cartesian tensors can be used to design cost-efficient-up to a certain tensor rank-and accurate equivariant models of many-body interactions; iv) We conduct different experiments to assess the effectiveness of equivariant message passing based on irreducible Cartesian tensors, and achieve state-of-the-art performance on benchmark data sets such as rMD17 [47], MD22 [48], 3BPA [49], acetylacetone [32], and Ta-V-Cr-W [50]. We hope our contributions will offer new insights into the use of Cartesian tensors for constructing accurate and cost-efficient MLIPs and beyond.

2 Background

Group representations and equivariance. A group ( G, · ) is defined by a set of elements G and a group product · . A representation D of a group is a function from G to square matrices such that D [ g ] D [ g ′ ] = D [ g · g ′ ] , ∀ g, g ′ ∈ G . This representation defines an action of G to any vector space X (of the same dimension as the dimension of square matrices) through the matrix-vector multiplication, i.e., ( g, x ) ↦→ D X [ g ] x for ∀ g ∈ G and ∀ x ∈ X . For vector spaces X and Y , a function f : X → Y is called equivariant to the action of a group G to X and Y iff

Invariance can be seen as a special type of equivariance with D Y [ g ] being the identity for all g . An important class of equivariant models focuses on equivariance to the action of the Euclidean group E(3) , comprising translations and the orthogonal group O(3) , i.e., rotation group SO(3) and reflections, in R 3 . MLIPs based on equivariant models usually focus on equivariance to the action of the orthogonal group O(3) and are invariant to translations.

Cartesian tensors. A vector x ∈ R 3 transforms under the action of the rotation group SO(3) as x ′ = R x , i.e., each component of it transforms as ( x ) ′ i = ∑ j R ij ( x ) j . Here, R = D X [ g ] ∈ R 3 × 3 denotes the rotation matrix representation of g ∈ SO(3) . Cartesian tensors of rank n are described by 3 n numbers and generalize the concept of vectors. A rankn Cartesian tensor T can be viewed as a multidimensional array with n indices, i.e., ( T ) i 1 i 2 ··· i n with i k ∈ { 1 , 2 , 3 } for ∀ k ∈ { 1 , · · · , n } . Furthermore, each index of ( T ) i 1 i 2 ··· i n transforms independently as a vector under rotation. For example, a rank-two tensor transforms under rotation as T ′ = R T R ⊤ , i.e., each component of

it transforms as ( T ) ′ i 1 i 2 = ∑ j 1 j 2 R i 1 j 1 R i 2 j 2 ( T ) j 1 j 2 . For Cartesian tensors, one defines an r -fold tensor contraction and an outer product, i.e., ( T ) j 1 ··· j s = ∑ i 1 , ··· ,i r ( U ) i 1 ··· i r j 1 ··· j s ( S ) i 1 ··· i r and ( T ) i 1 ··· i r j 1 ··· j s = ( U ) i 1 ··· i r ( S ) j 1 ··· j s , respectively. Cartesian tensors are generally reducible and can, thus, be decomposed into smaller representations that transform independently within their linear subspaces under rotation. For example, a rank-two reducible Cartesian tensor contains representations of dimension 1 (trace), 3 (anti-symmetric part), and 5 (traceless symmetric part): ∑ i 1 ( T ) i 1 i 1 , ( T ) i 1 i 2 -( T ) i 2 i 1 , and ( T ) i 1 i 2 +( T ) i 2 i 1 -2 / 3 ∑ i 1 ( T ) i 1 i 1 , respectively. Under rotation, the traceless symmetric part remains within its irreducible subspace, with a similar behavior for other irreducible components of ( T ) i 1 i 2 . Furthermore, for a reducible Cartesian tensor of rank n , an action of the rotation group SO(3) can be represented with a 3 n × 3 n -dimensional rotation matrix. This rotation matrix is also reducible, meaning that an appropriate change of basis can block diagonalize it into smaller subsets, the irreducible representations of the rotation group SO(3) .

Spherical tensors. Spherical harmonics (or spherical tensors) Y l m are functions from the points on a sphere to complex or real numbers, with degree l ≥ 0 and components -l ≤ m ≤ l . Collecting all components for a given l we obtain a (2 l + 1) -dimensional object Y l = ( Y l -l , . . . , Y l l -1 , Y l l ) . Spherical tensors transform under rotation as Y l m ( R ˆ x ) = ∑ m ′ ( D ) l mm ′ Y l m ′ (ˆ x ) , where ( D ) l mm ′ is the irreducible representation of SO(3) -the Wigner D -matrix-and R denotes the rotation matrix. Spherical tensors are irreducible and can be combined using the Clebsch-Gordan tensor product

with Clebsch-Gordan coefficients C l 3 m 3 l 1 m 1 ,l 2 m 2 and l 3 ∈ {| l 1 -l 2 | , · · · , l 1 + l 2 } . Furthermore, any reducible Cartesian tensor of rank n can be written in spherical harmonics as (ˆ x ) i 1 (ˆ x ) i 2 · · · (ˆ x ) i n = ∑ n l =0 ∑ l m = -l X lm i 1 i 2 ··· i n Y l m , with coefficients X lm i 1 i 2 ··· i n defined elsewhere [51].

3 Related work

Equivariant message-passing potentials. Equivariant MPNNs [25, 28-37, 39, 52-54] often outperform more traditional invariant models [55-60]. While many invariant and equivariant MPNNs rely on two-body features within a single message-passing layer, there is a growing interest in constructing higher-body-order features, such as angles and dihedrals, to model many-body interactions in atomic systems [57, 59, 60]. Note that equivariant models with two-body features in a single message-passing layer build many-body ones through repeated message-passing iterations, increasing the receptive field and, thus, computational cost. Recent work recognized the importance of higher-body-order features but faced challenges due to explicit summation over triplets or quadruplets [57, 59]. In contrast, MACE advances the current state-of-the-art by combining ACE and equivariant message passing, introducing cost-efficient many-body message-passing potentials [33]. With just two message-passing layers, MACE yields accurate potentials for interacting many-body systems [61].

Beyond Clebsch-Gordan tensor product. Despite the success of equivariant MPNNs based on spherical tensors, the high computational cost of the Clebsch-Gordan tensor product limits their computational efficiency [28, 30, 33-35, 53, 54, 62-64]. Thus, current research focuses on alternatives to the Clebsch-Gordan tensor product, which has a O ( L 5 ) complexity for tensors up to degree L . Recently, the relation of Clebsch-Gordan coefficients to the integral of products of three spherical harmonics, known as the Gaunt coefficients [41], has been exploited to reduce the computational cost of the tensor product of spherical tensors to O ( L 3 ) , compared to O ( L 6 ) of the full Clebsch-Gordan tensor product including all ( l 1 , l 2 ) → l 3 [36]. However, this approach excludes odd tensor products and, thus, restricts the expressive power and the range of possible architectures.

Cartesian tensors and their products offer another promising alternative to the Clebsch-Gordan tensor product, enabling the efficient construction of message-passing layers with two- and manybody features. Recent work has explored decomposing rank-two tensors into their irreducible representations and using higher-rank reducible tensors [30, 31]. These approaches, however, are limited to convolutions with invariant filters, restricting the range of possible architectures and, thus, the expressive power of resulting Cartesian models. Furthermore, they provide limited mechanisms for constructing higher-body-order features, restricted to three-body features obtained through matrixmatrix multiplication and invariant many-body features obtained through full tensor contractions, similar to moment tensor potentials and Gaussian moments [65-67]. Finally, using reducible Cartesian

even

odd

even

(a)

(b)

tensors during message-passing leads to mixing different irreducible representations, which can hinder the performance of resulting models compared to state-of-the-art spherical models [31].

4 Methods

We define an atomic configuration S = { r u , Z u } N at u =1 , where r u ∈ R 3 denotes Cartesian coordinates and Z u ∈ N represents the atomic number of atom u , with a total of N at atoms. Our focus lies on message-passing MLIPs, parameterized by θ , that learn a mapping from a configuration S to the total energy E , i.e., f θ : S ↦→ E ∈ R . Thus, we represent molecular and material systems as graphs in a three-dimensional Euclidean space. An edge { u, v } exists if atoms u and v are within a cutoff distance r c , i.e., ∥ r u -r v ∥ 2 ≤ r c . For more details on MPNNs, see Appendix A. The total energy of an atomic configuration is defined by the sum of individual atomic energy contributions [68], i.e., E = ∑ N at u =1 E u . Atomic forces are computed as negative gradients of the total energy with respect to atomic coordinates, i.e., F u = -∇ r u E .

4.1 Irreducible Cartesian tensor product

In the following, we explore irreducible Cartesian tensors and their products based on the threedimensional vector space and the three-dimensional orthogonal group O(3) , which comprises rotations and reflections. The respective tensors and tensor products are schematically illustrated in Fig. 1 (a) and (b), respectively, and are further employed in constructing MPNNs for atomic systems equivariant under actions of the orthogonal group. An irreducible Cartesian tensor T n ∈ ( R 3 ) ⊗ n of rank n and weight l ≤ n (related to the degree l of spherical tensors) can be represented by a tensor with 3 n components in a three-dimensional vector space. These 3 n components form a basis for a (2 l +1) -dimensional irreducible representation of the three-dimensional rotation group SO(3) [69, 70]. Thus, only 2 l +1 of the 3 n components are independent.

The rotation in the space of all tensors of rank n is induced through the n -fold outer product of a rotation matrix R ∈ R 3 × 3 , i.e., R ⊗ n = R ⊗ · · · ⊗ R . The obtained representation of the

̸

rotation group SO(3) is reducible for all n except n = { 0 , 1 } . Reducing a tensor of rank n yields a unique irreducible tensor with the same weight and rank ( n = l ), which is characterized by being symmetric, i.e., ( T n ) ··· i ··· j ··· = ( T n ) ··· j ··· i ··· , ∀ i = j ∈ { i 1 · · · , i n } , and traceless, i.e., ∑ i ( T n ) ··· i ··· i ··· = 0 , ∀ i ∈ { i 1 , · · · , i n } . An irreducible tensor of rank n and weight l with l < n can be viewed as a l -rank tensor embedded in the n -rank tensor space, e.g., by computing an outer product with the identity matrix. However, the embedding is often not unique. Thus, we construct tensors with the same weight and rank ( n = l ) in the following.

Irreducible Cartesian tensors from unit vectors. Hereafter, we denote the rank and the weight of irreducible Cartesian tensors by l to distinguish them from reducible counterparts. An irreducible Cartesian tensor of an arbitrary rank l can be constructed from a unit vector ˆ r in the form [45]

resulting in a symmetric and traceless tensor of rank l . Here, I denotes the 3 × 3 identity matrix, ˆ r ⊗ ( l -2 m ) = ˆ r ⊗··· ⊗ ˆ r and I ⊗ m = I ⊗··· ⊗ I are the corresponding ( l -2 m ) - and m -fold outer products. The curly brackets indicate the summation over all permutations of the l unsymmetrized indices [45], i.e., { T l } i 1 ··· i l = ∑ π ∈ S l ( T l ) i π (1) ··· i π ( l ) , with S l being the corresponding set of permutations. The expression in Eq. (1) involves three distinct outer products (ˆ r ⊗ ˆ r ) i 1 i 2 = ˆ r i 1 ˆ r i 2 , ( I ⊗ I ) i 1 i 2 i 3 i 4 = δ i 1 i 2 δ i 3 i 4 , and (ˆ r ⊗ I ) i 1 i 2 i 3 = ˆ r i 1 δ i 2 i 3 , where δ i 2 i 3 denotes the Kronecker delta. The normalization constant C = (2 l -1)!! /l ! is chosen such that an l -fold contraction of T l with the unit vector ˆ r yields unity. An example of an irreducible Cartesian tensor with rank l = 3 is ( T l =3 ) i 1 i 2 i 3 = 5 2 ( ˆ r i 1 ˆ r i 2 ˆ r i 3 -1 5 (ˆ r i 1 δ i 2 i 3 + ˆ r i 2 δ i 3 i 1 + ˆ r i 3 δ i 1 i 2 ) ) .

Irreducible Cartesian tensor product. The following defines the product of two irreducible Cartesian tensors x l 1 ∈ ( R 3 ) ⊗ l 1 and y l 2 ∈ ( R 3 ) ⊗ l 2 , yielding an irreducible Cartesian tensor of rank l 3 , i.e., z l 3 = ( x l 1 ⊗ Cart y l 2 ) l 3 ∈ ( R 3 ) ⊗ l 3 ∀ l 3 ∈ {| l 1 -l 2 | , · · · , l 1 + l 2 } , that is symmetric and traceless. The irreducible Cartesian tensor product is crucial for designing equivariant MPNNs and is used for equivariant convolutions and constructing many-body features in Section 4.2. For an even l 1 + l 2 -l 3 = 2 k , the general form of an irreducible Cartesian tensor of rank l 3 reads [45]

where ( x l 1 · ( k + m ) · y l 2 ) = ∑ i 1 , ··· ,i k + m ( x l 1 ) i 1 ··· i k + m ( y l 2 ) i 1 ··· i k + m denotes an ( k + m ) -fold tensor contraction, which results in a tensor of rank l 1 + l 2 -2( k + m ) . For simplicity, we skip the uncontracted indices in the above definition. For example, for l 1 = 4 and l 2 = 3 and a three-fold tensor contraction we obtain ( x l 1 · 3 · y l 2 ) i 4 = ∑ i 1 ,i 2 ,i 3 ( x l 1 ) i 1 i 2 i 3 i 4 ( y l 2 ) i 1 i 2 i 3 , i.e., the corresponding tensors are contracted along i 1 , i 2 , and i 3 . Note that the final result is independent of the index permutation, as the contracted tensors are symmetric. For an odd l 1 + l 2 -l 3 = 2 k +1 , we define [45]

with ε denoting the Levi-Civita symbol ( ε i 1 i 2 i 3 = -ε i 3 i 2 i 1 and ε i 1 i 1 i 3 = 0 ). The double contraction with the Levi-Civita symbol reads ( ε : x l 1 · ( k + m ) · y l 2 ) i 1 = ∑ i 2 ,i 3 ε i 1 i 2 i 3 ( x l 1 · ( k + m ) · y l 2 ) i 2 i 3 , and yields a tensor of rank l 1 + l 2 -2( k + m ) -1 . Details on the normalization constants C l 1 l 2 l 3 and D l 1 l 2 l 3 are provided in Appendix B.1.

4.2 Equivariant message-passing based on irreducible Cartesian tensors

The following section introduces the basic operations for constructing equivariant MPNNs based on irreducible Cartesian tensors. Using their irreducible tensor products, we demonstrate how to build equivariant two- and many-body features, crucial for modeling many-body interactions in molecular and materials systems. Particularly, we focus on MLIPs based on equivariant MPNNs and extend the state-of-the-art MACE architecture [33] to the Cartesian basis. Following the MACE architecture, we use only even tensor products. We split vectors r uv = r u -r v ∈ R 3 from atom u to atom v , schematically shown in Fig. 1 (a), into their radial and angular components (unit vectors), i.e.,

r uv = ∥ r uv ∥ 2 ∈ R and ˆ r uv = r uv /r uv ∈ R 3 , respectively. In the t -th message-passing layer, edges { u, v } are embedded using a fully connected neural network R ( t ) kl 1 l 2 l 3 : R → R with k output feature channels. The radial function R ( t ) kl 1 l 2 l 3 takes as an input radial distances r uv , which are embedded through Bessel functions and multiplied by a smooth polynomial cutoff function [60]. Finally, we use irreducible Cartesian tensors T l (ˆ r ) , similar to spherical tensors Y l (ˆ r ) , to embed unit vectors into the tensor space of maximal rank l max .

Equivariant convolutions and two-body features. Rotation equivariance in MPNNs is typically achieved by constraining convolution filters to be the products between learnable radial functions and spherical tensors, i.e., R ( t ) kl 1 l 2 l 3 ( r uv ) Y l 1 m 1 (ˆ r uv ) . The two-body features A ( t ) ukl 3 m 3 are further obtained through the tensor product-the point-wise convolution [39]-between the respective filters and neighbors' equivariant features h ( t ) ukl 2 m 2 . The permutational invariance is enforced by pooling over the neighbors v ∈ N ( u ) . Here, we use irreducible Cartesian tensors, with the rotation-equivariant filters given by R ( t ) kl 1 l 2 l 3 ( r uv ) ( T l 1 (ˆ r uv ) ) i 1 i 2 ··· i l 1 . Thus, two-body features ( A ( t ) ukl 3 ) i 1 i 2 ··· i l 3 are obtained using the irreducible Cartesian tensor product and are represented by rankl 3 irreducible Cartesian tensors. The Cartesian two-body features are defined by

where d t represents the number of feature channels in the node embeddings h ( t ) vk ′ l 2 of the t -th messagepassing layer. In the first message-passing layer, node embeddings are initialized as learnable weights W kZ v that are invariant to actions of the orthogonal group, i.e., are scalars or tensors of rank l 2 = 0 , and embed the atom type Z v . Thus, constructing equivariant two-body features simplifies to

Equivariant many-body features. The importance of many-body terms arises from the fact that the interaction between atoms changes when additional atoms are present; see Appendix A. Furthermore, many-body terms are often required to ensure the generalization of interatomic potentials, i.e., their ability to accurately predict energies and forces for temperatures and stoichiometries on which they were not trained [71]. Here, we construct ( ν +1) -body equivariant features from ( A ( t ) ukl ξ ) i 1 i 2 ··· i l ξ obtained using Eqs. (4) or (5). The ν -fold Cartesian tensor product, which yields ( ν + 1) -body features represented by an irreducible Cartesian tensor of rank L , reads

where η ν counts all possible ν -fold products of { l 1 , · · · , l ν } -rank tensors, yielding rankL irreducible Cartesian tensors, and ˜ A ( t ) ukl ξ = 1 √ d t ∑ k ′ W ( t ) kk ′ l ξ A ( t ) uk ′ l ξ with d t feature channels.

The irreducible Cartesian tensor product does not allow pre-computing the coefficients of the ν -fold tensor product, differing from spherical tensors that use the generalized Clebsch-Gordan coefficients, contracted with weights from Eq. (A1) and summed over the possible paths len( η ν ) , for this purpose [33, 40, 51]. Thus, we obtain the result of Eq. (6) by iteratively applying the irreducible Cartesian tensor product ( ν -1) times and refer to the respective models as irreducible Cartesian tensor potentials (ICTPs) with the full product basis or ICTP full . However, two-fold tensor products in Eq. (6) are symmetric to permutations of involved tensors. Thus, the number of the ν -fold tensor products, len( η ν ) , can be significantly reduced; we refer to the corresponding models as ICTP sym . Furthermore, we can reduce the computational cost of the Cartesian product basis by performing the calculations in a latent feature space. We use learnable weights W kk ′ l ξ to reduce the number of feature channels for the product basis calculation and then increase it again for subsequent steps; we refer to the corresponding models as ICTP lt . For more details on the model architecture, such as the construction of updated many-body node embeddings, readout functions, and different options for the Cartesian product basis, see Appendix B.2.

Runtime considerations. When choosing an architecture to implement MLIPs, the runtime per energy and force evaluation is crucial. The computational complexity as a function of rank L is

1

2

3

L

1

.

0

10

.

0

Inference time in ms

(a)

1

2

3

L

0

.

1

1

.

0

10

.

0

GPU memory in GB

(b)

1

2

3

4

5

6

ν

1

.

0

10

.

0

Inference time in ms

(c)

1

2

3

4

5

6

ν

0

.

1

1

.

0

10

.

0

GPU memory in GB

(d)

MACE (

ν

= 1

)

ICTP (

ν

= 1

)

MACE (

ν

= 3

)

ICTP (

ν

= 3

)

MACE (

ν

= 6

)

ICTP (

ν

= 6

)

MACE (

L

= 1

)

ICTP (

L

= 1

)

MACE (

L

= 2

)

ICTP (

L

= 2

)

MACE (

L

= 3

)

ICTP (

L

= 3

)

O ( 9 L L ! / ( 2 L/ 2 ( L/ 2)! )) for the irreducible Cartesian tensor product and O ( L 5 ) for the ClebschGordan one; see Appendix B.3 for more details. Thus, equivariant convolutions based on spherical tensors are more computationally efficient for L → ∞ than those based on irreducible Cartesian tensors. However, state-of-the-art models and physical properties typically require L ≤ 4 [33, 72], making sub-leading terms and implementation-dependent amplitudes crucial. Based on our analysis, for L ≤ 4 , we can expect equivariant convolutions based on irreducible Cartesian tensors to be more computationally and memory efficient than their spherical counterparts. For L ≤ 4 , the ν -fold tensor product in the Cartesian basis can also offer computational advantages. Its cost, as a function of rank L and correlation order ν , is K (9 L L ! / (2 L/ 2 ( L/ 2)!)) ν -1 and K L 1 2 ν ( ν +3) for ICTP and MACE, respectively. The pre-factor K , which counts all possible ν -fold tensor products, is removed in MACE through generalized Clebsch-Gordan coefficients, though these coefficients increase the memory spherical models use. Therefore, it is essential to consider the inference times for a fair comparison, which we provide in Section 5.

Equivariance and tracelss property of message-passing layers. We conclude this section by giving a theoretical result that ensures the equivariance of message-passing layers based on irreducible Cartesian tensors and their irreducible tensor products to actions of the orthogonal group. The proof is provided in Appendix C. We also prove in Appendix D that these message-passing layers preserve the traceless property of irreducible Cartesian tensors.

Proposition 4.1. The message-passing layers based on irreducible Cartesian tensors and their irreducible tensor products are equivariant to actions of the orthogonal group.

Proposition 4.2. The message-passing layers based on irreducible Cartesian tensors and their irreducible tensor products preserve the traceless property of irreducible Cartesian tensors.

5 Experiments and results

This section presents the results for the five benchmark data sets: rMD17, MD22, 3BPA, acetylacetone, and Ta-V-Cr-W. We describe data sets and training details in Appendices E.1 and E.2, respectively.

Scaling and computational cost. The expressive power and computational efficiency of equivariant many-body message-passing potentials depend on the tensor ranks employed in equivariant message passing and embedding local atomic environments, i.e., L and l max , respectively, as well as the correlation order ν . Recent work has shown that for identifying environments with L -fold symmetries, at least rankL tensors are required [73]. These symmetries are typically lifted in atomistic simulations, motivating the use of L ≤ 4 . Higher body-order correlations ν , in turn, are required if atomic environments are degenerate to a lower body-order correlation ν -1 [73, 74]. Figure 2 demonstrates inference times and memory consumption of models based on irreducible Cartesian and spherical tensors, i.e., ICTP and MACE, respectively, as a function of the tensor rank and the correlation order. Table A1 presents the corresponding numerical results. We find that irreducible Cartesian tensors

outperform spherical ones for most parameter values. In particular, irreducible Cartesian tensors allow spanning the ν -space more efficiently, in line with our theoretical results in Section 4.2.

Molecular dynamics trajectories. We assess the performance of ICTP models using the revised MD17 (rMD17) data set, which includes structures, total energies, and atomic forces for ten small organic molecules obtained from ab initio molecular dynamics simulations [47]. Table 1 shows that ICTPsym achieves accuracy on par with state-of-the-art spherical and Cartesian models. Notably, several methods exhibit similar accuracy when trained with 950 configurations. However, the achieved accuracy is much lower than the desired accuracy of 43 . 37 meV ≈ 1 kcal/mol, making a model comparison less meaningful. Therefore, we also compare ICTPsym with MACE and NequIP, trained using 50 configurations, making learning accurate MLIPs more challenging. From Table 1, we see that ICTPsym outperforms MACE and NequIP for most molecules in this scenario.

We further evaluate the performance of ICTP using the MD22 data set, which contains seven molecular systems with 42-370 atoms [48]. This data set spans four major classes of biomolecules and supramolecules and was designed to challenge short-range models. Table A2 shows that ICTP achieves an accuracy on par with or better than state-of-the-art models, including long-range ones.

Extrapolation to out-of-domain data. We continue to assess the performance of ICTP models using the 3BPA data set [49]. The training data set comprises 500 configurations, total energies, and atomic forces acquired from molecular dynamics at 300 K. The test data set is obtained from simulations at 300 K, 600 K, and 1200 K. We also test our models using energies and forces along dihedral rotations of the molecule. Table 2 shows that ICTP models trained using 450 configurations perform on par with state-of-the-art spherical models, similar to the results for rMD17. However, we were not able to reproduce the original results using the current MACE source code and the described training setup [33]. Therefore, for a fair comparison, we unified the training setup for ICTP and MACE (see Appendix E.2) and Table 2 reports the corresponding results for 450 training configurations. In Table A3, we present the results obtained for ICTP and MACE trained using 50 configurations.

From Table 2, we observe that ICTPfull slightly outperforms MACE in total energy and atomic force RMSEs but is ∼ 1 . 4 times less computationally efficient. This difference arises from MACE using the generalized Clebsch-Gordan coefficients and pre-computing their product with the weights in the linear expansion in Eq. (A1) [33], which reduces the effective number of evaluated tensor products. Thus, we may attribute the lower computational efficiency of ICTP full to the use of the

ff

fi

fl

0

100

200

300

γ

in

◦

-

20

0

20

40

60

80

Rel. energy in meV

α

= 71

.

0

◦

,

β

= 120

.

0

◦

0

100

200

300

γ

in

◦

α

= 67

.

0

◦

,

β

= 150

.

0

◦

0

100

200

300

γ

in

◦

α

= 151

.

0

◦

,

β

= 180

.

0

◦

DFT

ICTPfull

ICTPsym

ICTPsym

+

lat

MACE

MACE architecture, which results in a larger pre-factor K for Cartesian models but facilitates a fair comparison between irreducible Cartesian and spherical tensors. Using the symmetric Cartesian product basis and that in the latent space, for example, we further improve the runtime of our models while maintaining accuracy on par with MACE.

Table A4 presents additional results obtained for ν = 1 , i.e., focusing on models that rely exclusively on two-body interactions. We observe that Cartesian models exhibit shorter inference times than spherical ones, with MACE and ICTPfull achieving 2 . 96 ± 0 . 06 ms and 2 . 63 ± 0 . 02 ms, respectively. Regarding memory consumption, MACE and ICTP perform similarly despite the larger number of tensor products required for the Cartesian product basis in Eq. (6). This observation can be attributed to, for example, the Clebsch-Gordan tensor product requiring the computation of intermediate tensors with (2 L +1) 2 elements, whereas irreducible Cartesian tensors contain 3 L elements.

Figure 3 compares potential energy profiles obtained with ICTP and MACE trained using 50 configurations. For the potential energy cut at β = 120 ◦ (left panel), ICTP and MACE perform similarly, except for the energy barrier at γ ≈ 143 ◦ , which ICTP tends to underestimate stronger than MACE.

a Similar to Table 2, we re-run MACE experiments using the similar training setup as for ICTP; see Appendix E.2.

For β = 150 ◦ (middle panel), however, ICTPfull and ICTPsym outperform MACE across nearly the entire range of the dihedral angle γ . For β = 180 ◦ (right panel), all models perform similarly. Figure A1 shows the corresponding potential energy profiles for models trained with 450 configurations. All models perform similarly in this scenario, with energy profiles close to the reference (DFT).

Flexibility and reactivity. We further use the acetylacetone data set to assess the ICTP models' extrapolation capabilities to higher temperatures (similar to 3BPA), bond breaking, and bond torsions [32]. Table 3 shows that ICTP models achieve state-of-the-art results while employing fewer parameters than spherical counterparts. Appendix E.3 includes additional results for the acetylacetone data set, such as total energy and atomic force RMSEs for models trained with 50 configurations and details on the potential energy profiles for hydrogen transfer and C-C bond rotation. Overall, ICTP and MACE perform similarly, demonstrating excellent generalization capability. However, when trained using 50 configurations, ICTP full is the only MLIP consistently producing the potential energy profile for hydrogen transfer close to the reference (DFT).

Multicomponent alloys. We finally evaluate the ICTP and MACE models using the Ta-V-CrWdata set, designed to assess the performance of state-of-the-art MLIPs in modeling chemically complex multicomponent systems. In this evaluation, we attempt to predict energies and forces for Ta-V-Cr-W subsystems under two scenarios: The 0 K energies and forces in binary, ternary, and quaternary systems and near-melting temperature energies and forces in 4-component disordered alloys. Table A6 shows that ICTP sym outperforms MACE in nearly all subsystems, particularly in energy prediction. ICTP achieves an overall accuracy of 1.38 ± 0.09 meV/atom for energies and 0.028 ± 0.001 eV/Å for forces, compared to 2.19 ± 0.31 meV/atom and 0.029 ± 0.001 eV/Å by MACE. However, the K pre-factor from the ν -fold tensor product results in longer inference times for ICTP than MACE, in line with the discussion for 3BPA.

6 Conclusions and limitations

This work introduces many-body equivariant MPNNs based on higher-rank irreducible Cartesian tensors, offering an alternative to spherical models and addressing the limitations of state-of-the-art Cartesian models. We assess the performance of resulting MPNNs using five benchmark data sets, such as rMD17, MD22, 3BPA, acetylacetone, and Ta-V-Cr-W. In these experiments, MPNNs based on irreducible Cartesian tensors show a lower computational cost of individual operations compared to spherical counterparts. Furthermore, we demonstrate that these Cartesian models achieve accuracy and generalization capability on par with or better than state-of-the-art spherical models while memory consumption is comparable. Our results hold across the typical range of tensor ranks used in modeling many-body interactions and relevant physical properties, i.e., L ≤ 4 .

Limitations. We emphasize our focus on introducing MPNNs based on irreducible Cartesian tensors and prove their equivariance and traceless property. We adapted the MACE architecture, which uses only even tensor products, to enable a fair comparison with state-of-the-art spherical models. Further modifications to the architecture are possible and necessary, e.g., to reduce the pre-factor arising from the Cartesian product basis, before we can fully exploit the potential of irreducible Cartesian tensors.

Data availability

All data sets used in this study are publicly available: rMD17 ( https://doi.org/10.6084/ m9.figshare.12672038.v3 ), MD22 ( http://www.sgdml.org ), 3BPA ( https://github. com/davkovacs/BOTNet-datasets ), acetylacetone ( https://github.com/davkovacs/ BOTNet-datasets ), and Ta-V-Cr-W ( https://doi.org/10.18419/darus-3516 ).

Code availability

The source code is available on GitHub and can be accessed via this link: https://github.com/ nec-research/ictp .

References

K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh: Machine learning for molecular and materials science . Nature 559 , 547-555 (2018)

J. Vamathevan, D. Clark, P. Czodrowski, I. Dunham, E. Ferran et al.: Applications of machine learning in drug discovery and development . Nat. Rev. Drug Discov. 18 , 463-477 (2019)

J. A. Keith, V . Vassilev-Galindo, B. Cheng, S. Chmiela, M. Gastegger et al.: Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems . Chem. Rev. 121 , 9816-9872 (2021)

O. T. Unke, S. Chmiela, H. E. Sauceda, M. Gastegger, I. Poltavsky et al.: Machine Learning Force Fields . Chem. Rev. 121 , 10142-10186 (2021)

N. Fedik, R. Zubatyuk, M. Kulichenko, N. Lubbers, J. S. Smith et al.: Extending machine learning beyond interatomic potentials for predicting molecular properties . Nat. Rev. Chem. 6 , 653-672 (2022)

A. Merchant, S. Batzner, S. S. Schoenholz, M. Aykol, G. Cheon et al.: Scaling deep learning for materials discovery . Nature 624 , 80-85 (2023)

D. P. Kovács, J. H. Moore, N. J. Browning, I. Batatia, J. T. Horton et al.: MACE-OFF23: Transferable Machine Learning Force Fields for Organic Molecules . https://arxiv.org/ abs/2312.15211 (2023)

I. Batatia, P. Benner, Y. Chiang, A. M. Elena, D. P. Kovács et al.: A foundation model for atomistic materials chemistry . https://arxiv.org/abs/2401.00096 (2023)

G. D. Purvis and R. J. Bartlett: A full coupled-cluster singles and doubles model: The inclusion of disconnected triples . J. Chem. Phys. 76 , 1910-1918 (1982)

T. D. Crawford and H. F. Schaefer III: An Introduction to Coupled Cluster Theory for Computational Chemists , pp. 33-136. John Wiley & Sons, Ltd (2000)

R. J. Bartlett and M. Musiał: Coupled-cluster theory in quantum chemistry . Rev. Mod. Phys. 79 , 291-352 (2007)

P. Hohenberg and W. Kohn: Inhomogeneous Electron Gas . Phys. Rev. 136 , B864-B871 (1964)

W. Kohn and L. J. Sham: Self-Consistent Equations Including Exchange and Correlation Effects . Phys. Rev. 140 , A1133-A1138 (1965)

A. Micheli: Neural network for graphs: A contextual constructive approach . IEEE Trans. Neural Netw. 20 , 498-511 (2009)

F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini: The graph neural network model . IEEE Trans. Neural Netw. 20 , 61-80 (2009)

J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl: Neural Message Passing for Quantum Chemistry . Int. Conf. Mach. Learn. 70 , 1263-1272 (2017)

W. L. Hamilton, R. Ying, and J. Leskovec: Representation learning on graphs: Methods and applications . IEEE Data Eng. Bull. 40 , 52-74 (2017)

M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst: Geometric deep learning: going beyond Euclidean data . IEEE Signal Process. Mag. 34 , 18-42 (2017)

Z. Zhang, P. Cui, and W. Zhu: Deep Learning on Graphs: A Survey . IEEE Trans. Knowl. Data Eng. 34 , 249-270 (2022)

S. Zhang, H. Tong, J. Xu, and R. Maciejewski: Graph convolutional networks: a comprehensive review . Comput. Soc. Netw. 6 , 11 (2019)

D. Bacciu, F. Errica, A. Micheli, and M. Podda: A Gentle Introduction to Deep Learning for Graphs . Neural Netw. 129 , 203-221 (2020)

C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo et al.:

An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage . https: //arxiv.org/abs/2010.09435 (2020)

J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov et al.: Highly accurate protein structure prediction with AlphaFold . Nature 596 , 583-589 (2021)

J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte et al.: Robust deep learning-based protein sequence design using ProteinMPNN . Science 378 , 49-56 (2022)

S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa et al.: E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials . Nat. Commun. 13 , 2453 (2022)

A. Duval, S. V. Mathis, C. K. Joshi, V. Schmidt, S. Miret et al.: A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems . https://arxiv.org/abs/2312.07511 (2024)

M. F. Langer, A. Goeßmann, and M. Rupp: Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning . npj Comput. Mater. 8 , 41 (2022)

K. T. Schütt, O. T. Unke, and M. Gastegger: Equivariant message passing for the prediction of tensorial properties and molecular spectra . Int. Conf. Mach. Learn. 139 , 9377-9388 (2021)

M. Haghighatlari, J. Li, X. Guan, O. Zhang, A. Das et al.: NewtonNet: a Newtonian message passing network for deep learning of interatomic potentials and forces . Digital Discovery 1 , 333-343 (2022)

G. Simeon and G. D. Fabritiis: TensorNet: Cartesian Tensor Representations for Efficient Learning of Molecular Potentials . Adv. Neural Inf. Process. Syst. 36 , 37334-37353 (2023)

B. Cheng: Cartesian atomic cluster expansion for machine learning interatomic potentials . npj Comput. Mater. 10 , 157 (2024)

I. Batatia, S. Batzner, D. P. Kovács, A. Musaelian, G. N. C. Simm et al.: The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials . https://arxiv.org/abs/2205. 06643 (2022)

I. Batatia, D. P. Kovacs, G. N. C. Simm, C. Ortner, and G. Csanyi: MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields . Adv. Neural Inf. Process. Syst. 35 , 11423-11436 (2022)

A. Musaelian, S. Batzner, A. Johansson, L. Sun, C. J. Owen et al.: Learning local equivariant representations for large-scale atomistic dynamics . Nat. Commun. 14 , 579 (2023)

S. Passaro and C. L. Zitnick: Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs . Int. Conf. Mach. Learn. 202 , 27420-27438 (2023)

S. Luo, T. Chen, and A. S. Krishnapriyan: Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products . Int. Conf. Learn. Represent. https://arxiv.org/ abs/2401.10216 (2024)

P. Thölke and G. D. Fabritiis: Equivariant Transformers for Neural Network based Molecular Potentials . Int. Conf. Learn. Represent. https://arxiv.org/abs/2202.02541 (2022)

R. F. Snider: Irreducible Cartesian Tensors . De Gruyter, Berlin, Boston (2018)

N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li et al.: Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds . https://arxiv.org/abs/ 1802.08219 (2018)

R. Drautz: Atomic cluster expansion for accurate and transferable interatomic potentials . Phys. Rev. B 99 , 014104 (2019)

E. Wigner: Group Theory and its Application to the Quantum Mechanics of Atomic Spectra , volume 5. Elsevier (2012)

J. A. R. Coope, R. F. Snider, and F. R. McCourt: Irreducible Cartesian Tensors . J. Chem. Phys. 43 , 2269-2275 (1965)

J. A. R. Coope and R. F. Snider: Irreducible Cartesian Tensors. II. General Formulation . J. Math. Phys. 11 , 1003-1017 (1970)

J. A. R. Coope: Irreducible Cartesian Tensors. III. Clebsch-Gordan Reduction . J. Math. Phys. 11 , 1591-1612 (1970)

D. R. Lehman and W. C. Parke: Angular reduction in multiparticle matrix elements . J. Math. Phys. 30 , 2797-2806 (1989)

M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. S. Cohen: 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data . Adv. Neural Inf. Process. Syst. 31 (2018)

A. S. Christensen and O. A. von Lilienfeld: On the role of gradients for machine learning of molecular energies and forces . Mach. Learn.: Sci. Technol. 1 , 045018 (2020)

S. Chmiela, V. Vassilev-Galindo, O. T. Unke, A. Kabylda, H. E. Sauceda et al.: Accurate global machine learning force fields for molecules with hundreds of atoms . Sci. Adv. 9 , eadf0873 (2023)

D. P. Kovács, C. v. d. Oord, J. Kucera, A. E. A. Allen, D. J. Cole et al.: Linear Atomic Cluster Expansion Force Fields for Organic Molecules: Beyond RMSE . J. Chem. Theory Comput. 17 , 7696-7711 (2021)

K. Gubaev, V. Zaverkin, P. Srinivasan, A. I. Duff, J. Kästner et al.: Performance of two complementary machine-learned potentials in modelling chemically complex systems . npj Comput. Mater. 9 , 129 (2023)

R. Drautz: Atomic cluster expansion of scalar, vectorial, and tensorial properties including magnetism and charge transfer . Phys. Rev. B 102 , 024104 (2020)

B. Anderson, T. S. Hy, and R. Kondor: Cormorant: Covariant Molecular Neural Networks . Adv. Neural Inf. Process. Syst. 32 , 14537-14546 (2019)

V. G. Satorras, E. Hoogeboom, and M. Welling: E(n) Equivariant Graph Neural Networks . Int. Conf. Mach. Learn. 139 , 9323-9332 (2021)

J. Brandstetter, R. Hesselink, E. van der Pol, E. J. Bekkers, and M. Welling: Geometric and Physical Quantities improve E(3) Equivariant Message Passing . Int. Conf. Learn. Represent. https://arxiv.org/abs/2110.02905 (2022)

K. Schütt, P.-J. Kindermans, H. E. Sauceda Felix, S. Chmiela, A. Tkatchenko et al.: SchNet: A continuous-filter convolutional neural network for modeling quantum interactions . Adv. Neural Inf. Process. Syst. 30 , 991-1001 (2017)

O. T. Unke and M. Meuwly: PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges . J. Chem. Theory Comput. 15 (6), 3678-3693 (2019)

Y. Liu, L. Wang, M. Liu, Y. Lin, X. Zhang et al.: Spherical Message Passing for 3D Molecular Graphs . Int. Conf. Learn. Represent. https://arxiv.org/abs/2102.05013 (2022)

J. Gasteiger, S. Giri, J. T. Margraf, and S. Günnemann: Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules . Machine Learning for Molecules Workshop, Adv. Neural Inf. Process. Syst. https://arxiv.org/abs/2011.14115 (2020)

J. Gasteiger, F. Becker, and S. Günnemann: GemNet: Universal Directional Graph Neural Networks for Molecules . Adv. Neural Inf. Process. Syst. 34 , 6790-6802 (2021)

J. Gasteiger, J. Groß, and S. Günnemann: Directional Message Passing for Molecular Graphs . Int. Conf. Learn. Represent. https://arxiv.org/abs/2003.03123 (2020)

D. P. Kovács, I. Batatia, E. S. Arany, and G. Csányi: Evaluation of the MACE force field architecture: From medicinal chemistry to materials science . J. Chem. Phys. 159 , 044118 (2023)

F. Fuchs, D. Worrall, V. Fischer, and M. Welling: SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks . Adv. Neural Inf. Process. Syst. 33 , 1970-1981 (2020)

T. Frank, O. Unke, and K.-R. Müller: So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems . Adv. Neural Inf. Process. Syst. 35 , 29400-29413 (2022)

Y.-L. Liao, B. Wood, A. Das, and T. Smidt: EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations . Int. Conf. Learn. Represent. https://arxiv. org/abs/2306.12059 (2023)

A. V. Shapeev: Moment Tensor Potentials: A Class of Systematically Improvable Interatomic Potentials . Multiscale Model. Simul. 14 , 1153-1173 (2016)

V. Zaverkin and J. Kästner: Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials . J. Chem. Theory Comput. 16 , 5410-5421 (2020)

V. Zaverkin, D. Holzmüller, I. Steinwart, and J. Kästner: Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments . J. Chem. Theory Comput. 17 , 6658-6670 (2021)

J. Behler and M. Parrinello: Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces . Phys. Rev. Lett. 98 , 146401 (2007)

U. Fano and G. Racah: Irreducible Tensorial Sets . Academic Press Inc., New York (1959)

I. M. Gel'fand, R. A. Minlos, and Z. Y. Shapiro: Representations of the Rotation and Lorentz Groups and Their Applications . Pergamon Press, Inc. (1963)

S. V. S. Martin H. Müser and L. Pastewka: Interatomic potentials: achievements and challenges . Adv. Phys. X 8 , 2093129 (2023)

I. Grega, I. Batatia, G. Csanyi, S. Karlapati, and V . Deshpande: Energy-conserving equivariant GNN for elasticity of lattice architected metamaterials . Int. Conf. Learn. Represent. https: //arxiv.org/abs/2401.16914 (2024)

C. K. Joshi, C. Bodnar, S. V . Mathis, T. Cohen, and P. Lio: Onthe Expressive Power of Geometric Graph Neural Networks . Int. Conf. Learn. Represent. https://arxiv.org/abs/2301.09308 (2023)

S. N. Pozdnyakov, M. J. Willatt, A. P. Bartók, C. Ortner, G. Csányi et al.: Incompleteness of Atomic Structure Representations . Phys. Rev. Lett. 125 , 166001 (2020)

J. Behler: Atom-centered symmetry functions for constructing high-dimensional neural network potentials . J. Chem. Phys. 134 , 074106 (2011)

A. P. Bartók, M. C. Payne, R. Kondor, and G. Csányi: Gaussian Approximation Potentials: The Accuracy of Quantum Mechanics, without the Electrons . Phys. Rev. Lett. 104 , 136403 (2010)

A. P. Bartók, R. Kondor, and G. Csányi: On representing chemical environments . Phys. Rev. B 87 , 184115 (2013)

M. S. Daw and M. I. Baskes: Embedded-atom method: Derivation and application to impurities, surfaces, and other defects in metals . Phys. Rev. B 29 , 6443-6453 (1984)

Y.-M. Kim, B.-J. Lee, and M. I. Baskes: Modified embedded-atom method interatomic potentials for Ti and Zr . Phys. Rev. B 74 , 014101-014112 (2006)

A. Stone: Transformation between cartesian and spherical tensors . Mol. Phys. 29 , 1461-1471 (1975)

A. J. Stone: Properties of Cartesian-spherical transformation coefficients . J. Phys. A 9 , 485 (1976)

J. M. Normand and J. Raynal: Relations between Cartesian and spherical components of irreducible Cartesian tensors . J. Phys. A 15 , 1437 (1982)

K. He, X. Zhang, S. Ren, and J. Sun: Deep Residual Learning for Image Recognition . IEEE Conf. Comput. Vis. Pattern Recognit. https://doi.org/10.1109/CVPR.2016.90 (2016)

S. Chmiela, A. Tkatchenko, H. E. Sauceda, I. Poltavsky, K. T. Schütt et al.: Machine learning of accurate energy-conserving molecular force fields . Sci. Adv. 3 , e1603015 (2017)

K. T. Schütt, F. Arbabzadah, S. Chmiela, K. R. Müller, and A. Tkatchenko: Quantum-chemical insights from deep tensor neural networks . Nat. Commun. 8 , 13890 (2017)

S. Chmiela, H. E. Sauceda, K.-R. Müller, and A. Tkatchenko: Towards exact molecular dynamics simulations with machine-learned force fields . Nat. Commun. 9 , 3887 (2018)

S. Elfwing, E. Uchibe, and K. Doya: Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning . Neural Netw. 107 , 3-11 (2018)

P. Ramachandran, B. Zoph, and Q. V. Le: Searching for activation functions . Int. Conf. Learn. Represent. https://arxiv.org/abs/1710.05941 (2018)

S. J. Reddi, S. Kale, and S. Kumar: On the Convergence of Adam and Beyond . Int. Conf. Learn. Represent. https://arxiv.org/abs/1904.09237 (2018)

Y. Li, Y. Wang, L. Huang, H. Yang, X. Wei et al.: Long-Short-Range Message-Passing: A Physics-Informed Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics Simulation . Int. Conf. Learn. Represent. https://arxiv.org/abs/2304.13542 (2024)

J. T. Frank, O. T. Unke, K.-R. Müller, and S. Chmiela: A Euclidean transformer for fast and stable machine learned force fields . Nat. Commun. 15 , 6539 (2024)

Appendices

Appendix Contents

A Background

Message-passing neural networks. Message-passing neural networks (MPNNs) learn node representations in a graph by iteratively processing local information sent by the nodes' neighbors. The initial features of node u are represented as the vector x u , and undirected edges { u, v } connect pairs of nodes u, v . A node v belongs to the neighborhood of node u , denoted as N ( u ) , if there exists an edge { u, v } in the graph. Typically, the ( t +1) -th message-passing layer computes a new node u 's representation h ( t +1) u by applying a permutation invariant aggregation function over the neighbors [16, 21]

where ϕ ( t ) , ψ ( t ) are often implemented as learnable fully-connected neural networks (NNs), and h (0) u = x u . To learn a mapping from a learned representation h ( t ) u to the atoms' energies, we can couple T message-passing layers with corresponding readout functions R t , t ∈ { 1 , . . . , T } such that E u = ∑ T t =1 R t ( h ( t ) u ) .

Many-body interatomic potentials. Interatomic potentials approximate the potential energy of atoms-the electronic ground state energy-as a function of their coordinates [71]. Many-body potentials naturally arise because the interaction between two atoms is influenced by the presence of additional atoms, changing their electronic structure. This concept is formalized by expanding the atomic energy E u of a many-atom system into a series of two-body, three-body, and higher-body-order contributions

where r u represents the position of atom u and the superscript ν in E ( ν ) indicates the order of the many-body interaction. In the absence of external fields, E ( ν ) contributions to the atomic energy are invariant to rotations, and the two-body potential depends only on distances r uv = ∥ r u -r v ∥ 2

Expansions of this form have found a broad application in constructing machine-learned interatomic potentials (MLIPs), significantly advancing the field [40, 65].

Higher-body-order local descriptors. Recent advances in MLIPs have been influenced by moment tensor potentials (MTPs) [65] and atomic cluster expansion (ACE) [40]. These approaches enable systematic construction of higher-body-order polynomial basis functions, encompassing representations like atom-centered symmetry functions (ACSFs) [68, 75], smooth overlap of atomic positions (SOAP) [76, 77], Gaussian moments [66, 67], and embedded atom/multi-scale embedded atom method (EAM/MEAM) potentials [78, 79]. Furthermore, a reducible Cartesian tensor can be represented as a linear combination of irreducible spherical counterparts, and vice versa [40, 51]. More general expressions, including tensor contractions, have also been provided, hinting at the relationship between Cartesian and spherical models [80-82]. Despite the success of MTP and ACE, defining smaller cutoff radii and rigid architecture can result in limited accuracy compared to MPNNs.

Many-body message passing. Designing accurate and computationally efficient interatomic potential models for interacting many-body systems requires including higher-body-order energy contributions and, thus, higher-body-order learnable features. Recently, a new message construction mechanism has been proposed by expanding the messages m ( t ) u to include many-body contributions [33]

where ( ν +1) denotes the order of many-body interactions, defining the number of contracted tensors. Using ∑ v 1 , ··· ,v ν instead of ∑ v 1 < ··· <v ν circumvents the exponential increase in computational cost with ν . It allows exploiting the product structure of many-body features, which differs from other approaches [58-60].

B Methods

B.1 Normalization constants

The following provides the normalization constants C l 1 l 2 l 3 and D l 1 l 2 l 3 for even and odd irreducible Cartesian tensor products in Eqs. (2) and (3), respectively. The respective normalization constants read [45]

with L = l 1 + l 2 + l 3 and L i = L -2 l i -1 . Here, C l 1 l 2 l 3 is defined such that an l 3 -fold contraction of the tensor z l 3 , obtained through the irreducible Cartesian tensor product between x l 1 and y l 2 ( x l 1 and y l 2 are obtained using Eq. (1)), with the unit vector ˆ r yields unity. We refer to the original publication for the motivation behind D l 1 l 2 l 3 [45].

B.2 Further details on the equivariant message passing

The following provides additional details on the employed equivariant message-passing architecture. Learnable weights employed in Eqs. (4), (5), and (6) (i.e., W ( t ) kk ′ l 2 , W ( t ) kZ v , and W ( t ) kk ′ l ξ with k, k ′ running over feature channels) are initialized by picking the respective entries from a normal distribution with zero mean and unit variance.

Many-body message-passing. The many-body equivariant features, represented by an irreducible Cartesian tensor of rank L and obtained through the ν -fold tensor product in Eq. (6), are combined using the linear expansion

where W ( t ) Z u η ν kL denotes a learnable weight matrix which depends on the chemical element Z u and rank L and which elements are initialized by picking the respective entries from a normal distribution with zero mean and a standard deviation of 1 / len( η ν ) . The updated node embeddings are further obtained as a linear function of ( m ( t ) ukL ) i 1 i 2 ··· i L and the residual connection [33, 83]

where N Z denotes the number of atom types and learnable weights W ( t ) kk ′ L and W ( t ) Z u kk ′ L are initialized by picking the respective entries from a normal distribution with zero mean and unit variance.

Full and symmetric product basis. In the MACE architecture, the authors pre-compute products between the generalized Clebsch-Gordan coefficients, which define the interactions of { l 1 , · · · , l ν } -rank spherical tensors, and the learnable weight W ( t ) Z u η ν kL of the linear expansion in Eq. (A1) [33]. This approach reduces the computational cost of constructing the product basis with spherical tensors as the effective number of evaluated tensor products is smaller by K = len ( η ν ) . For irreducible Cartesian tensors, operations like matrix-vector and matrix-matrix products define the interactions between { l 1 , · · · , l ν } -rank tensors. Thus, an equivalent operation to those proposed for spherical tensors may be generally impossible for irreducible Cartesian tensors or would lead to an architecture different from MACE.

Note that one of our main goals is to demonstrate that a many-body equivariant message-passing architecture defined using higher-rank irreducible Cartesian tensors can be as expressive as the one using spherical tensors. Therefore, we compute m ( t ) ukL by iteratively evaluating all possible ν -fold tensor products, which effectively leads to a larger number of operations than for MACE. We refer to the models based on this architecture as irreducible Cartesian tensor potentials (ICTPs) with the full product basis, or ICTP full . We also noticed that the number of tensor products len( η ν ) leading to

the tensor of rank L , can be reduced by the symmetry of the two-fold tensor product in Eq. (6), i.e., A l 1 ⊗ A l 2 = A l 2 ⊗ A l 1 if A l 1 and A l 2 are symmetric. We refer to this design choice as ICTPs with the symmetric product basis or ICTP sym .

Coupled feature channels. Using coupled feature channels instead of uncoupled ones in Eq. (A1) can improve the performance of the final model. Additionally, the number of feature channels and, thus, the overall computational cost can be reduced when constructing the product basis. However, the number of parameters and, thus, the expressive power of the model can be preserved. Specifically, we can define a linear expansion for combining many-body equivariant features as

where W ( t ) Z u η ν kk ′ L denotes a learnable weight matrix which depends on the chemical element Z u and rank L and which elements are initialized by picking the respective entries from a normal distribution with zero mean and a standard deviation of 1 / ( √ d t × len( η ν )) . The construction of the product basis in the latent feature space requires encoding the two-body features A ( t ) ukl ξ using the learnable weight matrices in Eq. (6). The linear expansion is then constructed in the latent feature space and decoded using the learnable weight matrices of the linear function in Eq. (A2). We refer to this design choice as ICTPs with the product basis constructed in the latent feature space or ICTP lt . Combining it with the symmetric product basis, we obtain ICTP sym+lt .

Readout. Atomic energies expanded into a series of many-body contributions, E u = E (0) u + E (1) u + · · · + E ( T ) u , are obtained by applying readout functions R t to node features with L = 0 , which are invariant to rotations

with learnable weights W ( t ) k ′ or those of NN ( t ) initialized by picking the respective entries from a normal distribution with zero mean and unit variance. Linear readout functions for t < T preserve the many-body orders in h ( t ) uk ( L =0) , while a one-layer fully-connected NN is used for the last messagepassing layer and accounts for the residual higher-order terms in the expansion [32].

B.3 Computational complexity and runtime analysis

We analyze the computational complexity of the irreducible Cartesian tensor product with respect to the maximum rank L used. For each tuple of ( i 1 , · · · , i L ) of a rankL tensor in Eq. (2), the single contraction term ( x L · ( k + m ) · y L ) has a cost of 3 k + m . The total computational cost is 3 L for even tensor products and 3 L +1 for odd ones-due to the additional double contraction with the LeviCivita symbol. The computation of the set of permutations over the L unsymmetrized indices scales as L ! / ( 2 L/ 2 ( L/ 2)! ) for even tensor products and L ! / ( 2 ( L -1) / 2 (( L -1) / 2)! ) for odd ones [45]. Because each final rankL tensor has 3 L elements, the complexity for computing an irreducible Cartesian tensor of rank L is O ( 9 L L ! / ( 2 L/ 2 ( L/ 2)! )) . This expression is used throughout this work as it captures contributions from tensor contractions and index permutations, though it simplifies asymptotically to O ( L L ) using Stirling's approximation for factorials.

The number of calculations required to obtain a rankL spherical tensor through the Clebsch-Gordan tensor product is (2 L +1) 5 . Thus, the computational complexity of the Clebsch-Gordan tensor product is O ( L 5 ) . While the Clebsch-Gordan tensor product is more computationally efficient than the irreducible Cartesian tensor product for L →∞ , the latter is expected to be advantageous for smaller tensor ranks, i.e., L ≤ 4 , assuming similar multiplicative factors and negligible sub-leading terms. Tensors of rank L ≤ 4 are particularly relevant for equivariant message-passing architectures and representing physical properties [32, 33, 72]. Compared to the Gaunt tensor product with the computational complexity of O ( L 3 ) , including all ( l 1 , l 2 ) → l 3 [36], the irreducible Cartesian tensor product may be less advantageous for L ≥ 3 . However, the Gaunt-coefficients-based approach excludes odd tensor products, limiting the expressive power and the range of possible architectures.

As regards the cost of performing message passing at each layer, the general neighbors' aggregation of Eq. (4) has a cost of E N ch 9 L L ! / ( 2 L/ 2 ( L/ 2)! ) , where E is the number of edges in the atomic

system and N ch is the number of feature channels. Computing many-body features via Eq. (6) requires to iteratively perform ν -1 Cartesian tensor products for all K = len( η ν ) possible ν -fold tensor products, resulting in a computational cost of M N ch K (9 L L ! / (2 L/ 2 ( L/ 2)!)) ν -1 with M denoting the number of nodes. For the Clebsch-Gordan tensor product, the corresponding number of calculations is E N ch L 5 and M N ch K L 5( ν -1) . Spherical models, however, can use generalized Clebsch-Gordan coefficients, resulting in M N ch K L 1 2 ν ( ν +3) for the product basis. We can remove the factor K from the above expression by restricting the parameterization to uncoupled feature channels as in Eq. (A1) and obtain M N ch L 1 2 ν ( ν +3) . Thus, MACE with this specific choice of the product basis can be more computationally efficient than ICTP only for large N ch and small ν , given L ≤ 4 . In this work, we have shown that leveraging the symmetry of the irreducible Cartesian tensor product and coupled feature channels can improve the computational cost of ICTP. The former, in particular, reduces the effective number of ν -fold tensor products K . Further optimization of the architecture, however, is possible and is expected to fully exploit the advantage of Cartesian tensors.

Finally, memory consumption for Cartesian tensors is often believed to be less advantageous than that of spherical tensors. However, for contracting two rankL spherical tensors, the memory requirement is about (2 L +1) 2 , resulting from the definition of the Clebsch-Gordan tensor product in Section 2. For a Cartesian tensor, the memory requirement is about 3 L . Thus, the irreducible Cartesian tensor product can be expected to require the same or less memory for L ≤ 4 than the Clebsch-Gordan counterpart. The memory consumption of models based on spherical tensors is further increased by the use of generalized Clebsch-Gordan coefficients, as we demonstrate in Section 5.

C Proof of Cartesian message-passing equivariance to actions of the orthogonal group

We first recap the standard action of the O(3) group onto ( R 3 ) ⊗ l . Let D X be a representation of the orthogonal group O(3) (also in line with Section 2), i.e.,

We define the action of the O(3) group onto ( R 3 ) ⊗ l as follows

where T l ∈ ( R 3 ) ⊗ l . Wehereinafter denote ϕ ( g, T l ) also by R T l . The outer product ⊗ is equivariant to this action, and R acts trivially on the 3 × 3 -identity matrix.

Lemma C.1. Let R be a representation of an element of the orthogonal group O(3) . Then,

(ii) For the 3 × 3 -identity matrix I , we have

Proof. (i) We first show that R ( T l 1 ⊗ T l 2 ) = ( R T l 1 ) ⊗ ( R T l 2 )

Using Lemma C.1, we can show that the irreducible Cartesian tensor operator T l : R 3 → ( R 3 ) ⊗ l is equivariant to actions of the O(3) group.

Proposition C.2. Let l ≥ 0 be a positive integer. Then, the irreducible Cartesian tensor operator T l : R 3 → ( R 3 ) ⊗ l is equivariant to actions of the O(3) group.

Proof. It suffices to show that the map ˆ r ↦→ ˆ r ⊗ ( l -2 m ) ⊗ I ⊗ m is equivariant to actions of the O(3) group ∀ l ≥ 1 and ∀ m ≤ ⌊ l/ 2 ⌋ ( m ≥ 0 ).

Our next claim is that for l 3 ∈ {| l 1 -l 2 | , | l 1 -l 2 | +1 , · · · , l 1 + l 2 } the irreducible Cartesian tensor product ⊗ Cart defined in Eqs. (2) and (3) is equivariant to actions of the O(3) group, i.e., the following diagram commutes

/

/

/

/

The proof for this claim is very similar to Proposition C.2. The only difference is to show the equivariance for the ( k + m ) -fold tensor contraction.

Proposition C.3. The irreducible Cartesian tensor product ⊗ Cart : ( R 3 ) ⊗ l 1 × ( R 3 ) ⊗ l 2 → ( R 3 ) ⊗ l 3 makes the above diagram commute.

Proof. It suffices to show that the ( k + m ) -fold tensor contraction is equivariant to actions of the O(3) group. For x l 1 ∈ ( R 3 ) ⊗ l 1 , y l 2 ∈ ( R 3 ) ⊗ l 2 , and R being a representation of an element of the orthogonal group O(3) , we can write

GLYPH<15>

GLYPH<15>

GLYPH<15>

GLYPH<15>

A similar derivation applies to the odd case (3), which completes the proof.

We finally give a proof for Proposition 4.1, i.e., the equivariance of message-passing layers based on irreducible Cartesian tensors to actions of the O(3) group.

Proof of Proposition 4.1. Since a message-passing layer in Section 4.2 and Appendix B.2 is defined by stacking layers corresponding to Eqs. (4), (5), (6), (A1), (A2), and (A3), we show the equivariance for each of these equations. We show the equivariance by induction.

Case t = 1 :

Equivariance of Eq. (5): Since learnable weights W kZ v and learnable (invariant) radial basis R (1) kl 1 ( r uv ) are defined for each feature channel k , the weights are multiplied with ( T l 1 (ˆ r uv ) ) i 1 i 2 ··· i l 1 as scalars. Therefore,

is equivariant to actions of the O(3) group.

Equivariance of Eq. (6): The proof is similar to the case of Eq. (5), since we again have learnable parameter W (1) kk ′ l ν for each A (1) uk ′ l ν . Therefore, noting that the Cartesian tensor product is equivariant to actions of the O(3) group, the following function

is also equivariant to actions of the O(3) group. Equivariance of Eq. (A1): The equation defined by

is equivariant to actions of the O(3) group, since for each set of indices u , η ν , k , and L we have a scalar learnable parameter W (1) Z u η ν kL .

Equivariance of Eq. (A2): The respective parameters W (1) kk ′ L and W (1) Z u kk ′ L in Eq. (A2)

are multiplied as scalar with h (1) uk ′ L and m (1) uk ′ L , which are shown to be equivariant to actions of the O(3) group. Thus, h (2) ukL is equivariant to actions of the O(3) group as a function of h (1) uk ′ L and m (1) uk ′ L .

Equivariance of Eq. (A3): The equivariance of Eq. (A3) to actions of the O(3) group is also immediate since learnable parameters apply to each tuple of ( i 1 , i 2 , · · · , i L ) as a scalar.

General case t > 1 : The equivariance of Eqs. (4), (5), (6), (A1), (A2), and (A3) to actions of the O(3) group for arbitrary t > 1 follows in a similar way to those obtained for the t = 1 case. However, we need to show the equivariance of Eq. (4). Suppose that Eqs. (5), (6), (A1), (A2), and (A3) are equivariant to actions of the O(3) group for t > 1 . Then, for the ( t +1) -th message-passing layer,

is equivariant to actions of the O(3) group. Here, the irreducible Cartesian tensor product ⊗ Cart and T l 1 are equivariant to actions of the O(3) group, h ( t ) vk ′ l 2 is equivariant by the assumption, and W ( t ) kk ′ l 2 applies to h ( t ) vk ′ l 2 as a scalar.

D Proof of the traceless property for irreducible Cartesian tensors

This section provides a proof for Proposition 4.2 in the main text. The corresponding proposition states the following

Proposition D.1. The message-passing layers based on irreducible Cartesian tensors and their irreducible tensor products preserve the traceless property of irreducible Cartesian tensors.

Our proof of Proposition 4.2 comprises two main parts. First, we show that given two irreducible Cartesian tensors, the irreducible Cartesian tensor product yields again an irreducible Cartesian tensor. We then can use this result in the second part, where we show that expressions defined by Eqs. (4), (5), (6), (A1), (A2), and (A3) preserve the traceless property of irreducible Cartesian tensors. The proof of the second part is straightforward since the message-passing layers are defined by multiplications with scalars, summations, and the irreducible Cartesian tensor product of tensors defined in Eqs. (1), (2), and (3). Therefore, we provide the proof only for the first part in the rest of this section.

Irreducible Cartesian tensors from unit vectors. Recall that an irreducible Cartesian tensor of arbitrary rank l for a unit vector ˆ r ∈ R 3 is defined as

The trace of this tensor reads

Let { k 1 , k 2 , . . . , k l -2 m } be a subset of { 1 , 2 , . . . , l } with l -2 m distinct elements, and a subset I l -2 m with l -2 m distinct elements in { i 1 , . . . , i l } is written as I l -2 m = { i k 1 , i k 2 , . . . , i k l -2 m } . We also let J l -2 m be a set of m disjoint subsets with 2 elements in { i 1 , . . . , i l } \ I l -2 m , i.e., J l -2 m = {{ i k l -2 m +1 , i k l -2 m +2 } , . . . , { i k l -1 , i k l }} . While we introduce I l -2 m and J l -2 m as sets, we assume the order of elements in those sets if the order is necessary and there is no confusion. Furthermore, we introduce two notations used throughout this section

Then, we can write

where I l -2 m runs over all subsets of { i 1 , . . . , i l } with l -2 m elements and J l -2 m comprises all ( ∏ 1 p = m ( 2 p 2 ) ) /m ! combinations of m subsets with 2 elements of { i 1 , . . . , i l } \ I l -2 m . Depending on whether i 1 and/or i 2 belong to I l -2 m , the above expression may be further reduced to

We get the following traces for each pair of I l -2 m and J l -2 m

LemmaD.2. Let l ≥ 2 and 0 ≤ m ≤ ⌊ l/ 2 ⌋-1 . For each pair ( I l -2 m , J l -2 m ) , there exist 2 l -2 m -1 pairs of ( I l -2( m +1) , J l -2( m +1) ) whose trace equals to the trace for the pair ( I l -2 m , J l -2 m ) , i.e.,

Proof. Without loss of generality, we may assume I l -2 m = ( i 1 , i 2 , i k 3 , . . . , i k l -2 m ) and J l -2 m = {{ i k l -2 m +1 , i k l -2 m +2 } , . . . , { i k l -1 , i k l }} . For ∀ m , Tr(( a ) m ) has the following expression

For Tr ( ( b ) m +1 ) and m +1 , we have a set of ( I l -2( m +1) , J l -2( m +1) ) with the cardinality of l -2 m -2 and the corresponding trace equals to Tr(( a ) m ) . For 3 ≤ ∀ s ≤ l -2 m , let

Here, we note that we assume the order of elements in I ( s ) l -2( m +1) . Then, we obtain

The same derivation as above holds for Tr ( ( c ) m +1 ) . For Tr ( ( d ) m +1 ) , let

and we get

For Tr ( ( e ) m +1 ) and 1 ≤ ∀ s ≤ m , we let

and obtain

The additional factor of two originates from the permutation of indices i 1 and i 2 .

Using all the above results, we obtain

which completes the proof of the Lemma.

For each m , define A ( m ) = Tr (( a ) m ) and B ( m ) = Tr (( b ) m )+Tr(( c ) m )+Tr(( d ) m )+Tr(( e ) m ) . Then, the trace of T l (ˆ r ) may be written as a summation of A ( m ) and B ( m ) over m

By Lemma D.2, each pair of the diagonal terms ( A ( ˜ m ) , B ( ˜ m +1)) is cancelled, and B (0) and A ( ⌊ l/ 2 ⌋ ) are left, which by definition are trivial. This result completes the proof of the traceless property for Eq. (1).

Even irreducible Cartesian tensor product. The trace of an even irreducible Cartesian tensor product may be written as

In the following, we let

For simplicity, we omit the subscripts l 1 -( k + m ) , l 2 -( k + m ) , and l 3 -2 m for I , J , and K , if there is no confusion. We also introduce a new notation x j 1 ··· j p ,Q = x j 1 ··· j p ,q 1 ··· q u , where x is an irreducible Cartesian tensor and Q = { q 1 , . . . , q u } , which is used throughout this section. Then, non-trivial terms in ( ∗ ) even may be written as

We note that the trace is trivial when both of i 1 and i 2 belong to either of I or J , because x l 1 and y l 2 are traceless by assumption.

Lemma D.3. Let 0 ≤ m ≤ min( l 1 , l 2 ) -k -1 . For each triplet ( I l 1 -( k + m ) , J l 2 -( k + m ) , K l 3 -2 m ) , there exist 2 l 3 -2 m -1 triplets of ( I l 1 -( k + m +1) , J l 2 -( k + m +1) , K l 3 -2( m +1) ) such that the trace for each triplet equals to the trace for ( I l 1 -( k + m ) , J l 2 -( k + m ) , K l 3 -2 m ) .

Proof. Without loss of generality, we may assume

Then, Tr(( a ) m ) has the following expression

The same derivation applies to Tr(( b ) m ) , and it has the same result as above.

For Tr(( c ) m +1 ) and m +1 , we have l 3 -2 m -2 of ( I l 1 -( k + m +1) , J l 2 -( k + m +1) , K l 3 -2( k + m +1) ) triplets whose trace equals to Tr(( a ) m ) . Indeed, like in the proof of Lemma D.2, we let

Then, by letting ˜ J = { j 1 , . . . , j k + m , j k + m +1 } , we obtain

The same derivation applies to , and we get .

Tr(( d ) m +1 ) Tr(( d ) m +1 ) = ( l 3 -2 m -2) Tr (( a ) m ) For Tr(( e ) m +1 ) , the same derivation as for Tr(( d ) m +1 ) and Tr(( e ) m +1 ) in the proof of Lemma D.2 applies. Therefore, we have

Hence, we obtain

which completes the proof of the Lemma.

For each m , define A ( m ) = Tr (( a ) m )+Tr(( b ) m ) and B ( m ) = Tr (( c ) m )+Tr(( d ) m )+Tr(( e ) m ) . Then, the trace for ( x l 1 ⊗ Cart y l 2 ) l 3 may be written as a summation of A ( m ) and B ( m ) over m

· · ·

.

By Lemma D.3, each pair of the diagonal terms ( A ( ˜ m ) , B ( ˜ m +1)) is cancelled taking into account an additional factor of 2 m in front of B ( ˜ m +1) ; B (0) and A (min( l 1 , l 2 ) -( l 1 + l 2 -l 3 ) / 2) are left, which by definition are trivial. This result completes the proof of the traceless property for Eq. (2).

Odd irreducible Cartesian tensor product. The trace of an odd irreducible Cartesian tensor product may be written as

In the following, we let

For simplicity, we omit the subscripts l 3 , l 1 -( k + m ) , l 2 -( k + m ) , and l 3 -2 m for E , I , J , and K , if there is no confusion. Then, non-trivial terms inside ( ∗ ) odd can be written as

Note that the trace is trivial when i 1 or i 2 belongs to E and the remaining index belongs to I ∪ J . Indeed, let ˜ J = { j 1 , . . . , j k + m } and suppose i 1 ∈ E and i 2 ∈ I . Then, and we have

For the non-trivial terms inside ( ∗ ) odd we have the lemma below, whose proof is identical to that of Lemma D.3.

LemmaD.4. Let 0 ≤ m ≤ min( l 1 , l 2 ) -k -2 . For each tuple ( E l 3 , I l 1 -( k + m ) , J l 2 -( k + m ) , K l 3 -2 m ) , there exist 2 l 3 -2 m -1 tuples of ( E l 3 , I l 1 -( k + m +1) , J l 2 -( k + m +1) , K l 3 -2( m +1) ) such that the trace for each of the tuples equals to the trace for ( E l 3 , I l 1 -( k + m ) , J l 2 -( k + m ) , K l 3 -2 m ) .

Proof. Without loss of generality, we may assume

Then, Tr(( a ) m ) has the following expression

Here, we assume that a 2 ∈ I l 1 -( k + m ) and a 3 ∈ J l 2 -( k + m ) . The same derivation holds for Tr(( b ) m ) .

For Tr(( c ) m +1 ) and m +1 , we have l 3 -2 m -2 of ( E l 3 , I l 1 -( k + m +1) , J l 2 -( k + m +1) , K l 3 -2( m +1) ) triplets whose trace equals to Tr(( a ) m ) . Indeed, like in the proof of Lemma D.2, we let

Then, by letting ˜ J = { j 1 , . . . , j k + m , j k + m +1 } , we obtain

The same derivation applies to Tr(( d ) m +1 ) , and we get Tr(( d ) m +1 ) = ( l 3 -2 m -2) Tr (( a ) ) .

m For Tr(( e ) m +1 ) , the same derivation as for Tr(( d ) m +1 ) and Tr(( e ) m +1 ) in the proof of Lemma D.2 applies. Therefore, we obtain

Finally, we can write

which completes the proof of the Lemma.

Noting that Tr(( a ) m ) is doubled for every m due to the factor 2 m , the proof is completed similarly to the proof of Lemma D.3. This result completes the proof of the traceless property for Eq. (3).

E Experiments and results

E.1 Description of the data sets

rMD17 data set. The revised MD17 (rMD17) data set is a collection of structures, energies, and atomic forces of ten small organic molecules obtained from ab initio molecular dynamics (AIMD) [47]. These molecules are derived from the original MD17 data set [47, 84-86], with 100,000 structures sampled for each. Our models are trained using 950 and 50 configurations for each molecule randomly sampled from the original data set using five random seeds, with 50 additional configurations randomly sampled for early stopping. We use the remaining configurations to test the final models. Table 1 reports the mean absolute errors (MAEs) in total energies and atomic forces averaged over five independent runs, including the standard deviation between them.

MD22 data set. The MD22 data set includes structures, energies, and atomic forces of seven molecular systems derived from AIMD simulations at elevated temperatures, spanning four major classes of biomolecules and supramolecules [48]. These molecular systems range from a small peptide containing 42 atoms to a double-walled nanotube comprising 370 atoms. Characterized by complex intermolecular interactions, this data set was designed to challenge short-range models. The training set sizes used in this study are consistent with those in the original publication [48], selected to ensure that the sGDML model stays within a target accuracy of 1 kcal/mol ( ≈ 43 . 37 meV). We randomly selected an additional set of 500 structures for each molecule in the data set for early stopping, while the remaining configurations were reserved for testing the final models. The corresponding training and validation data sets were randomly selected using three random seeds. Table A2 reports MAEs in total energies per atom and atomic forces averaged over three independent runs, including the standard deviation between them.

3BPA data set. The 3BPA data set comprises structures, energies, and atomic forces of a flexible drug-like organic molecule obtained from AIMD at various temperatures [49]. The training data set consists of 500 configurations sampled at 300 K, while three separate test data sets are obtained from

AIMD simulations at 300 K, 600 K, and 1200 K. An additional test data set provides energy values along dihedral rotations of the molecule. This test directly assesses the smoothness and accuracy of the potential energy surface, influencing properties such as binding free energies to protein targets. Our models are trained using 450 and 50 configurations randomly sampled from the training data set using five random seeds, with further 50 configurations reserved for early stopping. Table 2 reports the root-mean-square errors (RMSEs) in total energies and atomic forces averaged over five independent runs for a training set size of 450 structures, including the standard deviation between them. Table A3 presents the corresponding results obtained with a training set size of 50 structures.

Acetylacetone data set. The acetylacetone data set includes structures, energies, and atomic forces of a small reactive molecule obtained from AIMD at various temperatures [32]. The training data set comprises configurations sampled at 300 K, while the test data sets are sampled at 300 K and 600 K. The generalization ability of final models is evaluated using an elevated temperature of 600 K and along two internal coordinates of the molecule: The hydrogen transfer path and a partially conjugated double bond rotation featuring a high rotation barrier. Our models are trained using 450 and 50 configurations randomly sampled from the training dataset using five random seeds, with further 50 configurations reserved for early stopping. Table 3 reports RMSEs in total energies and atomic forces averaged over five independent runs for a training set size of 450 structures, including the standard deviation between them. Table A5 presents the corresponding results obtained with a training set size of 50 structures.

Ta-V-Cr-W data set. The Ta-V-Cr-W data set includes 0 K energies, atomic forces, and stresses for binaries, ternaries, and quaternary and near-melting temperature properties in four-component disordered high-entropy alloys [50]. In total, this benchmark data set comprises 6711 configurations, with energies, atomic forces, and stresses computed at the density functional theory (DFT) level. More precisely, there are 5680 0 K structures: 4491 binary, 595 ternary, and 594 quaternary structures, along with 1031 structures sampled from MD at 2500 K. Structure sizes range from 2 to 432 atoms in the periodic cell. All models are trained using 5373 configurations, with 4873 used for training and 500 for early stopping. The remaining configurations are reserved for testing the models' performance. The performance is evaluated separately using 0 K binaries, ternaries, quaternaries, and near-melting temperature four-component disordered alloys. The original data set provides ten different training-test splits. In our experiments, each training set is further split into training and validation subsets using a random seed. Furthermore, we use additional binary structures strained along the [100] direction as a part of the test data set. Note that the final models, in this case, were obtained by training using the whole data set of 6711 configurations (training + test), 500 of which were reserved as a validation data set. Table A6 reports RMSEs in total energies per atom and atomic forces averaged over ten independent runs, including the standard deviation between them.

E.2 Training details

All ICTP and MACE models employed in this work were trained on a single NVIDIA A100 GPU with 80 GB of RAM. Training times for ICTP and MACE models typically ranged from 30 minutes to a few days, depending on the data set, the data set size, and the employed precision. We used double precision for 3BPA and acetylacetone data sets and single precision for rMD17, in line with the original experiments [33]. Double precision was also used for MD22, while single precision was employed in our Ta-V-Cr-W experiments. Unless stated otherwise, we used two message-passing layers and irreducible Cartesian tensors or spherical tensors of a maximal rank of l max = 3 to embed the directional information of atomic distance vectors.

For ICTP models with the full (ICTP full ) and symmetric (ICTP sym ) product basis and MACE, we employ 256 uncoupled feature channels. Exceptions include our experiments with the 3BPA data set, aimed at investigating scaling and computational cost, and the Ta-V-Cr-W experiments, where we used eight and 32 feature channels, respectively. For ICTP models with the symmetric product basis evaluated in the latent feature space (ICTP sym+lt ), we use 64 coupled feature channels for the Cartesian product basis and 256 for two-body features. Radial features are derived from eight Bessel basis functions with polynomial envelope for the cutoff with p = 5 [60]. These features are fed into a fully connected NN of size [64 , 64 , 64] . We apply SiLU non-linearities to the outputs of the hidden layers [87, 88]. The readout function of the first message-passing layer is implemented as a linear layer. The readout function of the second layer is a single-layer fully connected NN with 16 hidden

neurons. A cutoff radius of 5.0 Å is used across all data sets except MD22, where we used a cutoff radius of 5.5 Å for the double-walled nanotube and 6.0 Å for the other molecules in the data set.

All parameters of ICTP and MACE models were optimized by minimizing the combined squared loss on training data D train = ( X train , Y train ) , where X train = {S ( k ) } N train k =1 and Y train = { E ref k , { F ref u,k } N at u =1 , σ ref k } N train k =1

Here, σ ref k is the stress tensor defined as σ = 1 V ∇ ϵ E | ϵ = 0 , where E denotes total energy after a strain deformation with symmetric tensor ϵ ∈ R 3 × 3 and V is the volume of the periodic box.

When training ICTP models on rMD17, 3BPA, and acetylacetone data sets, we neglected the stress loss and set C e = 1 /N ( k ) at and C f = 10 Å 2 to balance the relative contributions of total energies and atomic forces, respectively. For MACE and 3BPA/acetylacetone, C e = 1 / ( B × N ( k ) at ) and C f = 1000 / ( B × 3 × N ( k ) at ) Å 2 were used with B denoting the batch size. For ICTP models trained on the MD22 data set, we set C e = 10 /N ( k ) at and C f = 1 Å 2 , using energies and forces in eV and eV/Å, respectively. For the Ta-V-Cr-W dataset, the stress loss was incorporated into the combined loss in Eq. (A5), along with the energy and force losses. For ICTP, we used C e = 1 /N ( k ) at , C f = 0 . 01 Å 2 , and C s = 0 . 001 /N ( k ) at to balance the relative contributions of total energies, atomic forces, and stresses, respectively. For MACE, we chose C e = 1 / ( B × N ( k ) at × N ( k ) at ) , C f = 1 / ( B × 3 × N ( k ) at ) Å 2 , and C s = 0 . 05 / ( B × 9 × N ( k ) at × N ( k ) at ) . Here, E ( S ( k ) , θ ) , F u ( S ( k ) , θ ) , and σ ( S ( k ) , θ ) are total energies, atomic forces, and stresses predicted by ICTP or MACE.

All models for rMD17, 3BPA, and acetylacetone were trained for 2000 epochs using the AMSGrad variant of Adam [89], with default parameters of β 1 = 0 . 9 , β 2 = 0 . 999 , and ε = 10 -8 . For MD22 and Ta-V-Cr-W, all models were trained for 1000 epochs. For rMD17, 3BPA, and acetylacetone data sets, we used a learning rate of 0.01 and a batch size of 5. For MD22 and Ta-V-Cr-W, we again chose a learning rate of 0.01 but a mini-batch of 2 and 32, respectively. For evaluations on the validation and test data sets, we used a batch size of 10 for rMD17, 3BPA, and acetylacetone, while mini-batches of 2 and 32 were used for MD22 and Ta-V-Cr-W, respectively.

The learning rate was reduced using an on-plateau scheduler based on the validation loss with a patience of 50 and a decay factor of 0.8 for all data sets except for MD22, for which we used a patience of 10. We utilize an exponential moving average with a weight of 0.99 for evaluation on the validation set and for the final model. Additionally, in line with MACE [33], we apply exponential weight decay of 5 × 10 -7 on the weights of Eqs. (6), (A1), and (A3). Furthermore, we incorporate a per-atom shift of total energies via the average per-atom energy over all the training configurations, including the energies of individual atoms for 3BPA and acetylacetone datasets. If no atomic energies are provided, as for rMD17, MD22, and Ta-V-Cr-W, the per-atom shift is obtained by solving a linear regression problem [67]. Additionally, a per-atom scale is determined as the root-mean-square of the components of the forces over the training configurations.

E.3 Additional results

Scaling and computational cost. Table A1 provides the numerical results complementing Fig. 2 in the main text. All results in Table A1 and Fig. 2 were obtained using irreducible Cartesian tensors with a maximal rank of l max = L to represent local atomic environments. Here, L denotes the tensor rank of employed equivariant messages. We facilitated the exploration of larger ν values by setting the number of uncoupled feature channels to eight. In the MACE model, intermediate spherical tensors with a rank of l > l max are used to construct the product basis. However, the pre-computation of generalized Clebsch-Gordan coefficients for ν > 4 , in some cases, would require more than 2 TB of RAM. Therefore, in our experiments, we fixed the maximum rank of intermediate tensors to l = l max . We also used the full product basis for ICTP to calculate the same number of ν -fold tensor

a ICTP, MACE, Equiformer, Allegro, and TorchMD-Net are short-range models (i.e., they use local or semi-local atomic representations), ViSNet-LSRM and SO3krates include long-range information, and sGDML is a global model.

b For consistency, these results are obtained for relative energy and force weights of C e = 10 /N ( k ) at and C f = 1 Å 2 , based on our experiments with the Ac-Ala3-NHMe molecule. For comparison, we also trained ICTP for 800 epochs with C e = 100 /N ( k ) at and C f = 1 Å 2 , achieving MAEs of 0.209 ± 0.007 meV/atom for energy and 13.98 ± 0.30 meV/Å for force.

products, i.e., K = len ( η ν ) , as used in MACE with l = l max . Finally, we note that ICTP and MACE use different approaches to optimize their runtimes; however, the scaling with respect to the tensor rank and the correlation order is independent of these optimization methods.

Molecular dynamics trajectories. Table A2 presents the energy and force mean absolute errors (MAEs) for the ICTP models trained on the MD22 data set. This data set was designed to challenge short-range potential models-i.e., those based on local or semi-local atomic representations-and includes large molecular systems with complex intermolecular interactions. We compare errors obtained for ICTP models with those of state-of-the-art approaches that incorporate global, short- and long-range information. From Table A2, we see that ICTP achieves accuracy in predicted energies and atomic forces on par with or better than state-of-the-art methods. The ICTP model uses a cutoff of 6.0 Å (5.5 Å for the double-walled nanotube), resulting in a receptive field of 12.0 Å (11.0 Å), considering the two message-passing layers. This receptive field is, in most cases, smaller than the diameter of molecular systems in the data set. Thus, ICTP is, at most, a semi-local potential model-similar to MACE, which used a 5.0 Å cutoff and two message-passing layers.

We chose a larger cutoff for ICTP than the one used by MACE motivated by the results in the recent work [7]. In particular, the authors compared MACE to VisNet-LSRM-the best model reported to date for MD22, which employs mixed short- and long-range information in their message passing-and reported that MACE can achieve lower force errors. However, VisNet-LSRM typically had lower energy errors, attributed to the improvement from considering atomic interactions beyond 10.0 Å. Using a larger cutoff radius for ICTP compared to MACE, we expected results closer to those of VisNet-LSRM. Table A2 shows that using a receptive field of 12.0 Å is, in most cases, sufficient to achieve accuracy in predicted energies close to the one obtained by VisNet-LSRM.

Extrapolation to out-of-domain data. Table A3 demonstrates total energy and atomic force RMSEs obtained for ICTP and MACE models trained using 50 configurations randomly drawn from the original data set. ICTP and MACE models perform similarly, considering the standard deviation obtained across five independent runs. However, ICTP models often have lower mean RMSE values compared to MACE. Furthermore, Table A4 presents the total energy and atomic force RMSEs for ICTP and MACE models that use ν = 1 to compute the product basis. Thus, we provide results for models which rely exclusively on two-body interactions. We note that for ν = 1 , ICTPfull and ICTPsym are identical; though, we include both results for completeness.

Figure A1 compares potential energy profiles obtained with ICTP and MACE models trained using 450 configurations. Potential energy cuts at β = 120 ◦ and β = 180 ◦ are easier tasks for MLIPs, as there are training points in the data set with similar combinations of dihedral angles [32]. In contrast, the potential energy cut at β = 150 ◦ is more challenging, with no training points close to it. Notably, Fig. A1 shows that all models produce smooth potential energy profiles close to the reference ones (DFT) for all values of β . These results again demonstrate excellent extrapolation capabilities of irreducible Cartesian models that are on par with the spherical MACE model.

Flexibility and reactivity. Table A5 demonstrates total energy and atomic force RMSEs obtained for ICTP and MACE models trained using 50 configurations randomly drawn from the original data set.

ff

fi

fl

0

100

200

300

γ

in

◦

0

20

40

60

Rel. energy in meV

α

= 71

.

0

◦

,

β

= 120

.

0

◦

0

100

200

300

γ

in

◦

α

= 67

.

0

◦

,

β

= 150

.

0

◦

0

100

200

300

γ

in

◦

α

= 151

.

0

◦

,

β

= 180

.

0

◦

DFT

ICTPfull

ICTPsym

ICTPsym

+

lat

MACE

Similar to the 3BPA data set, ICTP and MACE models demonstrate comparable accuracy in predicted energies and forces, considering the standard deviation obtained across five independent runs.

Figure A2 further investigates the generalization capabilities of ICTP models trained using 450 configurations, demonstrating potential energy profiles for the rotation around the C-C bond, i.e., the C-C-C-O dihedral angle ( α ), and for the hydrogen transfer (i.e., the O-H distance d OH in Fig. A2). The training data set encompasses dihedral angles less than 30 ◦ . Furthermore, the energy barrier of 1 eV is outside the energy range of the training data set obtained at 300 K. As for the hydrogen transfer, the training data does not contain transition geometries, but the reaction still occurs in a region that is not too far from the training data. Overall, ICTP models perform on par with MACE for predicting potential energy profiles for the rotation around the corresponding C-C bond and for the hydrogen transfer.

d

OH

α

0

250

500

750

1000

1250

Rel. energy in meV

(a)

0

5

10

15

(b)

0

50

100

150

Dihedral angle in

◦

0

200

Count

1

.

1

1

.

2

1

.

3

1

.

4

Distance in

˚ A

0

10

DFT

ICTPfull

ICTPsym

ICTPsym

+

lat

MACE

Figure A3 shows potential energy profiles for MLIPs trained with 50 configurations, similar to Fig. A2. Notably, Fig. A2 (b) demonstrates that ICTP full is the only MLIP consistently producing the potential energy profile for the hydrogen transfer close to the reference (DFT). This task is particularly challenging, as most data splits do not include configurations sufficiently close to the transition structure as in Fig. A2.

Multicomponent alloys. The Ta-V-Cr-W data set is designed to evaluate the performance of MLIPs across atomic systems with varying numbers of atom types/components, comprising both relaxed (0 K) and high-temperature structures [50]. In particular, this data set includes 0 K energies, forces, and stresses for 2-, 3-, and 4-component systems and 2500 K properties in 4-component disordered alloys. It contains 6711 configurations with sizes ranging from two to 432 atoms in the periodic cell. Table A6 demonstrates the energy and force RMSEs for the ICTP and MACE models, evaluated separately on 0 K binaries, ternaries, quaternaries, and near-melting temperature four-component disordered alloys. ICTP consistently outperforms state-of-the-art models for the Ta-V-Cr-W data set, i.e., MTP and GM-NN. In contrast, for MACE, we were not able to identify a set of relative weights for energy, forces, and virial losses that consistently yield results better than those of MTP and GM-NN in both energies and forces. Table A6 shows that MACE often matches the accuracy of ICTP on forces but is typically outperformed by a factor of ≤ 2.0 on energies.

d

OH

α

0

250

500

750

1000

1250

Rel. energy in meV

(a)

0

5

10

15

(b)

0

50

100

150

Dihedral angle in

◦

0

25

Count

1

.

1

1

.

2

1

.

3

1

.

4

Distance in

˚ A

0

2

DFT

ICTPfull

ICTPsym

ICTPsym

+

lat

MACE

We further compare the ICTP, MACE, MTP, and GM-NN models using the separate test data set containing binary structures strained along the [100] direction. We find that neither ICTP nor MACE consistently outperforms MTP and GM-NN in this case. However, because this test data set contains only a single configuration of 432 atoms per binary, it may not serve as a valuable benchmark in this study and is included merely for completeness. Additionally, we trained a Cartesian model with ν = 2 and L = l max = 2 , which resembles a TensorNet-like architecture [30], though it incorporates equivariant convolution filters. The corresponding results are provided in Table A6. We found that model configurations with ν = 3 and L = 2 (and often with L = 1 ) outperform the ν = 2 and L = l max = 2 configuration by factors of ≤ 1.4 and ≤ 1.2 in energy and force RMSEs, respectively.

Finally, similar to our results for the 3BPA data set, we observe that MACE can be more computationally efficient than ICTP. We attribute longer inference times of ICTP to the pre-factor K arising from the Cartesian product basis in Eq. (6). Thus, we attribute ICTP's lower computational efficiency to the use of the MACE architecture, which is optimized for spherical tensors. We expect that further modifications to the architecture can facilitate more efficient use of the advantages of operations based on irreducible Cartesian tensors.

F Broader social impact

This section discusses the broader social impact of the presented work. Our work has important implications for the chemical sciences and engineering, as many problems in these fields require atomistic simulations; we also discuss it in Section 1. Although this work focuses on standard benchmark data sets, our experiments demonstrate the scalability of our method to larger atomic systems. Beyond constructing machine-learned interatomic potentials, equivariant models based on irreducible Cartesian tensors can be applied for molecular property prediction, protein structure prediction, protein generation, ribonucleic acid structure ranking, and many more.

Our work has no obvious negative social impact. As long as it is applied to the chemical sciences and engineering in a way that benefits society, it will have positive effects.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction include the presented work's main contributions, assumptions, and results. Each of these claims are backed up through theoretical and empirical results in the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of the presented work in Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the proof of the equivariance and traceless property of MPNNs based on irreducible Cartesian tensors in Appendices C and D.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe the entire experimental setting and all details on the performed experimens in Section 5 and Appendix E. We also provide a detailed description of MPNNs employed in this work, including the weight initialization; see Section 4.2 and Appendix B.2. Furthermore, we describe the construction of irreducible Cartesian tensor products and introduce the irreducible Cartesian tensor products in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The source code is available on GitHub and can be accessed via this link: https://github.com/nec-research/ictp . All data sets used in this study are publicly available: rMD17 ( https://doi.org/10.6084/m9.figshare. 12672038.v3 ), MD22 ( http://www.sgdml.org ), 3BPA ( https://github.com/ davkovacs/BOTNet-datasets ), acetylacetone ( https://github.com/davkovacs/ BOTNet-datasets ), and Ta-V-Cr-W ( https://doi.org/10.18419/darus-3516 ).

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The details on the experimental setting are provided in Section 5 and Appendix E.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We run all experiments using several random seeds and provide the corresponding standard deviations and error bars in all tables and figures.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details on the computer resources in Section 5 and Appendix E. Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The presented work follows the NeurIPS Code of Ethics. We also include Appendix F, which discusses the broader social impact.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We include Appendix F, which discusses the broader social impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper does not release any data or models with a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We reference all employed data sets and source codes in the main text.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We provide a detailed description of the construction of irreducible Cartesian tensors, computation of their irreducible tensor products, and the resulting equivariant message-passing layers; see Section 4 and Section B. We also provide training details in Appendix E.2 and describe the employed data sets in greater detail in Appendix E.1. The source code includes comprehensive documentation as well.

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We performed neither crowdsourcing nor research with human subjects.

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We performed neither crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work builds upon advances on equivariant and many-body architectures for the construction of neural network potentials. It lays out the formalism to substitute the conventionally-used spherical tensors in higher-rank models for Cartesian tensors. Taking as a reference the MACE architecture, the authors intend to show that this is a competitive approach to SOTA models in terms of accuracy and computational efficiency.\n\nSTRENGTHS:\nThe exploration of new methods to more efficiently learn machine learning force fields is an active area of research, and the use of higher-rank Cartesian approaches is quite novel, in contrast to the use of the spherical basis. The authors demonstrate that they can obtain results on benchmarks datasets that compete with SOTA models (both spherical and higher-rank Cartesian). The exposition of mathematical concepts is quite clear for a reader familiar with the literature. In terms of accuracy, the model is very satisfactory.\n\nWEAKNESSES:\nAlthough the achievement of competitive performance when compared to SOTA models is relevant enough, and the mathematical machinery is novel for the neural network potential field, I am not sure whether the authors have been able to demonstrate in some way why their method should be chosen in contrast to MACE, for example. The comparison of inference times seems to not favor the use of ICTP. However, I am aware of the fact that the formalism laid out in the paper allows the construction of other architectures, and that the design space of these models could be further investigated to find even more efficient models.\n\nQUESTIONS:\n1) It seems that the first model to explore the idea of higher-rank Cartesian tensors was TensorNet [30], even though it is not flexible to incorporate arbitrary ranks, and it does not explicitly account for many-body interactions. I miss some more thorough discussion on their differences, even more taking into account that [30] seems to display competitive performance to ICTP without making use of those more sophisticated approaches. I would encourage the authors to include some discussion in this regard. Is ICTP a combination of CACE and TensorNet?\n\n2) I do not intend the authors to address the following question with more experiments, I acknowledge the limited time frame, but: experiments have been conducted on datasets consisting of single systems. Do the authors have any reference of how the model performs on datasets with varying chemical composition?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the use of higher-rank irreducible Cartesian tensors as an alternative to spherical tensors for equivariant message passing in machine learning interatomic potentials. The authors illustrate clearly on how to construct these tensors and their products, prove equivariance properties, and evaluate the approach empirically on several molecular datasets.\n\nSTRENGTHS:\n* The mathematical foundations are clearly illustrated, with detailed explanations of how to construct irreducible Cartesian tensors and compute their products.\n* The experiments on out-of-domain extrapolation, particularly on the 3BPA and acetylacetone datasets, provide valuable insights into the generalization capabilities of the proposed method.\n* The paper demonstrates that irreducible Cartesian tensor-based models can achieve comparable or sometimes better performance than state-of-the-art spherical tensor models.\n\nWEAKNESSES:\n* The empirical evaluation is limited to relatively simple molecular datasets. The paper would be strengthened by including experiments on more challenging datasets such as MD22 or heterogeneous datasets like QM9.\n\n*  The efficiency gain and the performance gain is not that appealing to my eye. It seems little more than “instead of using that math, you can use this math!” without a very strong theoretical justification for why to do so. The author could have done a better job of explaining what is the fundamental difference/advantage of the proposed cartesian tensors when compared with the sphereical tensors.\n\nQUESTIONS:\n* What are the core differences between this method and TensorNet? A clearer comparison would help position this work in the context of existing literature.\n* Is the proposed model compatible with Hamiltonian prediction? This could be an interesting avenue for future work.\n* Can the authors provide plots showing how their model scales with increasing L? \n* The paper mentions \"transferability\" in line 225. Could the authors clarify what they mean by this term in this context?\n* The claim that Cartesian tensors are advantageous to spherical tensors requires further explanation. From a representation power perspective, aren't they equivalent? Is it possible that the observed performance gains are due to hyperparameter tuning rather than fundamental differences in representation power?\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors proposed higher-rank irreducible Cartesian Tensor Product, and explored its usage in equivariant neural networks design in scientific applications such as molecular modeling. The authors firstly prove that irreducible Cartesian Tensor Product is equivariant to O(3) group, and further show that higher-rank (e.g., > 2) operations can be used in an efficient way for models using many-body interactions. Experiments are conducted to demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS:\n1. The problem this work aims to tackle is of great significance in real-world scientific applications. \n2. The proposed approach is interesting and can potentially improve a new class of equivariant neural networks for crucial tasks.\n3. The paper is easy to follow.\n\nWEAKNESSES:\n1. **The motivation of this work needs to be better explained and presented**. As stated in the Introduction and Related Works, the major disadvantage of spherical tensors is computationally demanding, which motivates the development of Cartesian-Tensor-Product-based approaches. However, the authors do not well discuss the disadvantages of existing Cartesian-Tensor-Product-based approaches (e.g., inefficiency in scaling tensor ranks up), which is necessary as a solid support for the motivation of this work. Besides, the lack of comprehensive discussion and comparisons between this work and existing Cartesian-Tensor-Product-based approaches makes the actual value of this work doubtful for readers who are not familiar with the context.\n\n2. **Experimental results are weak in demonstrating the superiority of the proposed approach**:\n    - Lack of detailed efficiency comparisons: in this work, the authors demonstrate that the proposed irreducible Cartesian Tensor Product can be used for both two-body and many-body feature interaction or equivariant convolutions with better theoretically-proved efficiency. However, there are no comprehensive comparisons covering different operations and also different rank L.\n    - Lack of large-scale experiments: all datasets used in this work (rMD17, 3BPA and Acetylacetone) have limited scales of molecular systems and number of samples. Since the proposed approach is claimed to bring benefits in efficiency, it would be necessary to verify it on larger-scale datasets such as OC20/22.\n    - Lack of experiments on applying iCTP for two-body operations only: MACE is mainly used to compare iCTP and spherical tensor products. However, two-body operations like equivariant feature interaction/equivariant convolution are also widely used in equivariant networks. It would be better to further verify the effectiveness of iCTP on these operations only to demonstrate its generality.\n\nOverall, it is of great significance to design more powerful and efficient equivariant networks for real-world applications. However, several issues exist in the current submission. My recommendation is Borderline Accept, and I will carefully read the rebuttal and other reviews to decide whether to decrease or increase my scores.\n\nQUESTIONS:\n1. Could you detailedly explain the difference in computational complexity between iCTP and spherical Tensor Products on both two-body and many-body interactions?\n\n2. Could you comprehensively compare this work with TensorNet and compare the strengths and weaknesses of them?\n\nLIMITATIONS:\nThe authors have discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-23"}