A General Framework for Equivariant Neural Networks on Reductive Lie Groups
Authors: Ilyes Batatia
Mario Geiger
Jose M Munoz
Tess Smidt
Lior Silberman
Christoph Ortner
Venue: NeurIPS 2023 poster (2023)
Keywords: equivariance
point clouds
machine learning
particle physics

Abstract: Reductive Lie Groups, such as the orthogonal groups, the Lorentz group, or the unitary groups, play essential roles across scientific fields as diverse as high energy physics, quantum mechanics, quantum chromodynamics, molecular dynamics, computer vision, and imaging. In this paper, we present a general Equivariant Neural Network architecture capable of respecting the symmetries of the finite-dimensional representations of any reductive Lie Group. Our approach generalizes the successful ACE and MACE architectures for atomistic point clouds to any data equivariant to a reductive Lie group action. We also introduce the lie-nn software library, which provides all the necessary tools to develop and implement such general G-equivariant neural networks. It implements routines for the reduction of generic tensor products of representations into irreducible representations, making it easy to apply our architecture to a wide range of problems and groups. The generality and performance of our approach are demonstrated by applying it to the tasks of top quark decay tagging (Lorentz group) and shape recognition (orthogonal group).

A General Framework for Equivariant Neural Networks on Reductive Lie Groups
Tlyes Batatia
Mario Geiger
Engineering Laboratory, University of Cambridge Cambridge, CB2 IPZ UK Department of Chemistry, ENS Paris-Saclay, Université Paris-Saclay 190 Gif-sur- Yvette, France ilyes. batatia@ens -aeaaaeennnnniiieeeeeeeeee wwr--parrssssiieee fr
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology Cambridge, MA, USA
Jose Munoz
Tess Smidt
EIA University, FTA Group Antioquia, Colombia
Department of Electrical Engineering and Computer Science, Massachusetts Institute Technology Cambridge, MA, USA
Lior Silberman Department of Mathematics University of British Columbia Vancouver, BC, Canada V6T 1Z2
Christoph Ortner Department of Mathematics University of British Columbia Vancouver, BC, Canada V6T 1Z2
Abstract
Reductive Lie Groups, such as the orthogonal groups, the Lorentz group, or the unitary groups, play essential roles across scientific fields diverse as high energy physics, quantum mechanics, quantum chromodynamics, molecular dynamics, computer vision, and imaging. In this paper, we present general Equivariant Neural Network architecture capable of respecting the symmetries of the finite- dimensional representations of any reductive Lic Group G. Our approach gener- alizes the successful ACE and MACE architectures for atomistic point clouds to any data equivariant reductive Lie group action. We also introduce the lie-nn software library, which provides all the necessary tools to develop and implement such general G-equivariant neural networks. It implements routines for the reduc- tion of generic tensor products of representations into irreducible representations, making easy to apply architecture to wide range of problems and groups. The generality and performance of our approach are demonstrated by applying it to the tasks of top quark decay tagging (Lorentz group) and shape recognition (orthogonal group).
Introduction
Convolutional Neural Networks (CNNs) (LeCun et al.) 1989) have become widely used and powerful tool for computer vision tasks, in large part due to their ability to achieve translation equivariance. This property led to improved generalization and a significant reduction in the number of parameters. Translation equivariance is one of many possible symmetries occurring in machine learning tasks.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

A wide range of symmetries described by reductive Lie Groups is present in physics, such as O(3) in molecular mechanics, (0)), in Engryy Physics, SU(2") in quantum mechanies, and SU(3) in quantum chromodynamics. Machine learning architectures that respect these symmetries often lead to significantly improved predictions while requiring far less training data. This has been demonstrated in many applications including 2D imaging with O(2) symmetry (Cohen and Welling 2016a Esteves et al. 2017). machine learning force fields with (()) symmetry (Anderson et al.
2019: Bartók et al. 2013 Batzner et al., 2022 Batatia al. 2022a or jet tagging with S0*(1,3)
symmetry (Bogatskiy et a.. 2022] Li et al. 2022).
One way to extend CNNs to other groups (Finzi al. 2020 Kondor and Trivedi 2018) is through harmonic analysis on homogencous spaces, where the convolution becomes an integral over the group. Other architectures work directly with finiteddimensional representations. We follow the demonstration of Bogatskiy et al. (2020a) who constructed a universal approximation of any equivari- ant map with feed-forward neural network with vector activations belonging to finiteddimensional representations of a wide class of Lie groups. In this way, one can avoid computational challenges created by fiinit-dimmesional representations.
$O(,,3)
((3)
SU (3)
Figure 1: Examples of natural science problems and associated reductive Lie groups. For high energy physics, the Lorentz group 30;; for chemistry, the Euclidean group E(3); for quantum- chromodynamics, the SU(3) group.
Alternatively, our current work can be thought of as a generalization of the Atomic Cluster Expansion (ACE) formalism of Drautz (2019) general Lie groups. The ACE formalism provides complete body-ordered basis of -invarana features. By combining the concepts of ACE and E(3)- equivariant neural networks, Batatia al. (2022a) proposed the MACE architecture, which achieves state-of-the-art performance on learning tasks in molecular modelling. The present work generalizes the ACE and MACE architectures arbitrary Lie groups in order propose generic architecture for creating representations of geometric point clouds in interaction.
Concretely, our work makes the following contributions:
We develop the G-Equivariant Cluster Expansion. This framework generalizes the ACE (Drautz) 2019) and MACE (Batatia er al. 20226) architectures to parameterize proper- ties point clouds, equivariant under a general reductive Lie group G.
We prove that our architecture is universal, even for a single layer.
We introduce lie-nn, a new library providing all the essential tools to apply our framework to variety of essential Lie Groups in physics and computer visions, including the Lorentz group, SUNN), (C) and product groups
We illustrate the generality and efficiency of our general-purpost approach by demon- strating excellent accuracy on two prototype applications, jet tagging, and 3D point cloud recognition.
2 Background
We briefly review a few important group-theoretie concepts: A real (complex) Lie group is group that is also Anite-dimensional smooth (complex) manifold in which the product and inversion of the group are also smooth (holomorphic) maps. Among the most important Lie groups are Matrix Lie groups, which are closed subgroups of (n,,, C) the group of invertible entries. This includes well-known groups such as Sp(2n, R) consisting of matrices of determinant one, that is relevant in Hamiltonian dynamics A finiteddimensional representation of the Lie group
n matrices with complex

---

G' is saite-dimensioall vector space V endowed with smooth homomorphism p: G - GV). Features in the equivariant neural networks live in these vector spaces. An irreducible representation V is representation that has no subspaces which are invariant under the action of the group (other than {0} and V itself). This means that V can not be decomposed non-trivially as the direct sum of representations. A reductive group over a field F is (Zariski-) closed subgroup of the group of matrices GL(n, F') such that every minteedimmesional representation of G on an F-vectorspace can be decomposed sum of irreducible representations.
3 Related Work
Convolutional neural networks (CNNs), which are translation equivariant, have also been generalized to other symmetries. For example, G-convolutions (Cohen and Welling 2016b) generalized CNNs to discrete groups. Steerable CNNs (Cohen and Welling, (20 gener- alized CNNs to (22 equivariance and Spherical CNNs (Cohen et al. (018)) 0(3) equivariance. general theory of convolution on any compact group and symmetric space was given by Kondor and
Lie group convolutions
Trivedi (2018). This work was further extended to equivariant convolutions on Riemannian manifolds
by Weiler et al. (202T).
ACE The Atomic Cluster Expansion (A (ACE) (Drautz 2019) introduced a systematic framework for constructing complete (33)-invariant high body order basis sets with constant cost per basis function, independent of body order (Dusson et al. 2022).
The e3nn library (Geiger and Smidi 2022) provides a complete solution to build (3) equivariant neural networks based on irreducible representations. The Equivariant MLPs (Fizzi et al. 2021) include more groups, such as S0,,, Zn. but are restricted to reducible representations making them much less computationally efficient than irreducible representations.
e3nn+ + Equivariant MLPs
Equivariant MPNNs and MACE Equivariant MPNNs (Kondor al. 2018; Anderson et al. 2019
Bogatskly et al. 2020a Satorras et al. 2021 Brandstetter et al. [022 Batzner et (022) have
emerged as powerful architecture to learn on geometrie point clouds. They construct permutation invariants and group equivariant representations of point clouds. Successful applications include sim- ulations in chemistry, particle physics, and 3D vision. MACE (Batatia al. (2022a) generalized the ((3) Equiivvara MPNNs to build messages of arbitrary body order, outperforming other approaches on molecular tasks. (Batatia el al. (002b) showed that the MACE design space is large enough to include most of the previously published equivariant architectures.
4 The G-Equivariant Cluster Expansion
We are concerned with the representation of properties of point clouds. Point clouds are described as multi-sets (unordered tuples) X = [aa]; where each particle z; belongs to a configuration domain ft We denote the set of all such multi-sets by mSsessSt).. For example, in molecular modeling, I; might describe the position and species of an atom and therefore x; (r:, Zi) E R3 x Z. while in high energy physics, one commonly uses the four-momentum F; = (F,, Pc) E RAA, but one could also include additional features such as charge, spin, and so forth. A property of the point cloud is map
Φ: msets(()) + Z
i... X ((X) E Z. usually a scalar or tensor. The range space is application dependent and left abstract throughout this paper. Expressing the input as a multi-set implicitly entails two important facts: (1) can have varying lengths; (2) invariant under the permutations of the particles. The methods developed in this article are also applicable to fixed-length multi-sets, in function defined on some ΩA Mappings that are not
which case & is simply a permutation-invaricn germutation-ivvaiiat are special cases with several simplifications.
In many applications, especially in the natural sciences, particle properties satisfy additional symme- tries. When a group G acts on Ω as well as on Z we say that 4 is G-equivariant if
① o g pz(
GEG
where (z(g) is the action of the group element g on the range space Z. In order effectively incorporate exact group symmetry into properties O we consider model architectures of the form
T ms)
Z,
(3)
embedding
parmmettssiatio
readout
3

---

where the space V into which we "embed" the parameterization is possibly vector space which convenient representation of the group is available. For simplicity we will sometimes assume that Z = V.
The Atomic Cluster Expansion (ACE) framework (Drautz 2019 Dusson et al. 2022 Drautz 2020)
produces a complete linear basis for the space of all "smooth" G-equivariant properties d for the specific case when G (3) and r; are vectorial interatomic distances. Aspects of the ACE framework were incorporated into E(3)-equivariant message passing architectures, with significant improvements in accuracy (Batatia et al. 2022a). In the following paragraphs we demonstrate that these ideas readily generalize to arbitrary reductive Lie groups.
4.1 Efficient many-body expansion
The first step is to expand + in terms of body orders, and truncate the expansion a finite order N:
(x) po + Σ (1x)))) + Σ pa( Fb,)) +... +
( Tiy):
(4)
21.52
$1....5
where yn defines the n--odd interaction. Formally, the expansion becomes systematic in the limit as N + co. The second step is the expansion of the n-particle functions ynn in terms of symmetrized tensor product basis. To define this we first need to specify the embedding of particles æ: A countable family (ox) ) a 1-particle basis if they are linearly independent on 52 and any smooth 1-particle function 1i (not necessarily equivariant) can be expanded in terms of (Øx)x. i.e,
(x( Σ (x
(5)
For the sake of concreteness, assume that Φ& Ω + C, but the range in principle be any field. We provide concrete examples of 1 1-particle bases in Appendix A.2 Let complex vector space V be given, into which the particle embedding maps, i.e.,
(ov(z)) V
E
As consequence of (5) any smooth scalar r-particle function yo can be expanded in terms of the corresponding tensor product basis,
Pn (x) Fnd
Σ Wky... Ao
II s=1
ob, (I,).
(6)
Inserting these expansions into 44) and interchanging summation (see appendix for the details) we arrive at model for scalar permutation-symment properties,
A
Σ (x),, TEX
A
II A ==1
(NN) =
Σ wk A
kEK
where IC is the set all tuples indexing the features Ag. Since A& is invariant under permuting &. only ordered & tuples retained. The features Aş are embedding of msets(?)) into the space V. The tensorial product features (basis functions) Aş form a complete linear basis of multi-set functions on Ω and the weights Wk be understood symmetric tensor. We will extend this linear cluster expansion model ((N) message- passing type neural network model in $
While the standard tensor product embeds (0"=10k,) : Ω + V", the n-correlations A are symmetric tensors and embed (A&)a: msots((?) - Sym" V.
The evaluation of the symmetric tensor features A₁ is the computational bottleneck in most scenarios, but efficient recursive evaluation algorithms (Batatia et al.) [2022a] Kaliuzhnyi and Ortner 2022) are available. See Appendix A. 13.2 for further discussion of model computational costs.
4.2 Symmetrisation
With (7) we obtained a systematic linear model for (smooth) multi-set functions. It remains to incor- porate G-equivariance. We assume that a reductive Lie group with a locally finite representation in V. In other words we choose representation p (pxa)): G + GL(V) such that
dk o g
Σ Pkk (g)
(8)

---

where for each & the sum over k' is over a finite iedex-set depending only on k. Most Lie groups one encounters physical applications belong to this class, the affine groups being notable exceptions. However, those can usually be treated in an ad hoc fashion, which is done in all F(3) (e3)-equivaiiant architectures we are aware of. In practice, these requirements restrict how we can choose the embedding (Φx)). If the point clouds X = [xg]] are already given in terms of a representation of the group, then one may simply construct V to be iterative tensor products of Ω2 see e.g. the MTP (Shapeev 2016) and PELICAN (Bogatskiy al. (022) models. To construct equivariant two-particle basis we need to first construct the set of all intertwining operators from X V.
Concretely, we seek all solutions Ca.
ky
to the equation
Σ Ca.K RE (g)PkçRo (0) = E K
or, written in operator notation, Ca p8p= pCa. We will call the C2. generalized Clebsch-Gordan coefficients since in the case G = SO(3 acting the spherical harmonics embedding 6im Y those coefficients are exactly the classical Clebsch-Gordar coefficients. The index a enumerates normally
basis of the space of all solutions to this equation. For the most common groups. identifies a canonical basis Ca and assigns a natural meaning to this index (cf. $(..5). Our abstract notation is chosen because of its generality and convenience for designing computational schemes. The generalization of the Setbcch-Gordan equation (9) to n. products of representations acting on the symmetric tensor space Sym" (V)
(V) becomes (cf. A.9
Cock K Park Ce'
Σ K
Ca.KK' CK
AK, == (NI....,KN), g GEG,
(10)
where
PELk
Σ k" TES.
Pank
and
Plek
Pkcke
Due to the symmetry of the (AR)& tensors Ch. need only be computed for ordered k tuples and the sum En also runs only over ordered & tuples. Again, the index a enumerates a basis of the space of solutions. Equivalently, (TO) can be written in compact notation C*p pCa, These coupling operators for products can often (but not always) be constructed recursively from couplings of pairs (9) We can now define the symmetrized basis
BK a
Ec Aw. k'
(11)
The equivariance of 11 is easily verified by applying a transformation E G to the input (ef $ A.6)
Universality: In the limit the correlation order N - xc, the features [B K. form complete basis of smooth equivariant multi-set functions, in a sense that we make precise Appendix A.7 Any equivariant property dy Ω V can be approximated by a linear model
Σ
K BK
(12)
within arbitrary accuracy by taking the number of terms in the linear combination to infinity.
4.3 Dimension Reduction
The tensor product of the cluster expansion in taken all the indices of the one-particle basis. Unless the embedding (Ø*)x ) is very low-dimensional often preferable to "sketch" this tensor product. For example, consider the canonical embedding of atom 2; = (r,, Z,).
(x;) @zntm (I;) = 6.z, (r..)
Only the (Im) channels are involved in the representation of O(3) hence there considerable freedom in "compressing" the (z, n) channels.
Following Darby el al. 2022) we construct a sketched G-equivariant cluster expansion: We endow the one-particle basis with an additional index referred to as the sketched channel, replacing the index with the index pair (c, k), and renaming the embedding (dka)),, the case of three-dimensional particles one may, for example, choose (z, n). In general, it is crucial that the representation
5

---

remains in terms of Pk,k", that is, becomes Ock o Ex PRe Phtrgg)oes Therefore, manipulating only the channel does not change any symmetry properties of the architecture. Generalizing Darby el al. 2022), the G-TRACE (tensor-reduced ACE) basis then becomes
B ca
Co.
K Ackk,
where
(13)
12
Ack
II
Σ
Wee'
Σ =EX
deekk (x)
(14)
t=1
This construction is best understood as an cetivariance-preeeviinn canonical tensor decomposi- tion (Darby er al. 2022). There are numerous variations, but for the sake of simplicity, we restrict our presentation to this one case.
Following the proof of Darby er al. (2022) one can readily see that the G-TRACE architecture inherits the universality of the cluster expansion, in the limit of decoupled channels #c + oc. A smooth equivariant property & may be approximated to within arbitrary accuracy by an expansion $ (X) = >ca c B (X). Since the embedding A. is learnable, this is nonlinear model. We refer to $ A.7 for the details.
Universality:
4.4 G-MACE, Multi-layer cluster expansion
The G-equivariant cluster expansion is readily generalized to a multi-layer architecture by re- expanding previous features a new cluster expansion (Batatia et al, (022b)) The multi-set X is endowed with extra features, h =
that are updated for E [I, ..., T) iterations. These features themselves are chosen to a field of representations such that they have a well-defined transformation under the action of the group. This results in
z = (x,, h2)
(15)
6e (z h2) = Σ w
Ch th Σ kr.k"
(16)
The recursive update of the features proceeds as in a standard message- wessage-passing passing framework but with the unique aspect that messages are formed via the G-TRACE and in particular can contain arbitrary high correlation order:.
mi.c Ewack B
(17)
The gathered message mf = (m) k..ck is then used to update the particle states,
o,,
x = (z,, h
h = Um(m))
(18)
where U, can be an arbitary fixed or learnable transformation (even the identity). Lastly, a readout function maps the state of a particle to a target quantity of interest, which could be local to each particle or global the mset X.
T
T y Σ Relob ((a5))
V/ Σ Rlog (af),
respectively,
(19)
This multi-layer architecture corresponds to general besssage-passingg neural network with arbitrary body order of the message at each layer. We will refer to this architecture as G-MACE. The G-MACE architecture directly inherits universality from the G-ACE and G-TRACE architectures:
Theorem 4.1 (Universality of G-MACE), Assume that the one-particle embedding (Φx)k is a complete basis. Then, the set of G-MACE models, with fixed finite number of layers T, is dense in the set of continuous and equivariant properties of point clouds X E mscts({?), in the topology of pointwise convergence. It is dense in the uniform topology on compact and boundedddd subsets.
5 lie-nn Generating Irreducible Representations for Reductive Lie Groups
In order to construct the G-cluster expansion for arbitrary Lie groups, one needs to compute the generalized Clebsch-Gordan coefficients (10p for given tuple of representations (see [T]. To facilitate this task, we have implemented an open source software library, lie-n In this section we review the key techniques employed in this library.
'https://githuu http://:tttuu.ccooos///ss/tt

---

5.1 Lie Algebras of Reductive Lie Groups
Formally, the Lie algebra of a Lie group is its tangent space at the origin and carries an additional structure, the Lie bracket. Informally the Lie algebra can be thought of as a linear approximation to the Lie group but, due to the group structure, this linear approximation carries (almost) full information about the group. In particular the representation theory of the Group is almost entirely determined by the Lie algebra, which is a simpler object to work with instead of the fully nonlinear Lic group.
Lie algebra The Lie groups we study can be realized closed subgroups G GL, (RR) of the general linear group. In that case their Lie algebras can be concretely realized as g = Li) = {X E (R)) Vi R exp(I.X) E G} where exp(X) exponential. It turns out that g C Mn (R) is a linear subspace closed under the commutator bracket [X,Y] = XY YX.
X + _ is the standard matrix
Structure theory We fix a linear basis (X:} C g. called a set of generators for the group. The Lie algebra structure is determined by the structure constants Asjk defined by [X,, X3] Σx Aijx-Xx. in Xx. The classification of reductive groups provides convenient generating sets for their Lie algebras (or their complexifica- tions). One identifies large commutative subalgebra h C g (sometimes of gc = g Ø C) with basis {H,} so that most (or all) of the other generators E. can be chosen so that [H, Eal (HH,) ) E. for a linear function a on h. These functions are the so-called roots of g. Structural information about is commonly encoded pictorially via the Dynkin diagram of a finite graph the nodes of which are a certain subset of the roots. There are four infinite families of simple complex Lie algebras A, su(n + 1). B, so + 1), C, s2n) Dr so so(2n) and further five exceptional simple complex Lie algebras (a general reductive Lie algebra the direct sum of several simple ones and its centre). The Lie algebra only depends on the connected component of G. thus when the group G' is disconnected in addition to the infinitesimal generators {X,} one also needs to fix so-called "discrete generators", subset H C G containing representative from each connected component.
that if X Σ: a;X; and Y = by then [X, Y] = Σx Ei.AAjaaa
A, SU
D. SO(2n)
Figure 2: Examples of Dynkin diagrams and their associated group class.
Representation theory The representation theory of complex reductive Lie algebras is completely understood. Every finite-dimensionl representation (isomorphic to) the direct sum of irreducible representations (" ("inreps"), with the latter parametrized by appropriate linear functional on h ("highest weight"). Further given highest weight there is construction of the associated irrep with an explicit action of the infinitesimal generators chosen above. The Weyl Dimension Formula gives the dimension of an irrep in terms of its highest weight.
5.2 Numerical Computations in lie-nn
The most basic class of the lie-nn library encodes a group G and infinitesimal representation dp of g using the tuple
p = (A, n, (dp(X:)): {o(h))eeH)
(20)
with A the structure constants of the group, the dimension of the representation, and dp(X,) and p(h) being n x n matrices encoding the action of the infinitesimal and the discrete generators respectively. The action of infinitesimal generators is related to the action of group generators by the exponential, VX E g. p. only discrete generators.
ddo(X), For finite groups, we assume that Xx) O as they have
As the building blocks of the theory irreps are treated specially; the package implements functionality for the following operations for each supported Lie group:
Constructing the irrep with given highest weight.
7

---

Determining the dimension of an irrep.
Decomposing the tensor product of several irreps into irreps up isomorphism (the selection
rule, giving the list of irreducible components and their multiplicities).
Decomposing the tensor product of several irreps into irreps explicitly via a change of basis
("generalized Clebsch-Gordan coefficients").
Computating the symmetrized tensor product of the group (see. 5.3 and A.9 for details).
To construct an irrep explicitly as in (20) one needs to choose a basis in the abstract representation space (including labeling scheme for the basis) so that we can give matrix representations for the action of generators. For this purpose, we use in lie-nn the Gelfand-Tsetlin (GT) basis (Gelland and Tsetlin 1950) and associated labeling of the basis by GT patterns (this formalism was initially introduced for algebras of type A, but later generalized to all classical groups). Enumerating the GT patterns for given algebra gives the dimension of given irrep, the selection rules can be determined combinatorially. and it is also possible to give explicit algorithms to compute Clebsch-Gordan coefficients (the case of A, is treated by Alex et al. (201T)) For some specific groups, simplifications to this procedure are possible and GT patterns are not required.
In some cases, one wants compute coefficients for reducible representations for representations where the analytical computation with GT patterns is too complex. In these cases, numerical algorithm to compute the coefficients is required. Let dpı, dp2 two Lie aglebra representations of interest. The tensor product on the Lie algebra dp1 @ dp can be computed as,
dp dp2 (X) dp (X) @ & dp2(X)
(21)
Therefore, given sets of generators of three representations dp1, dp2, dps. the Clebsch-Gordan coefficients the change of basis between (dp1 (X) & 1 & (X)(X)) and dxs(X). One can compute this change of basis numerically via null space algorithm. For some groups, one can apply an iterative algorithm that generates all irreps starting with single representation, using the above-mentioned procedure (see A.
5.3 Symmetric Powers
Let V be vector space and } f be basis of V. Vhe symmetric power of V, Sym" V, can be regarded the space of homogeneous polynomials of degree n in the variables e;. The product basis in Equation 10 spans exactly this space. A basis of Sym V can be constructed as,
fer 622 -....... c,, f1 S ... S in}
(22)
If V if an irreducible representation of a reductive Lie group G with highest weight X, Sym" Va admits decomposition into irreducible representations,
Sym" " VA= = + CA
(23)
The generalized Clebsch Gordan coefficients in 11 represent the change of basis between Sym" Va and one of the Vu. The following steps are taken to obtain these coefficients:
Construct the symmetric power basis as in
Compute the coefficients cx using Freudenthal's Formula or GT patterns.
For any a with CX non-zero, find basis of V,,. and compute the change of basis between the basis of Sym" VA and Vi
Alternatively, if one simply has the Clebsch Gordan coefficients, the change of basis from VA some Vn, a new algorithm outlined in Appendix A,9 implemented in lie-nn can construct the change of basis from Sym" VA Vja-
Applications
6.1 Lie groups and their applications
In 6.1 give a non-exhaustive overview of Lie groups and their typical application domains, which methodology naturally applies.
Benchmarking our method on all of these applications beyond the scope of the present work, in particular, because most of these fields do not have standardized benchmarks and basclines to
8

---

Table I: Lie groups of interests covered by the present methods and their potential applications to equivariant neural networks. The groups above the horizontal line are already available in lie-nn. The ones below the line fall within our framework and can be added.
Group
Application
Reference
U(1)
Electromagnetismm
Laggrave al.
2021
SU(3)
Quantum Chromodynamics
(aavoni et al.
20/2
SO(3)
3D point clouds
Batatia el 2022a
$07(1,3)
Particle Physics
Bogatskiy al.
2020b
SL(3, RR
Point cloud classification Entangled QP
SU(2N)
Sp(N)
Hamiltonian dynamics
SO(2N 1)
Projective geometry
compare against. The MACE architecture has proven to be state of the art for large range of atomistic modeling benchmarks (Batatia et al. 2022a). In the next section, we choose two new prototypical applications and their respective groups to further assess the performance of our general approach.
6.2 Particle physics with the S0(1,3)
Jet tagging consists identifying the process that generated collimated spray of particles called jet after a high-energy collision occurs at particle colliders. Each jet can be defined as a multiset of mour-momenta [(E,, p.) where E, E R+ and P; R
Current state-of-the-art models incorporate the natural symmetry arising from relativistic objects, e.g, the Lorentz symmetry, model invariance. To showcase the performance and generality of the G-MACE framework we use the Top-Tagging dataset (Butter et al. (019). where the task is to differentiate boosted top quarks from the background composed of gluons and light quark jets. Table 6.22 we can see that G-MACE achieves excellent accuracy, being the only arbitrary equivariant model reach similar accuracy as PELICAN, which is invariant model. We refer to Appendix I.I. for the details of the architecture.
Table 2: Comparisson between state-of-the of-the-artt- art metrics on the Top- Tagging dataset. Scores were taken from (Bogatskiy et al.) [2022] Qu et al. 2022 Qu and Gouskos 2020 Munoz er al.) 2022 Bogatskiy
al. 2020a Komiske al. 2019 Pearkes al. 2017).
Architecture
#Params
Accuracy
AUC
Rejao%
PELICAN
45k
0.942
0.987
2289 + 204
partT
2.14M
0,940
0.986
1602 81
ParticleNet
498k
0.938
0.985
1298 = 46
LorentzNet
224k
0.942
0.987
2195 + 173
BIP
4k
0,931
0.981
853 = 68
LGN
4.5k
0,929
0.964
435 95
EFN
82k
0,927
0.979
888 = 17
ToPoDNN
59k
0.916
0.972
295 + 5
LorentzMACE
228k
0.942
0.987
1935 = 85
6.3 3D Shape recognition
3D shape recognition from point clouds is of central importance for computer vision. We use the ModelNet10 dataset (Wu er al. 2015) to test our proposed architecture in this setting. As rotated objects need to map to the same class, we use a MACE model with ((3) symmetry. To create an encoder version of G-MACE, we augment PointNet++ implementation (Yan) 2019) with G-MACE layers. See the appendix 11.2 more details on the architecture. We see in Table that MACE outperforms the non-equivariant baseline.

---

Table 3: Accuracy in shape recognition. Scores were taken from
Qi et al. 2016), (Qi et al.
2017).
Shen et 2018
Li (2018) Kumawat and Raman (2199)
Architecture
PointMACE: (ours)
PointNet
PointNet ++
KCN
SO-Net
LP-3DCNN
Accuracy
96.1 Point Cloud
94.2 Point cloud
95.0 Point cloud
94.4
95.5
94,4
Representation
Point Cloud
Point Cloud
Voxel grid
7 Conclusion
We introduced the G-Fquivariant Cluster Expansion, which generalizes the successful ACE and MACE architectures symmetries under arbitrary reductive Lie groups. We provide an open-source Python library lie-nn that provides all the essential tools to construct such general Lie-group cquivariant neural networks. We demonstrated that the general G-MACE architecture simultaneously achieves excellent accuracy in Chemistry, Particle Physics, and Computer Vision. Future development will implement additional groups and generalize to new application domains.
Acknowledgments and Disclosure of Funding
IB's work supported by the ENS Paris Saclay. CO's work supported by NSERC Discovery Grant IDGRO1 9381 and NERF Exploration Grant GR022937. This work was also performed using resources provided by the Cambridge Service for Data Driven Discovery (CSD3).IB would like to thank Gábor Csányi for his support.
References
Y. LeCun, B. Boser, S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Neural Computation 1, 541 (1989)
S. Cohen and M. Welling, ICLR 2017 (2016a), 11w.10/8///AAXX..61100.6622
C. Esteves, C.
X. Zhou, and K. Daniilidis, "Polar transformer networks." (2017).
B. Anderson, T. S. Hy, and R. Kondor, in Advances in Neural Information Processing Systems Vol. 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, M'Alcéé-Buc, E. Fox, and R. Garnett (Curran Associates, Inc., 2019).
A. P. Bartók, R. Kondor, and G. Csányi, Physical Review B 87, 184115 (2013).
S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt, and B. Kozinsky, Nature Communications 13, 2453 (2022).
I. Batatia, D. P. Kovács, G. N. C. Simm, C. Ortner, and G. Csányi, "Mace: Higher order equivariant message passing neural networks for fast and accurate force fields." (2022a).
A. Bogatskiy, T. Hoffman, D. W. Miller, and J. T. Offermann, Machine Learning and the Physical Sciences workshop, NeurIPS 2022 (2022), arXiv:2211.0000 [hep-ph]
C. Li. H. Qu, S. Qian, Q. Meng. S. Gong, J. Zhang, T.-Y. Liu, and Q Li, (2022), ar arXiv:2208.078: [hep-ph]]
M. Finzi, Stanton, P. Izmailov, and G. Wilson, in Proceedings of the 37th International Conference on Machine Learning Proceedings of Machine Learning Research, Vol. 119, edited by H. D. III and Singh (PMLR, 2020) pp. 3165-3176.
R. Kondor and S. Trivedi, in Proceedings of the 35th International Conference on Machine Learning Proceedings of Machine Learning Research, Vol. 80, edited by J. Dy and A, Krause (PMLR, 2018) pp. 2747-2755.
A. Bogatskiy, B. Anderson, J. T. Offermann, M. Roussi, W. Miller, and Kondor, "Lorentz Group Equivariant Neural Network for Particle Physics." (2020a), arXiv:2006.044 04780 [hep-ph]
R. Drautz, Phys. Rev. B 99, 014104 (2019)
10

---

I. Batatia, S. Batzner, D. P. Kovács, A. Musaclian, G. N. C. Simm, R. Drautz, C. Ortner. B. Kozinsky. and G. Csányi, "The design space of eq3)vvauiaa atom-centered interatomic potentials. (2022b).
T. Cohen and M. Welling, in Proceedings of The 33rd International Conference Machine Learning Proceedings of Machine Learning Research, Vol. 48, edited by M. F. Balcan and K. Q. Weinberger (PMLR, New York, New York, USA, 2016) pp. 2999.2999.
T. S. Cohen, M. Geiger, J. Köhler, and M. Welling, in International Conference Learning Representations (2018).
M. Weiler, P. Forré, E. Verlinde, and M. W "Coordinate independent convolutional networks
isometry and gauge equivariant convolutions on riemannian manifolds," (2021).
G. Dusson, M. Bachmayr, G. Csányi, R. Drautz, S. Etter, C. van der Oord, and Ortner, Journal of Computational Physics 454, 110946 (2022)
M. Geiger and T. Smidt, "e3nn: Euclidean neural networks," (2022).
M. Finzi, M. Welling, and A. G. Wilson, "A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups." (2021).
R. Kondor, Z. Lin, and S. Trivedi, in Advances in Neural Information Processing Systems, Vol. 31, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Curran Associates, Inc., 2018).
V. G. Satorras, E. Hoogeboom, and M. Welling, "E(n) equivariant graph neural networks," (2021).
J. Brandstetter, R. Hesselink, E. van der Pol, E. J. Bekkers, and M. Welling, "Geometric and physical quantities improve e(3) equivariant message passing." (2022), 1r.00022 [cs.LG]
R. Drautz, Phys. Rev. B 102, 024104 (2020)
I. Kaliuzhnyi and C. Ortner, ArXiv e-prints 2202.04140 (2022)
A. Shapeev, Multiscale Model. Simul. 14, 1153 (2016).
J. P. Darby, D. P. Kovács, I. Batatia, M. A. Caro, G. L. W. Hart, C. Ortner, and G. Csányi, TTnsorr reduced atomic density representations," (2022).
M. Gelfand and M. L. Tsetlin, 825828, (1950).
A. Alex, M. Kalus, A. Huckleberry, and J. von Delft, Journal Mathematical Physics 52, 023507 (2011)
P.-Y. Lagrave, Y. Cabanes, and F. Barbaresco, in Geometric Science of Information, edited by F. Nielsen and F. Barbaresco (Springer International Publishing, Cham, 2021) 1) pp. 577-584.
M. Favoni, A. Ipp, D. I. Müller, and D. Schuh, Phys. Rev. Lett. 128, 032003 (2022)
A. Bogatskiy, B. Anderson, J. Offermann, M. Roussi, D. Miller, and R. Kondor, in Proceedings of the 37th International Conference on Machine Learning Proceedings of Machine Learning Research, Vol. 119, edited by H. D. III and A. Singh (PMLR, 2020) pp. 992-1002.
A. Butter el al., SciPost Phys. 7, 014 (2019) ar arXiv: 1902.0992991 [hep-ph]
H. Qu, C. Li, and S. Qian, Particle transformer for jet tagging." (2022).
H. Qu and L.. Gouskos, Phys. Rev. D 101, 056019 (2020) arXiv: 1902.998777000 [hep-ph].
J. M. Munoz, I. Batatia, and C. Ortner, "Bip: Boost invariant polynomials for efficient jet tagging." (2022).
P. T. Komiske, E. M. Metodiev, and J. Thaler, JHEP 01, 121 (2019). ar Xiv: 10::181000515 [hep-ph]
J. Pearkes, W. Fedorko, A. Lister, and C. Gay, (2017),
[hep-ex]

---

Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang. X. Tang, and J. Xiao, in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE Computer Society, Los Alamitos, CA. USA, 2015) pp. 1912-1920.
X. Yan, github.com/yanx27 (2019).
C. R. Qi, H. Su, K. Mo, and L. Guibas, arXiv preprint arXiv: (2016).
C. R. Qi, L. Yi, H. Su, and L.. J. Guibas, arXiv preprint arXiv: (2017).
Y. Shen, Feng. Y. Yang, and D. Tian, CVPR (2018).
Li, B. M. Chen, and G. H. Lee, CoRR (2018).
Kumawar and S. Raman, in CVPR (2019).
A. I. Molev, "A weight basis for representations of even orthogonal lie algebras," (1999),
ar #IXIv:math/9900060 [math.RT]].
R. Steinberg, Bull. Amer. Math. Soc. 67, 406 (1961).
M. Bachmayr, G. Dusson, C. Ortner, and Thomas, ArXiv e-prints 2109.14771 (2021)
Thomas, H. Chen, and C. Ortner, Arch. Ration. Mech. Anal. 246 (2022).
Broecker, Representation of Compact Lie Groups (Springer Berlin Heidelberg, 1985).
N. Bourbaki, Lie Groups and Lie Algebras (Springer Berlin, Heidelberg, 1989).
12

---

A Appendix
A.1 Background on Lie groups and Tensor Products of Representations
A.1.1 Lie groups
Lie groups generalize the idea of continuous symmetry such as the rotation symmetry of the round sphere St (given by the action of the orthogonal group O(n)), the symmetry of a vector space under changes of basis (given by the action of the general linear group GL (R)), or the invariance of Minkowski spacetime under boosts (given by the Lorentz group). The symmetry groups of specific geometries were studied in the 19th century, leading to Sophus Lie's 's study of general continuous symmetry. Other major contributors to our understanding include Klein, Riemann, Hilbert, Poincarré, Noether, Cartan, among many others. In particular the solution of Hilbert's 's 5th problem by Gleason, Montgomery, and Zippin shows that under very general conditions a continuous symmetry group must be a Lie group. Symmetry being central to mathematics Lie groups appearl in many areas of mathematics, from differential geometry to number theory.
Let us give the general formal definition, as well as a more practical concrete one:
Definition A.1. A real (complex) Lie group is group that is also finitediimessinn smooth (complex) manifold in which the product and inversion of the group are also smooth (holomorphic) maps.
linear Lie group is a subgroup of the group GL,, (R) which also a closed subset (in the sense of real analysis).
It is a fact that every closed subgroup of a Lie group is Lie group, so every group of the second type is indeed also a group of the first type. Conversely while in general a Lie group is only locally isomorphic
linear group (by a surjective group homomorphism), every compact Lie group is isomorphic to a linear group, and most Lie groups that arise in practice are linear.
When the Lie group G acts space X (c.g. as symmetry group) it naturally also acts spaces of functions on X (by translation), and this action linear. In general. an action of G on a (suually complex) vector space by linear map is called a linear representation (of G). In the context of our paper the space of functions on is the space of features, and we achieve efficient computation by decomposing this representation into simpler constituents in other words by understanding the representation theory of G.
Describing the representation theory of arbitrary Lie groups is very difficult problem and active area of research in pure mathematics. However, for certain classes of groups it is possible to say quite bit (for example classical harmonic amalysis can be viewed from the lens of the representation theory of commutative groups). An important class of groups whose representation theory can be completely understood is the class of reductive Lie groups, on which we focus in this paper.
For now let us give a concrete definition; an abstract will follow later once we introduce the necessary language.
Definition A.2. linear Lie group G C GT,,, (R) reductive it is also closed under transpose: for every G we have that g" E G.
Most of the groups that are of interest in natural sciences reductive Lie groups This includes well-known groups such as the symplectic group Sp2,(R) relevant in Hamiltonian dynamics, and orthogonal group O(n) which parametrizes rotations RR.
A.1.2 Linear representations
Fix a Lie group G. A linear representation of is a pair (T, V) where V is complex vecetor space and is an action of G' on V by linear maps. In other words for each g E G we have a complex-linear map x (g): V + V such that (()192) = ((91) ( and such that (1aa) = Idy (in other words, T: G + GL(V) is group homomorphism).
For an example let the familiar rotation group 0(3) act on the usual round sphere 5 Then (()) acts on the space of functions on S This space is infinitediimensional, but it turns out we can approximate it by a sum of finite-dimeniionl pieces. Indeed for each degree 2 let Va be the space of polynomials in the variables 2, y, Z which are homogeneous of degree d (for example
13

---

(zr, zy. ....... Letting ((3) act on R$ by linear change of variable, it acts on each Va since such change of variable does not change the degree of homogeneity. Now let Wa C Va be the subspace of harmonic polynomials, those polynomials that satisfy 8ºp 8x + 8y² + az
Vi = span V span y. 2), V22 = span(2²
Then Wa is itself (3) invariant since the Laplace operator is iuation-invariane making Wa subrepresentation of Va. It a (not immediately obvious) fact that each Wa is in fact an irreducible representation: it has no subrepresentations of its own other than itself and the zero subspace. Further, we interpret the harmonic polynomials in Wa as functions on the unit sphere 8 C R' we can view Wa as space of functions the sphere. It then turns out that the spaces Wa are linearly independent of eaceh other, and that their sum ① d2o Wa is dense in any resaonable space of functions on the sphere (the space of continuous functions, the space of smooth functions, or the space of square-intege iquare-integrable functions, each with its respective notion of convergence). In other words, we have morally decomposed the space of functions on S as the sum of ((3) -invariant subrepresentations, cach of which is irreducible.
8'p
8"p
Decomposing representation into its irreducible constituents is major goal of representation theory (as well as a computational goal of this paper). When this is possible we can understand a general representations V of group G by first enumerating its irreducible representations and then counting how many times each irreducible occurs as summand in V, known as the (this is very much analogous to understanding general positive integers via their prime factorization here the irreducible representations play the role of the prime numbers).
Reductive groups are particularly amenable to this approach because their representations indeed do decompose as direct sums in this fashion. It is fact that a linear Lie group G' is reductive if and only if each of its finite-dimensional representations is isomorphic to a direct sum of irreducible representations.
Before we delve further into representation theory we need brief detour into the structure theory of Lie groups
A.1.3 Lie algebras
As Lie groups are manifolds, understanding their representations theory usually leads to advanced geometrical and topological results. We will now describe how these difficult questions can be transformed into simpler linear algebra questions using the Lie algebras associated with Lie groups
Lie algebra vector space g over a field F together with bilinear map [+++] : g x g ++ called
the Lie bracket, satisfying the following axioms:
The Lie bracket is bilinear, that is for all x, v. E g and a, 3 E F, [aa + By, =]
[xx 2] + B[y, x] and az + 32] = a(z, y] + B|z, 2].
The Lie bracket is antisymmetric, that is for all x, y g. [z, vl =[y,.
The Lie bracket satisfies the Jacobi identity, that is for all x, y, E [x,, [v, a|] + [vs, [2, a]]]
[z, [z, w]]
O.
Lie algebras can alternatively be seen as the tangent space of a Lie group to the identity:
Theorem A.3. Let G be a Lie group viewed smooth manifold, the tangent space T,G is Lie algebra called the Lie algebra of G noted g. Moreover in the case of connected complex Lie groups, the category of representation of G, Rep(G), is equivalent to the category of representation of g. Rep(g).
This theorem is at the heart of the representation theory of Lie groups As Lie algebra are vector spaces, the functorial equivalence between the two categories of representation translates geometrical and topological questions into simpler linear algebra questions.
A classification of Lie algebras was carried out by Cartan and later Gelfand work. In the case of complex Lie algebras, this classification is complete. Two major classes of Lie algebras emerged from Cartan's 's work, semi-simple Lie algebras and nilpotent Lie algebras.
Definition A.4 (Central series). A central series of a Lie algebra g is a sequence of ideals (g. keN such that go = g and gi+1 = [g:, g).
Definition A.5. Lie algebra is called semi-simple it does not contain any non-trivial ideals.
Lie algebra is called nilpotent if its central series terminates to zero.
14

---

An important result of this classification is that any Lie algebras over an algebrically closed field can be decomposed into semisimple parts and a nilpotent part. We call this the Jordan decomposition. Therefore, understanding arbitrary representations of Lie algebra boils down to understanding representation of both semi-simple and nilpotent algebras.
Most reductive Lie groups of interest have a semi semi-simple Lie algebra as associated Lie algebra. These algebras are also the one for which the representation theory is the best understood.
The study of Lie algebras is decomposed two subfields. The structure theory studies the classifica- tion of Lie algebras in terms of their eigenspaces (as vector spaces). The representation theory builds on the structure theory to classify representations of semi-simple Lie algebras. We will give a very short introduction both of them.
A.1.4 Structure theory of semi-simple Lie algebras
Let be a semi-simple complex I Lie algebras. Let h be one maximal set of mutually commutative element of g. Maximal means here that no other such set contains strictly more element than h. We call h the Cartan algebra of g. As elements of h are mutually commutative, they share the same eigenspaces.
Therefore one can decompose g in terms of eigenspaces of elements of h. Let a E h" be a function in the dual of the Cartan algebra such that Vh, h, a(h;)) a, with a; the eigenvalue associated to h4. We call this linear operator the roots of g. Let be the space of all this functions, called the root space. One therefore has the the following decomposition of the Lie algebra;
g = h Da
(24)
aER
The structure theory of semi-simple complex Lie algebras is the study of the root space R and the associated eigenspaces Ja. One can clasify complex semi-simple Lie algebras based on their type of root spaces. For real semi-simple Lie algebras, the structure theory essential reduces down to the complex case with a few extra steps.
A.1.5 Representation theory of complex semi-simple Lie algebras and Lie groups
The representation theory of complex reductive Lie algebras is completely understood. Every imite-dimensional representation is the direct sum irreducible representations. Every irreducible representation is uniquely determined by highest weight A, which is an element of the dual of the Cartan algebra h. The dimension of each irreducible representation can be computed using the Weyl dimension formula.
dim IAERR (A p.a)
(25)
II (p)
where Rt is subset of the root space R, called the positive roots, and p is the half sum of the positive roots. Obtaining explicitly a basis for each representation is much harder. Gelfand and Tselin proposed basis for the general case of (N) groups. This basis was recently generalized to other complex semi-simple algebras Moley 1999).
A.1.6 Representations of real semi-simple Lie groups
The representation theory of real semi-simple Lie groups can be elucidated by examining the complex semi-simple case through the application of the so-called Weyl unitary trick. This technique estab- lishes connection between the representations of real Lie groups and those of the complexification of their universal covering groups
Definition A.6. Given Lie algebra g. there exists unique simply connected group & possessing g as its Lie algebra. A group homomorphism vp exists from this unique simply connected Lie group to any group G sharing the same Lie algebra. We refer to as the universal covering Lie group, and it unique up to isomorphism.
An essential observation is that the irreducible representations of G form subset of the irreducible representations of G. Understanding the irreducible representations of & suffices. Take the case of
15

---

the su(2) Lie algebra, for instance. The universal covering group is SU(2), corresponding to the special unitary two-by-two matrices. The group (3) of special orthogonal matrices also has su(2) as its Lie algebra. Consequently, the irreducible representations of SO(3) are subset of those of su(2). Representations of su(2) can indexed by a half-integer, known as the quantum number j. The integer representations are also irreducible representations of SO(3), with non-integer ones being referred to as the spin representations.
Since the representation theory of complex Lie groups is well understood, an optimal approach to understanding the representation theory of a real Lie group involves studying the complexification of its universal cover.
Definition A.7. For a universal covering Lie group G with Lie algebra g. the universal covering Lie group Gc, having the associated Lie algebra gc = g @ C, is termed the complexification of G.
The group of interest G, its universal cover G and the complexification of the universal cover Gc, all share the similar irreducible representations. Therefore one obtains the following chain inclusion of the respective representations of the groups.
Rep(G) C Rep(CG) C. Rep(Gc)
(26)
Let K be the maximal compact subgroup of GO and pA a finite dimensional irreducible representation of K, with \ as its highest weight. It is possible analytically continue Ps into irreducible Ge representation of Gg, denoted as _x=, and also into the analytical conjugate P> GG Every finite-
dimensional representation of &c can be constructed as the tensor product form PX & px Due
Ge
to the fact that finite-diments dimensional representations of the universal covers and complexification of semi-s semi-simple Lie group are isomorphic, one can express:
P(A.)) == P(A.))
=(
(27)
As universal covering groups contain more representations than the original cover, this mapping is not an isomorphism for all lambda. To illustrate this, consider the example of the Lorentz group 30,,,
The universal cover of SO(1,3) is SL(2, C), and the maximal compact subgroup of SL(2, C) is (U) The irreducible representations of SU(2) are indexed by 1 N/2 and correspond to the Wigner D matrices D' (g),) E SU(2). The SU( group elements are parameterized by Buler angles that can be analytically continued to SL(2, C*) as follows:
= Ø + in, 3 0 + ic, 7 i
(28)
[0, 2m), E [0, m],
m,, R
This enables the construction of the fundamental representation of SL(2, C). resulting in representa- tions of the Lorentz group corresponding the product of Wigner D matrices:
(a, 68 (122(a, (0, 3,r)
(29)
The irreducible representations of (0,,, 3) are indexed by pair of integers (1, corresponding to the associated (U) representations the tensor product.
A.1.7 Gefland-Tsee Basis
The enumeration of finite-dimensional representations of complex Lie groups has been well under- stood. However, the derivation of explicit matrix representations in specific bases and the tensor product decomposition of representations within these bases remain challenging and subject to ongoing research. The Gelfand-Tselin basis provides explicit structure for irreducible representa- tions, initially designed for the SU SUN group Gelfand and Tsetlin (1950); Alex et al. (2011). and subsequently generalized to encompass other classical Lie groups Molev (1999).
Central to the concept of GT patterns is the application of branching rules, whereby representations of a larger group are constructed by enumerating induced representations on its subgroups. Consider, for instance, the SU(N) case where irreducible representations are characterized by the highest weight vector X, vector of length N terminated by a zero, À (Ai...., AN-1, 0). This highest weight
16

---

vector may spawn several induced representationn of the subgroup SU 1), each described by N vectors. This process can be iteratively executed in a descending chain:
SU(N) - SU - 1) + ... (2) - (1)
(30)
This chain commences with the group of interest and culminates with the smallest subgroup. From this branching chain, triangular arrays known as GTT-pattenns can be constructed, with each row corresponding to vector of representation ranging from SU (N) to (()):
AN11
AN.
AN.N-1 0 AN-1.N1
AN-11
AN-12
(31)
X21
A22
An
Adjacent rows within these patterns must adhere to a so-called "snake rule" to form an admissible GT pattern:
Ar.1 A6-1,1 AR. > A .... A AR-1R-1 > AR.
(32)
The dimension of a specific irreducible representation corresponds to the count of such admissible arrays, with the top row representing the highest weight vector. Representations sharing the same highest weight vectors interrelate through ladder operators. One key attribute of GT patterns is that the action of ladder operators on a GT pattern is analytically known, enabling the construction of the matrix representation for all representations using only the highest weight representation. Furthermore, constructing the highest weight matrix representation equals solving línear system when the ladder operators are known analytically. Extending this methodology to other classical Lie groups (such as (n), Sp(n). SL(n)) presents challenges as the ladder operators' entries become increasingly complex and recursive.
A.2 Example of one particle basis for 0(3) and SO(1,3) groups
In the paper, the one- me--paticle basis ø is left abstract, as it depends on the specific type of input point cloud and the group under consideration. We now describe two concrete examples, one involving isometry the other Lorentz equivariance.
A.2.1 Isometry equivariance
In the first example we consider a point cloud comprised of atoms, for example representing a molecule. Here, the point states are denoted by z; = (r:, Z₁) E R E, where r; corresponds to the position in 3D and Z,, its atomic number, a categorical variable. The one-partich. basis must form complete basis for R Z., which is both rotationally equivariant and translationally invariant. The translation typically handled by referring to the atom's state in relation a central value or another point. For simplicity, we can assume r; the particle's position relative to the point cloud's centre of mass, ensuring translational invariance.
The space R$ can be represented as the product I x S2, with S2 being the sphere. As such, we choose a basis for IR such as Bessel functions, Chebyshev polynomials Fourier basis, denoted as Rn For S we utilise the spherical harmonics Yim, which form a basis of functions in the sphere. Consequently, the particle basis can be expanded as:
@ntm (I,) = Rr, (r.) (f.f(Zz))
(33)
where r; corresponds to the norm of r; and A; r: r
The case of the Lorentz group is more complex than the (3) case. Consider point cloud made up of elementary particles, represented by their 4-momenta x; = (Fg, P:) E TR1. Although the 4-momenta is vector, the natural metric in this space is not the Fuclidean metric but rather a pseudo metric known as the Minkowski metric. This pseudo-metric encapsulates the unique role that time plays in special relativity. In natural units, the pseudo norm 17 of 4-vector u = (wg, U1 U2 u3) is defined as
nuu, u) 2i 2f 2 uz
(34)
To construct the one-particle basis, it is necessary to expand a four-vector terms of the basis of representations of the Lorentz groups As outlined in Appendix A.1.6 inreuucble representations
17

---

of the Lorentz group are characterised by tuple of integers (l, The v-ectosss correspond to the fundamental, irreducible representation, namely the (1,1) representation. Analogous structures to the spherical harmonics can be obtained by examining the symmetric powers of the irreducible representations. These are analogous harmonic polynomials on Minkowski space, much as spherical harmonics are harmonic polynomials on the sphere 5². We thus define the Minkowski spherical harmonics as the following symmetric tensor product,
Yum (p.) = Sym' P.
(35)
These spherical harmonics form harmonic polynomial basis on Minkowski space. Therefore, one can create the one- one-particle basis in manner akin to the 0(3) case, as follows:
@llm (p,) = R., ( (mm())
(36)
Where pæ = 7p,, p:). and p:: P: is zero due to the non- non-positivvness of the Minkowski norm.
P._ with being a small constant that prevents divergence when Pite'
A.3 Extended Related Work
Convolutional Neural Networks (CNNs), which translation equivariance, initiated the utilization of data symmetry in machine learning architectures. Throughout time, CNNs have been extended to include other symmetries as well. Central to all these generalizations the group averaging operation,
(x) /
/gec f(g z)dg.
(37)
Where æ denotes the input signal or feature, f is the convolution kernel, G represents the group of interest, and dg is invariant measure on G. This transformation is essential, as it converts any convolution into a group invariant convolution. The feasibility of this approach largely depends on the computational simplicity of the integral. The most straightforward instance occurs for finite groups, as explained by G-convolutions (Cohen and Welling, 2016b), where the integral simplifies to a sum,
Avg(f)(x) = Σ f(g-x)) x).
(38)
gEG
In the context of G being a compact group, the invariant measure dg is unique, referred to as the Haar measure of the group, allowing the integral to be computed numerically, c.g., grid. Steerable CNNs (Cohen and Welling 2016a). LicConv Finzi et al. (2020) and Spherical CNNs (Cohen er al. 2018) extended CNNs to (2) and O(3) equivariance, respectively. A general theory for convolution on any compact group and symmetric space was developed by Kondor and Trivedi (2018), and further extended to equivariant convolutions on Riemannian manifolds by Weiler et al. (2021. However, this approach has several limitations,
The direct computation of the integral can be numerically unstable and inefficient, even for relatively small groups like O(3).
In the case of non-compact groups, a unique invariant measure is absent, and the integral diverges.
Across these methods, the convolution kemel is usually constrained to a two-body operator
In the case of compact groups, the integral over the group may be calculated via an alternative means. There exists linear operator, called the Clebsch Gordan operator C, such that,
Avg(J) (z))
(39)
Therefore, the complex integral over the group becomes a linear operation. The form of this operator depends on the basis in which ff is expended. the case of G (3)) and if this basis is carefully chosen, the entries of this operator are known analytically. This approach is numerically stable and
more efficient and was taken by numerous works, including ACE Dusson el al. (2022), Cormorant Anderson et al. (2019). NequlP Batzner el al. (2022), or MACE Batatia ef al. (2022b). The central aim of our work is to show that this approach can also be generalized all reductive Lie groups, non-compact ones, and provide tools to do
18

---

A.4 Proof of 7
This statement follows closely the arguments by Dusson er al.
(2022)
Drautz 2020) and others.
ΣΣ wk
DR
(1
Σ
we Σ
Il
dx, (TJ.)
J1...an
Σ Wk
Dke (xz)
6=1
Wk II
A
Σ
W// Ak-
A.5 Custom notation and indexing
We briefly contrast our notation for Gordan coefficients (TO) with the standard notation. By means of example, consider the group 5O(3) in which case the Clebsch-Gordan equations are written
Σ G 29
CLM Camilamm Ping mi (1) mym my (g) Ep'sse
(40)
In this setting, our index a simply enumerates all possible such coefficients. One can often assign natural meaning to this index, e.g., for the group 50(3) it is given by the pair of angular quantum numbers (2). Specifically, in this case, we obtain
C m2mmmaa
Ca.LM LM
Immilama 0,
CLM
if (11,12)
(41)
otherwise,
where CHM, fammaam are the Clebsch-Gordan coefficients in the classical notation. Thus, the additional index a is not really required in the case of SO(3), nor our other main example, SO(1, 3). Our
notation is still useful to organize the computations of equivariant models, especially when additional channels are present, which is usually the case. Moreover, it allows for easy generalization to other groups where such simple identification is not possible (Steinberg 1961.
A.6 Equivariance of G-cluster expansion
The equivariance of the G-cluster expansion easily verified by applying a transformation g to the input.
BK gg
Σ
Ch
K
A
og
Σ ca.K EII &k (g)
Σ pa.K II [Pmnne(s))
AN
(42)
h
Σ
(9)
Ak
PKK (K)
19

---

A.7 Completeness of the basis and Universality of G-MACE
We explain in which sense the basis B is a complete basis, and briefly sketch how to prove this claim. The argument is contained almost entirely in (Dusson et al. (202) and only requires a single modification, namely Step below, using a classical argument from representation theory. We will therefore give only very brief summary and explain that necessary change.
We start with an arbitrary equivariant property ΦK embedded in V where have representation, i.e. the actual target property is & is then given as a linear mapping from V to Z. For technical reasons, we require that only finitely many entries & x may be non-zero, but this is consistent with common usage. For example, if G 0()) and if Φ is scalar, then @ Q while all other & 0. If + is covariant vector, then & YY is non-zero if and only if L = 1; and so forth. For other groups, the labeling may differ but the principle remains the same.
I. Convergence of the cluster expansion. The first step in our parameterisation is to approximate V in terms of a truncated many-body expansion (4). It highly applicatioo-dependeer on how fast this expansion converges. Rigorous results in this direction in the context of learning interatomic potentials can be found in (Bachmayr et al.) 2021; Thomas et al.) 2022). A generic statement can be made if the number of input particles is limited by upper bound, in which case the expansion becomes exact for a finite N. This case leads to uniform density result stated in Theorem 4.1 We adopt this setting for the time being and return to the pointwise convergence setting below.
In the uniform convergence setting we also require that the domain compact.
Throughout the remainder of this section we may therefore assume that an N can be chosen as well as smooth components cp(n) such that the resulting model + V.N approximates Φy to within a target accuracy
T V.N (x)
(x))) Sc
Vz E msets([2).
2. The density of the embedding. As already stated in the main text, if the components yex)) are smooth, and the embedding {dh is dense in the space of one-particle functions (5) then
(n) follows that the 4k can be expanded in terms of the tensor product basis du == 8 oR. to within
arbitrary accuracy. The precise statement the following standard result of approximation theory:
span (be) &a are dense in C(S2), then span (bx) } are dense in C(S2).. That is, for any >> 0, there
(n) exist approximants Pk such that
11 (n)
PK 11x C.
(n) 3. The density of the symmettried basis. The next and crucial step is show that, if the (k equivariant, then the Pk may be chosen equivariant as well without loss of accuracy. If the group G is compact then the representations p can be chosen unitary (Broecker) 1985). In that case, the argument from Dusson et al., 2022) can be used almost verbatim: let
n (x) = [ p(g))
[p()) (m) (gx) Hdgg)
where H is the normalized Haar measure then p(n) is equivariant by construction and l (z) p(")
1 p(g) (p()) -10)) (gx)) H(dg)
< / (pli) (gar) -p'" (gaz) H(dg)
< m1((m)) - -i Hdg)) S
If the group is not compact, then can apply "Weyl's Unitary Trick" (see (Bourbaki 1989), Ch. 3);
first, one complexifies the group (if it is real) and then constructs a maximal compact subgroup Ke of the complexification. This new group K will have the same representation as G and because is compact, that representation may again be chosen unitary. Therefore, symmetrizing (()) with
20

---

respect to Ke results in an approximant that is not only equivariant w.r.t. Ke but also equivariant w.I.t. G.
The density of the basis a As the last step, one can readily observe that the symmetrization and cluster expansion steps can be exchanged. l.e. first symmetrizing and then employing the steps 7 result in the same model. Letting - 0 in the foregoing argument while fixing the number of particles # results in all errors vanishing. Note that this will in particular require taking N co.
Pointwise convergence. To obtain density in the sense pointwise convergence we first introduce the canonical cluster expansion without self-interacting terms
xx(x)=
Σ
(n) "K (T)) ..... F.,).
The difference here is that the summation is only over genuine sub-clusters. Because of this restriction, series finite for all multi-set inputs æ. other words, converges in the pointwise sense.
One can easily see that can be chosen (explicitly) to make this expansion exact. After truncating (n) the expansion at finite S N and then expanding the potentials oj one exactly transform the canonical cluster expansion into the self-interacting cluster expansion. This procedure is detailed in Drautz 2020)).
(Dusson el al.) 2022
The arguments up this point establish the claimed universality for the linear ACE model. The corresponding universality of the TRACE model follows immediately from (Darby et al. 2022)). Since a single layer of the MACE model is a TRACE model, this completes the proof of Theorem 4.1
A.8 Product of groups
Let G1 and G2 be two reductive Lie groups (or finite groups). Let A₁ and A2 be the two structure constants of 11 and G2, (dp1, p1) and (dp2, P2) their continuous and discrete generators. One can define a representation of the direct product group G1 x G2 as
p p:= (A1 A2, nIne, (X.) & 12 + 1 & dp2(X,))ss (p(h)) & 1 + e oot,))),,ee, (43)
The following essential property holds: if and p2 are irreducible representations of G1 and G2 then A1 @ P2 is irreducible representation of G1 x G2. Moreover, for reducible Lie groups, all the irreps of G1 x G2 are of this form. Therefore, one can construct all the irreps of G1 x G2 this way. It is of particular interest in the case of equivariant message passing networks on points clouds, where the group of interest is G x Sn.
Let give non-trivial application of lie-nn for computing invariants for the product group of (3) x Sz. We consider a set of three vectors,
Qx
0xz 0gy
a
ag
b
ag
=
00=_
[Oz
given vector belong to the product representation the / vector of the 0(3) group and the natural representation of Sg. In lie-nn, we can define the product representation as follows,
rep lie ggggoruuuroduuce ( lie finite Sn_natural (3), lie irreps 03 p=-1))
If one wants know the unique permutation invariant vector that one can construct from the three vectors a, and one can use the following code,
qs lie infer change_ __o__oo_______ of_oofaaiia. basis
lie grodp_product (lic. finite. Sn_trivial (3), lic. irreps 03(111, p=-1)) ==-1)), rep,
As expected, the output is the sum of the three vectors, bb+ corresponding to the only permutation invariant vector that can be constructed from the three vectors.
Consider now the case where we want to extract a permutation invariant scalar from the set of products of two vectors. In this case, can use the following code,
21

---

qs lie infer infer_change change of ( lie group_produet (lie. finite. Sn Sn_trivial (3), lie. lie productt (rep, rep),
p=1)),
In this code, we first construct the tensor product representation and then seek the permutation invariant scalar in it. The result two distinct scalars,
a² + a. + a + 6% + by + b c + + C2 b + a,Cr aybs + aacy + azd= + a.c= + c2 + bocy + b,cN
Finally we consider case that would very hard to do by hand. It is the case extracting permutation invariant = vector for this tensor product of the original set of three vectors.
qs = lie infer _aarrrehaa______ of basis lic www.g-ruuupppo product (lie. finite. Sn trivill (3), lic iirepsrppp... 03(103(1))) p=1)), lic tensor prrsuctprouuct(ee (rep, rep)
The computation tells us that there exists two such vectors,
2a
V bb
V2200
22.ay
bb
C22zy
26a
/oa
/6a
V6b2
Vob.
v662
voc.
Voe
Vic
01
+
6
6
6
/20 02
v 22
b
/2000
Cz
V2
2
V
V
V2e²
VIc
2
2
2
2
2
b, C2 a.b₂ O,Cr b,C₂ b,Cz
0g by - AgCD a,b2 OyC C2 drCy C2
/3a,
-3
2v 2/3a,b, 2 2v /3a..b v3a,C, + v3b 3
2v3by
v3b.e.
1/2
3
3
b, CyCz a,b, O2Cy byC2 byCy
ag b + Cn a az a₂C₂ + bnCo bzC₂
A.9 Symmetric Tensor products
The permutation group is an important concept in the context of tensor products. It can be useful to focus on a subset the full tensor product space that exhibits certain permutation equivariance, For example, the spherical harmonics are defined as the permutationn-nvariiat part of a tensor product.
The symmetric tensor product can be thought of change of basis, or projector, from the tensor product to the symmetric part of the tensor product. In the case of tensor product of correlation order four we have,
S. = BusijkErr Fk
(44)
where B is the change of basis that satisfies:
Buzikl = Byio Ya E S4
(45)
propose lie-nn a new algorithm used to calculate B. The Symmetric Tensor Product calculated using tree structure, starting at the leaves and progressing towards the trunk. The leaves are the basis of the individual indices, and they are combined and constrained each step to impose symmetry.
A.10 Computing the irreps from input representations
For some groups, the computation of the generators X can become very involved task. However in most applications, the data itself is already given in a form of representation. One approach proposed by (Finzi et al. (202T) is to not work in the space of irreps but the space of polynomials of the input representation. This approach has the advantage of requiring little previous knowledge of
22

---

the group. However it is also much less efficient than using inreps. One alternative way is to consider polynomials of the input representation, that are reducible and then compute the block diagonalisation to project down to irreps subspace. One can then work directly polynomials in this subspace and compute Seebshh-Gordan coefficients numerically. We provide routines 1ie e ennn to earry out these operations from any given input representation.
A.11
Details of numerical experiments
A.11.1 Jet Tagging
The dataset (Butter el al.) 2019) was generated using Pythia, Delphes, and FastJet (using cuts for the jet's kinematics on An 2. R 0.8) to simulate the response of the ATLAS detector at the Large Hadron Collider (LHC). The dataset is released under the "Creative Commons Attribution 4.0" license. The entire dataset contains millions jets with 60/20/20 for training, validation, and testing balanced splits.
Dataset
Model The model uses 3 layers of the G-MACE architecture to generate the Lorentz group equivariant representation of each jet. For the particle basis, we use product of radial features on the Minkowski distances, and S0(1,3) spherical harmonics. The radial features are computing by passing logarithmic radial basis in (Bogatskiy al. 2022) into [64, 64, 64, 512] MLP using SiLU nonlincarities on the outputs of the hidden layers. The internal representations used are (0, 0) and (1, 1). We use 72 channels for each representation. For the embedding, and readout out, we use similar achitectures to LorentzNet.
Training Models were trained on an NVIDIA A100 GPU in single GPU training. Typical training time for the dataset is up to 72 hours. Models were trained with AMSGrad variant of Adam, with default parameters of 31 = 0.9, 82 = 0.999, and = 10-8 We used a learning rate of 0.0035 and batch size of 64. The model and followed by a phase of cosine annealing LR scheduling.
trained for 80 epochs with 2 epochs of linear learning rate warmup
A.11.2 3D shape recognition
Dataset ModelNet10 (Wu et al. 2015) is synthetic 3D object point clouds dataset containing 4,899 pre-aligned shapes from 10 categories. The dataset is split into 3,991 (80%) shapes for training and 908 (20%) shapes for testing. We were unable to find license.
Model The model uses three-layer encoder architecture following the PointNet++ one. We use an encoder of the full point cloud into sub-point clouds sizes [1024, 256, 128). Each PointNet layer maps point cloud of size No to one of size No+1, We compute the node features as the sum of the PointNet output and the MACE output,
h (1+1) (zintNet(zyz20)) (00) + (A (1))))
(46)
Training Models were trained on an NVIDIA A100 GPU in single GPU training. The typical training time for the dataset is up to 12 hours. Models were trained with the AMSGrad variant of Adam, with default parameters of 81 = 0.9, B3 = 0.999, and = 10
A.12 Limitations and Future Work
The spectrum of potential applications of the present method is very large. In this paper, focus a subset of applications that have known benchmarks and baselines. A broader range of groups is implemented in the lie-nn library. Future work should focus applying this architecture to tasks with comainn-specite knowledge.
23

---

A.13
Computational cost
A.13.1 Clebsch Gordan generation
Generation time The Clebsch Gordan generation time depends on the size of the representations. In Table we plot the generation time as a function of the size of the representations for different groups. The generation time is found to range from a matter of milliseconds for smaller representations to a few seconds for larger ones. The generation of CGs constitutes preprocessing step, separate from the model inference computations. Therfore, the results can be stored for subsequent use, ensuring that this phase does affect the model's overall performance.
SU(2)
SU(3)
SU(4)
0.200
25
25
0.175
20
20
0.150
Time (s)
0.125
15
15
0.100
0.075
10
10
0.050
5-
0.025
25
50
75
100
1000
2000
3000
500 1000
1500 2000
Representations
Representatiinss
Representations
Figure 3: Generation time in seconds of Clebsch- Gordcn coefficients as a function of the size of the representations for different groups. The representation size is computed as the product of the three representation sizes involved the Clebsch-Gordan.
Sparsity One distinguishing feature of the Clebsch-Gordan coefficient is their sparsity. This sparsity comes from the selection rule, which is unique for each group. These selection rules also give rise to sparsity structure that can exploited to achieve the best efficiency. In Figure [ we compare the sparsity percentage as function of the size of the representations for different groups
SU(2)
SU(3)
SU(4)
0.225
0.325
0.18
0.303
8 0.275
Sparsity
0.290
0.12
0.10
0.300
0.08
0.075
0.05
0.025
0.02
Resrrssntaaiion
Representatione
Figure 4: Sparsity in the percentage of non-zero entries of Clebsch-Gordat coefficients as function of the size of the representations for different groups. The representations size is computed as the product of the three representation sizes involved in the Clebsch-Gordan.
A.13.2 G-MACE computational cost
The computational bottleneck of traditional equivariant MPNNs is the equivariant tensor product on the edges of the graph used to form the message. In G-MACE the edge operation is the pooling operation to compute the features Ag. Correlations A₃ are computed through the tensor product of the product basis, an operation that is carried out on nodes. In typical models, the correlations are the bottleneck. Since the number of nodes is orders of magnitudes smaller than the number of edges a significant computational advantage to organize the computation in this way. We return to reviewing the cost of the product basis below.
in most applications we are envisioning,
24

---

Jnse 11 Ims 12,21 Juas 13,31
3.0010
3.0005
Corraistion
Figure 5: Inference time for a single jet for a two layer LorentZMACE model as a function of the correlation order in the product basis and the maximal angular resolution. Timing made on a Nvidia A100 and averaged over a 100 runs.
Obtaining the equivariant basis BK involves simply linear operation. The Clebsch-Gordee coef- ficients are very sparse, with a well defined structural sparsity, which can be easily exploited for constructing highly efficient low level code on both CPUs and GPUs.
Thus, return the product basis A which is normally the computational bottleneck. It has structural sparsity due to its symmetry under permuting the & tuples and due to the structural sparsity in the Clebsch-Gordan coefficients. Despite that sparsity it was shown Kaliuzhnyi and Ortner (022)) that in theory the product basis can be evaluated recursively with O operations per feature, however, this requires effective use of sparse tensor formats which challenging to optimize for GPU architectures since a naive implemention relies on random memory access patterns. Instead, the current MACE code employs similar recursive evaluation algorithm described in (Batatia al. (2022a), which employs full tensor formats, but avoids the random memory access. For the most common case of 3-correlation models we estimate that this code reaches within factor 3-5 of the optimal performance of hypothetical sparse tensor implementation.
25


Reviews:
[{"review_id": "ZEwgDR2Zpn", "reviewer": "Reviewer_jt5w", "summary": "This paper proposes a framework for building equivariant neural networks on reductive lie groups. The proposed method first constructs a linear model for multi-set functions which is then symmetrised to generate a complete basis of equivariant multi-set functions. The model is also extended to a multi-layer architecture by a message-passing scheme. ", "questions": "* The computations in Eqs. (7) together with (16) seem to be expensive. How these can be done in practice ?\n* It would be interesting to show the impact of higher order messages on the performance of G-MACE in terms of accuracy and computation time. I didn't find this study in the paper and supplemental material. ", "limitations": "Limitations are discussed in the supplemental material.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The idea of generalizing ACE and MACE frameworks to arbitrary reductive Lie groups is new. \n* Proofs are given in the supplemental material to support theoretical claims in the paper. \n* A library is provided for developing G-equivariant neural networks.  \n", "weaknesses": "* The paper has some minor errors in writting. For instance, Table 2 caption should be on top of the table. Tables (e.g., Tabs. (2) and (3)) that report experimental results should be mentioned in the text. \n* Lack of comparison against state-of-the-art methods for validating the effectiveness of the proposed method on 3D shape  recognition.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper. We appreciate that you find our work\nnovel and our open-source library interesting to the community.\nBelow we respond to your questions and suggestions to further improve\nthe paper.\n\n## Formatting of tables\n\n> Table 2 caption should be on top of the table. Tables (e.g., Tabs. (2)\n> and (3)) that report experimental results should be mentioned in the\n> text.\n\nWe thank you for spotting these problems. We have fixed the caption and\nwe are now cross-referencing the tables in the text.\n\n## Baseline for 3D shape recognition\n\n> Lack of comparison against state-of-the-art methods for validating the\n> effectiveness of the proposed method on 3D shape recognition.\n\nWe now compare to other state-of-the-art models for 3D shape recognition\nin Table 3. We have selected the best models we could find that use\npoint cloud or voxel representations. If we are missing any, we would be\nhappy to reference it and, if appropriate, add it to the benchmark. Here is a copy of this updated table,\n\n| Architecture |  **PointMACE** (ours) | **PointNet**  |  **PointNet ++**    | **KCN**  | **SO-Net** | **LP-3DCNN**|\n| --- | --- | --- | --- | --- | --- | --- |\n| Accuracy | **96.1** | 94.2 |  95.0 | 94.4 | 95.5 | 94.4 |\n| Representation | Point Cloud | Point cloud | Point cloud | Point Cloud | Point Cloud | Voxel grid |\n\nNote that we compare only to other point cloud methods. The best current\nmodel we know of uses additional information in images at different\nangles, achieving an accuracy of about $98$ %. We believe that this\nomission is fair.\n\n## Efficient implementation of Equivariant Product Basis \n\n> The computations in Eqs. (7) together with (16) seem to be expensive.\n> How these can be done in practice ?\n\nIn general, it depends on hyperparameter choices, but especially for\nlarger models the product basis $\\textbf{A}_{\\textbf{k}}$ is indeed the\ntheoretical and practical bottleneck. *In theory* the product basis can\nbe computed at O(1) cost per feature (see arXiv:2202.04140, in fact it\nis proven that the cost is asymptotically two operations per feature), but\nthe current G-MACE implementation does not leverage this algorithm yet as\nit requires efficient use of sparse tensors, which is difficult on GPU\narchitectures as a naive implementation would require random memory\naccess.\n\nAt present, the G-MACE code uses a highly performant dense tensor format\nimplementation, which we believe achieves within a factor of 3-5 of the\nhypothetical, optimal performance of a sparse code, in the low to moderate correlation order regime that is relevant in applications.\nThis renders its cost very attractive. The code published with this article generalizes\nthis efficient implementation used in $O(3)$-MACE to handle contractions\nin any group.\n\nWe added a discussion to the appendix of the paper and a brief sentence\nin Sec 4.1. to reference that discussion.\n\n## Impact of higher order in accuracy and speed\n\n> The impact of higher order messages on the performance of G-MACE in\n> terms of accuracy and computation time.\n\nWe refer to the previous question: a theoretical optimal evaluation\nscheme requires only O(1) operations per feature, independent\nof the correlation order. In practice, the current implementation's cost depends on correlation order. Since all our models\nseem to be optimal in the range of correlation order 2, 3, 4 (typically\n3) and since we are actively working towards the implementation of a\na quasi-optimal algorithm, we prefer not to emphasize this too much in the\npaper. In the Appendix A.13.2, we have added an analysis of the computational cost of\nG-MACE as a function of the correlation order in the case of the Lorentz\ngroup.  Please find the figures of this section in the pdf attached to the general response.\n\nWhile in theory, higher correlation means more expressiveness, the relationship between accuracy and the correlation order depends on the dataset. Please find here a table summarizing the accuracy and computational time for different correlation orders on the jet tagging dataset,\n\n|  **Correlation** | 1 | 2 |  3    |  4|\n| --- | --- | --- | --- | --- |\n| **Accuracy** | 93.6 | **94.2** |  **94.2** | **94.2** |\n| **Timings** (ms / jet) | **0.35** | 0.58 | 0.71 | 0.93|\n\nIn the case of the jet dataset, we see significant improvement going from correlation order one to two and then saturation. In molecular applications, correlation order three is routinely used. It is essential to note that two layers of correlation order three at each layer, give rise to functions of correlation 12. The convergence of the many-body expansion highly depends on data. Low body order is likely enough if the physics behind the data is close to a mean-field limit."}, {"review_id": "PyaWSE7hoG", "reviewer": "Reviewer_Mk5u", "summary": "The authors generalize MACE, a point cloud network that uses higher-order interactions via tensor products of basis expansions of the features, to being equivariant to arbitrary reductive Lie groups. The paper shows that this setup inherits universality properties from MACE. A generic method to compute the a basis for the equivariant linear maps between tensor products of the representations is implemented.\nWhile the paper is mostly theoretical, the authors show strong performance on a SO(1, 3) equivariant task and a point cloud task.", "questions": "- It would be helpful to include/repeat the definition of B used in equation (20).", "limitations": "The authors should clarify better that not all Lie groups of interest are reductive.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The proposed method is an elegant generalization of a popular prior method\n- The paper is well-written and mostly easy to read.\n- The code is included\n- The proposed method could be a useful turn-key equivariance solution for researchers with a niche symmetry problem.", "weaknesses": "- Figure 1 of the paper appears to suggest that the method works on E(3), but this can't be a reductive group, as the 4D homogenous representation is not decomposable as a sum of irreducibles (contradicting lines 67-68). I suppose the authors mean to write O(3) and treat the translation by canonicalization, but they should clarity that.\n- The point could experiment should include more baselines.\n- As the key contribution of the paper mostly lies in implementing the necessary computations for generic reductive Lie groups (sec 5), it would make sense to allocate more space to that and less to the prior sections, which are mostly already covered in prior works. Alternatively / in addition, it would be good to give more background on the material in section 5 in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your positive review and excellent comments. We\nappreciate that you find our work interesting and well written. Below we\nrespond to your questions and suggestions.\n\n## Clarification on $E(3)$\n\n> Figure 1 of the paper appears to suggest that the method works on\n> E(3), but this can't be a reductive group, as the 4D homogenous\n> representation is not decomposable as a sum of irreducibles\n> (contradicting lines 67-68). I suppose the authors mean to write O(3)\n> and treat the translation by canonicalization, but they should clarity\n> that.\n\nWe thank you for spotting this typo. As you said, the translation\ngroup is not reductive, and we usually use canonicalization to\nconstruct invariants. Now we refer to the $O(3)$ group in the figure.\nOne interesting aspect of the translation group is that it is an abelian\ngroup. Therefore the irreducible representations are still well\nunderstood, and our framework could extend to it. We intend to include\nrepresentations of such groups in further work.\n\n## Baseline on point clould experiment \n\n> Point could experiment should include more baselines.\n\nWe now compare to other state-of-the-art models for 3D shape recognition\nin Table 3. We have selected the best models we could find that use\npoint cloud or voxel representations. If we are missing any, we would be\nhappy to reference it and, if appropriate, add it to the benchmark. Please find here a\ncopy of this updated table,\n\n| Architecture |  **PointMACE** (ours) | **PointNet**  |  **PointNet ++**    | **KCN**  | **SO-Net** | **LP-3DCNN**|\n| --- | --- | --- | --- | --- | --- | --- |\n| Accuracy | **96.1** | 94.2 |  95.0 | 94.4 | 95.5 | 94.4 |\n| Representation | Point Cloud | Point cloud | Point cloud | Point Cloud | Point Cloud | Voxel grid |\n\n\nNote that we compare only to other point cloud methods. The best current\nmodel we are aware of uses additional information in images at different\nangles and achieves an accuracy of about 98 %. We believe that this\nomission is fair.\n\n## Balance in the sections\n\n> As the key contribution of the paper mostly lies in implementing the\n> necessary computations for generic reductive Lie groups (sec 5), it\n> would make sense to allocate more space to that and less to the prior\n> sections\n\nFollowing your remark, we have shortened the first sections and added a\nnew subsection to the main text discussing the symmetric powers of\nrepresentations of reductive Lie groups and how to generate the\ngeneralized Clebsch Gordan. Moreover, we have added an extensive (5 pages)\ndiscussion of the background of reductive Lie groups, Lie algebras, and\nGT patterns. Please find that in the\nnew Appendix section A.1. We want to emphasize that we tried to put the\nmost technicalities in the appendix, as we want this paper to be\naccessible to a broad audience. For such an audience, we think that a\ncertain detail of the general $G$-equivariant cluster expansion\ntechniques is important. In particular, it makes precise how the ideas\nused in the $O(3)$ case generalize."}, {"review_id": "NyInof0ebQ", "reviewer": "Reviewer_5uTH", "summary": "The paper proposes a class of G-equivariant neural networks for reductive lie groups that generalizes the previous ACE and MCAE models (which are designed to be equivariant with respect to the orthogonal group O(3)) to more general irreducible Lie groups.  A software library (Lie-NN) has also been developed and released for implementation of the proposed method. The experimental results on jet tagging and 3d point cloud classification support the authors' claims about the equivariant model improving model performance.", "questions": "These have been mentioned for the most part in the weaknesses section.", "limitations": "No potential negative societal impact of this work as far as I can tell. As mentioned earlier, computational aspects of the method would be helpful. I don't want the authors to feel compelled to show that this method is immediately computationally practical, as that is not the only measure of the value and worth of any new idea, but it would be helpful to indicate to other researchers on what the computational limitations are, and to possibly spur interest in finding improvements.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed approach seems novel; I am not aware of other works that design a G-equivariant network in this way.\n- The paper is generally well-written and technically correct (although I have not performed an in-depth check for the proof of the universality theorem). Occasional variable definitions are missing but I had no trouble following the general flow of the paper.\n- The experimental results appear to validate the claimed advantages of the proposed method (although more discussion on the computational aspects is needed to assess practicality; see below).", "weaknesses": "- Since this work is a generalized version of the previous ACE and MACE methods, a brief review of ACE and MACE would be helpful to understand and evaluate the novelty of the method. The current version is presented in a way that contains elements of previous existing methods, without a clear delineation of the new and original aspects of the proposed method. Exactly what is the precise nature of the extensions to ACE and MACE should be made clearer.\n- While the technical contents seem correct, some effort at providing intuitive explanation and justifications at key places would be helpful to understanding the paper.  Descriptive figures, intuitive examples (e.g., basis including 1-particle basis or Clebsch-Gordan coefficients) come to mind as examples. In particular, I am curious about how to construct a 1-particle basis for specific Lie groups, e.g., SO(3).\n- The formulation in the case of the product group in Section 4 seems to be missing, although there is a mention of the product group in the introduction and in Section 5. Is it trivial to design an equivariant model for the product group G_1 x G_2 with the formulation used in this method?  My initial impression is that there may be some subtleties involved, such as the order of the group actions.\n- A discussion of the computational aspects of this method is missing.  When using this model, the Clebsch-Gordan coefficients must be calculated numerically (as mentioned in Section 5), and it would be helful to mention calculation times and errors. Does the calculation time vary depending on the number of basis components? I am also curious about how much time it takes for iteration on backpropagation compared to classic MLP models.\n- Continuing with the above comment, since the Clebsch-Gordan coefficients are numerically calculated, some numerical calculation errors are inevitable. Is the equivariance of the model maintained in the presence of numerical errors? If not, some discussion, even qualitative, should be provided (e.g., to what extent the model remains equivariant) although quantitative results (e.g., experimental results, figures) would clearly be preferable.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your time and effort in reviewing our paper. We\nsincerely appreciate your thoughtful review. We appreciate that you have\nfound our paper to be novel and well-written. You will find here-after\nour responses to your questions and remarks, including the changes we\nmade to the paper following your review.\n\n## Clarification on extension of ACE and MACE\n\n> Exactly what is the\n> precise nature of the extensions to ACE and MACE should be made\n> clearer.\n\nThere are two aspects to this question: (1) the general architectures;\nand (2) the generation of the generalized Clebsch-Gordan coefficients.\n\n\\(1\\) The architecture of the models we present here is a\ngeneralization of the special case of $O(3)$-equivariance treated in\nprevious works. However, for most researchers in the field, it\nis unclear whether this extension is possible or how to formulate generalization. The\npurpose of Section 4 is to make precise that this generality is indeed\npossible and how it can be achieved. To the best of our knowledge this\nhas not yet been communicated anywhere else.\n\n\\(2\\) To make the general framework from Section 4 practical, the most\nchallenging aspect is generating the symmetrisation operation, i.e., the\ngeneralized Clebsch--Gordan coefficients. This is achieved in Section 5.\nThis is technically challenging and novel work leading to new software\nthat has a potentially widespread impact across many application areas, as\nindicated throughout the manuscript.\n\nIn response to the question, we lightly edited the list of contributions\nat the end of Section 1, but in terms of writing style, we prefer to keep\nit a bit understated.\n\n## Examples of 1-particle basis\n\n> Some effort at providing intuitive explanation and justifications at\n> key places would be helpful to understanding the paper. \\[\\...\\] In\n> particular, I am curious about how to construct a 1-particle basis for\n> specific Lie groups, e.g., SO(3).\n\nFollowing your remark, we have added an extensive discussion of the\n1-particle basis in the Appendix, giving examples for the $SO(3)$ group and the\nLorentz group. We hope this will clarify your questions.\n\n## On product of groups \n\n> Is it trivial to design an equivariant model for the product group\n> $G_1 \\times G_2$ with the formulation used in this method? My initial\n> impression is that there may be some subtleties involved, such as the\n> order of the group actions.\n\nOur library, `lie-nn`, functions at the level of matrix representations,\nmaking the integration of product groups a natural step. We now provide\na more comprehensive explanation of this in the Appendix (see Product of\ngroups), where we include a practical example of calculating non-trivial\ninvariants for the product groups $O(3) \\times S_{n}$. It is worth\nnoting that the direct product of groups is commutative. Specifically,\nfor any two groups $G_{1}$ and $G_{2}$, we have the isomorphism\n$G_{1} \\times G_{2} \\cong G_{2} \\times G_{1}$.\n\n## Computational cost\n\n> A discussion of the computational aspects of this method is missing.\n> Does the calculation time vary depending on the number of basis\n> components?...\n\nWe added a discussion to the Appendix of the paper and a brief sentence\nin Sec 4.1. to reference that discussion. In the Appendix, we make an\nanalysis of the computational cost of G-MACE as a function of the\ncorrelation order and order of expansion in the 1-particle basis, in the\ncase of the Lorentz group.\n\nIn general, it depends on hyperparameter choices, but especially for\nlarger models the product basis $A_{k}$ is the theoretical and\npractical bottleneck. *In theory* the product basis can be computed at\nO(1) cost per feature (see arXiv:2202.04140, in fact it is proven that\nthe cost is asymptotically 2 operations per feature), but the current\nG-MACE implementation does not leverage this algorithm yet as it requires\nefficient use of sparse tensors, which is difficult on GPU architectures\nas a naive implementation would require random memory access. At present\nthe G-MACE code uses a highly performant dense tensor format\nimplementation which we believe achieves within a factor of 3-5 of the\nhypothetical, optimal performance of a sparse code in the low to moderate correlation order regime that is relevant in applications. The code published with this article generalizes\nthis efficient GPU implementation used in $O(3)$-MACE to handle contractions\nin any groups.\n\nBackpropagation is of comparable cost to inference. The computational kernels we employ are relatively simple. The\nbackward pass is 2-5 times more expansive than the forward pass.\n\nClassical MLP models exploit BLAS3 and similar dense tensor operations\nthat have been optimized *ad nauseum* by generations of researchers. Our codes do not yet reach a similar level\nof peak FLOPs performance and hence the cost of our models will be larger *per parameter*. But note that our physical priors (such as symmetries) usually result in\nmodels that are much smaller and more data-efficient.\n\n## Numerical error and generation of Clebsch Gordan\n\n> Since the Clebsch-Gordan coefficients are numerically calculated, some\n> numerical calculation errors are inevitable. It\n> would be helpful to mention calculation times and errors.\n\nWhile numerical errors are generally inevitable, we use numerically stable solvers\nfor solving the linear systems, thereby achieving machine precision\nerrors. Moreover, for $SU(N)$, the Clebsch-Gordan (CG)\ncoefficients are roots of rational fractions. We employ a\nrounding scheme designed to round to the nearest rational fractions,\nreaching exact accuracy in this case. \n\nFollowing your remark regarding calculation times, we have \nadded a section to\nthe Appendix including a comparison of CG\ngeneration across various sizes of representations and different groups (see pdf in general response).\nWe would like to underscore that the generation of CGs constitutes a\npreprocessing step; so, this phase does not affect the model's performance. "}, {"review_id": "bLeyFIcSFe", "reviewer": "Reviewer_KJSX", "summary": "The paper presents a novel and highly general Equivariant Neural Network (ENN) architecture that is capable of respecting the symmetries of the finite-dimensional representations of any reductive Lie Group G. The proposed approach generalizes the successful ACE and MACE architectures for atomistic point clouds to any data equivariant to a reductive Lie group action. The authors demonstrate the generality and performance of their approach by applying it to two different tasks, namely top quark decay tagging (Lorentz group) and shape recognition (orthogonal group). The results presented in the paper are convincing and showcase the potential of the proposed architecture.", "questions": "* A more comprehensive discussion comparing methods based on Lie Groups, such as LieConv, should be included in the paper.\n* The results presented in the paper might not adequately convey the effectiveness of the proposed method. In order to reinforce the claims and emphasize the practical applicability of the approach, it would be valuable for the authors to incorporate comparisons of experimental results from methods like MACE and LieConv, using examples such as those from the QM9 dataset.", "limitations": "There are no potential negative societal impacts of this work.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-organized and well-written, providing a clear overview of the problem and proposed solution.\n* The paper addresses a significant and challenging problem by proposing a highly general ENN architecture that can handle symmetries of the finite-dimensional representations of any reductive Lie Group G, which contributes to the field of equivariant neural networks.", "weaknesses": "Related concerns are discussed in the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in reviewing our paper. We appreciate\nthat you find our paper well-organized and well-written and addresses a\nsignificant and challenging problem. You will find our response to your\nremarks and questions here. We respectfully hope that our responses will\nbe satisfactory and will increase your mark.\n\n## More in depth discussion of LieConv\n\n> A more comprehensive discussion comparing methods based on Lie Groups,\n> such as LieConv, should be included in the paper.\n\nIn response to your comment, we have incorporated a new section in the\nappendix, entitled \\\"Extended Related Work.\\\" This section delves into a\ndetailed examination of previous literature, including the LieConv\nmethod. It is important to note that our work stands out in providing a\nmore comprehensive framework than any previously developed architecture,\nboth in terms of the diversity of groups covered and the level of\nexpressiveness achieved. To our knowledge, our method represents the\nfirst instance of a higher-order equivariant neural network for generic\nreductive Lie groups.\n\nMoreover, the applicability of LieConv is confined to compact groups, as\nit necessitates the explicit computation of an integral over the group.\nThe convolution in LieConv is limited to considering two-body\ninteractions, whereas our approach, G-MACE, accommodates interactions of\narbitrary order. To this day and for the reasons outline above, LieConv\nis only implemented for a restricted number of groups namely abelian groups and subgroups of\nthe $O(3)$ group. Our method and library covers a much broader range of\napplications in physics and beyond. To the best of our knowledge, no\nprevious work has provided such a general framework.\n\nConvolutional Neural Networks (CNNs), which are translation equivariance, initiated the utilization of data symmetry in machine learning architectures. Throughout time, CNNs have been extended to include other symmetries as well. Central to all these generalizations (including LieConv [2]) is the group averaging operation,\n\n$$\n    \\text{Avg}(f)(x) = \\int_{g \\in G} f(g \\cdot x) dg,\n$$\n\nWhere $ x $ denotes the input signal or feature, $f$ is the convolution kernel, $G$ represents the group of interest, and $dg$ is an invariant measure on $G$. This transformation is essential, as it converts any convolution into a group invariant convolution. The feasibility of this approach largely depends on the computational simplicity of the integral. This approach has several limitations,\n\n- The direct computation of the integral is unstable and inefficient, even for relatively small groups like $O(3)$.\n- For non-compact groups, a unique invariant measure is absent, and the integral diverges.\n- The convolution kernel $f$ is usually constrained to a two-body operator .\n\nIn the case of compact groups, the integral over the group may be calculated via an alternative means. There exists a linear operator, called the Clebsch Gordan operator $\\mathcal{C}$, such that,\n\\begin{equation}\n     \\text{Avg}(f)(x) = \\mathcal{C}(f)(x)\n\\end{equation}\nTherefore, the complex integral over the group becomes a linear operation. The central aim of our work is to show that this approach can also be generalized to all reductive Lie groups, even non-compact ones, and provide tools to do so including the basis to expand $f$ and tools to compute $\\mathcal{C}$ in this basis. \n\n## More comparaison to LieConv\n\n> In order to reinforce the claims and emphasize the practical\n> applicability of the approach, it would be valuable for the authors to\n> incorporate comparisons of experimental results from methods like MACE\n> and LieConv, using examples such as those from the QM9 dataset.\n\nMACE is a subset of the presented G-MACE architecture, when the group is\nthe rotation group and the task is on molecular data. Independently to\nthis work, the MACE force field architecture has been recently\nbenchmarked on QM9 and compared to LieConv in \\[1\\]. Here are selected\ncomparisons relevant to this paper,\n\n|                      | $\\textbf{Gap}$ | $\\textbf{Homo}$ | $\\textbf{Lumo}$ | $C_{V}$ | $\\mu$ | $\\textbf{ZPVE}$ |              |\n|----------------------|--------------|---------------|---------------|--------------|------------|---------------|--------------|\n|                      | meV          | meV           | meV           | cal/mol K    | D          | meV           | $\\alpha_0^2$ | $\\alpha_0^3$ | meV | meV | meV | meV |\n| $\\textbf{LieConv}$ [2] | 49           | 30            | 25            | 0.038        | 0.032      | 2.28          | 0.800        | 0.084        | 22  | 24  | 19  | 19  |\n| $\\textbf{MACE}$ [1]    | 42           | 22            | 19            | 0.021        | 0.015      | 1.23          | 0.210        | 0.038        | 5.5 | 4.7 | 4.1 | 4.1 |\n\nFrom this table, you can observe that MACE far outperforms LieConv,\nthanks to more expressiveness due to its higher-order interactions. In\nthe paper, we compare G-MACE to an extensive range of methods for the\nLorentz group and have now included more baselines for the point-cloud\nclassification. In both of these tasks, G-MACE also achieves\nstate-of-the-art performance. Beyond QM9, the other application\nbenchmark of LieConv is the toy image dataset RotMNIST, corresponding to\nthe group $SO(2)$. This group is implemented in our library. However, as\nit is both an abelian and compact group, specialized architecture has\nbeen constructed for this case that performs very well, including\nsteerable convolution. In the paper, we have preferred to focus on\nproblems in which the point cloud representation is more natural than\nfor images and the group theory more challenging.\n\n\\[1\\] Evaluation of the MACE Force Field Architecture: from Medicinal\nChemistry to Materials Science, D.P Kovacs, I. Batatia, E.S. Arany, G.\nCsanyi\n\n\\[2\\] Generalizing Convolutional Neural Networks for Equivariance to Lie\nGroups on Arbitrary Continuous Data, M. Finzi, S. Stanton, P. Izmailov,\nA. G. Wilson"}]

Author rebuttal:
We thank all reviewers for their time and effort in reviewing our paper.
We are glad you think that our work is "new" (R1) and that our "proposed method is an elegant generalization of previous methods" (R2), addressing "a significant and challenging problem" (R4) of the field of equivariant neural networks. We appreciate that you find our manuscript "well-written" and "well-organized" (R3).

In summary, we have updated our manuscript with the following changes. An updated manuscript incorporating those changes will be made available on arxiv within a few days of the deadline.

-  We have improved the overall clarity of the writing and referencing of results.
-  We have added a new subsection to section 5 in the main text on symmetric powers of representations and the generation of generalized Clebsch Gordan coefficients for reductive Lie groups.
-  Fixed the typo in Figure 1.
-  We added a more extensive number of baselines for the 3D shape recognition task. Here you will find the updated table:
| Architecture |  **PointMACE** (ours) | **PointNet**  |  **PointNet ++**    | **KCN**  | **SO-Net** | **LP-3DCNN**|
| --- | --- | --- | --- | --- | --- | --- |
| Accuracy | **96.1** | 94.2 |  95.0 | 94.4 | 95.5 | 94.4 |
| Representation | Point Cloud | Point cloud | Point cloud | Point Cloud | Point Cloud | Voxel grid |

- We added an extensive background on Lie groups and their representation in the Appendix. We expose in more detail the theoretical foundations of our work. In particular, we give a more in-depth summary of the Gelfand-Tsetlin patterns.

- We added a new section to the Appendix that gives concrete examples of one particle basis for the case of the $O(3)$ group and the Lorentz group.

- We give more details for constructing irreducible representations of the product of groups. We also provide a concrete application of the lie-nn library for computing invariants of the product groups $O(3) \times S_{3}$.

- We have added a section in the Appendix on computational cost, both on Clebsch Gordan generation for a wide range of groups and comparing the cost of ablated versions of G-MACE for the Lorentz group. Please find the figures in the pdf attached.

- We have incorporated an Extended Related Work section to the Appendix, providing an extended background on methods for constructing equivariant neural networks. This section also helps to clarify our contributions.

- Finally, we have published our code on GitHub for complete reproducibility.

Below we respond to the reviewers' comments individually.

Meta-review: The paper introduces a general method for construction of equivariant networks for arbitrary reductive Lie groups, generalizing the ACE / MACE methods. Reviewers found the work novel, clearly written, and elegant, and appreciated the release of code. The main concerns raised by the reviewers were addressed in the rebuttal/discussion, and so I recommend this paper to be accepted.
Decision: Accept (poster)