SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For an example let the familiar rotation group  $\mathrm{O}(3)$  act on the usual round sphere  $S^2$ . Then  $\mathrm{O}(3)$  acts on the space of functions on  $S^2$ . This space is infinite-dimensional, but it turns out we can approximate it by a sum of finite-dimensional pieces. Indeed for each degree  $d \geq 0$  let  $V_d$  be the space of polynomials in the variables  $x, y, z$  which are homogeneous of degree  $d$  (for example
$V_{0} = \operatorname{span}\{1\}, V_{1} = \operatorname{span}\{x, y, z\}, V_{2} = \operatorname{span}\{x^{2}, xy, y^{2}, \ldots\}$ . Letting O(3) act on  $\mathbb{R}^3$  by linear change of variable, it acts on each  $V_{d}$  since such change of variable does not change the degree of homogeneity. Now let  $W_{d} \subset V_{d}$  be the subspace of harmonic polynomials, those polynomials  $p$  that satisfy  $\frac{\partial^2 p}{\partial x^2} + \frac{\partial^2 p}{\partial y^2} + \frac{\partial^2 p}{\partial z^2} = 0$ . Then  $W_{d}$  is itself O(3) invariant since the Laplace operator is rotation-invariant, making  $W_{d}$  a subrepresentation of  $V_{d}$ . It is a (not immediately obvious) fact that each  $W_{d}$  is in fact an irreducible representation: it has no subrepresentations of its own other than itself and the zero subspace. Further, if we interpret the harmonic polynomials in  $W_{d}$  as functions on the unit sphere  $S^2 \subset \mathbb{R}^3$  we can view  $W_{d}$  as a space of functions on the sphere. It then turns out that the spaces  $W_{d}$  are linearly independent of each other, and that their sum  $\bigoplus_{d \geq 0} W_{d}$  is dense in any reasonable space of functions on the sphere (the space of continuous functions, the space of smooth functions, or the space of square-integrable functions, each with its respective notion of convergence). In other words, we have morally decomposed the space of functions on  $S^2$  as the sum of O(3)-invariant subrepresentations, each of which is irreducible.
Decomposing a representation into its irreducible constituents is a major goal of representation theory (as well as a computational goal of this paper). When this is possible we can understand a general representations  $V$  of a group  $G$  by first enumerating its irreducible representations and then counting how many times each irreducible occurs as summand in  $V$ , known as the (this is very much analogous to understanding general positive integers via their prime factorization – here the irreducible representations play the role of the prime numbers).
Reductive groups are particularly amenable to this approach because their representations indeed do decompose as direct sums in this fashion. It is fact that a linear Lie group  $G$  is reductive if and only if each of its finite-dimensional representations is isomorphic to a direct sum of irreducible representations.
Before we delve further into representation theory we need a brief detour into the structure theory of Lie groups.
A.1.3 Lie algebras
As Lie groups are manifolds, understanding their representations theory usually leads to advanced geometrical and topological results. We will now describe how these difficult questions can be transformed into simpler linear algebra questions using the Lie algebras associated with Lie groups.
A Lie algebra is a vector space  $\mathfrak{g}$  over a field  $F$  together with a bilinear map  $[\cdot, \cdot]: \mathfrak{g} \times \mathfrak{g} \to \mathfrak{g}$  called the Lie bracket, satisfying the following axioms:
- The Lie bracket is bilinear, that is for all  $x,y,z\in \mathfrak{g}$  and  $\alpha ,\beta \in F$ $[\alpha x + \beta y,z] = \alpha [x,z] + \beta [y,z]$  and  $[x,\alpha y + \beta z] = \alpha [x,y] + \beta [x,z]$ .
- The Lie bracket is antisymmetric, that is for all  $x,y\in \mathfrak{g},[x,y] = -[y,x]$
- The Lie bracket satisfies the Jacobi identity, that is for all  $x, y, z \in \mathfrak{g}$ ,  $[x, [y, z]] + [y, [z, x]] + [z, [x, y]] = 0$ .
Lie algebras can alternatively be seen as the tangent space of a Lie group to the identity:
Theorem A.3. Let  $G$  be a Lie group viewed as a smooth manifold, the tangent space  $T_{\Gamma}G$  is a Lie algebra called the Lie algebra of  $G$  noted  $\mathfrak{g}$ . Moreover in the case of connected complex Lie groups, the category of representation of  $G$ ,  $\operatorname{Rep}(G)$ , is equivalent to the category of representation of  $\mathfrak{g}$ ,  $\operatorname{Rep}(\mathfrak{g})$ .
This theorem is at the heart of the representation theory of Lie groups. As Lie algebra are vector spaces, the functorial equivalence between the two categories of representation translates geometrical and topological questions into simpler linear algebra questions.
A classification of Lie algebras was carried out by Cartan and later Gelfand work. In the case of complex Lie algebras, this classification is complete. Two major classes of Lie algebras emerged from Cartan's work, semi-simple Lie algebras and nilpotent Lie algebras.
Definition A.4 (Central series). A central series of a Lie algebra  $\mathfrak{g}$  is a sequence of ideals  $(\mathfrak{g}_i)_{i\in \mathbb{N}}$  such that  $\mathfrak{g}_0 = \mathfrak{g}$  and  $\mathfrak{g}_{i + 1} = [\mathfrak{g}_i,\mathfrak{g}]$ .
Definition A.5. A Lie algebra is called semi-simple if it does not contain any non-trivial ideals. A Lie algebra is called nilpotent if its central series terminates to zero.
An important result of this classification is that any Lie algebras over an algebraically closed field can be decomposed into semisimple parts and a nilpotent part. We call this the Jordan decomposition. Therefore, understanding arbitrary representations of a Lie algebra boils down to understanding representation of both semi-simple and nilpotent algebras.
Most reductive Lie groups of interest have a semi-simple Lie algebra as associated Lie algebra. These algebras are also the one for which the representation theory is the best understood.
The study of Lie algebras is decomposed in two subfields. The structure theory studies the classification of Lie algebras in terms of their eigenspaces (as vector spaces). The representation theory builds on the structure theory to classify representations of semi-simple Lie algebras. We will give a very short introduction to both of them.
A.1.4 Structure theory of semi-simple Lie algebras
Let  $\mathfrak{g}$  be a semi-simple complex Lie algebras. Let  $\mathfrak{h}$  be one maximal set of mutually commutative element of  $\mathfrak{g}$ . Maximal means here that no other such set contains strictly more element than  $\mathfrak{h}$ . We call  $\mathfrak{h}$  the Cartan algebra of  $\mathfrak{g}$ . As elements of  $\mathfrak{h}$  are mutually commutative, they share the same eigenspaces.
Therefore one can decompose  $\mathfrak{g}$  in terms of eigenspaces of elements of  $\mathfrak{h}$ . Let  $\alpha \in \mathfrak{h}^*$  be a function in the dual of the Cartan algebra such that  $\forall h_i \in \mathfrak{h}, \alpha(h_i) = \alpha_i$  with  $\alpha_i$  the eigenvalue associated to  $h_i$ . We call this linear operator the roots of  $\mathfrak{g}$ . Let  $R$  be the space of all this functions, called the root space. One therefore has the following decomposition of the Lie algebra;
$$
\mathfrak {g} = \mathfrak {h} + \bigoplus_ {\alpha \in R} g _ {\alpha} \tag {24}
$$
The structure theory of semi-simple complex Lie algebras is the study of the root space  $R$  and the associated eigenspaces  $g_{\alpha}$ . One can classify complex semi-simple Lie algebras based on their type of root spaces. For real semi-simple Lie algebras, the structure theory essential reduces down to the complex case with a few extra steps.
A.1.5 Representation theory of complex semi-simple Lie algebras and Lie groups
The representation theory of complex reductive Lie algebras is completely understood. Every finite-dimensional representation is the direct sum of irreducible representations. Every irreducible representation is uniquely determined by a highest weight  $\lambda$ , which is an element of the dual of the Cartan algebra  $h$ . The dimension of each irreducible representation can be computed using the Weyl dimension formula.
$$
\dim V _ {\lambda} = \frac {\prod_ {\alpha \in R ^ {+}} (\lambda + \rho , \alpha)}{\prod_ {\alpha \in R ^ {+}} (\rho , \alpha)} \tag {25}
$$
where  $R^{+}$  is a subset of the root space  $R$ , called the positive roots, and  $\rho$  is the half sum of the positive roots. Obtaining explicitly a basis for each representation is much harder. Gelfand and Tselin proposed a basis for the general case of  $SU(N)$  groups. This basis was recently generalized to other complex semi-simple algebras Molev (1999).
A.1.6 Representations of real semi-simple Lie groups
The representation theory of real semi-simple Lie groups can be elucidated by examining the complex semi-simple case through the application of the so-called Weyl unitary trick. This technique establishes a connection between the representations of real Lie groups and those of the complexification of their universal covering groups.
Definition A.6. Given a Lie algebra  $\mathfrak{g}$ , there exists a unique simply connected group  $\hat{G}$  possessing  $\mathfrak{g}$  as its Lie algebra. A group homomorphism  $\psi$  exists from this unique simply connected Lie group to any group  $G$  sharing the same Lie algebra. We refer to  $\hat{G}$  as the universal covering Lie group, and it is unique up to isomorphism.
An essential observation is that the irreducible representations of  $G$  form a subset of the irreducible representations of  $\hat{G}$ . Understanding the irreducible representations of  $\hat{G}$  suffices. Take the case of
the  $\mathfrak{su}(2)$  Lie algebra, for instance. The universal covering group is  $\mathrm{SU}(2)$ , corresponding to the special unitary two-by-two matrices. The group  $\mathrm{SO}(3)$  of special orthogonal matrices also has  $\mathfrak{su}(2)$  as its Lie algebra. Consequently, the irreducible representations of  $\mathrm{SO}(3)$  are a subset of those of  $\mathfrak{su}(2)$ . Representations of  $\mathfrak{su}(2)$  can be indexed by a half-integer, known as the quantum number  $j$ . The integer representations are also irreducible representations of  $\mathrm{SO}(3)$ , with non-integer ones being referred to as the spin representations.
Since the representation theory of complex Lie groups is well understood, an optimal approach to understanding the representation theory of a real Lie group involves studying the complexification of its universal cover.
Definition A.7. For a universal covering Lie group  $G$  with Lie algebra  $\mathfrak{g}$ , the universal covering Lie group  $\hat{G}_{\mathbb{C}}$ , having the associated Lie algebra  $\mathfrak{g}_{\mathbb{C}} = \mathfrak{g} \otimes \mathbb{C}$ , is termed the complexification of  $\hat{G}$ .
The group of interest  $G$ , its universal cover  $\hat{G}$  and the complexification of the universal cover  $\hat{G}_{\mathbb{C}}$ , all share the similar irreducible representations. Therefore one obtains the following chain of inclusion of the respective representations of the groups,
$$
R e p (G) \subset R e p (\hat {G}) \subset R e p (\hat {G} _ {\mathbb {C}}) \tag {26}
$$
Let  $K$  be the maximal compact subgroup of  $\hat{G}_{\mathbb{C}}$  and  $\rho_{\lambda}$  a finite dimensional irreducible representation of  $K$ , with  $\lambda$  as its highest weight. It is possible to analytically continue  $\rho_{\lambda}$  into an irreducible representation of  $\hat{G}_{\mathbb{C}}$ , denoted as  $\rho_{\lambda}^{\hat{G}_{\mathbb{C}}}$ , and also into the analytical conjugate  $\bar{\rho}_{\lambda}^{\hat{G}_{\mathbb{C}}}$ . Every finite-dimensional representation of  $\hat{G}_{\mathbb{C}}$  can be constructed as the tensor product form  $\rho_{\lambda}^{\hat{G}_{\mathbb{C}}}\otimes \bar{\rho}_{\lambda^{\prime}}^{\hat{G}_{\mathbb{C}}}$ . Due to the fact that finite-dimensional representations of the universal covers and complexification of a semi-simple Lie group are isomorphic, one can express:
$$
\rho_ {\lambda} ^ {\hat {G} _ {\mathbb {C}}} \otimes \bar {\rho} _ {\lambda^ {\prime}} ^ {\hat {G} _ {\mathbb {C}}} := \rho_ {(\lambda , \lambda^ {\prime})} ^ {\hat {G}} := \rho_ {(\lambda , \lambda^ {\prime})} ^ {G} \tag {27}
$$
As universal covering groups contain more representations than the original cover, this mapping is not an isomorphism for all lambda. To illustrate this, consider the example of the Lorentz group  $SO(1,3)$ .
The universal cover of  $\mathrm{SO}(1,3)$  is  $\mathrm{SL}(2,C)$ , and the maximal compact subgroup of  $\mathrm{SL}(2,C)$  is  $\mathrm{SU}(2)$ . The irreducible representations of  $\mathrm{SU}(2)$  are indexed by  $l\in \mathbb{N} / 2$  and correspond to the Wigner D matrices  $D^{l}(g),g\in \mathrm{SU}(2)$ . The  $\mathrm{SU}(2)$  group elements are parameterized by Euler angles that can be analytically continued to  $\mathrm{SL}(2,C)$  as follows:
$$
\alpha = \phi + i \kappa , \beta = \theta + i \epsilon , \gamma = \phi + i \xi
$$
$$
\phi \in [ 0, 2 \pi), \theta \in [ 0, \pi ], \phi \in [ 0, 2 \pi), \kappa , \epsilon , \xi \in \mathbb {R} \tag {28}
$$
This enables the construction of the fundamental representation of  $\mathrm{SL}(2,\mathbb{C})$ , resulting in representations of the Lorentz group corresponding to the product of Wigner D matrices:
$$
D ^ {k / 2} (\alpha , \beta , \gamma) \otimes \bar {D} ^ {l / 2} (\alpha , \beta , \gamma) \tag {29}
$$
The irreducible representations of  $\mathrm{SO}(1,3)$  are indexed by a pair of integers  $(l,k)$  corresponding to the associated  $SU(2)$  representations in the tensor product.
A.1.7 Gefland-Tselin Basis
The enumeration of finite-dimensional representations of complex Lie groups has been well understood. However, the derivation of explicit matrix representations in specific bases and the tensor product decomposition of representations within these bases remain challenging and subject to ongoing research. The Gelfand-Tselin basis provides an explicit structure for irreducible representations, initially designed for the  $SU(N)$  group Gelfand and Tsetlin (1950); Alex et al. (2011), and subsequently generalized to encompass other classical Lie groups Molev (1999).
Central to the concept of GT patterns is the application of branching rules, whereby representations of a larger group are constructed by enumerating induced representations on its subgroups. Consider, for instance, the  $SU(N)$  case where irreducible representations are characterized by the highest weight vector  $\lambda$ , a vector of length  $N$  terminated by a zero,  $\lambda = (\lambda_{1},\dots,\lambda_{N - 1},0)$ . This highest weight
vector may spawn several induced representations of the subgroup  $SU(N - 1)$ , each described by  $N - 1$  vectors. This process can be iteratively executed in a descending chain:
$$
S U (N) \rightarrow S U (N - 1) \rightarrow \dots \rightarrow S U (2) \rightarrow U (1) \tag {30}
$$
This chain commences with the group of interest and culminates with the smallest subgroup. From this branching chain, triangular arrays known as GT-patterns can be constructed, with each row corresponding to a vector of a representation ranging from  $SU(N)$  to  $U(1)$ :
$$
\left( \begin{array}{c c c c c} \lambda_ {N, 1} & \lambda_ {N, 2} & \dots & \lambda_ {N, N - 1} & 0 \\ \lambda_ {N - 1, 1} & \lambda_ {N - 1, 2} & \dots & \lambda_ {N - 1, N - 1} \\ \vdots & \vdots & & \ddots \\ \lambda_ {2 1} & \lambda_ {2 2} & & \\ \lambda_ {1 1} & & & \end{array} \right) \tag {31}
$$
Adjacent rows within these patterns must adhere to a so-called "snake rule" to form an admissible GT pattern:
$$
\lambda_ {k, 1} > \lambda_ {k - 1, 1} > \lambda_ {k, 2} > \lambda_ {k - 1, 2} > \dots > \lambda_ {k, k - 1} > \lambda_ {k - 1, k - 1} > \lambda_ {k, k} \tag {32}
$$
The dimension of a specific irreducible representation corresponds to the count of such admissible arrays, with the top row representing the highest weight vector. Representations sharing the same highest weight vectors interrelate through ladder operators. One key attribute of GT patterns is that the action of ladder operators on a GT pattern is analytically known, enabling the construction of the matrix representation for all representations using only the highest weight representation. Furthermore, constructing the highest weight matrix representation equals solving a linear system when the ladder operators are known analytically. Extending this methodology to other classical Lie groups (such as  $(n),\mathrm{Sp}(n),\mathrm{SL}(n))$  presents challenges as the ladder operators' entries become increasingly complex and recursive.
A.2 Example of one particle basis for  $\mathbf{O}(3)$  and  $\mathrm{SO}(1,3)$  groups
In the paper, the one-particle basis  $\phi$  is left abstract, as it depends on the specific type of input point cloud and the group under consideration. We now describe two concrete examples, one involving isometry the other Lorentz equivariance.
A.2.1 Isometry equivariance
In the first example we consider a point cloud comprised of atoms, for example representing a molecule. Here, the point states are denoted by  $x_{i} = (r_{i},Z_{i})\in \mathbb{R}^{3}\times \mathbb{Z}$ , where  $r_i$  corresponds to the position in 3D and  $Z_{i}$ , its atomic number, is a categorical variable. The one-particle basis must form a complete basis for  $\mathbb{R}^3\times \mathbb{Z}$ , which is both rotationally equivariant and translationally invariant. The translation is typically handled by referring to the atom's state in relation to a central value or another point. For simplicity, we can assume  $r_i$  the particle's position relative to the point cloud's centre of mass, ensuring translational invariance.
The space  $\mathbb{R}^3$  can be represented as the product  $\mathbb{R}\times S^2$ , with  $S^2$  being the sphere. As such, we choose a basis for  $\mathbb{R}$  such as Bessel functions, Chebyshev polynomials or a Fourier basis, denoted as  $R_{n}$ . For  $S^2$ , we utilise the spherical harmonics  $Y_{lm}$ , which form a basis of functions in the sphere. Consequently, the one-particle basis can be expanded as:
$$
\phi_ {n l m} \left(x _ {j}\right) = R _ {n} \left(r _ {i}\right) Y _ {l m} \left(\hat {r} _ {i}\right) f \left(Z _ {i}\right) \tag {33}
$$
where  $r_i$  corresponds to the norm of  $\pmb{r}_i$  and  $\hat{r}_i = \frac{\pmb{r}_i}{r_i}$ .
The case of the Lorentz group is more complex than the (3) case. Consider a point cloud made up of elementary particles, represented by their 4-momenta  $x_{i} = (E_{i},\pmb{p}_{i}) \in \mathbb{R}^{4}$ . Although the 4-momenta is a vector, the natural metric in this space is not the Euclidean metric but rather a pseudo metric known as the Minkowski metric. This pseudo-metric encapsulates the unique role that time plays in special relativity. In natural units, the pseudo norm  $\eta$  of a 4-vector  $u = (u_0,u_1,u_2,u_3)$  is defined as
$$
\eta (u, u) = u _ {0} ^ {2} - u _ {1} ^ {2} - u _ {2} ^ {2} - u _ {3} ^ {2} \tag {34}
$$
To construct the one-particle basis, it is necessary to expand a four-vector in terms of the basis of representations of the Lorentz groups. As outlined in Appendix A.1.6, irreducible representations
of the Lorentz group are characterised by a tuple of integers  $(l,k)$ . The 4-vectors correspond to the fundamental, irreducible representation, namely the  $(1,1)$  representation. Analogous structures to the spherical harmonics can be obtained by examining the symmetric powers of the irreducible representations. These are analogous to harmonic polynomials on Minkowski space, much as spherical harmonics are harmonic polynomials on the sphere  $S^2$ . We thus define the Minkowski spherical harmonics as the following symmetric tensor product,
$$
Y _ {l m} \left(\mathbf {p} _ {i}\right) = \operatorname {S y m} ^ {l} \mathbf {p} _ {i} \tag {35}
$$
These spherical harmonics form a harmonic polynomial basis on Minkowski space. Therefore, one can create the one-particle basis in a manner akin to the  $O(3)$  case, as follows:
$$
\phi_ {n l m} \left(\mathbf {p} _ {j}\right) = R _ {n} \left(p _ {i}\right) Y _ {l m} \left(\hat {p} _ {i}\right) \tag {36}
$$
Where  $p_i = \eta(\pmb{p}_i, \pmb{p}_i)$ , and  $\hat{p}_i = \frac{\pmb{p}_i}{p_i + \epsilon}$ , with  $\epsilon$  being a small constant that prevents divergence when  $p_i$  is zero due to the non-positiveness of the Minkowski norm.
A.3 Extended Related Work
Convolutional Neural Networks (CNNs), which are translation equivariance, initiated the utilization of data symmetry in machine learning architectures. Throughout time, CNNs have been extended to include other symmetries as well. Central to all these generalizations is the group averaging operation,
$$
\operatorname {A v g} (f) (x) = \int_ {g \in G} f (g \cdot x) d g, \tag {37}
$$
Where  $x$  denotes the input signal or feature,  $f$  is the convolution kernel,  $G$  represents the group of interest, and  $dg$  is an invariant measure on  $G$ . This transformation is essential, as it converts any convolution into a group invariant convolution. The feasibility of this approach largely depends on the computational simplicity of the integral. The most straightforward instance occurs for finite groups, as explained by G-convolutions (Cohen and Welling, 2016b), where the integral simplifies to a sum,
$$
\operatorname {A v g} (f) (x) = \sum_ {g \in G} f (g \cdot x). \tag {38}
$$
In the context of  $G$  being a compact group, the invariant measure  $dg$  is unique, referred to as the Haar measure of the group, allowing the integral to be computed numerically, e.g., on a grid. Steerable CNNs (Cohen and Welling, 2016a), LieConv Finzi et al. (2020) and Spherical CNNs (Cohen et al., 2018) extended CNNs to  $O(2)$  and  $O(3)$  equivariance, respectively. A general theory for convolution on any compact group and symmetric space was developed by Kondor and Trivedi (2018), and further extended to equivariant convolutions on Riemannian manifolds by Weiler et al. (2021). However, this approach has several limitations,
- The direct computation of the integral can be numerically unstable and inefficient, even for relatively small groups like  $O(3)$ .
- In the case of non-compact groups, a unique invariant measure is absent, and the integral diverges.
- Across these methods, the convolution kernel  $f$  is usually constrained to a two-body operator
In the case of compact groups, the integral over the group may be calculated via an alternative means. There exists a linear operator, called the Clebsch Gordan operator  $\mathcal{C}$ , such that,
$$
\operatorname {A v g} (f) (x) = \mathcal {C} (f) (x) \tag {39}
$$
Therefore, the complex integral over the group becomes a linear operation. The form of this operator depends on the basis in which  $f$  is expended. In the case of  $G = O(3)$  and if this basis is carefully chosen, the entries of this operator are known analytically. This approach is numerically stable and more efficient and was taken by numerous works, including ACE Dusson et al. (2022), Cormorant Anderson et al. (2019), NequIP Batzner et al. (2022), or MACE Batatia et al. (2022b). The central aim of our work is to show that this approach can also be generalized to all reductive Lie groups, even non-compact ones, and provide tools to do so.
A.4 Proof of (7)
This statement follows closely the arguments by Dusson et al. (2022); Drautz (2020) and others.
$$
\begin{array}{l} \sum_ {j _ {1}, \dots , j _ {n}} \sum_ {\boldsymbol {k}} w _ {\boldsymbol {k}} \prod_ {s} \phi_ {k _ {s}} \left(x _ {j _ {s}}\right) = \sum_ {\boldsymbol {k}} w _ {\boldsymbol {k}} \sum_ {j _ {1}, \dots , j _ {n}} \prod_ {s} \phi_ {k _ {s}} \left(x _ {j _ {s}}\right) \\ = \sum_ {\boldsymbol {k}} w _ {\boldsymbol {k}} \prod_ {s = 1} ^ {n} \sum_ {j} \phi_ {k _ {s}} (x _ {j}) \\ = \sum_ {\boldsymbol {k}} w _ {\boldsymbol {k}} \prod_ {s = 1} ^ {n} A _ {k} \\ = \sum_ {k} w _ {k} A _ {k}. \\ \end{array}
$$
A.5 Custom notation and indexing
We briefly contrast our notation for Clebsch-Gordan coefficients (10) with the standard notation. By means of example, consider the group  $SO(3)$  in which case the Clebsch-Gordan equations are written as