SlowGuess/NeurIPS_2023_OCR huggingface dataset:
David Ruhe
AI4Science Lab, AMLab, API
david.ruhe@gmail.com
Johannes Brandstetter
AI4Science
brandstetter@ml.jku.at
Patrick Forre
AI4Science Lab, AMLab
p.d.forre@uva.nl
We introduce Clifford Group Equivariant Neural Networks: a novel approach for constructing  $\mathrm{O}(n)$ - and  $\mathrm{E}(n)$ -equivariant models. We identify and study the Clifford group: a subgroup inside the Clifford algebra tailored to achieve several favorable properties. Primarily, the group's action forms an orthogonal automorphism that extends beyond the typical vector space to the entire Clifford algebra while respecting the multivector grading. This leads to several non-equivalent subrepresentations corresponding to the multivector decomposition. Furthermore, we prove that the action respects not just the vector space structure of the Clifford algebra but also its multiplicative structure, i.e., the geometric product. These findings imply that every polynomial in multivectors, including their grade projections, constitutes an equivariant map with respect to the Clifford group, allowing us to parameterize equivariant neural network layers. An advantage worth mentioning is that we obtain expressive layers that can elegantly generalize to inner-product spaces of any dimension. We demonstrate, notably from a single core implementation, state-of-the-art performance on several distinct tasks, including a three-dimensional  $n$ -body experiment, a four-dimensional Lorentz-equivariant high-energy physics experiment, and a five-dimensional convex hull experiment.
Incorporating group equivariance to ensure symmetry constraints in neural networks has been a highly fruitful line of research [CW16, WGTB17, CGKW18, KT18, WC19,  $\mathrm{WGW}^{+}18$ , Est20, BBCV21, WFVW21, CLW22]. Besides translation and permutation equivariance [ZKR  $^{+}17$ , SGT  $^{+}08$ ], rotation equivariance proved to be vitally important for many graph-structured problems as encountered in, e.g., the natural sciences. Applications of such methods include modeling the dynamics of complex physical systems or motion trajectories [KFW  $^{+}18$ , BHP  $^{+}22$ ]; studying or generating molecules, proteins, and crystals [RDRL14, GLB  $^{+}16$ , CTS  $^{+}17$ , SSK  $^{+}18$ , ZCD  $^{+}20$ , TVS  $^{+}21$ , AGB22]; and point cloud analysis [WSK  $^{+}15$ , UPH  $^{+}19$ ]. Note that many of these focus on three-dimensional problems involving rotation, reflection, or translation equivariance by considering the groups O(3), SO(3), E(3), or SE(3).
Such equivariant neural networks can be broadly divided into three categories: approaches that scalarize geometric quantities, methods employing regular group representations, and those utilizing irreducible representations, often of O(3) [HRXH22]. Scalarization methods operate exclusively on
Code is available at https://github.com/DavidRuhe/clifford-group-equivariant-neural-networks
Figure 1: CGENNs (represented with  $\phi$ ) are able to operate on multivectors (elements of the Clifford algebra) in an  $O(n)$ - or  $E(n)$ -equivariant way. Specifically, when an action  $\rho(w)$  of the Clifford group, representing an orthogonal transformation such as a rotation, is applied to the data, the model's representations corotate. Multivectors can be decomposed into scalar, vector, bivector, trivector, and even higher-order components. These elements can represent geometric quantities such as (oriented) areas or volumes. The action  $\rho(w)$  is designed to respect these structures when acting on them.
scalar features or manipulate higher-order geometric quantities such as vectors via scalar multiplication [SSK $^{+}$ 18, CCG18, KGG20, KKN20, SHW21, JES $^{+}$ 21, GBG21, SUG21, DLD $^{+}$ 21, HHR $^{+}$ 22, TF22]. They can be limited by the fact that they do not extract all directional information. Regular representation methods construct equivariant maps through an integral over the respective group [CW16, KT18]. For continuous Lie groups, however, this integral is intractable and requires coarse approximation [FSIW20, Bek19]. Methods of the third category employ the irreducible representations of O(3) (the Wigner-D matrices) and operate in a steerable spherical harmonics basis [TSK $^{+}$ 18, AHK19, FWFW20, BHP $^{+}$ 22, BMS $^{+}$ 22]. This basis allows a decomposition into type- $l$  vector subspaces that transform under  $D^l$ : the type- $l$  matrix representation of O(3) [Len90, FA $^{+}$ 91]. Through tensor products decomposed using Clebsch-Gordan coefficients (Clebsch-Gordan tensor product), vectors (of different types) interact equivariantly. These tensor products can be parameterized using learnable weights. Key limitations of such methods include the necessity for an alternative basis, along with acquiring the Clebsch-Gordan coefficients, which, although they are known for unitary groups of any dimension [KR67], are not trivial to obtain [AKHvD11].
We propose Clifford Group Equivariant Neural Networks (CGENNs): an equivariant parameterization of neural networks based on Clifford algebras. Inside the algebra, we identify the Clifford group and its action, termed the (adjusted) twisted conjugation, which has several advantageous properties. Unlike classical approaches that represent these groups on their corresponding vector spaces, we carefully extend the action to the entire Clifford algebra. There, it automatically acts as an orthogonal automorphism that respects the multivector grading, enabling nontrivial subrepresentations that operate on the algebra subspaces. Furthermore, the twisted conjugation respects the Clifford algebra's multiplicative structure, i.e. the geometric product, allowing us to bypass the need for explicit tensor product representations. As a result, we obtain two remarkable properties. First, all polynomials in multivectors generate Clifford group equivariant maps from the Clifford algebra to itself. Additionally, grade projections are equivariant, allowing for a denser parameterization of such polynomials. We then demonstrate how to construct parameterizable neural network layers using these properties.
Our method comes with several advantages. First, instead of operating on alternative basis representations such as the spherical harmonics, CGENNs (similarly to scalarization methods) directly transform data in a vector basis. Second, multivector representations allow a (geometrically meaningful) product structure while maintaining a finite dimensionality as opposed to tensor product representations. Through geometric products, we can transform vector-valued information, resulting in a more accurate and nuanced interpretation of the underlying structures compared to scalarization methods. Further, we can represent exotic geometric objects such as pseudovectors, encountered in certain physics problems, which transform in a nonstandard manner. Third, our method readily generalizes to orthogonal groups regardless of the dimension or metric signature of the space, thereby attaining  $\mathrm{O}(n)$ - or  $\mathrm{E}(n)$ -equivariance. These advantages are demonstrated on equivariance bench-
marks of different dimensionality. Note that specialized tools were developed for several of these tasks, while CGENNs can be applied more generally.
2 The Clifford Algebra
Clifford algebras (also known as geometric algebras) are powerful mathematical objects with applications in various areas of science and engineering. For a complete formal construction, we refer the reader to Appendix D. Let  $V$  be a finite-dimensional vector space over a field  $\mathbb{F}$  equipped with a quadratic form  $\mathfrak{q}: V \to \mathbb{F}$ . The Clifford algebra  $\operatorname{Cl}(V, \mathfrak{q})$  is the unitary, associative, noncommutative algebra generated by  $V$  such that for every  $v \in V$  the relation  $v^2 = \mathfrak{q}(v)$  holds, i.e., vectors square to scalars. This simple property solely generates a unique mathematical theory that underpins many applications. Note that every element  $x$  of the Clifford algebra  $\operatorname{Cl}(V, \mathfrak{q})$  is a linear combination of (formal, non-commutative) products of vectors modulo the condition that every appearing square  $v^2$  gets identified with the scalar square  $\mathfrak{q}(v): x = \sum_{i \in I} c_i \cdot v_{i,1} \cdots v_{i,k_i}^2$ . Here, the index set  $I$  is finite,  $c_i \in \mathbb{F}$ ,  $k \in \mathbb{N}_0$ ,  $v_{i,j} \in V$ . The Clifford algebra's associated bilinear form  $\mathfrak{b}(v_1, v_2) := \frac{1}{2} (\mathfrak{q}(v_1 + v_2) - \mathfrak{q}(v_1) - \mathfrak{q}(v_2))$  yields the fundamental Clifford identity:  $v_1 v_2 + v_2 v_1 = 2\mathfrak{b}(v_1, v_2)$  for  $v_1, v_2 \in V$  (Lemma D.3). In this context, the quantity  $v_1 v_2$  represents the geometric product, which is aptly named for its ability to compute geometric properties and facilitate various transformations. Note that when  $v_1, v_2$  are orthogonal (e.g., for orthogonal basis vectors),  $\mathfrak{b}(v_1, v_2) = 0$ , in which case  $v_1 v_2 = -v_2 v_1$ . The dimensionality of the algebra is  $2^n$ , where  $n := \dim V$  (Theorem D.15). Let  $e_1, \ldots, e_n$  be an orthogonal basis of  $V$ . The tuple  $(e_A)_{A \subseteq [n]}$ ,  $[n] := \{1, \ldots, n\}$ , is an orthogonal basis for  $\operatorname{Cl}(V, \mathfrak{q})$ , where for all such  $A$  the product  $e_A := \prod_{i \in A}^{<} e_i$  is taken in increasing order (Theorem D.26). We will see below that we can decompose the algebra into vector subspaces  $\operatorname{Cl}^{(m)}(V, \mathfrak{q})$ ,  $m = 0, \ldots, n$ , called grades, where  $\dim \operatorname{Cl}^{(m)}(V, \mathfrak{q}) = \binom{n}{m}$ . Elements of grade  $m = 0$  and  $m = 1$  are scalars  $(\operatorname{Cl}^{(0)}(V, \mathfrak{q}) = \mathbb{F})$  and vectors  $(\operatorname{Cl}^{(1)}(V, \mathfrak{q}) = V)$ , respectively, while elements of grade  $m = 2$  and  $m = 3$  are referred to as bivectors and trivectors. Similar terms are used for elements of even higher grade. These higher-order grades can represent (oriented) points, areas, and volumes, as depicted in Figure 1.
Clifford algebras provide tools that allow for meaningful algebraic representation and manipulation of geometric quantities, including areas and volumes [RDK21, DM02, DFM09]. In addition, they offer generalizations such as extensions of the exterior and Grassmannian algebras, along with the natural inclusion of complex numbers and Hamilton's quaternions [Gra62, Ham66]. Applications of Clifford algebras can be found in robotics [BCRLZE06, HZBC08], computer graphics [WCL05, BTH22], signal processing [Hit12, BMQQS $^{+}$ 21], and animation [HP10, CC12]. In the context of machine learning, Clifford algebras and hypercomplex numbers have been employed to improve the performance of algorithms in various tasks. For example, [MFW21] learn an equivariant embedding using geometric neurons used for classification tasks. Further, [Spe21] introduce geometric algebra attention networks for point cloud problems in physics, chemistry, and biology. More recently, [BBWG22] introduce Clifford neural layers and Clifford Fourier transforms for accelerating solving partial differential equations.  $\mathrm{[RGdK^{+}23]}$  continue this direction, strengthening the geometric inductive bias by the introduction of geometric templates. Concurrently with this work, [BDHBC23] develop the geometric algebra transformer. Further, [LK14, PRM $^{+}$ 18, ZXXC18] introduce complex-valued and quaternion-valued networks. Finally, [KIdHN23] define normalizing flows on the group of unit quaternions for sampling molecular crystal structures.
3 Theoretical Results
In order to construct equivariant multivector-valued neural networks, we outline our theoretical results. We first introduce the following theorem regarding the multivector grading of the Clifford algebra, which is well-known in case the algebra's metric is non-degenerate. Although the general case, including a potentially degenerate metric, appears to be accepted, we were unable to find a self-contained proof during our studies. Hence, we include it here for completeness.
Theorem 3.1 (The multivector grading of the Clifford algebra). Let  $e_1, \ldots, e_n$  and  $b_1, \ldots, b_n$  be two orthogonal bases of  $(V, \mathfrak{q})$ . Then the following sub-vector spaces  $\operatorname{Cl}^{(m)}(V, \mathfrak{q})$  of  $\operatorname{Cl}(V, \mathfrak{q})$ ,
2If  $k_{i} = 0$  the product of vectors  $v_{i,1}\dots v_{i,k_i}$  is empty, and, in this case, we mean the unit 1 in  $\operatorname {Cl}(V,\mathfrak{q})$
$m = 0,\dots ,n$  , are independent of the choice of the orthogonal basis, i.e.,
$$
\operatorname {C l} ^ {(m)} (V, \mathfrak {q}) := \operatorname {s p a n} \left\{e _ {A} \mid A \subseteq [ n ], | A | = m \right\} \stackrel {!} {=} \operatorname {s p a n} \left\{b _ {A} \mid A \subseteq [ n ], | A | = m \right\}. \tag {1}
$$
The proof can be found in Theorem D.27. Intuitively, this means that the claims made in the following (and the supplementary material) are not dependent on the chosen frame of reference, even in the degenerate setting. We now declare that the Clifford algebra  $\mathrm{Cl}(V,\mathfrak{q})$  decomposes into an orthogonal direct sum of the vector subspaces  $\mathrm{Cl}^{(m)}(V,\mathfrak{q})$ . To this end, we need to extend the bilinear form  $\mathfrak{b}$  from  $V$  to  $\mathrm{Cl}(V,\mathfrak{q})$ . For elements  $x_{1},x_{2},x\in \mathrm{Cl}(V,\mathfrak{q})$ , the extended bilinear form  $\bar{\mathfrak{b}}$  and the extended quadratic form  $\bar{\mathfrak{q}}$  are given via the projection onto the zero-component (explained below), where  $\beta :\mathrm{Cl}(V,\mathfrak{q})\to \mathrm{Cl}(V,\mathfrak{q})$  denotes the main anti-involution of  $\mathrm{Cl}(V,\mathfrak{q})^3$ :
$$
\bar {\mathfrak {b}}: \operatorname {C l} (V, \mathfrak {q}) \times \operatorname {C l} (V, \mathfrak {q}) \rightarrow \mathbb {F}, \quad \bar {\mathfrak {b}} \left(x _ {1}, x _ {2}\right) := \left(\beta \left(x _ {1}\right) x _ {2}\right) ^ {(0)}, \quad \bar {\mathfrak {q}} (x) := \bar {\mathfrak {b}} (x, x). \tag {2}
$$
Note that by construction, both  $\bar{\mathsf{b}}$  and  $\bar{\mathfrak{q}}$  reduce to their original versions when restricted to  $V$ . Using the extended quadratic form, the tuple  $(\mathrm{Cl}(V,\mathfrak{q}),\bar{\mathfrak{q}})$  turns into a quadratic vector space in itself. As a corollary (see Corollary D.30), the Clifford algebra has an orthogonal-sum decomposition w.r.t. the extended bilinear form  $\bar{\mathsf{b}}$ :
$$
\operatorname {C l} (V, \mathfrak {q}) = \bigoplus_ {m = 0} ^ {n} \operatorname {C l} ^ {(m)} (V, \mathfrak {q}), \quad \dim \operatorname {C l} ^ {(m)} (V, \mathfrak {q}) = \binom {n} {m}. \tag {3}
$$
This result implies that we can always write  $x \in \operatorname{Cl}(V, \mathfrak{q})$  as  $x = x^{(0)} + x^{(1)} + \dots + x^{(n)}$ , where  $x^{(m)} \in \operatorname{Cl}^{(m)}(V, \mathfrak{q})$  denotes the grade- $m$  part of  $x$ . Selecting a grade defines an orthogonal projection:
$$
\left(_ {-}\right) ^ {(m)}: \operatorname {C l} (V, \mathfrak {q}) \rightarrow \operatorname {C l} ^ {(m)} (V, \mathfrak {q}), \quad x \mapsto x ^ {(m)}, \quad m = 0, \dots , n. \tag {4}
$$
Let us further introduce the notation  $\mathrm{Cl}^{[0]}(V,\mathfrak{q})\coloneqq \bigoplus_{m\text{even}}^{n}\mathrm{Cl}^{(m)}(V,\mathfrak{q})$ , whose elements are of even parity, and  $\mathrm{Cl}^{[1]}(V,\mathfrak{q})\coloneqq \bigoplus_{m\text{odd}}^{n}\mathrm{Cl}^{(m)}(V,\mathfrak{q})$  for those of odd parity. We use  $x = x^{[0]} + x^{[1]}$  to denote the parity decomposition of a multivector.
3.1 The Clifford Group and its Clifford Algebra Representations
Let  $\operatorname{Cl}^{\times}(V, \mathfrak{q})$  denote the group of invertible elements of the Clifford algebra, i.e., the set of those elements  $w \in \operatorname{Cl}(V, \mathfrak{q})$  that have an inverse  $w^{-1} \in \operatorname{Cl}(V, \mathfrak{q})$ :  $w^{-1}w = ww^{-1} = 1$ . For  $w \in \operatorname{Cl}^{\times}(V, \mathfrak{q})$ , we then define the (adjusted) twisted conjugation as follows:
$$
\rho (w): \operatorname {C l} (V, \mathfrak {q}) \rightarrow \operatorname {C l} (V, \mathfrak {q}), \quad \rho (w) (x) := w x ^ {[ 0 ]} w ^ {- 1} + \alpha (w) x ^ {[ 1 ]} w ^ {- 1}, \tag {5}
$$
where  $\alpha$  is the main involution of  $\operatorname{Cl}(V, \mathfrak{q})$ , which is given by  $\alpha(w) := w^{[0]} - w^{[1]}$ . This map  $\rho(w): \operatorname{Cl}(V, \mathfrak{q}) \to \operatorname{Cl}(V, \mathfrak{q})$ , notably not just from  $V \to V$ , will be essential for constructing equivariant neural networks operating on the Clifford algebra. In general,  $\rho$  and similar maps defined in the literature do not always possess the required properties (see Motivation E.1). However, when our  $\rho$  is restricted to a carefully chosen subgroup of  $\operatorname{Cl}^{\times}(V, \mathfrak{q})$ , many desirable characteristics emerge. This subgroup will be called the Clifford group<sup>4</sup> of  $\operatorname{Cl}(V, \mathfrak{q})$  and we define it as:
$$
\Gamma (V, \mathfrak {q}) := \left\{w \in \mathrm {C l} ^ {\times} (V, \mathfrak {q}) \cap \left(\mathrm {C l} ^ {[ 0 ]} (V, \mathfrak {q}) \cup \mathrm {C l} ^ {[ 1 ]} (V, \mathfrak {q})\right) \mid \forall v \in V, \rho (w) (v) \in V \right\}. \tag {6}
$$
In words,  $\Gamma(V, \mathfrak{q})$  contains all invertible (parity) homogeneous elements that preserve vectors ( $m = 1$  elements) via  $\rho$ . The special Clifford group is defined as  $\Gamma^{[0]}(V, \mathfrak{q}) \coloneqq \Gamma(V, \mathfrak{q}) \cap \mathrm{Cl}^{[0]}(V, \mathfrak{q})$ .
Regarding the twisted conjugation,  $\rho(w)$  was ensured to reduce to a reflection when restricted to  $V$  - a property that we will conveniently use in the upcoming section. Specifically, when  $w, x \in \mathrm{Cl}^{(1)}(V, \mathfrak{q}) = V$ ,  $w \in \mathrm{Cl}^{\times}(V, \mathfrak{q})$ , then  $\rho(w)$  reflects  $x$  in the hyperplane normal to  $w$ :
$$
\rho (w) (x) = - w x w ^ {- 1} \stackrel {!} {=} x - 2 \frac {\mathfrak {b} (w , x)}{\mathfrak {b} (w , w)} w. \tag {7}
$$
Next, we collect several advantageous identities of  $\rho$  in the following theorem, which we elaborate on afterwards. For proofs, consider Lemma E.8, Theorem E.10, and Theorem E.29.
3Recall that any  $x\in \operatorname {Cl}(V,\mathfrak{q})$  can be written as a linear combination of vector products.  $\beta$  is the map that reverses the order:  $\beta \left(\sum_{i\in I}c_i\cdot v_{i,1}\dots v_{i,k_i}\right)\coloneqq \sum_{i\in I}c_i\cdot v_{i,k_i}\cdot \cdot \cdot v_{i,1}$  . For details, see Definition D.18.
4We elaborate shortly on the term Clifford group in contrast to similar definitions in Remark E.14.
$$
\begin{array}{c} \overbrace {\operatorname {C l} (V , \mathfrak {q}) \times \cdots \times \operatorname {C l} (V , \mathfrak {q})} ^ {\ell \text {t i m e s}} \xrightarrow {F} \operatorname {C l} (V, \mathfrak {q}) \\ \Biggl \downarrow \rho (w) \Biggl \downarrow \rho (w) \Biggl \downarrow \rho (w) \\ \operatorname {C l} (V, \mathfrak {q}) \times \dots \times \operatorname {C l} (V, \mathfrak {q}) \xrightarrow {F} \operatorname {C l} (V, \mathfrak {q}) \end{array}
$$
$$
\begin{array}{c} \operatorname {C l} (V, \mathfrak {q}) \xrightarrow {(-) ^ {(m)}} \operatorname {C l} ^ {(m)} (V, \mathfrak {q}) \\ \Biggl \downarrow \rho (w) \quad \Biggl \downarrow \rho (w) \\ \operatorname {C l} (V, \mathfrak {q}) \xrightarrow {(-) ^ {(m)}} \operatorname {C l} ^ {(m)} (V, \mathfrak {q}) \end{array}
$$
Figure 2: Commutative diagrams expressing Clifford group equivariance with respect to the main operations: polynomials  $F$  (left) and grade projections  $(\underline{\_})^{(m)}$  (right).
Theorem 3.2. Let  $w_{1}, w_{2}, w \in \Gamma(V, \mathfrak{q})$ ,  $x_{1}, x_{2}, x \in \mathrm{Cl}(V, \mathfrak{q})$ ,  $c \in \mathbb{F}$ ,  $m \in \{0, \dots, n\}$ .  $\rho$  then satisfies:
1. Additivity:  $\rho (w)(x_1 + x_2) = \rho (w)(x_1) + \rho (w)(x_2)$
2. Multiplicativity:  $\rho (w)(x_1x_2) = \rho (w)(x_1)\rho (w)(x_2),\quad and:\quad \rho (w)(c) = c,$
3. Invertibility:  $\rho (w^{-1})(x) = \rho (w)^{-1}(x)$
4. Composition:  $\rho(w_2)(\rho(w_1)(x)) = \rho(w_2w_1)(x)$ , and:  $\rho(c)(x) = x$  for  $c \neq 0$ ,
5. Orthogonality:  $\bar{\mathfrak{b}}\left(\rho (w)(x_1),\rho (w)(x_2)\right) = \bar{\mathfrak{b}}\left(x_1,x_2\right)$
The first two properties state that  $\rho(w)$  is not only additive, but even multiplicative regarding the geometric product. The third states that  $\rho(w)$  is invertible, making it an algebra automorphism of  $\operatorname{Cl}(V, \mathfrak{q})$ . The fourth property then states that  $\rho: \Gamma(V, \mathfrak{q}) \to \operatorname{Aut}_{\mathbf{Alg}}(\operatorname{Cl}(V, \mathfrak{q}))$  is also a group homomorphism to the group of all algebra automorphisms. In other words,  $\operatorname{Cl}(V, \mathfrak{q})$  is a linear representation of  $\Gamma(V, \mathfrak{q})$  and, moreover, it is also an algebra representation of  $\Gamma(V, \mathfrak{q})$ . Finally, the last point shows that each  $\rho(w)$  generates an orthogonal map with respect to the extended bilinear form  $\overline{\mathfrak{b}}$ . These properties yield the following results (see also Theorem E.16, Corollary E.18, Figure 2).
Corollary 3.3 (All grade projections are Clifford group equivariant). For  $w \in \Gamma(V, \mathfrak{q})$ ,  $x \in \operatorname{Cl}(V, \mathfrak{q})$  and  $m = 0, \ldots, n$  we have the following equivariance property:
$$
\rho (w) \left(x ^ {(m)}\right) = \left(\rho (w) (x)\right) ^ {(m)}. \tag {8}
$$
In particular, for  $x \in \mathrm{Cl}^{(m)}(V, \mathfrak{q})$  we also have  $\rho(w)(x) \in \mathrm{Cl}^{(m)}(V, \mathfrak{q})$ .
This implies that the grade projections:  $\operatorname{Cl}(V, \mathfrak{q}) \to \operatorname{Cl}^{(m)}(V, \mathfrak{q})$  are  $\Gamma(V, \mathfrak{q})$ -equivariant maps, and, that each  $\operatorname{Cl}^{(m)}(V, \mathfrak{q})$  constitutes an orthogonal representation of  $\Gamma(V, \mathfrak{q})$ . The latter means that  $\rho$  induces a group homomorphisms from the Clifford group to the group of all orthogonal invertible linear transformations of  $\operatorname{Cl}^{(m)}(V, \mathfrak{q})$ :
$$
\rho^ {(m)}: \Gamma (V, \mathfrak {q}) \rightarrow \mathrm {O} \left(\operatorname {C l} ^ {(m)} (V, \mathfrak {q}), \bar {\mathfrak {q}}\right), \quad \rho^ {(m)} (w) := \rho (w) | _ {\operatorname {C l} ^ {(m)} (V, \mathfrak {q})}. \tag {9}
$$
Corollary 3.4 (All polynomials are Clifford group equivariant). Let  $F \in \mathbb{F}[T_1, \ldots, T_\ell]$  be a polynomial in  $\ell$  variables with coefficients in  $\mathbb{F}$ ,  $w \in \Gamma(V, \mathfrak{q})$ . Further, consider  $\ell$  elements  $x_1, \ldots, x_\ell \in \mathrm{Cl}(V, \mathfrak{q})$ . Then we have the following equivariance property:
$$
\rho (w) \left(F \left(x _ {1}, \dots , x _ {\ell}\right)\right) = F \left(\rho (w) \left(x _ {1}\right), \dots , \rho (w) \left(x _ {\ell}\right)\right). \tag {10}
$$
To prove that  $\rho(w)$  distributes over any polynomial, we use both its additivity and multiplicativity regarding the geometric product.
By noting that one can learn the coefficients of such polynomials, we can build flexible parameterizations of Clifford group equivariant layers for quadratic vector spaces of any dimension or metric. We involve grade projections to achieve denser parameterizations. More details regarding neural network constructions follow in Section 4.
3.2 Orthogonal Representations
To relate to the orthogonal group  $\mathrm{O}(V, \mathfrak{q})$ , the set of invertible linear maps  $\Phi : V \to V$  preserving  $\mathfrak{q}$ , first note that Equation (5) shows that for every  $w \in \mathbb{F}^{\times}$  we always have:  $\rho(w) = \mathrm{id}_{\mathrm{Cl}(V, \mathfrak{q})}$ . So the action  $\rho$  on  $\operatorname{Cl}(V, \mathfrak{q})$  can already be defined on the quotient group  $\Gamma(V, \mathfrak{q}) / \mathbb{F}^{\times}$ . Moreover, we have:
Theorem 3.5 (See also Remark E.31 and Corollary E.32). If  $(V, \mathfrak{q})$  is non-degenerate<sup>5</sup> then  $\rho$  induces a well-defined group isomorphism:
$$
\bar {\rho} ^ {(1)}: \Gamma (V, \mathfrak {q}) / \mathbb {F} ^ {\times} \stackrel {{\sim}} {{\longrightarrow}} \mathrm {O} (V, \mathfrak {q}), \quad [ w ] \mapsto \rho (w) | _ {V}. \tag {11}
$$