GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
Authors: Ekaterina Filimoshina
Dmitry Shirokov
Venue: ICML 2025 poster (2025)
Keywords: equivariant neural network
geometric deep learning
geometric algebra
Clifford algebra
pseudo-orthogonal groups
Lipschitz groups
spin groups
weight sharing
equivariance
Primary area: deep_learning
TL;DR: New equivariant neural network that balances between expressiveness of geometric algebra-based models and parameter-efficiency.

Abstract: We propose, implement, and compare with competitors a new architecture of equivariant neural networks based on geometric (Clifford) algebras: Generalized Lipschitz Group Equivariant Neural Networks (GLGENN). These networks are equivariant to all pseudo-orthogonal transformations, including rotations and reflections, of a vector space with any non-degenerate or degenerate symmetric bilinear form. We propose a weight-sharing parametrization technique that takes into account the fundamental structures and operations of geometric algebras. Due to this technique, GLGENN architecture is parameter-light and has less tendency to overfitting than baseline equivariant models. GLGENN outperforms or matches competitors on several benchmarking equivariant tasks, including estimation of an equivariant function and a convex hull experiment, while using significantly fewer optimizable parameters.

GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
Ekaterina Filimoshina 12
Dmitry Shirokov 13
Abstract
protein structures (Townshend et al., 2021; Liu et al., 2024;
Satorras et al., 2021; Pepe et al., 2024; Fuchs et al., 2020), estimating arterial wall-shear stress (Brehmer et al., 2023), processing tasks involving point clouds (Thomas et al., 2018;
We propose, implement, and compare with com- petitors architecture of equivariant neural networks based on geometric (Clifford) algebras: Generalized Lipschitz Group Equivariant Neu- ral Networks (GIGGENN.. These networks are equivariant to all pseudo-orthogonal transforma- tions, including rotations and reflections, vec- tor space with any non-degenerate degenerate symmetric bilinear form. We propose weight- sharing parametrization technique that takes into account the fundamental structures and operations of geometric algebras. Due to this technique, GLGENN architecture is parameter-light and has less tendency to overfitting than baseline equiv- ariant models. GLGENN outperforms or matches competitors several benchmarking equivariant tasks, including estimation of an equivariant func- tion and convex hull experiment, while using significantly fewer optimizable parameters.
Fuchs et al., 2020), motion capture (Liu et al., 2024), robotic planning (Brehmer al., 2023), etc. Early works addressing equivariance with respect pseudo-orthogonal transforma- tions (pseudo-orthogonal groups) include Cohen & Welling, 2016; 2017; Weiler & Cesa, 2019; Thomas et al.. 2018; 8; Weiler et al., 2018; Anderson et al., 2019; Finzi et al., 2021.
Veriac
Trieches
Voders
Vesten
arft it
JoNecton
ID-Veten
Orthogonal
Neural Neework (GLGENN)
Trunduraation
1. Introduction
Stalux
Vesteen
Binectar Nesturs
Iifecttor Visters
Sake V-eesa
Vartes S-Veeere
Bivernas Shedte
Tavato PYeass
Equivariant neural networks are a class of neural networks that explicitly incorporate symmetries into their architec- ture, making them well-suited for tasks that inherently re- quire equivariance transformations with respect some group's action (e.g., rotations, permutations, translations, etc.). Equivariant neural networks were introduced in Cohen & Welling, 2016 and have since been extensively developed and applied in a wide range of tasks in computer and natural sciences. These applications include modeling dynamical systems (Finzi et al., 2021), particle physics (Ruhe et al., 2023; Finzi al., 2021), analyzing molecular properties and
Vostoss
S-Vertao
Orthegonal
Netwoek (GLGENN)
Scalers
Vietues
Strecter
Soledee
V-Veroe
Figure 1. GLGENN is architecture of neural networks equivari- ant with respect to any pseudo-orthogonal transformation. Inputs and outputs are represented as multivectors (elements of geomet- ric algebras), which encade various geometric quantities such as scalars, vectors, oriented areas (bivectors) and volumes (trivectors). and nigher-dimensional objects (4-vectors, etc.). GLGENN are parameter- light, since they operate in unified manner across fundamental subspaces of geometric algebras defined by the grade involution (") and reversion ("); they processes geometric objects in groups with a step size of 4.
'HSE University, Moscow, Russia Skolkovo Institute of Science and Technology. Moscow, Russia Institute for In- formation Transmission Problems of the Russian Academy Ekate-
of Sciences, Moscow, Russia. Correspondence rina Filimoshina <fllimoshincck comi,,com>, Dmitry Shirokov <dm.shirokove @ gmai..omm>
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Design and implementation of GLGGNNN. We construct a novel, parameter-light architecture of pseudo-orthogonal groups equivariant neural networks based on geometric algebras. Code is available at GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
This work introduces a novel neural network architecture that is equivariant with respect any pseudo-orthogonal transformation, including rotations and reflections. Our ap- proach is based on the well- well-known mathematical framework of (Clifford) geometric algebras (GAs), which have found applications in various scientific fields, including physics (Doran & Lasenby, 2003; Hestenes & Sobezyk, 1984), com- puter science Bayyr-Corroochao, 2019; Brandstetter al., 2023; Dorst et al., 2002), engineering (Dorst et al., 2002), and other scientific fields. Works on applications of GAs in neural networks include Pearson & Bisset, 1994; Bayro- Corrochano & Buchholz, 1997; Buchholz & Sommer, 2008; Kuroe, 2011; Brandstetter et al., 2023.
of this work is present novel, parameter-light equivari- ant neural network architecture, which balances between expressiveness of CGENN and parameter efficiency. We call this architecture Generalized Lipschitz Group Equivari- ant Neural Networks (GIGENN)), see Fig. 1. Our approach introduces a new parameter-light parametrization technique, which enables our model to either outperform or match ex- isting equivariant models while significantly reducing the number of trainable parameters. The key idea is to design novel weight-sharing approach (Lecun et al., 1998) for GA-based neural networks that respects the fundamental algebraic structures of GAs, thereby improving efficiency without sacrificing expressive power.
In particular, GAs provide an efficient and elegant repre- sentation of pseudo- orthogonal transformations via spin groups, Lipschitz groups (sometimes also called Clifford groups in the literature), and twisted adjoint representations. Specifically, for any pseudo-orthogonal matrix, there ex- ists a corresponding element of the Lipschitz group in GAs that induces the same transformation of vectors through the twisted adjoint action. The transition from matrix formal- ism to GA formalism is highly beneficial and inspiring for applications, as it introduces a rich set of operations that are naturally defined within the GA framework.
The key contributions are as follows:
Introduction of generalized Lipschitz groups. We introduce and study new class of Lie groups in arbi- trary geometric algebra, which are related pseudo- orthogonal groups and useful for construction of equiv- ariant neural networks.
The idea of leveraging GAs to construct pseudo-orthogonal group equivariant neural networks was first proposed by the Amsterdam Machine Learning Lab (University of Amster- dam). Their pioneering work Ruhe et al., 2023 introduced Clifford Group Equivariant Neural Networks (CGENN) and served as the foundation for many subsequent research works. Brehmer et al., 2023 propose Geometric Algebra Transformer (GATr), which incorporates GA into ordinary transformer architecture and outperforms traditional non- geometric baselines in N-body modeling and robotic plan- Stterrrbbee Con-
Superior performance. GLGENN achieve state-of- the-art performance on benchmark equivariant tasks with significantly fewer trainable parameters.
The paper is organized as follows. Section 2 provides all necessary definitions related to GAs and equivariant neural networks. In Section 3, we present our main theoretical results and introduce and study generalized Lipschitz groups Section 4 applies these results for GLGENN construction. In Section 5, we evaluate GL GLGGENN through experiments. All the mathematical details and proofs can be found in the appendix.
ning. Zhdanov et al., 2024 introduce volutional Neural Networks based on CGENN approach, demonstrating superior performance in tasks related to fluid dynamics and relativistic electrodynamics forecasting. Liu et al., 2024 introduce Clifford Group Equivariant Simplicial Message Passing Networks, a method for steerable E(n)- equivariant message passing on simplicial complexes. Pepe et al., 2024 use CGENN to predict protein coordinates to estimate the 3D structure of a protein.
'Application of the generalized Lipschitz groups instead of ordinary Lipschitz groups allows to achieve parameter efficiency. The generalized Lipschitz groups are important because they pre- serve the four fundamental subspaces of GAs under the significant operation of the twisted adjoint representation. We prove that equivariance of a mapping w.r.t. these groups, as well as the ordi- nary Lipschitz groups, implies its orthogonal groups equivariance. The key distinction is that the generalized Lipschitz groups contain ordinary Lipschitz groups as subgroups. As result, the set of operations cquivariant w.r.t. the generalized Lipschitz groups a subset of the set of operations cquivariant w.r.t. ordinary Lipschitz groups This reduction in the number of 'degrees of freedom' encourages us to parametrize operations in layers in a more 'eco- nomic' way (there is smaller number of parameters that we can place). Specifically, in all GLGENN layers, we employ such equiv- ariant operations as projections of inputs-multivectt four fundamental subspaces of GAs mentioned above, whereas CGENN relies on projections onto the subspaces of fixed grades. GLGENN and CGENN layers parameterize linear combinations, products, and normalizations the corresponding projections.
The above mentioned papers demonstrate the importance of Lipschitz groups in GA as fundamental tool for build- ing expressive ththe-of-the-art equivariant neural networks. However, one major challenge in the design of GA-based equivariant neural networks is overparameterivation. This issue often leads to a tendency to overfit, particularly in cases where the training dataset is small - a common sce- nario in natural science applications where such networks typically employed. Additionally, the excessive number of parameters results in inefficient training times. The goal
onto the
2

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
2. Theoretical Background
Table I. Signs of the projections of U = (U)a + (U)x + (U)) +
2.1. Geometric (Cliffiord) Algebras
(U)s E Clp.a.r for the grade involution (*), ) reversion ( ("), and Clifford conjugation acting on it.
Let us consider (Clifford) geometric algebra (GA) (Hestenes & Sobezyk, 1984; Lounesto, 1997; Porteous, 1995) CB(V) = Cp P 4 = n 2 1, over a vector space V with a symmetric bilinear form b and the corresponding quadratic form 9, where V can be real RP... complex Cetauu, We use If to denote the field of real numbers IR in the first case and the field of complex numbers C in the second case. In this work, we consider both the case of the non-degenerate GAs Clp. == p. r = 0, and the case of the degenerate GAs Cpp..r. r * O. We use A, to denote the subalgebra Cdo,,,r. which is the Grassmann (ex- terior) algebra (Doran & Lasenby, 2003; Lounesto, 1997). A well- example of geometric algebra is the spacetime algebra 1,1 = CT associated with the Minkowski space IR which plays a central role in relativistic physics. For the readers' convenience, in Appendix B, we illustrate all key thcoretical concepts introduced in this section with examples in the setting of C(1,3-
Cep.g.r
k =
0
k= 1
k= 2
k == 3
+
+
+
+
+
We can represent any element U E Cl C.p.q.r as a sum = (U) + (U)(), (U)() E
Cel9 p.g.rs = 0, We use angle brackets ())) to denote the operation of projection of multivectors and sets onto Ce The grade involution and reversion define four subspaces Cfp. CL Cl Crar and CP p.g.r (they are called the subspaces of quaternion types 0, 1, 2, and 3 respectively in Shirokov, 2012a;b):
Ce Cf.rr (U E Clp.p::: Ú=
Ü (-1 = U}.
k= 0,2,,,
(4)
Ct. Cbp... C2p. @ C2 @ for k = 0, 1, A discussion on the significance of the grade involution, reversion, and these subspaces is provided in Appendix C. Note that the GA Clp.g.r can be represented as direct sum of the subspaces Cē C.p.g.. k 0, 1, 2,3, and viewed as Z x Z2 -gggggd algebra with respect to the commutator and anticommutator (Shirokov, 2018), 8). We can represent any element U E &CEp.g.r as a sum of 4 elements: U = (U) + + +UU), where (U) E Ofp...- m- = 0, 1, 3,3. The action of the grade involution, the form above is summarized in Table 1. Note that Ceff p.a.r = Cpkk p.g.r h 0, 2, in the cases S 3.
In other words, Cek C.
The identity element of Cg is denoted by e = 1, the generators are denoted by ea. I,..., n. The generators satisfy the following conditions: ca + cbCa- = sase for any a, b = ,,...,,,, n. where 1 = (7as) is the diagonal matrix with times- +1, q times - 1, and r times O on the diagonal in the real case C(((B....) and + q times +1 and r times 0 on the diagonal in the complex case CE(Crta0:r).
Let us consider the subspaces Cb fixed grades k = O,..., Their elements are linear combinations of the basis elements ca....a = Ca. ... Eag. a1 < .... < ag. The grade- O subspace can be denoted by CC without the lower indices P, 9, r, since it does not depend on the GA's signature. We have Oek p.q. = {0} for < and >> n. We can represent any element (multivector)) U Clp.rr as a sum of n + elements U- (U)o +... + (U)m. (U)x Cek Ca ks
reversion, and Clifford conjugation on a multivector
2.2. Lipschitz Groups and Twisted Adjoint Representations
0, O,...,9h. We call any operation of the form
We use the upper index x to denote the subset Hx of all invertible elements of any set H. Let us consider the adjoint representation ad acting on the group of all invertible ele- ments ad CPX p.g.r + Aut(CCpmar) as T + ady, where adr C2p.r + Clp...:
U C (U) K=0
=1=
(L)
a conjugation operation in Ctp.g.-- Consider conjugation operations called grade involution and reversion. The grade involute of an element U Clpar is denoted by U and the reversion is denoted by Ü. and they are defined for an arbitrary U E Clp.grr as
ad)(UU)=TUTT
== -UTT1, U.E.pp T CPX Clpar
(5)
Also let us consider the rwisted adjoint representation. This notion is introduced by Atiyah, Bott, and Shapiro (Atiyah et al., 1964) in particular case, and there are two ways how generalize it (see motivation in Appendix E). The first approach (Choi et al., 2002; Harvey, 1990; Lundholm & Svensson, 2009) is to define it as the operation ad acting on the group of all invertible elements add CPX p.g.r + Aut(Clp...r) as T + ady with adr Clpa. + Opaari
n
Ô (-))
Ú== k=
(U)
(2)
The composition of grade involution and reversion is called Clifford conjugation. The Clifford conjugate of U E Cl
is denoted by Ù. The grade involution defines the even celou r and odd Copldr subspaces:
CVCL p.q.r = (U E Cbpar: Ū= (-1)U,, 0, (3)
adr(U)= = TUT-1, UE E Clpar T CPX C.
(6)
3

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
The second approach (Helmstetter & Micali, 2008; Knus, 1991; Walpuski, 2022) is to define the twisted adjoint rep- resentation as the operation ad: CP P.g.r + Aut(Clparr) (ppar)
degenerate or non-degenerate geometric algebra Cl ppp..r For GLGENN, we are mainly interested in ITT p.q.rr however other groups I'VE p.q.r k = 3, are necessary to prove the main statements about 1 p.g.r and serve as auxiliary tools. The (ordinary) Lipschitz groups p (10) preserve the subspace C6 of the first grade under the twisted adjoint representation ad (6). The generalized Lipschitz groups preserve the subspaces Cb. p.g.r k = 0, 1, 2, 3, determined by the grade involution and reversion (4), under the same representation ad:
acting as T + adr with adr: .lpai + Oppari
= TU)) (+
(7)
for any U E Clp..rr and T E .. The representation ad
has the following properties, which we prove in Lemma F.2
and apply in construction of equivariant a layers in Section 3.3.
The properties of ad and ad, are considered in Lemma E.2 as well.
Lemma 2.1. Let W& E Opg.. U.V E Clp.prr and
i& p.g.r := (T E CE
adr (Cep..))))) S OC p. (13)
TE (Cepax U ) A. where the notation (Ce U
Cep ) := {ab
E Cepax U pir b E A
The groups I'VE p.g.r. k = 0,1, can be considered as generalizations of Lipschitz groups because of the following theorem.
stands for the product of two groups. Then adw satisfies:
adw (U + +V adw (U) - (V). adw (c) c (8)
Theorem 3.1. The (ordinary) Lipschitz groups are sub- groups of the generalized Lipschitz groups and coincide with some of them in the low-dimensional case:
for any c E Ct Moreover, adl satisfies multiplicativity:
(UV) adr(U))adr())
(9)
Consider the Lipschitz groups i" p.g.r (Benn & Tucker, 1987:
[ p.g.r S F.p.rr S F
p.g. k 0,,,,,,, Vn; (14)
Lounesto, 1997; Ablamowiez, 1986; Brooke, 1978; 1980):
Fl
rT p.q.r
5
(15)
r p.g.r := (T where ady (Ce p...r.) = TCK, p.g.r P.. T-1. These groups are
=T E CEX p.g.r: adT((C), p..r)) S CP g. (10)
p.g.r
Proof. The proof is based on the equivalent definitions of TE p.g.r. k 0, 1, 3, and is provided in Appendix F (see Corollary F.4, Remark F6,, and Theorem F.7).
often considered in the literature in the case of the non-
degenerate geometric algebra Clp.g- The definition (10) straightforwardly generalizes the common definition to the case of arbitrary Clp.g.r- The well-known spin groups (see details in Appendix E) in the non-degenerate (Benn & Tucker, 1987; Lounesto, 1997; Porteous, 1995) and de- generate (A (Ablamowicz, 1986; Crumeyrolle, 1990; Brooke, 1978; 1980; Dereli et al., 2010) cases, which have vari- ous applications in physics, are normalized subgroups of iı p.g.r defined using the following norm functions v. X : Cl Cl par + Cl Clp.r of the GA elements:
In this work, construct neural networks that are equivari- ant with respect to the action ad of the generalized Lipschitz groups is p.g.r (13). To prove the main statements for the de- sign of the layers, we need Theorem 3.2 about these groups.
Consider the following groups preserving the subspaces Cgl p.g.rr k 0, 2, under the adjoint representation ad (5) and twisted adjoint representation ad (6) respectively:
Id p.g. = (T E CF
adr (CEk p.g.)) S Ceh a. (16)
v(U):= := ÜU, x(U) ÚU.
(11)
A p.g.r = (T E CPX g ady (CE5 p.g.r) S CP Co (17)
2.3. Equivariant Mappings and Neural Networks Suppose G is group and ox and oy its actions on the sets X and Y respectively. A function (network) L X + Y is called G-equivariant (see, e.g., Bredon, 1972; Pitts, 2013; Yarotsky, 2022) iff it commutes with these actions:
where adr (Cr (Cr)...)) TCEE P......T- and aCr (.r
TCy p.g.- p.g..--- T- defined in an equivalent way, using only the norm functions vé and (11) applied in the theory of spin groups
Further, we show that these groups can be
Let us introduce the families of Lie groups Q. OZ Q3 Oz and 2z
Tig ox x)- = oy L(x), Vg E G, VX X. (12)
The definition means that we get the same output if we trans- form the input to the neural network or transform the output. We prove several general statements about equivariance with respect to an arbitrary group in Appendix G.
Op == (T E CPX
TT = Z
(18)
Q (T E CP
TT. TT b
(19)
and Oa 0g.g.r Qg Qo.rri
3. Theoretical Results
OF Op.9. {T E CbX
TT,T7 b }
(20)
3.1. Generalized Lipschitz Groups
S == {T E CPx :
TT E (4 (21)
In this section, we introduce and study the generalized Lip- schitz groups FR
p.q.r. k = 0, 1, 2, 3, in the case of arbitrary
with k: = 1,2,3, where the sets
4

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
pag.- (X Op...:: XV VV=VX, We WEC p.rr) are the centralizers of the subspaces Of p.g.r in Clp.grr and
the generalized Lipschitz groups i p.g.r (Theorem 3.1), (2) elements of the generalized Lipschitz groups F p.g.r preserve not only the subspace CP but also all other subspaces C pig.r m72 0, 2, 3, under the twisted adjoint representa- tion ad (see (28)-(29)), and (3) elements of I'T have a special form (Theorem 3.3).
2 == {X E Olpar: XV VX, VV E C p.r.)
are the swisted centralizers of Clpa.r in Clpa. These sets are studied in detail in Filimoshina & Shirokov, 2024b. We provide explicit forms of 4m and p.g.r m S 4, in Remark D.I.
p.g.r
p.g.
3.2. Pseudo-orthogonal Groups, Complex Orthogonal Groups, and Generalized Lipschitz Groups Equivariance
Theorem 3.2. In degenerate and non-degenerate geometric algebras p.p. we have for k- 2,
Let us consider the degenerate and non-degenerate pseudo- orthogonal group (in the real case V = -R----" or complex orthogonal group (in the complex case V CP+...0.r) de- noted by O(V, q). which is the Lie group of all linear trans- (24) formations of an n--imensional vector space V that leave invariant quadratic form q of signature (p, r) if V is real and (p + q. 0, r) if V is complex (see Appendix H):
- p.q.r Opan. r p.g.r = Opgnr
(22)
I'T p.q.r Opeg.r
E c p.g.r p.g.rs
(23)
1
r p.g.r
Q Qp.g.
p.g.r Qp.gers
Moreover, for m. = 0, 1,
rl p-g.r
S Im
c p.g.r
(25)
pon.r
O(V.9) { V + V
+ is linear, invertible,
P p.g.r
C r
IT P.g.r
r p.g.r
(26)
p.g.rt
(()(v)) = q()),
EV
(30)
Proof. The proof relies on the relation between the preser- vation of the subspaces CP C.p.g.r k = 0, 1,2,3, 3, under ad and the values of the norm functions a and x (11). is provided in Appendix F (see Theorems F.I, F.2 and Remark F.3). D
When considering both the real and complex cases, we refer to the group O(V, q) as the orthogonal group. Consider the following subgroup of the orthogonal group, which leaves invariant the radical subspace A, := Ci
The generalized Lipschitz groups F pag.r (13), h: = 0, 2, are related to the groups M p.g.r
Oa (V, { E O(V, g): & =dx;}- (31)
(16) and 1 p.g.r (17):
We have the following relation between the Lipschitz group f pgg.r and the corresponding restricted orthogonal group
Fut p.g.
{
D P.g.r
i gam
k= 0=
k: k= 1,
(27)
O (V. q):
I p.ar / A: = (V,q).
ad
(32)
since adr(CCr (.gr..))) (C (..r).))) in the cases k = 0,2
where ad' C.,rr + Aul (C is ad restricted to V (a real 18--. or complex Catu,orr vector space, which can be identified with the space of vectors Ct. Cxp. see page 3),
and ad, (Crk p. = adr((C)) (.ar)) in the cases h = 1,3.
As a corollary of Theorem 3.2 and (27), the generalized Lipschitz groups
1 p.q.r (13) satisfy
+ CC plp.gr defined as (d) = π-T-1 for any vector CC C.p.grr From (32), i' we have that for any T E p.g.r there exists E O ( (V,q) q)
i.e. it acts as T + ad,
CP p.a.r
FT = -QT.ar C - p.g.r = oz yaar P p.g.r PI
p.g. = Q
Opa.r
(28)
P.g. C f p.g.r = 02
(29)
such that adj = & and vice versa. We prove (32) in The- orem 10 In the particular case Clp.g the statement (32)
Note that the group F phar can be regarded as the most significant among the generalized Lipschitz groups i'v p.g.r. k = 0, 1, 3, because it contains the (ordinary) Lipschitz group as a subgroup and is itself a subgroup of all other generalized Lipschitz groups (see Theorem 3.1 and (28)- (29)). We prove that its elements are of special form:
is well-knows (see, for example, Benn & Tucker, 1987). and the similar statements are proved in Crumeyrolle, 1990; Ruhe et al., 2023.
The relation (32) implics that OA₃ (V, q) acts on the whole Clp.g.r in a well-defined way (Ruhe et al., 2023). Namely, for an arbitrary element x = E, u.U;,1 ... Ug,e; E Clp with Va.j E V, u4 E F, for any T E I'l p.g.r with the corre- sponding &b = adly, we have
Theorem 3.3. We have I'T p.g.r S (Cxe))
p.g.r U CCp4. )AX.
Proof. The proof is based on the equivalent definition of the
group (Cr (U)(()x )x and is provided in Theorem F.5.
ady ad) = Σ usad (15,1) (dd
(33)
The key takeaways from this section for constructing the generalized Lipschitz groups F'l p.g.r- equivvaraat layers are as follows: (1) the ordinary Lipsehitz groups are subgroups of
= Σ (u, ....k..))) =: ((x).
(34)
5

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
The following theorem discusses the relation between ON (V,(V,g) (V,9)- 9)- and FJ rquivvarrin....
In Theorem 3.7, we prove that several conjugation opera- tions (1) in Cpp.. are generalized Lipschitz group i equivariant. Note that it does not hold for any conjugation operation. However, we prove that any conjugation op- eration is (ordinary) Lipschitz group I'1 Theorem 3.8.
p.g.r"
Theorem 3.4. I mapping J -Cpar~CCparr is equiv- ariant with respect to any group H that contains the Lips- chitz group I p.g.r as a subgroup, then f is equivariant with respect to the corresponding orthogonal group. That is, if E H and x E Cpp.gr then f()(z)) = ((((x)) for any &b E OV.(V,9) q) and sense (34).
p.g. requivaraat in
adr (f(x)) = (x)r((x) for any T
Theorem 3.7. The grade involution, reversion, and Clif- ford conjugation are p.g.r requvvriaan:: adr(3)
E Clp.g.r. where & acts on
adr - - add((z), adr(x) adr(2) for any T E ET p.g.r and x: E Clp.g.r-
Proof. The proof relies on the direct relation between i'1 p-g-r" and Ox₂(M (V, 9) -eqquivaare (see Theorem H.11).
0
Proof. Since the grade involute, reversion, and Clifford con- jugate of an element x E Ctp.g. are linear combinations of the projections onto the subspaces CC m = 0, 1, 2, 3, (see Table 1) which are IT p.g.r -quiivaquu,r we get the state- ment by Lemma G.3 and Theorem 3.6.
p.g.r - FT p.g.r by Theorem 3.1, all generalized Lip- schitz group FT p.g.r requivarqa mappings are restricted orthogonal group Oa (V, -)-eqivvarinnt as well.
Since F
O Theorem 3.8. Any conjugation operation in Clp.g.r is Lips- nn p.g.r p.g.vvaquvarian.
3.3. Generalized and Ordinary Lipschitz Groups Equivariant Mappings
chitz group I
This section proves that several mappings are generalized Lipschitz groups 1 p.q.r equivariant.
adrr An(()))) - - Aa (a(adr(z))) A = #1,
K=O
k=O
Theorem 3.5. Let T E pO U CE and F E F[Ti, "[TT,,.., Ti] be a polynomial in I variables with coefficients in F. Consider I multivectors Ti,...,21 22 Clp.gr We have the following equivariance property with respect to the group (CX U (e())))
for any T p-q.r. and x E =Cpp.r..
Proof. By definition (1), any conjugation operation is a
linear combination of projections onto the subspaces Cek p.q.r
k 0, .......1 n2, which are I'1 -qquivqquvar (see Corollary 3.3 Ruhe et al., 2023): adr((x)x = (adr(z)) ke for any T E Fi p.g.r and x E C2p.g.- Then, we get the statement by Lemma G.3.
(a)d (x))). (35)
Proof: This fact follows directly from additivity and mul- tiplicativity of adr for T (CO,2x U (x which proved in Lemma 2.1.
Theorem 3.9. The norm functions vb and x (11) are F'l pig.""
equivariant:
LI
(=)()))=(((a))) (x)), adz (((x)))>((ad)) (z)) (41) for any T E iT p.g.r and x E Clpar.
Note that the statement that all polynomials are p...- equivariant, which follows as a corollary of Theorem 3.5, is well-known (Ruhe et al., 2023).
Proof. Since i F S ady is multiplicative for any T E fT p.q.r by Lemma 2.1. The mappings (2) and x(x) are iT p.g.r eqqivvaaann by Lemma G.4, since they are products of the [J -.g.vvvvavarian map- pings: + ā. x + 2. and x + (Theorem 3.7).
P.i:) (Theorem 3,3),
Theorem 3.6. For any x E Cp and m = 1,3,, we have the following equivariant property with respect to the generalized Lipschitz. groups:
adr((x) (()) = (adr (z))
VT E
IT
(36)
p.g.r
4. Methodology
Proof. Suppose T E FI
p.g.r and x = (0) x() Chp.r
where I(() == (z)(0)) and T(1)= == (1) For any
In this section, we describe the architecture of Gener- alized Lipschitz Groups Neural Networks layers. In this and the following sections, we consider the real geometric algebra CE(RPA.)). Code is available at com//aaaalllimnshinggggom We plan to con- tinue developing this repository.
3, we get
(ady(x)) = (T T-1 + (T /m
(37)
= ( Σ T(a)eT + Σ (1)))
(38)
002
1.3
Σ (T
m + Σ (1))
m
(39)
4.1. Geometric Algebra Data Embeddings
GLGENN, along with other GA-based equivariant neural networks such as CGENN (Ruhe et al., 2023), can be ap- plied to any data representable as multivectors. For example, consider the following task in R We have data objects T1.... T1 where each 2; contains a scalar (c.g. a E IR, rep- resenting mass, temperature, or a hot-eededdd feature,
0,2
1,3
(T(x) T
|/m
T is even, 1m = Txxm m is odd,
(40)
(T
where in (39) we apply linearity of projection ().. and in (40), we use Theorem 3.2 and the notes below it.
6

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
etc.), vector (e.g. (b, d) E R where b, e, R. rep- resenting position, velocity, etc.), and pseudoscalar (e.g-. signed volume value f E IR). These components can be embedded in GA as ac + bej + ce2 + de3 + fe123 E Clo,3 and processed by our layers. Additional grades beyond 0, 1, and 3 can be utilized as well. 2 The output can be obtained by projecting the resulting multivectors onto the appropriate subspace based on the task. E.g. projecting onto CC allows for scalar predictions (which will be both equivariant and invariant), or onto CP p.g.r if vector predictions are required.
Cout are used to denote the number of the input and the output channels respectively. In other words, for any fixed k = 0, 2,33, 3, the value (Mo.us) is a linear combination of (Ze..) where Gi = I,...,,, One CC. p.g.- CCp.q.. -lg.g.-- layer is parametrized with 4/m optimizable coefficients, where and m are the numbers of input and output channels respectively. Such layers are I'T p.g.r -equivaraa by Theorems 3.5 and 3.6.
4.4. Cek -Geometric Product Layers p.g.r
Now let us parameterize product interaction terms. We consider only the second-order interactions, because the higher-order interactions are indirectly modeled via multi- plc successive layers (Ruhe et al., 2023). A second-order interaction term for the pair of multivectors 21 and 22 has the form ((21);((2))))) where i, k - 0, 1, All the terms from Ceki p.q.r resulting from the interaction of x) and D2 are parameterized as
4.2. Conjugation Operations Layers
These layers are based the concept of conjugation opera- tions in Cl p.r. (1). Suppose , Ii E plp.gr are input data multivectors, where / is a number of input channels. For an input data multivector Te.. E C.p.g.r. Cin = 1,....,4 a conjugation operations layer is constructed using
Tein
Deink (Fesn)
Pank = =1,
(42)
P(11,
Degn (1gn((11))((22)))) (44)
where Pern k E 1 are optimizable coefficients. The conjugation operation layer is [ p.g.r -qquivaraat by The- orem 3.8 and contains I(n + 1) optimizable parameters. We suggest the following method to make the optimiza- tion of conjugation layers with discrete parameters possible. We first apply a linear transformation of the projections of the input multivector as in (42) but with any parameters. Then we round them to 1 or 1 either by directly applying &cnnk + sgn( or by firstly applying the sigmoid func- tion, then scaling the value to 1]. and applying sign function to the result. Conjugation operations, including and play significant role in GAs and their applica- tions (see Appendix C for a detailed discussion), making these layers highly promising for experiments.
==
where Disk E R are optimizable coefficients. P(#1, 22){ is I -qquivarrrnn by Theorem 3.5 and Theorem 3.6. We get 4 p parameters Φist for each geometric product of a pair of multivectors. Since we have 12 possible pairs of multi- vectors for input channels, parameterizing such number of coefficients can be computationally expensive. There- fore, if number of channels / large, let us compute geometric products for less than 1 pairs of multivectors by an approach proposed in Ruhe et al., 2023. Firstly, we apply to ; 21 Clp.rr a linear map to get .i..... V1 E Clp.gr and then compute the products of the pairs x;, yi for i= 1,...,,, which we denote by 21,..., 21 respectively. Using such a trick, the terms that will be mul- tiplied get learned. Now, we need to caleulate only geo- metric products. We get (Econed == P( yern )², where (e=o( and Can = Cout = -I...... L. In to- tal, for a Ctk p.q.r -geometric product layer, we get 412 + 491 parameters for I input channels.
4.3. Otk p.g.r -Linerr Layers
A Cpk P.g.- -iinerr layer is constructed using
-linear
(Werr) == Σ & &ooutin ( D c
(43)
Ecout := 3
Cn==0
where Brout = Ex (oruah is an output channel, contonsk E R are optimizable coefficients, and ca. and
4.5. CeT p.g. p. Normmalittinn Layers
"Note that in the case of Cla.3, the equivariant model proposed this paper has some connection with equivariant models based on irreducible representations (irreps) of O(3) (Han et al., 2022). Generally speaking. these are two different approaches: irreps of O(3) follow the tensor approach, whereas the GAs operate with multivectors. However, there are connections between the two approaches: scalars, vectors, bivectors, and trivectors in GA are equivalent to = 0 irreps with even parity, 1 = 1 irreps with odd parity, / = 1 irreps with even parity, and I = 0 irreps with odd parity, respectively. The dimensions and respectively), the behavior under rotations/reficctions of the coordinate system matches, and the coupling operations (such as scalar multiplication, vector scalar and cross products in various cases) are also the same. Further study of this connection deserves separate study and is not the topic of the current work, but seems important for building bridges between the two communities.
For numerical stability of Ci -geometrc product lay- ers, we apply normalization to the four projections (rea.) 'o- (Ze.n). (Fein)z, and (Ie.n)z for each multivector Fe... Can = 1,...., bl..or, before geometric product:
(Fein)
(Den)
(45)
alo,, e, (F 1 where o(x) := Ite E (0, is the logistic sigmoid func- tion and Per.m E R are optimizable parameters. The sim- ilar approach is applied in Ruhe et al., 2023. Note that ((ee. (eee.. ) is IT. p.q.-- equivvarant by Theorems 3.6 and 3.9. Each normalization layer contains 4/ optimizable parameters, where / is the number of input channels.
the same (1. 3. 3,
7

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
5. Experiments
Ruhe et al., 2023; Liu et al., 2024. The average convex hull volume in training dataset is 11.4. The results, which are presented in Figure 2 (middle) and Table 3, demon- strate that GLGENN either outperform or match CGENN, which itself outperforms ordinary MI.,, Geometric Vector Perceptrons (Jing et al., 2021), I) and Vector Neurons (Deng et al., 2021). In Table 3, MSE for K = 16 are averaged over 5 runs, number of iterations is the same for both algo- rithms. Notably, as highlighted Ruhe et al., 2023, CGENN tends to overfit when trained on small datasets. In contrast, GL GENN demonstrate a reduced tendency to overfitting. The reason might be that GLGENN achieve these superior results with more than twice the parameter efficiency, us- ing significantly fewer parameters than CGENN. Note that when CGENN is scaled down to approximately the same size as GLGENN (around 25K trainable parameters), its performance deteriorates compared to its original configura- tion (see Table 9). In the case = 7, we get similar results (see Appendix I).
We demonstrate that GLGENN either outperforms or matches the performance of other equivariant models, while using significantly fewer optimizable parameters. For CGENN (Ruhe et al., 2023), the training setups are aligned with its public code release. For other models, we use the loss values from the corresponding code repository (Finzi et al., 2021). All experimental details and discussions can be found in Appendix and our code repository. For simplicity, the current examples focus on the case of the non-degenerate GA Cpp.g. Experiments involving C.p.g. will be explored in future research. We construct our models to closely re- semble the CGENN architecture, replacing the CP% p.a -linear, CPk pp. geoomeric product, and CP nnormaliaainn layers from Ruhe ct al., 2023 with the same number of GLGENN's p.q counterparts (see Section 4).
CF
(55, o0-Regressonn Task
We consider an (55, O)-invariant regression task (Finzi et al., 2021) to estimate the function sin([|=ı])) || 1 =f x2 [0 22|
Following the suggestion of one of the anonymous review- ers, we further increase the difficulty and real-world rel- evance of the experiment by estimating the convex hull volumes of K- = 256 and 512 points in R5.0, In this setup, the average convex hull volumes in the training datasets are R 430.3 and 694 respectively. GLGENN consistently outperforms the best-performing CGENN architecture, with the results summarized in Table Due its significantly smaller number of trainable parameters, GLGENN also re- quire less training time (see Table 7 in Appendix I).
where ;1; 32 IR5.0 are vectors sampled from standard Gaussian distribution. The loss function used is Mean squared error (MSE). We evaluate the performance us- ing four different training dataset sizes and compare against the ordinary MLP, MLP with augmentations, the O55, 0)- and S5 0) equivvariant MLP architectures proposed in (Finzi et al., 2021), and with CGENN (Ruhe et al., 2023). The results, presented in Tables 2 and and Figure 2 (left), demonstrate that GLGENN achieves performance on par with CGENN, while significantly outperforming the other models. Moreover, GL.GENN attains these results with fewer parameters and reduced training time compared to CGENN. In this experiment, CGENN has approximately 1.8K parameters associated with GA layers, while GI .GENN has around 0.6K parameters. Other models have approxi- mately the same number of parameters as CGENN. In Ta- bles 2 and 6. MSE for CGENN and GLGENN are averaged over runs. MSE for other models are averaged over 3 runs (from Finzi et al., 2021). Number of iterations is the same for all algorithms. Note that when CGENN has the same size as GLGENN (with R 0.6K GAaassociated parameters), we get the similar results (Table 10).
Combining GLGENN with Typical Activation Functions Standard activation functions (e.g. SiLU or ReLU) can be applied to elements of Cto (scalars) without breaking equiv- ariance. For simple tasks, the best performance is achieved by combining GLGENN (applied to all grade subspaces) with standard neural network, such as MLP, acting on elements of Ct E.g., in the O(5, -)-Regrsssion Task: (I) a standalone MLP with three ReLU-activated layers performs poorly; (2) GLGENN alone performs reasonably well but converges slower than (3) the combination of GI.GENN (to all grades) MLP (to scalars). With 300 training samples, this improvement is shown in Figure 3 and Table 11. The main limitation is that applying nonlincarities only to tain subspaces (e.g., scalars) prevents interaction between different grades (e.g.. vectors and bivectors), effectively iso- lating them. In contrast, the nonlinearities introduced via geometric product layers inherently mix all grades, creating strong interactions.
055, 0)- and O Convvx Hull Volume Estimation
In these experiments, the task is to estimate the volume of convex hull generated by K points in 5,50 and 7 respec- tively. We first consider the setup with K' = 16 points, as in
Table 2. MSE (L) on the O(5, Regressson Experiment
MODEL
# OF TRAINING SAMPLES
O
Experiment
3 3-10'
3- 10
3.10 100
3- 10
In this benchmarking equivariant experiment (Ruhe et al., 2023; Brehmer et al., 2023), we consider a system of N = 5
GLGENN
0.1055
0.0020
0.0031
0.0011
MLP
28.1011
0.2482
0.0623
0.0622
MLP+AUG
0.4758
0.0936
0.0889
0.0672
charged particles (bodies) with given masses, initial po- sitions, and velocities. The objective is to predict the fi- nal positions of the bodies after the system evolves under
EMLP-O(5)
0.152
0.0344
0.0310
0.0273
EMLP-S0(5)
0.1102
0.0384
0.032
0.0279
CGENN
0.0791
0.0089
0.0012
0.0003
8

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
EMLP-O|S)
MLP-Aug
20
OGENN
GLGENN (ours)
CGENN
GLGENN (ours)
10a
EMLP:5015)
CGENN
0.0200
MLP
GLGENN (ours)
10a
15
Test Loss
10
10
0.0100
10
0.0055
10
0.0005
30
300
3000
30000
2
2
2
2
30
300
3000
Training Set Size
Training Set Size
Training Size
Figure 2. Leftt: O(5, 0) Regressson Middle: (55, O)-Coveex Hull, 16 points. The shaded regions depict 95% confidence intervals taken over 5 runs. Right: (55, -o-yy The shaded regions depict 95% confidence intervals taken over runs.
Table 3. MSE (L) on the (55, 0) O)-Convex Hull Experiment (K points). The average convex hull volume in the training dataset for K = 16,
256, 512 points is = 11.4, ze 430.3, and = 694, respectively. The number of trainable parameters is as follows: for K 16, GLGENN
21.1K vs. CGENN 58 for K = 256, GLGGNN 791K vs. CGENN 1.72M; for K = 512, GLGENN 922K vs. CGENN 1.75M.
K
16
256
512
MODEL
# TRAIN
SA
SAMPLES
# TRAIN SAMPLES
# TRAIN SAMPLES
2
2
2
2
2
2
2
2
2
2
2
GLGENN
16.94
10.40
6.2
4.46
3.62
3.04
2.61
2908.
2918.09
8538.86
4872.39
CGENN
18.71
11.93
6.1
4.11
3.23
2.52
2.08
5176.7
3384.62
14727.6
7212.44
GAP
1.77
1.53
0.1
0.35
0.39
0.52
0.53
2268.54
-66.53
6188.74
2340.05
1.0
GLGENN (w/o MLP)
cralized Lipschitz Groups Equivariant Neural Networks (GLGENN). These networks are equivariant with respect to pseudo-orthogonal transformations of a vector space with any non-degenerate or degenerate bilinear form. GL.GENN are parameter-light, since they operate in unified manner across four fundamental subpaces of GAs defined by the grade involution and reversion. GLGENN strike a balance between the expressiveness of Clifford Group Equivariant Neural Networks (CGENN) and parameter efficiency by respecting the fundamental algebraic structures of GAs?.
GLGENN (w/ MIPi
0.8
CGENN (w/a MLP)
CGENN (w/ ML?)
Test Loss
0.E
MLP
0.4
0.2
0.c
300
600
900 1200 1500 1800 2100 2400 2700 3000 Iteration of Training
Figure 3. Combination of MLP with GLGENN and CGENN in (55, Regressson..
To develop GLGGNN layers, we introduce and study the no- tion of generalized Lipschitz groups in GAs. We prove that the equivariance of a mapping with respect to these groups implies equivariance with respect to pseudo-orthogonal groups. As a result, GLGENN are equivariant with re- spect to a larger group than the pseudo-orthogonal group. However, empirically, we find that these networks are suffi- ciently expressive for tasks that require pseudo-orthogome group equivariance. Experimental results demonstrate that GLGENN either outperform or match state-of-the-ane mod- els, while having significantly fewer trainable parameters and lower training time. Future research will focus on apply- ing GLGENN to more complex real-world data experiments. Given their lower training time and promising results, we are optimistic about their potential.
Newtonian gravity for 1000 Fuler integration steps. Unlike standard low-dimensional setups, we simulate the system in 1R5.0 and embed the data into Cla.o. We construct a graph neural network (GNN) based on the message-passing paradigm (Gilmer et al., 2017), where bodies are treated as modes in graph, and their pairwise interactions are modeled as edges. For each edge, we compute a message and then aggregate at each node to update the particle states. The message and update networks are equivariant GLGENN. We compare against CGENN, which itself outperforms several tate-of-the-art method (see Appendix I). To ensure a fair comparison, we use the perforeefornnng CGENN architec- ture and then replace its layers with analogous GL.GNNN counterparts to obtain the GLGENN architecture, which au- tomatically has two times fewer trainable parameters. The results are presented in Table 12 and Figure 2 (right).
*GLGENN goes beyond CGENN in the following: (1)
surameter-sharinne approach for GA- neural networks is pre- sented for the first time. (2) New layers with parameter sharing are proposed, theoretical justification is provided (Section 4). (3) General idea is proposed that if need orthogonal groups equiv- ariance, then we may search for broader groups cquivariance, such as new generalized L Lipschitz groups, and get reasonable results (Theorem 3.4 and experiments).
6. Conclusions
We introduce a new equivariant neural networks archi- tecture based on geometric algebras (GAs), called Gen-

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Acknowledgements
Brandstetter, J., van den Berg, R., Welling, M., and Gupta, J. K. Clifford neural layers for PDE modeling. In The Eleventh International Conference Learning Repre- sentations, 2023.
The article was prepared within the framework of the project 'Mirror Laboratories' HSE University (" ("Quaternions, geo- metric algebras and applications"..
Bredon, G. Introduction to Compact Transformation Groups. Academic Press, 1972.
The authors are grateful to the four anonymous reviewers for their careful reading of the paper and helpful comments on how to improve the presentation.
Brehmer, J., De Haan, P., Behrends, S., and Cohen, T. Geo- metric algebra transformer. Advances in Neural Infor- mation Processing Systems, volume 37, 2023.
Impact Statement
The broader impact of this work is in improvement of the efficiency and effectiveness of equivariant neural networks. The significance of these networks is substantial, especially in the field of natural sciences, where the tasks inherently in- volve equivariance with respect to pseudo-orthogonal trans- formations. Parameter-light equivariant architectures re- duce the risk of model's overfitting in case of small training datasets, which is a common scenario in the natural sci- ence field. Efficient equivariant neural networks have great potential to stimulate breakthroughs and innovations in mod- eling and simulations of physical systems, robotics, material science molecular biology, geoscience, etc.
Brooke, J. A Galileian formulation of spin: Clifford algebras and spin groups. Journal of Mathematical Physics, 19: 952-959, 1978.
Brooke, J. Clifford Algebras, Spin Groups and Galilei Invariance- -Nww Perspectives. PhD thesis, University of Alberta, 1980.
Buchholz, S. and Sommer, G. On clifford neurons and Clifford multi-layer perceptrons. Neural Networks, 21(7): 925- 935, 2008.
Choi, H. I., Lee, D. S., and Moon, H. P. Clifford alge- bra, spin representation, and rational parameterization of curves and surfaces. Advances in Computational Mathe- matics, 17:5-48, 48, 2002.
References
Ablamowicz, R. Structure of spin groups associated with degenerate Clifford algebras. Journal of Mathematical Physics, 27(1):1-6, 1986.
Cohen, T. and Welling, M. Group equivariant convolutional networks. In Proceedings of The 33rd International Con- ference on Machine Learning, volume 48 of PMLR, pp. 2990-299 2999, 2016.
Anderson, B., Hy, T. S., and Kondor, R. Cormorant: Covari- ant molecular neural networks. In Advances in Neural Information Processing Systems (NeurIPS), pp. 14510- 14519, 2019.
Cohen, T. and Welling, M. Steerable CNNs. In Interna- tional Conference on Learning Representations, 2017.
Crumeyrolle, A. Orthogonal and Symplectic Clifford Alge- bras. Springer, Netherlands, 1st edition, 1990.
Atiyah, M., Bott, R., and Shapiro, A. Clifford modules. Topology, 3:3-38, 1964.
Deng, C., Litany, O., Duan, Y., Poulenard, A., Tagliasac- chi, A., and Guibas, L. J. Vector neurons: A general framework for SO())-qqiivarian networks. In IEEE In- ternational Conference on Computer Vision (ICCV), pp. 12180-12189. IEEE, 2021.
Eayro-Corrochen E. Geometric Algebra Applications, vol. I, Computer Vision, Graphics and Neurocomputing. Springer, 2019.
Coyrr-Corrochaoo, E. and Buchholz, S. Geometric neural networks. In Sommer, G. and Koenderink, J. J. (eds.), Algebraic Frames for the Perception-Action Cycle, pp- 379-394, Berlin, Heidelberg, 1997. Springer Berlin Hei- delberg.
Dereli, T., Kocak, S., and Limoncu, M. Degenerate spin groups as semi-direct products. Advances in Applied Clifford Algebras, 20:565-573, 10 2010.
Dicudonné, J. La géométrie des groupes classiques. Springer- Verlag, 1971.
Benn, I. and Tucker, R. An Introduction to Spinors and Geometry with Applications in Physics. Bristol, 1987.
Doran, C. and Lasenby, A. Geometric Algebra for Physicists. Cambridge University Press, Cambridge, UK, 2003.
Brandstetter, J., Hesselink, R., van der Pol, E., Bekkers, E. and Welling, M. Geometric and physical quantities im- prove E(3) equivariant message passing. In International Conference on Learning Representations, 2022.
Dorst, L., Doran, C., and Lasenby, J. Applications of Ge- ometric Algebra in Computer Science and Engineering. Birkhäuser, 2002.
10

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Filimoshina, E. and Shirokov, D. On some Lie groups in degenerate Clifford geometric algebras. Advances in Applied Clifford Algebras, 33(44), 2023.
Köhler, J., Klein, L.,, and Noé, F. Equivariant flows: exact likelihood generative learning for symmetric densities. In Proceedings of the 37th International Conference Machine Learning, ICML'20, 2020.
Filimoshina, E. and Shirokov, D. On generalization of Lipschitz groups and spin groups. Mathematical Methods in the Applied Sciences, 47(3):1355-140M, 2024a.
Kuroe, Y. Models of Clifford recurrent neural networks and their dynamics. In The 2011 International Joint Conference on Neural Networks, pp. 1035-1041, 2011.
Filimoshina, E. and Shirokov, D. A note on centralizers and twisted centralizers Clifford algebras. Advances in Applied Clifford Algebras, 34(50), 2024b.
Langley, Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference Machine Learning (ICML,000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Filimoshina, E. and Shirokov, D. Generalized degenerate Clifford and Lipschitz groups in geometric algebras. Ad- vances Applied Clifford Algebras, 35(29), 2025.
Lecun, Y., Bottou, L.,, Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Proceed- ings of the IEEE, 86(1 1):22782-2224, 1998.
Finzi, M., Welling, M., and Wilson, A. G. Practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups. volume 139 of PMLR, pp. 3318- 3328, 2021.
Liu, C., Ruhe, D., Eijkelboom, F,, and Forré, P. Clifford group equivariant simplicial message passing networks. ICLR, 2024.
Fuchs, F., Worrall, D., Fischer, V., and Welling, M. SE(3)- transformers: 3D tnto-translation equivariant attention networks. Advances in Neural Information Processing Systems, volume 33. pp. 1970-1981, 2020.
Lounesto, P. Clifford Algebras and Spinors. Cambridge University Press, 1997.
Lundholm, D. and Svensson, L. Clifford algebra, geomet- ric algebra, and applications, 2009. https: :atpss///aa //arxiv. org/ / abs / 00007 6356.
Garling, D. Clifford Algebras: An Introduction. Cambridge University Press, Cambridge, UK, 2011.
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chem- istry. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Ma- chine Learning Research, pp. 1272- PMLR, 2017.
Pearson, J. and Bisset, D. Neural networks in the Clifford domain. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94) volume 3, pp. 1465-1444 1469 vol.3, 1994.
Pepe, A., Buchholz, S., and Lasenby, J. Clifford group equivariant neural network layers for protein structure prediction. In Northern Lights Deep Learning Confer- ence, 2024.
Han, J., Rong, Y., Xu, T., and Huang, W. Geometrically equivariant graph neural networks: A survey. ArXiv, 2202.07230. 2022.
Harvey, F. Spinors and Calibrations. Academic Press, 1990.
Pitts. A. M. Nominal Sets: Names and Symmetry in Com- puter Science, volume 57 of Cambridge Tracts in Theo- retical Computer Science. Cambridge University Press, 2013.
Helmstetter, and Micali, A. Quadratic Mappings and Clifford Algebras. Birkhäuser, 2008
Hestenes, D. and Sobczyk, G. Clifford Algebra to Geometric Calculus A Unified Language for Mathematical Physics. Reidel Publishing Company, 1984.
Porteous, I. Clifford Algebras and the Classical Groups. Cambridge University Press, 1995.
Ruhe, D., Brandstetter, J., and Forré, P. Clifford group equiv- ariant neural networks. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023.
Isaacs, I. M. Algebra: A Graduate Course. American Mathematical Society, 2009,
Jing, B., Eismann, S., Suriana, P., Townshend, R., and Dror, R. Learning from protein structure with geometric vector perceptrons. In ICLR, 2021.
Satorras, V. G., Hoogeboom, E., and Welling, M. E(n)- equivariant graph neural networks. In ICML, 2021.
Shirokov, D. Quaternion typification of Clifford algebra elements. Advances in Applied Clifford Algebras, 22(1): 243-256, 2012a.
Knus, M.-A. Quadratic and Hermitian forms over rings. Springer, Berlin, 1991.

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Shirokov, D. Development of the method of quaternion typification of Clifford algebra elements. Advances in Applied Clifford Algebras, 22(2):483-497, 2012b.
Shirokov, D. Clifford algebras and their applications to Lie groups and spinors. In Proceedings of the 19th In- ternational Conference Geometry, Integrability and Quantization, pp. 11 11-53, 2018.
Shirokov, D.
On inner automorphisms preserving fixed subspaces of Clifford algebras. Advances in Applied Clifford Algebrus, 31:30, 2021.
Thomas, N., Smidt, T. E., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., and Riley, Tensor field networks: Rotation- and canslation-equivarian neural networks for 3D point clouds. CoRR, 2018.
Townshend, R. L., Vögele, M., Suriana, P. A., Derry, A., Powers, A., Laloudakis, Y., Balachandar, S., Jing, B., Anderson, B. M., Eismann, S., Kondor, R., Altman, R., and Dror, O. ATOM3d: Tasks molecules in three dimensions. In Thirty-fithh Conference Neural Infor- mation Processing Systems Dulasets and Benchmarks Track (Round 1), 2021.
Walpuski, T. Differential geometry IV, Lecture notes, 2022. https::/walpp..kki/ wwt.:/aciit/e/eeeeeeeeeaaaa DifferentiialGemeetryy
Weiler, M. and Cesa, General E(2) equivaaara steerable CNNs. In Neural Information Processing Systems, 2019.
Weiler, M., Geiger, M., Welling, M., Boomsma, W., and Cohen, T. 3d stecrable cnns: Learning rotationally equiv- ariant features volumetric data. ArXiv, abs/1807.00 0054072 2018.
Yarotsky, D. Universal approximations of invariant maps by neural networks. Constructive Approximation, 55(2): 407, 2022.
Zerouali, A. Twisted conjugation on connected simple Lie groups and twining characters. 2020.
Zhdanov, M., Ruhe, D., Weiler, M., Lucic, A., Brandstetter, J.. and Forré, P. Cliffond-steerable convolutional neural networks. volume 235 of PMLR, 2024.
12

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Brief Summary of Appendix
In Appendix A, we provide an overview of notation that we use throughout the paper.
Appendix B provides an example of the main theoretical constructions considered in Section using the geometric algebra C11.3 of the Minkowski space I...3.
Appendix C discusses the significance of grade involution and reversion in geometric algebras (GAs) CC We show that these operations are in some sense unique in geometric algebras. This paragraph serves as motivation for considering the four subspaces of geometric algebras determined by the grade involution and the reversion. Our proposed architecture GLGENN actively operates with these subspaces in its structure.
In Appendix D, we consider the notion of centralizers and twisted centralizers of the subspaces of fixed grades and subspaces determined by the grade involution and reversion. We write out explicit forms of these sets. The centralizers and twisted centralizers play crucial role in main theoretical results generalized Lipschitz groups.
Appendix E considers the (ordinary) Lipschitz groups, spin groups, and adjoint and twisted adjoint representations in arbitrary degenerate and non-degenerate geometric algebras. We prove several properties of these representations and find their kernels.
Appendix F presents and proves the key theoretical results necessary to constructing GLGENN. Specifically, we introduce and study generalized Lipschitz groups in arbitrary degenerate or non-degenerate geometric algebra. These groups preserve the subspaces determined by the grade involution and reversion under the twisted adjoint representation. We prove that these groups can be defined equivalently, using only the norm functions, which are applied in the theory of spin groups. This results allows us to prove that the (ordinary) Lipschitz groups are subgroups of the generalized Lipschitz groups
Appendix G proves several properties of equivariant mappings, which are applied in Section 4.
Appendix H considers the relation between the well-known degenerate and non nen--egenerate meeudo-orhogoone (or complex orthogonal) groups and generalized Lipschitz groups. The obtained results enable to prove that equivariance of a mapping with respect the generalized Lipschitz groups implies its equivariance with respect the corresponding pseudo-orthogonal group. This result is crucial for the construction of GLGENN, which are equivariant with respect to pseudo-orthogonal transformations.
Appendix provides experimental details and discussions.
A. Notation
In Table 4, we provide overview of notation used throughout the paper. We write out the notation, its meaning, and the place where it is mentioned for the first time.
B. Example of Geometric Algebra C(1,3 of Minkowski Space R13
In this section, we provide an example of the key theoretical concepts and constructions considered in Section 2. Let us consider the Minkowski space V RR. 1.3 equipped with the Minkowski metric r) = diag(1, 11 1, -1). In this case, we have p- = q = 3, r 0, and = p + q 4. The corresponding geometric (Clifford) algebra is Chi,3 CE(.) which is the real non-degenerate 2 16- di-dimensional algebra with the identity element and generators c1, E2, c3, e, that satisfy:
Cg + EbCa 21aase a,
(46)
which implies that the distinct generators anticommute, the first generator squared gives +e, and the other three generators squared give- -6:
ca
Ebbar a,b= 1, 4, a + by
(ei)² = te,
= = (e,)² -e.
(47)
The basis elements of the algebra Cli,s are constructed as ordered products of distinct generators, such as 6123 616203 and, in general, Ca....a. Caj ea, for o1,..., ag {1,2,3, 4) and a <... ag- In the case of Cl1,,, we have five non-trivial subspaces of fixed grades:
Ct9,a = span(e) = R. Ct2,3 span e3,
e2, ea) =V, CG.,3 = span G13, C14, 623, 624, e34): (48)
13

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Table Summary of notation.
NOTATION
MEANING
FIRST MEN-
TION
Cpaa
(CLIFFORD) GEOMETRIC ALGEBRA (GA) OVER REAL RP:0-" OR COM-
PAGE
PLEX CP+..,00 VECTOR SPACE
NON-DEGENERATE GEOMETRIC ALGEBRA Clp.a.D
PAGE
A
GRASSMANN SUBALGEBRA Cba.o,r OF Cly.ar
PAGE
REAL RR-44 OR COMPLEX CP+9.00 VECTOR SPACE
PAGE
F
FIELD OF REAL COMPLEX NUMBERS IN THE CASES R
AND CP
PAGE
RESPECTIVELY
7
-F MATRIX OF A BILINEAR FORM
PAGE
b
SYMMETICC BILINEAR FORM,
b:VxV
PAGE
9
QUADRATIC FORM, 9 V +F IDENTITY ELEMENT
PAGE
PAGE
GENERATORS OF Clp.g.r
PAGE
CP
SUBSPACE
O
PAGE
Ce
p-gir
SUBSPACE OF GRADE k, WHERE l = I,...,
PAGE
SUBSPACE p. SPANNED E CC AND b É A
ab, WHERE
PAGE
PROJECTION ONTO THE
GRADE
PAGE
U
GRADE INVOLUTE OF U Clp.air REVERSION OF U E Clpag
FORMULA (2)
FORMULA (2)
CLIFFORD CONJUGATE U E C.P.o..
PAGE
Cb Cepio
EVEN AND ODD SUBSPACES OF Clpar
FORMULA (3)
(Cex U C
PRODUCT TWO GROUPS: (ab | a CX1O CX E AØ PROJECTION ONTO THE SUBSPACE CX WHERE 0, SUBSPACES DETERMINED BY GRADE INVOLUTION AND REVERSION
LEMMA 2.1
Og
PAGE
Ce Pag
FORMULA (4)
Qm
PROJECTION ONTO THE SUBSPACE Cpp.arr WHERE m 0, 1,2,3
PAGE
H
SUBSET ALL INVERTIBLE ELEMENTS OF SET H
PAGE
ADJOINT REPRESENTATION (U) = TUT
FORMULA (5)
ad
TWISTED ADIOINT REPRESENTATION adr(U) TUT-1
FORMULA (6)
ad
TWISTED ADJOINT REPRESENTATION adz (U) 11U00'
FORMULA
ad
ad RESTRICTED TO CP C.
PAGE
T1
ORDINARY LIPSCHITZ GROUPS
FORMULA (10)
p.g.r
X
NORM FUNCTIONS Claa. ELEMENTS
FORMULA (II)
F
GENERALIZED LIPSCHITZ GROUPS, 0,1,2,3
FORMULA (13)
par
p.g.r Oua...
GROUPS PRESERVING THE SUBSPACES Cb & = 0,1,2,3 UNDER ad
FORMULAS
(16) AND (18)-
(19)
[E
p.g.r S5.0
GROUPS PRESERVING THE SUBSPACES CC k 0, UNDER ad
FORMULAS
(17) AND (20)-
(21)
Loaa Uya. Dzag
CENTER OF Clpag
FORMULA (70)
CENTRALIZERS AND TWISTED CENTRALIZERS RESPECTIVELY Cit.....
PAGE
k = O,...,n
Z 2. 27.98
CENTRALIZERS AND TWISTED CENTRALIZERS RESPECTIVELY OF CE.
FORMULAS
k = 0, 2,3
GL(n,F)
GENERAL LINEAR GROUP ACTING ON T--DIMENIINALL VECTOR SPACE
FORMULA (30)
OVER F
0(V.9)
ORTHOGONAL GROUP
FORMULA (30)
(V,
SUBGROUP O(V,q) THAT LEAVES INVARIANT A!
FORMULA (31)
CH, = span(e123, 6124, 6134, e234), C01 = span(e12a4
(49)
which we call the subspaces of scalars, vectors, bivectors, trivectors, and 4-vectors respectively. An arbitrary element
14

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
U E 1,1 can be written
U = ue +
Σ
UaCa +
Σ
ab +
Σ
UabeCabe + U123161231
U..... 11231 E R.
(50)
a
a. a<b
a.b.c=1,...4 acb<c
The even C and odd 1,3 subspaces are defined as
1 = Cf a Cha @ Ctt
- = Ct @ C/
(51)
In 1, the four subspaces determined by the grade involution and the reversion (the subspaces of quaternion types) defined as
Ct1,3 = Cf + C,1 = GL = -0 Ch = Cf
(52)
and are very similar to the subspaces of fixed grades because of small 4.
Let us consider the following example on how the projections ()x onto the subspaces of fixed grades CC h: = ......4,
the projections (n)) onto the even and odd subspaces C 1,3 I = 0, 1, and the projections () onto CCT,3 m = 0, 1, 2, 3 act
on the element W = é + 2e1 + 302 + 46234 + 561234:
(W)o e, (W)) (W); 2e1 + 3e2, (W) (W)2 0, (W)a (W)3 4E234, (W)4 5e 1234, (53)
(W)o (W)(o) = 501234. (W)(1) 2e1 3c² + 46234.
(54)
For an arbitrary U E C,1,3 (50), we have
Ü = -U))a))
(U)
-= (U)) (U))
(55)
Now let us consider the example on how the adjoint representation ad (5) and twisted adjoint representations ad (6) and ad (7) act. Let us consider the element T = e1 + 6123 E ClT,3, which is invertible because 1 (e1 + e123) - -e12)) = e.
Suppose U = e 64 Ch133 We have
(U) = TUT = - - TesT = e 1 (er + e12))ea(e) - 6123)
(56)
2
1
e 2 (el
+ +
= (e) + (12))((e -12))e [a C4:
(57)
aU)(U) =
(58)
aUT(U) T(U) + T(U)) +W TeT TesT e e4.
(59)
Note that the results are different for different operations ad, ad, ad.
The Lipschitz group in the case Cl 3 has the form
Fl.3 = (T Clia: TCH S CC1.3
(60)
simple example an element T is any basis element ea ca....s. since - Ca....ax = tc...... ..... where the sign depends on a1.....d ag and k, and Cai..a Cae=l Caba.... = tea E Cf for any 1,..., 4. A slightly more complicated example of the Lipschitz group Fl,3 element is T e, + 6123 E CK considered the paragraph above. We have:
1 (ei + e12aJei e123) -E1s Te2T e3,
-e =
TeaT = -2. TexT-1 = C4.
(61)
Therefore, Te,T-1
-1 E Ct2 for a = 1,...,,4 4; thus, T TE
15

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
C. Grade Involution and Reversion as Fundamental Operations in CC p.g.
This work introduces the neural networks that equivariant with respect the groups i p.g.r (27) preserving the subspace CeT p.g.ar (4) determined by the grade involution and reversion under the twisted adjoint representation ad. These operations
can be considered as fundamental operations in geometric algebras Clp. In this section, we discuss their sigmificance.
In Subsection 2.1, the grade involute and reversion an arbitrary U E Clp.gr are defined in the following way respectively:
-=
(U))
Ú=
(-= k(k-1)
(U)
(62)
These mappings have the following well-known properties, which we apply for many times in work:
(UV) =V OV, (UV) -U, VU.V E Clp.g.-
(63)
Let show that these operations in some sense unique in Clp.gr-- For this purpose, need to consider another equivalent definition of geometric algebras Clp.grr. More detailed discussion of definitions and statements of this section can be found, for example, in Lounesto, 1997; Helmstetter & Micali, 2008
The geometric (Clifford) algebra CE(V) is an associative algebra with the unity over vector space V with a symmetric V + F and the corresponding quadratic form + F, with a linear map j CE(V) such that
bilinear form b
((x) = q()),,
Va E V,
(64)
and for any other associative algebra A over V with the unity е and any linear map k: : - A, such that k² = q for any z E V, there exists a unique algebra homomorphism f CE(V) + A, such that f foj = k, i.c. the corresponding diagram is commutative:
V
CC(V)
A
Theorem C.1. In any geometric algebra CL(V), there is unique antiauttomophhism 7: C((V) CLVV, such that
r(zy) T(y)T(x), TOT id, T((()) j(v). Vz E CI(V), Yu V.
(65)
Proof. Let us consider the opposite algebra CE(V)", in which the product of and y is given by yz. Then there exists unique algebra homomorphism 7: C((V) - CE(V)) in the diagram below
C((V) 1-
Ce(V)
This completes the proof.
Since in our case, V has finite dimension n with a basis 61,..., Cn. the mapping completely determined by its action on the basis elements. Namely, 7 is defined by
T(es) -ii
Ei ...)).))))))))))))))))).) Cig Cig Cip
T(e) = e,
(66)
where 1 S in < < i n. The mapping 7 coincides with the reversion defined as in (62).
Theorem C.2. In any geometric algebra CL(V). there is unique automorphism a CE(V) + CV(V), such that
(xwy) a())a(y),
aoa a id, alj(v)) -j(),, Vx, V.
(67)
16

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Proof. Let consider the linear mapping og V CP(V) defined (o(v) -i(v) for any V. We get a unique homomorphism as in the diagram below
V 3 ce(V)
C((V)
Furthermore, any x E C((V) can be represented as
x Σ 21 Em.
.1.... Zm (VV,
(68)
and since a<);))) -t for any j 1,..., m. we get id. This completes the proof.
n
Again, since V has finite dimension, the mapping a completely determined by its action on the basis elements. Namely, is defined by
a(e;)= -c, ale, Co2 ... ...)) (-1)* ... Capt a(e) = e,
(69)
where </ < 12 <.... < in S n. The mapping a coincides with the grade involution defined as in (62).
D. Centralizers and Twisted Centralizers
We denote the center of the algebra ppp by p.g..r is well-known fact (Crumeyrolle, 1990) that the center has the following form
(A. @ C )* n odd; (()x n is even.
Lp.
(70)
Consider the centralizers (see, for example, Garling, 2011; Isaacs, 2009) of the subspaces of fixed grades Cfm O,..., n, in Cl Cp..ri
p.g.r. m
Z.g.r (X E Cppar: XV VX, AV E Cp
(71)
The centralizer Zmggr contains all the elements of p.g.r that commute with any grade-m clement. The center of the Clifford algebra Clpar is the centralizer of the entire Clifford algebra Cb
Similarly, we consider the twisted centralizers of Cppg.. m O,... n, Opp.rri
Zg..r (X Cpar::
XV VX, AV E Cgm p.......
(72)
Note that Zm p 7m Clpar for m < O and m > n. The particular case 7/ is considered in the papers Ruhe et al., 2023; Brooke, 1980; Filimoshina & Shirokov, 2024b; 2023.
Note that the projections (2 and (pmar)o)) 4gg,r and p.g.r respectively onto the even subspace Cepo coincide by definition:
(Z)g.)) = (2m Vm = 0, I....,n.
(73)
The paper Filimoshina & Shirokov, 2024b studies explicit forms of the sets Zm and 7m Zp.g.r in the of arbitrary O...., In Remark D.I, for the readers' convenience, write out several particular cases of these sets, which we further use. A few words about the notation in this remark and below. The spaces Cikk p.g and A,. k = 0,... are regarded subspaces of Clp...- By {Cek (Ce,,yA,) A/)), we denote the subspace of Ct p.p. spanned by the elements of the form ab, where p.a
E Clkl and b E A..
17

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Remark D.I (Explicit forms of centralizers and twisted centralizers). We have:
())
Ce @
n is odd,
Z...r
(74)
A
n even;
A, ①
Cen
+ n,
Z2 p.g.r
(75)
Ar:
n
A OA7- @ (CP) ng (A7-3 e A.- ① p. A + CP part
n is odd,
Z p.g.r
(76)
A @A7 @ (CK) p.q A 2 1
n even;
A, @ {CE) (1p @ 1T- @ (0L na (A, (A7- @ A A2-3)} }} @ par
r # n r n
p.g.r
Ar.
(77)
and:
2n p.g.i Ari
(78)
A!O) AZ @ 10% p
n is odd,
Vp pigir
1 @ A @ (OE) 1 @ CC pig.r
n. is even, r # n n is even, r == n
(79)
A A7-1,
7 = A, @ {CP) p. A3" @
(CP ( 32
(80)
p.q
Remark D.2. In the particular case of the non-degenerate geometric algebras Cpp.g we have
CE
Cen pag
is odd,
4p.a-
Lp.
Ct @ CPP.qi p.q
(81)
CE
is even;
CC Clp.g
n = 2,3,
Zz.4 = Sknor
n 3,
Cep
n = 4,
(82)
|ce,
n + 2,3;
CP Dpp.g
n + 3, 4;
Clp
1,2,
1,2, Cl p.g.
2/19 = CE
p.. =
CO
* 1 is odd,
"p.0
p.g. 3,
(83)
CC @ p.g.
n + 2 even;
CC,
n 2 4.
Using the definition of the even subspace p.p.g and explicit forms of centralizers of fixed grades, presented in Filimoshina Shirokov, 2024b, the following lemma can be obtained.
Lemma D.3. We have
A, + Ce
r x n, T n,
(X E Ct Charr:
XV VX, VV e Colo)
=
{ Ans
(84)
A
(X E Ct Charr:
XV= VX, W CPto) (0
n is odd; r = v is even;
(85)
A + CCn
n. is even, + n.
Now let us consider the centralizers and twisted centralizers respectively of the subspaces CFE por (4), k = 0, 1,2,, defined by the grade involution and reversion, in Clp
7 Pp..r == (X E Clp.r.
XV VX, V CFT a..
(86)
2 2p = {X E Cpaar:
XV VX,
V Cek )
(87)
Theorem D.4. (Filimoshina & Shirokov, 2024b) We have
7. 7.p..r
7.p..r
7m p.g.r:
m ===1 1,2,3
70 p.g.r
7.1 7.p..r
(88)
18

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
The centralizers p...r and twisted centralizers Zm formula (88), we have
p.g.r m 1,2,3,4 4, written out explicitly in Remark D.I. In the
A @ (Ce) p.
11172-16000 20 (CP, A.
is odd n; n is even, r # n.
(.p.r)) (0)
(89)
A @ (Cep. A 37 e (C6, p.o A7 + Ce p.g.
Let us consider the following Lemma D.5 about T E Cpp.gr such that (T) and x(T) (11) are in the centralizers Z p.g.r (86) and twisted centralizers 2he Z.g.rr (87). The proofs of theorems in Appendix F are based on this lemma.
Lemma D.5. (Filimoshina & Shirokov, 2025) For any T P.p. in the cases (I:,1) (0, 1), (1, (3.3)
Pap Par TC T S
CEAT p.g.r
xx + TT E
(90)
T T-1 CP
TT
Z p.9.."
(91)
p.g.r
and in the cases (k. = (0,3), 0), (1, (2,1), 1), we have:
TC p.a. T
CPRI
TT E pgr
(92)
p.g.r
TCET pas T-1
C
p.g.r
TT
p.g."
(93)
E. Ordinary Lipschitz Groups, Spin Groups, Adjoint and Twisted Adjoint Representations
In the non- non-degenerate geometric algebras Clp.g the Lipschitz group is defined as the group of all invertible elements preserving the grade-1 subspace under the twisted adjoint representation ad. Generalizing this definition to the of the degenerate Cl p.g.r we similarly define the Lipschitz group 17 pp.a.-
fi pig. (T E CP aar=
p. C CP....
(94)
The definition (94) of the degenerate Lipschitz group is used, for example, in the works Brooke, 1980; 1978; Crumeyrolle, 1990.
The well-known spin groups (see, for example, Lounesto, 1997; Crumeyrolle, 1990) are defined as normalized subgroups of the Lipschitz groups [ p.g.r. using the norm functions va and X (11). For example, in the non-degenerate geometric algebras CP Clp.g they are defined as
Pinp.g = (T E
fl
ÎT = te} = E pa TT = tel,
(95)
p.u
Pin p.g {T E iu
TT te}.
(96)
p.g
Pink = (T E
i
TT +e},
=
(97)
p.g
Spinp.y p.y = (T's E
fu
C)(0)x
TT = =e] = (T E Ml
C pg
TT = =e),
(98)
p.q
p.
Spin = (T Fi p.a
n C
ÎT -e) = (T E r
c
2
TT +c}.
(99)
p.
Let us consider the adjoint representation ad acting on the group of all invertible elements ad CXX p.g.r > Aut(Clp.gr) as T + adr, where adr: Clp.r + Clpari
ady(U) = - U E Clp.gr
T E CP p.g..
(100)
Since (UV) (V) for all U.V E Clp.g.r the mapping adr is an algebra homomorphism, which is moreover an algebra automorphism for all T E CPX Ofg... It is called inner automorphism (or conjugation).
Also let us consider the twisted adjoint representation. which is introduced by Atiyah, Bott, and Shapiro in Atiyah et al., 1964. In this paper, the twisted adjoint representation is defined the case of the non-degenerate algebra Clo acts
19

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
on the Lipschitz group I'd 0. (94) in the way ad Too 0.g.0 - Aut(Ce as T adz, where ady Cb. + Cet Cbo.g. is defined for elements of the grade-1 subspace (vectors)
aU)(U)= TUT U.E Cli Clu.g.
T E 1 0.g.0*
(101)
There are two ways how to generalize the definition (101) of the twisted adjoint representation to the case of any degenerate and non-degenerate Clp.2. and arbitrary T E CP p.q.r U E Cpp. The first approach (Choi et al., 2002; Harvey, 1990; Lundholm & Svensson, 2009) is to define it as the operation ad acting the group of all invertible elements ad P.4.r + Aut(C(p.g.)) as T + adr with adr Cbp.r Cpparr
aU)(U) = - 1 U UECCpa
T E CP
(102)
The operation a (102) is similar some operations considered in representation theory of Lie groups (Zerouali, 2020). Note that unlike adr (100) the mapping adr (102) is not multiplicative in U and, therefore, is not algebra homomorphism for all T E CP* p.g.r
The second approach (Helmstetter & Micali, 2008; Knus, 1991; Walpuski, 2022) is to define the twisted adjoint representation as the operation acting on the group of all invertible grade-m elements ad CPp p.g.r + Aut(Clp as + adr with adr: Clp.arr + p.... defined as:
(U) (-1)6mTUTT) UE Ct p.g.
T E CE
(103)
where we use another notation ad, not to confuse it with ad, (102). The mapping adr is also called twisted inner antomorphism, for example, in Helmstetter & Micali, 2008. The operation adr can be extended by linearity adr + V) adr (U) + (V), and we finally obtain
aU) T(U) (-)) VU E Clpar
T E CP P.p.
(104)
Note that for any elements of fixed parity (()) and U(), we have
(U)((U())) (U)(U())), (U()) = (U)))
WU( CP(0) p.g.rs
T P.girr
(105)
(U() E Cell) p.g.r
T CP p.g.r'
(106)
Note that both generalizations ad (102) and ad (104) of the twisted adjoint representation (101) to the case of arbitrary T gr.. and U Olp.n. are used the literature for various purposes and have some advantages indicated in the works cited above.
Let us consider the kernels of the adjoint and the twisted adjoint representations:
Gard) = {T' E C Ch
TUT-1 = U, VU E Olpart
ker(ai)= (T Cbp..
TUT-1 = U,, YU Cppar):
ker(ad) {T OPP...
(U T + TU,TT U, VU Uo) U() E Cpar U1o CF U(s E CP(l)
Lemma E.1. (Filimoshina & Shirokov, 2023) We have
(A @ C Cppggr) 1(0)x
if n is odd, if n. is even;
ker(ad) = Z p.g.r
(107)
kad) A (0)x; ker(ad) = A
(108)
(109)
Proof. We obtain (107) from (70).
Let us prove (0)x C ker (ad). Suppose T' E (0)x; then TUT-1 U for any U E Clpa.. Since T is even, we have T= therefore, -VT-1 for any U E Ct Clp.r.
20

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Let us prove ker (ad) _ ,0 Suppose E CP
p.g.r satisfies TUT-1 U for any U P.p.. Substituting the element U e, we obtain f T: hence, T E O62 and TUT U for any U E Clp.rrr r. In other words, T E C Oker (ad).
Using (107), we obtain T' e Cpp n ()) C parr) and the proof is completed.
A, 1 in the case of odd T E Ar [0)x in the case of even
Let us prove kcr(ad) S A, Suppose T E ker(ad); then TU,T = U1 for any U1 E CC Thus, T E AØ by the statement (78) of Remark D.I.
Now we must only prove that A% C kor(ad). Suppose T E A then TCP = . T and TCP) p-a.r = CP ph T by Lemma D.3 and the statement (78) of Remark D.I respectively. Since any odd basis element can be represented as product of an odd number of generators, we obtain TCe = C.....T Thus, (U T- + (U
U for all
U (() + U1)) E plpar where U() E CCp.g.r and U(1) E CCL)) and the proof is completed.
In the next lemma, we study the properties of ad, ad, and ad. Note that the multiplicativity of ady (115) proved in the particular case T E C U Cfp.g in Ruhe et al., 2023, and we generalize this statement to T E p.g. U C
Lemma E.2 (Lemma 2.1). Let H E Cep4 adw, adw, and adw satisfy additivity:
x uc p.g.r T (CEp2) U CO AA3 W E CPX p.g.r and U,V V E Cl Then
adw (U + V) = adw(U) adw (V),
(110)
adw (U + V) = adw (U) + adw (V), adw (U + V) = adw (U) + adw (V),
(111)
(112)
adw, add, and adw satisfy for any E Ct
adw (c) c, (c) (-1)/e,
(113)
where l = Of E [0 and = E Cell)x Moreover, adw and adr satisfy multiplicativiny:
adw (UV) = adw (U (U)adw(V)) (V), adr(UV) aidr(U)acdr(V).
(114)
(115)
Proof. The formulas (110)-113) can be easily checked using the definitions of ad (100), ad (102), and add (104). Let us prove (114) and (115). We have
adw (U)ad (V))dd(()= wyUw--vVe = (UV).
Since U = (0)) + (L), we get
aUr(U))iir (V) = (T() 71
(116)
= <T) (V) ""
+T(U)o "P "(())))) TWWN (117)
Now we apply the result of Theorem 4.7 Filimoshina & Shirokov, 2023:
(C U UCEU) = (T' CPX Cpp.r:::
T-TEA
and get T-1T E A,S which implies T- T- E AW. Since A = ker(ad) by Lemma E.I, we obtain that
(T (T- -T) for any even X E C. and (T (T-) = X(T for any odd X E C..... Therefore,
( TT (T) (U) (V),, -f (V)))
(118)
-"V -T- - (VU)
(119)
Finally,
ady (Ulr(UUadd (V) = (o) (V) +"
+ T(U) (V)) + T(U) (1) ()))))(((()))))))) (V)r) = adr(UV)
where we use that the product of two elements of the same parity is even and the product two elements of different parity odd.
21

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
F. Generalized Lipschitz Groups in Clp.r
this section, we introduce and study degenerate and non-degenerate generalized Lipschitz groups in geometric algebras ppp. of arbitrary dimension and signature.
The generalized Lipschitz groups are defined as the groups preserving the subspaces OE p.g.r (4), h: 0, 2,3,, determined by the grade involution and reversion, under the twisted adjoin representation ad (104):
i've p.g.r {T's E CC p.r.: (CT (C....))) S CC. C.p......
(120)
These groups are setwise stabilizers (see, for example, Isaacs, 2009) the subspaces C2 (4), = 0, 2,23, the group p.g.r under the group action ad. To study these groups, we introduce and study the following groups, which are setwise stabilizers of CF. p.g,r k 0, 0,,,2,3, 3, in CP p.g.r under ad (100) and ad (102) respectively:
n p.q.r
(T C p.g.
adT(CIp p.g.r) TC E p.g.r
T C Cb. part)
(121)
F {T E CP pag. (Ct( ) TC T C CCt p.g.rr
(122)
P-g.r
p.q."
and are related 1 p.g.r
(120) as follows:
F
F p.g.
h: 1,3,
(123)
P-g.r
T P.g.r
= 0, 2,
since (CPl (106)-(10).
p.g.r) = adr adr( p.g.r) in the cases k- = 0,2 and adz (CE p.g.)) = adr(Ckk p.g.r) in the cases k = 1, by
In Subsections F.1 and F22 we find the equivalent definitions of the groups I p.g.r and DT p.g.r. k = 0, 2,23
F.1 The Groups Qpg..r Ogar ogrr Oz
P r p.g.r p.g.r 12 p.g.rs and N P.g.r
Let us consider the groups Qb
OZ Oz Qgar and QU Quari
Q (T Cpp...
TT Zp...
ker(ad),
TT Z,x, ker(ad)),
(124)
p.g. (T E CEX p.g.r
2
TT Z2x
pig.r
TT E 22x )
(125)
Q2 = (T E C
TT 73x ppitirr
ीT E p.g.r }
(126)
P.g.
Op.g.r (T' E Ofp...
TT zAx 4.g.r:
TT ZAx )
(127)
where ker(ad) (107) is the kernel of the adjoint representation ad (100), 7 Tp 7 and 7.,.a.r (see Remark D.1) are the centralizers of the subspaces CP1 Clp.g.r. Cp CB3 Cpp..r- and C.p.rr respectively considered in Section D.
The groups Qg...r+ Q Qg.g.r Op...r. and Qp.g.r are generalizations of the groups Q and Q' (Shirokov, 2021) in the non- degenerate geometric algebras Cl p. to the case of the degenerate geometric algebras Cpp.g. and coincide with them if r
oT -0 = Q,
0u = Q
=
Q 1,2,3 mod 4, + 2,3,
Q' = 0 mod 4,
and Q1 = 0% if n 3.
Theorem F.1 (Formulas (22) and (25) of Theorem 3.2). In the degenerate and non-degenerate geometric algebras Ct Clp.g.r. we have
o
II pg.m
2z
1
1 p.g.r.
QU p.g.r
p.g.r
where
rI
D P.n.r
m 2.3.
(128)
p.g.r
P.g.r.
22

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Proof. For fixed k- 0,3, mod 4 and (k 1) mod 4, 1 (k 1) mod get
F p.g.r
{T E Ce
pag
TC p.g. T _ (C p.g.r n not (.g.))
(129)
{T E CP Pay
TC P.g.
P. T S
CP p.p. TCP p.g. T... C Ce p.pr
(130)
{T E CC p.g. TT
Lex
TT p.g.r
Q Op.gir.
(131)
p.g.r
where we use Lemma D.5 in the first equality (131) and Zp.g.- = p.... = 1,2,3, and Zp...- Zp...r by Lemma D.4 in the second equality (131). The inclusions in (128) follow from the definitions explicit forms of centralizers and twisted centralizers implied by Remark D.1:
and the following facts about
ker(ad)
p... S P..go S p.g.
Z4x
2,3.
(132)
This completes the proof.
F.2. The Groups Op.g.r Opair 0gg.r p.g.rt p.g.r. p.g.r pig.rs and 1
Q3 1
IT 1
p.g.
Consider the groups QJ p.p.r OZ op. O G....r and Op.n.:
p. (T E C6X piai:
TT 71x ker(ad),
TT p...r ker(ad)),
(133)
02 pgg = {T E O p.g.r
TT 22x
Îr E Z,x p.r.)
(134)
p.g.rt
0 p.g.r (T E Opp.g..
TT 73x p.g.rt
TT 73x r
(135)
02.9.- == (T E CC P.air
TT (7 (()) (0))
iT E (Z
(136)
where ker(ad) (109) is the kernel of the twisted adjoint representation ad (104), the sets 22 Zg.grr (79) and 23 (80) the twisted centralizers of the subspaces CP2 p.g.r and CP3 p.g.r respectively, and Z g.... (Remark D.1) is the centralizer of the
subspace CF1 p.g.r (see Appendix D). We use in the notation of the groups (133)-(136 due to Theorem F.2 below. We have (see (109)):
A + A @ g A3-3),
is odd;
(
A @ (Cel Ar 3 @ (CP, Ar n- + Ce p.g.rt n is even, r " n;
(137)
(0)
is even, n.
In the particular case of the non-degenerate geometric algebra Clp.g we obtain the groups Q and Q' considered in Filimoshina & Shirokov, 2024a:
0o = -Z = oO+ = =1000
O+,
n == 1,2,3 mod 4, n + 1,2,
2 = 0220 = =Q+,
Q'
n n= 0 mod 4,
and QJ p.g Qp, Q+ in 1,2.
Theorem F2 (Formulas (23)-(24) of Theorem 3.2). In degenerate and non-degeneaate geometric algebras Clp.gr we have
Op..1 -i p.g.r S CG..r = f p.g.rt
1 p.g.rt
&p.g.r = -00- p.g.r
Proof. We get &pgrr S &g..r using Z....r (k- 1) mod 4, 1 = (k: + 1) mod 4, obtain
S 733 T.p.g.r (see Remark D.I). For fixed 0, 0,1,2,3 mod and m
F n-g.r
{T E CPx
par:
TC pg. T- (Cokmn p.ger
TCEK T
D Cuki mgrr
(138)
{T E CPx p.g.r
TCek T C
TCP p.g.T T
Optm p.g.r
TCpk TC
p.g.r - C p.g.r
Cpll
(139)
(T E CPx pig.r
7Tx TT "pigrr
TT TTx OF O....r
(140)
where we use Lemma D.5 in the first equality (140) and 2, = Lt 74 p.gr On pep by Lemma D.4 and (73) the second equality (140).
Dt = 1, 3, and 2T
2par n Ce
p.g.r
23

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Remark F.3 (Formula (26) of Theorem 3.2). (122), k 0,333
We have the following relations between the groups I p.g.r (121) and I
P-g."
Imm p.g.r C
r P-g.r
m
2,
FI P.g.r
C r P.g.r"
(141)
The statements follow from Theorems F.1 and F22 the definitions of the groups Op7 (133)- 00 2,,,, Qp (127), 2. (125), 2a C
7 p.g.r and 7 p.g.r S 2 p.g.r = 1, 2, 3, by Remark D.I.
NT F.3. The Groups Lo p.g. p.g.19 p.0.r and
[ P.g.
In Corollary F.4, we summarize the results on the equivalent definitions of the generalized Lipschitz groups I
p.g.r (120),
= 0, 1,3,3 They follow from Theorems F.I and F.2 on the equivalent definitions the groups 1 p.g.r (121) and i'
p.g.r
(122), k = 0,1, respectively, Remark F.3, and the relation (123) between these groups and the generalized Lipschitz groups I'V ".q.."
Corollary F.4 (Formula (14) of Theorem 3.1). In the case of arbitrary Olp. we have
N = Op..r S C
= Op.ai
-
C 1 p.g.r Qp.g.r
Q
(142)
p.g.r
p.q.r
AΣ C f3 p.g.r
Q3 Qp.gr
(143)
p.g.r
Theorem ES (Theorem 3.3). The elements of the generalized Lipschitz groups I'1 p.g.r have the following special form:
pT p.g S (C U (x p.ger
(144)
Proof. Suppose E FT p.g.ri then TT W ker(ad) and TT U E kcr(ad)) Therefore, T= T- and 7 7-1 --IIT.
Thus, T-T (U-1T)(T- U-lW ker(ad) A, Now we apply Theorem 4.7 (Filimoshina & Shirokov, 2023):
(C U CX(U)) (T Cb p.gar
FiT
and obtain T E (Cep2) or U p.ggr))
F.4. Relation Between Ordinary and Generalized Lipschitz Groups
The group [T p.g.r may be considered the most important among the generalized Lipschitz groups M p.g.r (120), k = 1,2,3, because its elements preserve under ad not only the subspace CeT Cf but also all other subspaces Cok P.g.r- = 1,2,3, by Corollary F.4. Moreover, the group i' p.g.r coincides with the ordinary Lipschitz group I'l p.g.. (10) in the case of the now-dimensional geometric algebras Cl Cl,.rr (Remark F.6 below) and contains it as subgroup in the case of arbitrary CC Clp.a. (Theorem F.7).
Remark F.6 (Formula (15) of Theorem 3.1). In the case of the small dimensions n I 4, the (ordinary) Lipschitz groups (10)
coincide with the generalized Lipschitz groups I p.g.r
due to CPI p.g.r = Ce1 p.q.r
M
PT P.g.rt
n S
(145)
P.g.r
Note that in the particular case of the non-degenerate geometric algebra Clp.gr is proved (Filimoshina & Shirokov, 2024a) that the groups coincide in the case as well:
Fl
PI
n S
(146)
P.9
p.0
Theorem F.7 (Formula (14) of Theorem 3.1). In arbitrary Clpar the (ordinary) Lipschitz group F (10) is a subgroup of the generalized Lipschitz group iT p.g.r (120):
F1 p.g.r
FT P.g.r-
(147)
24

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Proof. Suppose T E i1 p.g.r then T E P.g.r p.g.r and TUTT-L E CA. pig.r for any U1 Ct. p.q.r by definition. We get
=
VUI CP p.
(148)
= ---------------------
VU1 C.p.g.m
(149)
2
where we use CP p.a.r C CCT p.g.rr Table 1, and the properties of the reversion UV VU, Clifford conjugation UV VU,
and grade involution 0 = U, for any U,V E Cpp.rr We multiply both sides of the equality (148) on the left by T and on the right by T, both sides of the equality (149) the left by T and on the right by T, and get
(TTJ (TTY
= (TT)
(TT);= = UTTT. VU1 E CP! p.g.r
(150)
Therefore, by the definition (72) of the twisted centralizers, the elements T7 and TT belong the twisted centralizer of the grade-1 subspace CV1 phg Ct p.r.
TT E % p.g.
TT E P.p..
(151)
We have 2, Zp.gr Ap by Remark D.I. Therefore, दT, TT E A and T e 0p by definition (133). By Corollary F.4, Qp.., = = p.r. Thus, T Fi ".g. and the proof is completed.
G. Equivariant Mappings
Let G be group and X be set. (left) group action is a map: o: o: G X, (g.2z) go x that satisfies associativity (i.e. (gh) 9 (h o x) for any 9, h E G and
X) and identity condition (i.e. x = x for any X).
Suppose G is a group and ox and are its actions on the sets X and Y respectively. A function (network) L. X - Y is
called G-equivariant iff it commutes with these actions:
L(g x) or L(z),
Vg E G,
Va trex X.
(152)
Example G.1. Consider the generalized Lipschitz group iT This function [i p.g.r eqquivaraat iff
with the action ad and function L Clpar - Olpar.
p.grr
(x)d = =a))))))))>((()))))) (Lx)))
VT E pT
Vx E Cl
(153)
pam
i.c.
I(T(s) - +
VT E FT p.g.r.
Vr E Clp.gr
(154)
We prove several general statements about equivariance with respect to an arbitrary group. Suppose G is a group, is its action defined on a set X. Lemma G.2. Consider & G-equivariant mappings fi Xi-1 + X, i = 1....,k .k. Their composition is G-equivariant:
fa( (Fa(((a(( x))))) gox, fal ((2(((()))
(155)
for any g E G and x Xo-
Proof. We prove the statement by induction, using
felfi(g ox, +)) f2 ox, (f((z) = g ox, fz(f(
25

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Lemma G.3. Suppose group G action is linear, i.e. for any g E G, 2, E X, and a F
g oy (az) alg ox x), gr (a + y) g xoxx+ y.
Consider & G-equivariant mappings fI...., fk X -. Their linear combinations are G-equivariant:
a.fi(z))
(156)
for any 9 G. e.
Proof. We have
anfi(gox ox z) + a2f2(g ox x) = ailgoy oy fi(z)) ag fa(z)) gor + (arfi(z)
(157)
and obtain the result similarly for k summands.
Lemma G.4. Suppose group G action is multiplicatiiv, i.e.
9 oy (zy) (g (g v), The product of two G-equivariant mappings f1. X + Y G-equivariant:
Vg G, z,yeX.
oo (2(()) fi ox f2
(158)
for any g E G and x E X.
Proof. We have
g oy (fi(=)f(()) oy fi(x)) fe(x)) = fi x)
(159)
H. Degenerate and Non-degenerate Lipschitz, Generalized Lipschitz, and Orthogonal Groups Equivariance
In this section, consider the relation between the degenerate and non-degenerate pseudo-orthogone (complex orthogonal) groups and the ordinary and generalized Lipschitz groups. We extend the work presented in Crumeyrolle, 1990; Brooke, 1980; Dereli et al., 2010; Ruhe et al., 2023, etc.
Consider the degenerate and non-degenerate pstudo-orthogonal group (in the real case V = IR:::-") or complex orthogonal group (in the complex case V CD+).0.r) denoted by O(V, 4). It defined as the Lie group of all linear transformations of an n-dimensional vector space V that leave invariant a quadratic form q of signature (p. g. r) if V real and (p + q. 0, V is complex:
O(V.g) = V -V
+ is linear, invertible, q(Φ(1)) qu)), Ve Ve V).
(160)
Note that
O(V q) = (A FL), ATTA -1]
When considering both the real and complex cases, we refer to the group O(V, q) as the orthogonal group.
Let us use the following notation. The radical subspace is denoted by A7 == CP We have V = CP1 p.g.r CA The subalgebra generated by the basis elements of AA is denoted by A, and is a Grussmann (exterior) algebra.
P.g A7.
Consider the following subgroup of the orthogonal group, which leaves invariant the radical subspace A.:
O (V.4) = f E O(V.))
+ E = idA!)-
(161)
The following statement about the matrix forms of the orthogonal O(V, q) and restricted orthogonal O (V, q) groups well-known and considered, for example, in Crumeyrolle, 1990; Ruhe et al., 2023. We use the proof of the main statements about the relation between the Lipschitz and orthogonal groups below.
26

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Lemma H.1. We have the following isomorphisms
O(V.g) = (
A
(ppta)xr)
J.
A O(CC p.g. 9log..)))))))
M E Matrx(pto)(E),
G GL(r,
(162)
G
(V, = ( (p)+))x))
A A OCCP p.g. 9 9lceyy...
M Mabrx(pta)
(163)
M I
where O p.g 9 oxp..) is the non-degenerate orthogonal group of transformations of Cf, p.g with non-degenerate quadratic form a|cr; xr is the zero matrix of size (p + q) x v; (abr (F) is set of arbitrary matrices of size x (p 1 q) with coefficients in F; I, is the identity matrix of size v; GL(r, F) is the general linear group acting an v-dimensional vector space over E.
Theorem H.2 (Kernel of ad ). The kernel of the restricted twisted adjoint representation ad acting the Lipschitz group - Aut(Cpp.gr) has the following form:
ad f1 pag
ker(ad
F p.g. + Aut(CEparr) A,
(164)
Proof. We have
ker(ad' -f p.g.r - Aut(CCpgrr) = E p.q.: ToT =v Ve E p...rr
(165)
ri p.g.r n (T CPX par: T-T v, Vu E Ce! )arr
(166)
NA*
(167)
where in (167), we apply the statement (109) of Lemma E.1. Note that A. C fa u E CH1 pag. we have TTT uTT-1 u E Cf
p.g.r since for any T E A and any par. again by Lemma E.1. Thus, we obtain (164).
Remark H.3. Note that for any vectors U1, 02 E V, we have
26(11, 02)e U12 + 12 q(o)e o
(168)
In Remark H.4, consider how reflections be represented in geometric algebras. We this particular case of the relation between elements of the orthogonal groups OVV, q) and the Lipschitz groups iul below.
p.g.r in the proof of Theorem H.9
p.q Remark H.4. The mapping ad where E Ce1x x an invertible vector, acts on an arbitrary vector æ E CP. p.g.r as
reflection of vector x across the hyperplane orthogonal the vector v:
ad = erv vru (rv + vz)u = x 26(x, v)
22 q(v)
b(x, v) b(u, v)
U,
(169)
where in the right-han hight-handaaa side, we have the difference between the vector æ and twice the projection of the vector onto the vector
In Remarks H.5 and H.6, we consider several examples of elements that belong to the Lipschitz group fu pag. (10).
Remark H.5. All invertible non-degenerate vectors belong the Lipschitz group:
C il p.q p.g.r
(170)
since, by Remark H.4, we have (x) 2 b(a,,))
CP1 p.g.r
for any CC. p.g.r and Ct1x
b(v,e)
Remark H.6. Any element U E Clp.gr of the form
U + mcab CED + CP Cl
Vm E E, Vea Cf.rr Wes A
(171)
27

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
invertible and belongs to the Lipschitz group:
U E I p.gr'
(172)
Let us prove these statement. Firstly, U is invertible, since (e meat) meab) c. Secondly U preserves the grade- 1 subspace under ad, since
ÜeaU~I= = (e + meas) - meab) = (ea -maaa meab) = 2mmaaa E Ce! P.g.rr
(173)
Ūe.-1 (e meab)et(e meah) eb CP p.g.m
(174)
ŪeA-- = (e meab)eele meab) (ec + meable) meat) = ce E Ce par =e= 1........ aFa.b. a. (175)
Thus, by definition of the Lipschitz group ř1 p.g.r (10), we get (172).
Let us prove auxiliary Lemmas H.7 and H.8, which we use in the proof of Theorem H.9. Lemma H.7. In the case and n, consider any matrix of the form
(Ip+9 Mis
=
-Mis
I.) E (v.(V, =x
(176)
where Ipto and I, are the identity matrices of the sizes (p + q) x (p + q) and r r respectively, and Ma, E MMt,, p.g (F) has at most one non-zero element may. which is in the i-th row and j-th column, for fixed .,...,,....................... r. Then for the following multivector
q.
UNj = == Ciejeptoti E (CE @ C2 p.q.r)
Cij ==
Maj E F, 2nja
E
OP! p.g²
eptots E Ap,
we have
Bij adu...
(177)
In the cases r = n or r = 0, for the identity matrices I, or Ipta respectively, we have I, = ad, and 1p-4 = ad respectively.
Proof. In the Further in the proof, we consider the case + 0 and + Without loss of generality, let us prove the statement for B
0, we have add (v) eve- 1 for any E ph.rr and get the statement.
Since B1,1 E (V.q) (163), leaves invariant A/. Thus, B1,1 = idv for any e E A,. On the other hand, for U1,1 == + c1141pptot1. we have add (v) = V U statement (84) of Lemma D.3, since U1, CPp and E A7.
for any Gij E F, where we apply the
Let us consider how B1,1 acts on the vectors from canonical basis of Ct! p.2"
For the vector 01 = (1, 0,.,,) E V corresponding to 61, we have
B1,1 = (1, 0, 0, m11 000000..... O),
(178)
where mi1 on the (p + q + 1)-th position. On the other hand, for U1,1 and eq, we get
aāl,, (e1) (c1 (11h11ptttt)(( (1111pttt11 e1 2m11111p4ttt1 G1 + M110p-g-1,
(179)
which corresponds to (178).
For any other vector Up E V, .,...,P. + 4. corresponding to cx. we have
B1, = (O...., 1,0,...,00) where 1 on the k-th position. On the other hand, for U1,1 and ex. we get
(180)
add,, (ex) c1e10pttttt (eue c1111ppt411) Ck.
(181)
which corresponds to (180). We have proved that B1,1V = ady, (v) for any v from a canonical basis of V. By linearity, we get the statement (177). This completes the proof.
28

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Lemma H.8. We have
XV = VX, e e POX U C(LLK)
VV W
(182)
Proof. Suppose V A and X WH, where W (CE (0) U CPp2)r) and H E A.A.
Note that HV VH by the statement (109) Lemma E.I.
W E Clp..r then we have WV = WV = VW by the statement (84) of Lemma D.3.
Consider the case W C p.g.r. I then W A, and get WV VW again by the statement (109) of Lemma E.1. r * n, then there exists such generator e; that (e,)² o O, and we can always represent W as W = (Tae,; Since mae, W E Cep.g.r. we again get (neeWW) = ((nae:W by the statement (84) of Lemma D.3. Also we have = = -e,V Ve,, since V Ar does not contain e,. So, in this case, we get WV (nae,WW &.nnee Vemae; VW as well.
Using the notes above, finally get
WWV=WV=VV=VVV WWW
(183)
and the statement is proved.
Theorem H.9 (Image of ad ). The image of the restricted twiste adjoint representation ad' acting the Lipschitz group ad' p.g + Aut(CC has the form:
innad 1 p.go Aut(C(pa.r)) (V.q).
(184)
Proof. Firstly, we prove that imda" r p.g.r + Aut(Clp.ar)) S OV. i.. that ady (v, for any i Suppose T iu p.g.rt then ady is linear by Lemma 2.1. Moreover, adl is invertible with the inverse ady since
p.g.
aad = 1)== u
Ve E Ce. Ce
(185)
Now, we need to show that
(v) q(v),
Vr E Cpp...
(186)
We have
q(ad(())) = (air (w)) = adr(d)) = (x²),
(187)
where we apply Remark H.3, since adr(v) E V, and multiplicativity of ad7 (Lemma 2.1), since ce(U)))A by Theorem 3.3. Since 2 qu)) again by Remark H.3, we finally get
fu p.g.r S (Ce(0)x
q (v)) (q(x)) q(v).
(188)
where in the last equality we use the property that ady leaves invariant the scalars by the statement (8) of Lemma 2.1. The 11 statement (186) is proved, i.e. we have shown that ady E O(V, q). To prove adl E O (V,9), we finally need show that lA = idag. This statement true because for any v A7 and fu p.g.r C (O)()) (( (Theorem 3.3), we have a (v) = T = vTT = by Lemma H.8.
Now let us prove Oa (V.q) C ii((dd' fu -fa. + Aut(C(p.g.))), i.e. let is prove the surjectivity the mapping st We have by Lemma H.l:
Ov: = (1 0) 9)), A E O(Ce) paa Alcop,))
p
M E Matrx(p+o) (F).
(189)
29

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Suppose & Oa (V, Then, there exist A E O(C
p.g. Alay, and M E Malrx((tg)) (IF) such that
A 0 M
A
p+g 0 M I,
A
0 I, (o.
0) Ipta
2.)... M1
". Ipto 0 (190)
(Mrpte I, 2.).
I.)
I,
0
where A A1.. ......A AA. k Σ + g. and A Ak E O(CE pa: 9lcr,.) are matrices representing reflections, by the theorem (Dieudonné, 1971), which states that every orthogonal transformation in a (p + q)- dimensional space with a non-degenerate symmetric bilinear form can be represented as a composition of at most + P 9 is a matrix with at most one non-zero element,
well-known
reflections; and Ma E Mat, (x+g) (F), i = 1.....r. r. I.................. which is on the i-th row and j-th column, such that M Ei=t.... Ej=t....p4g Ma.j-
Note that each reflection matrix A,, i = I,..., k, can be associated with some vector iú E CC. p.g. so that A, ad by Remark H.4. Any matrix
1p--g 1)) T.)
Big
(191)
Mij
can be associated with some multivector 7i. mireseptori Ct @ CC Ubg. where maj F. Ej CA p.g. and
Eptgmi Ar, so that Big a by Lemma H.7. Consider the following multivector, which consists of + + factors:
T 01 VK.1 Trpto
(192)
Note that T E Fi
since ,1...., Uk E Ti D.g.r
by Remark H.5 and - f pig.r by Remark H.6. We obtain
p.g.
for any E V = CPl
p.g.ri
(u)) = A1
(AR
0 I.)
9)) ... (Mrptg I,) 1.) M1,1
Ipta
Ipto
(193)
I,-
ad.,
(ndux (aid).., (... (ad ( (wi)))
(194)
)o क ... (. (Tnpto ",, 1uk -1 ...
(195)
(V1 1. " "a..a..nhhhhh///a (nptg)u(or 21711 Trptg)
(196)
ad
(197)
(o).
(198)
Thus, for any & Oxg (V. q), there exists T E fu p.g.r such that Φ- adr,, and this completes the proof.
Theorem H.10. In the case of any degenerate or non-degenerate Clp.rr- the mapping ad defines the following isomor- phism:
ad'
FI Par / A 00;
(199)
Proof. We have the homomorphism sdd of the groups F p.g.r and O (V, which is surjective by the statement (184) of Theorem H.9. By the fundamental homomorphism theorem, the group Ox₁ (V, q) is isomorphic to the quotient group fl p.g.r /ker(ad)) ). By applying Theorem H.2, we complete the proof.
Note that in the particular case of the den-negeeee geometric algebra Clp.g- Theorem H. 10 has the form
ad i' P.a /cx0x & O(V.q)
(200)
and is well-known (see, for example, Benn & Tucker, 1987).
30

---

Therefore, f is O(V, requivarva, and the statement is GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Theorem H.11 (Theorem 3.4). If a mapping f ..p.g.r + Ppp r is equivariant with respect any group H. that contains the Lipschitz group [ p.g.r as a subgroup, then f is equivariant with respect to the corresponding restricted orthogonal group. In other words, if
adr(s(x)) = f(ar(<))) EHH Vx E Cla
then
v$ E (V, Ve Cppar
where & acts on x in a sense (34) by applying an orthogonal transformation to its vector components.
Proof. Suppose H. is a group, f fp.. S H. and adr(f(z)) = ((dr(()) for some mapping f. for any T E H, and Clp...- Then holds, in particular, for any T E [' p.g.r and is f p...--quiivarinn By Theorem H. mapping is equivariant w.r.t I p.g.r iff it is equivariant w.r.. proved.
I. Experimental Details
The implementation of GLGENN and summary of each experiment https//githubbcommkkkayyiiiiiiip
are
available
For CGENN (Ruhe et al., 2023), we follow the training setups from the corresponding public code release. For other models, we use the loss values from the corresponding code repository (Finzi et al., 2021).
We construct our models to closely resemble the CGENN architecture, replacing the CPk -linear, CCt, geeomerii product, and Oikk p.a"" -ormalizzaiin layers from Ruhe et al., 2023 with the same number of GLGENN's CFE p.g counterparts (see Section 4). This design choice results in GLGENN having significantly fewer parameters than CGENN, they operate in unified manner across 4 fundamental subspaces of geometric algebras defined by the grade involution (*) and reversion ("); they processes geometric objects in groups with step size of 4.
However, this parameter efficiency difference between GLGENN and CGENN becomes apparent only for tasks with n > 3. In lower-dimensional cases (n - 3), subspaces of fixed grades coincide with the subspaces determined by grade involution and reversion (4), i.e., Cek p.g.r Ci p.g.r for k = 0, 1, 2, 3. Nonetheless, experiments show, even in 5-dimensional cases, the performance gap between CGENN and GLGENN is substantial. Moreover, GLGENN's parameter efficiency advantage increases with dimension n, the subspaces CPF g...r diverge further from Cyk p.g.r
( Regressiin Task
In our first experiment, consider O(5, -iiinrariant regression task proposed in (Finzi et al., 2021). The task is to estimate the function sin[[ II) || || /2
I [| where 21, E 15.0 vectors sampled from a standard Gaussian
distribution. The loss function used is Mean squared error (MSH).
We evaluate the performance using four different training dataset sizes and compare against the ordinary MLP, MLP with augmentations. the O(5, 0)- and SO(5, 0) -jequiivariant MLP architectures proposed in (Finzi et al., 2021), and with CGENN (Ruhe et al., 2023). CGENN, MIP, MLP with augmentations, O(5, 0)- and SO(5, --equiivari MIP architectures have approximately the same number of parameters, we use the setup from the similar experiment provided Ruhe et al., 2023 (they state that they compare their model CGENN with other models with the same number trainable parameters).
The number of parameters in Table (O(5, 0) Reggression Experiment) is as follows. All the models besides GLGENN and CGENN have = 150.3K parameters in total. In CGENN and GLGENN, the architecture contains sequentially applied geometric algebra-based layers (applied to the subspaces of all grades) and ordinary MI.P layers (applied only to the subspace of O-grade (scalars)). The most significant layers and at the same time the most consuming for training are geometric algebra-basee ones, and CGENN possess = 1.8K of parameters associated with such layers, while GLGENN has se 0.6K of such parameters. For the MLP part, which is very fast and easy for training, both CGENN and GLGENN both have 148.5K parameters.
31

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Table 5. MSE ()) on the (55, -)-eegrsssion Experiment.
MODEL
OF TRAINING
SAMPLES
3-10 10
3 10
3 -00
3 10*
GLGENN
0.1055
0.0020
0.0031
0.0011
MLP
28.1011
0.2482
0.0623
0.0622
MLP+AUG
0,4758
0.0936
0.0889
0.0672
EMLP-O(5)
0.152
0.0344
0.0310
0.0273
EMLP- SO(()
0.1102
0.0384
0.032
0.0279
CGENN
0.0791
0.0089
0.0012
0.0003
Table 6. MSE (1) on the O55, -)Regrsssooon Experiment in the case of large training set size.
MODEL
# OF TRAINING SAMPLES
6 10
1-10
GLGENN
0.0001
0.0001
CGENN
0.0002
0.0002
The results, presented in Tables 5, 6 and Figure (left), demonstrate that GL,GENN achieves performance on par with CGENN, while significantly outperforming the other models. Moreover, GLGENN attains these results with fewer parameters and reduced training time compared to CGENN. In Table 5, MSE for CGENN and GLGENN are averaged over 5 runs. MSE for other models are averaged over 3 runs (from Finzi et al., 2021). Number of iterations is the same for all algorithms.
(55, 0)-Coneex Hull Volume Estimation
In this equivariant experiment, the task is to estimate the volume of convex hull generated by K points in R5.0 We consider three different values of K: 16 (as used in (Ruhe et al., 2023; Liu et al., 2024)) and two additional settings, K = 256 and 512, which are more relevant for real-world applications.
In the case K = 16, we consider 4 different sizes of the training set: 256, 1024, 4096, and 16384 samples. We compare GLGENN with the stae-of-the--rr model CGENN (Ruhe al., 2023). While both models share a similar architecture, GLGENN has 24 trainable parameters, whereas CGENN has 58.8K. The number of training steps is the same for both models. The results presented in Table and Figure 2 (middle). GLGENN either outperforms or matches CGENN across all training set sizes. The most noticeable difference occurs with smaller training sets. We attribute this to GLGENN's lower parameter count, which reduces its tendency to overfit an issue commonly observed in small datasets.
To study the behavior of CGENN and GL.GNN on different iterations of training, we provide Figure 4. This figure shows the training and test loss at different iterations. Notably, CGENN tends to achieve lower training loss compared to GLGENN. However, at certain point, while CGENN's training loss continues to decrease, its test loss plateaus and remains almost constant. In contrast, GLGENN'S 's training loss decreases slower rate, but its test loss consistently lower than that of CGENN, indicating better generalization.
For K = 256 and K = 512, the results are also presented in Table 3. To ensure a fair comparison, we first select CGENN architectures that perform best for each setup. We then construct corresponding GLGENN models by replacing CGENN layers with their GLGENN counterparts, resulting in significantly fewer trainable parameters. Note that GLGENN consistently outperform CGENN in these real-world settings.
There a contrast in GLGENN and CGENN generalization behavior in cases of large K. We illustrate it in Figure 5, which shows the training and test losses for K 256 in the case of 1024 (left) and 16384 (right) training set sizes. GLGENN demonstrate stable convergence without signs of overfitting: the training and test losses decrease in similar way throughout the optimization trajectory. CGENN show a clear overfitting pattern: while the training loss quickly drops to near-zero, the test loss plateaus early and remains significantly higher, especially in the small-data regime.
Following the recommendation of one of the anonymous reviewers, in Table 7, we report the average wall-clock time required to process one training batch for GL ELGENN and CGENN across different training set sizes and number of points K.
32

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
A
(Trai)
35
OGENN (Trwe)
CSENN (Test)
(Te4)
(ours) (Train)
GLGENN (ours) (train) GLGENN lours) (Test)
GLGENN (curs) (Tasti
25
20
Loss
15
5
64
128
192
256
320
384
448
512
Iteration
Iteration
C
D
35
OSENN (Train)
35
OSENN (Trai)
OGENN (Test]
OCENN (Test)
GLSENI
GLGENN tours) (Train)
GLGENN (curs) (Testi
GLGENN (ours) (Test)
20
LOSS
15
0
0
Iterraton
Iteration
Figure ((5, 0)-Convex Hull, K 16. The plots illustrate the training and loss curves for CGENN and GLGENN across different training iterations. Subfigures (A)(D) correspond to different training set sizes: 256 (A), 1024 (B), 4096 (C), and 16384 (D), respectively.
The results demonstrate that GLGENN consistently achieve faster training times compared to CGENN. The performance gap remains notable across training dataset scales. The majority of memory usage is attributed to the Python environment and PyTor'h's 's internal components rather than to the models themselves; result, the memory footprint comparable between GLGENN and CGENN.
Table 7. Average time (in scconds) for processing one training batch in O(5, 0)-Convex Hull Experiment. In case of the number of points
K 16 and 256, for datasets with 29, 20 and 21 training samples, batch size is 128; for 2 samples, batch size is 256. case of K 512, for datasets with 20 and 22 samples, batch sizes are 256 and 512 respectively.
16
256
512
MODEL
# TRAIN SAMPLES
# TRAIN SAMPLES
# E'TRIIN SAMPLES
2
2
212
214
2
2
210
214
GLGENN
0.22352
0.16708
0.1712
0.3408
3,9931
12,8758
13.3317
27.4383
CGENN
0.33692
0.2331
0.23418
0.43854
8.658
17.7424
17.6603
35,5605
GAP
-0.1134
0.06602
0.06298
-0.09744
-4.6649
-486666
4..228
-8.1222
O77, -onCnx Hull Volume Estimation
In this experiment, we extend the previous task with number of points K 16 to higher-dimensional setting and evaluate the performance of GLGENN and CGENN in estimating the volume of convex hull formed by 16 points in 7,700 We consider three different training set sizes: 256, 512, and 1024 samples.
While both models share a similar architecture, GLGENN maintains 24.1K trainable parameters, as the previous experiment, whereas CGENN's parameter count increases to 83.7K. The number of training steps remains the same for both models.
33

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
B
200000
GLGENN (ours) (Train)
30000
GLGENN (ours) (Triin)
175000
GLSENN (ours) (Test)
GLCENN (ours) (Test)
COENN (Trein)
CGENN 25000
CGENN
150000
CGENN (Test)
125000
Loss
100000
15000
75000
10000
50000
25000
5000
30
60
90
120
150 180 210 270 300 330
Iteration
Iteration
200000
GLGENN (ours) (Train)
14000
GLGENN (ours)
175000
CGENN (Train)
GUGENN (ours) (Test)
CGENN
GLCENN (ours) (Test)
150000
CGENN (Test)
CGENN (Test)
8000
Loss
100000
Loss
8000
75000
50000
4000
25000
2000
30
60
90
120
150
180
240
300 330
240
270
300
330
Figure 5. O5, 0)-Convex Hull, 256. The plots illustrate the training and test loss curves for CGENN and GLGENN across different
training iterations. Subfigures (A)-(B) correspond training with 1024 samples; subfigures (C)(D) correspond training with 16384 samples. The right-hand plots zoom in on the final iterations of training.
20
CGENN (Train)
CGENN (Tain)
CGENN (Taini
CGEKN (Tacc)
GLGENN (ours) (Triin)
GLGENN fours) (Tein))
ELGENN (oursi [Tran]]
GLGENN (ours) (Testi
CLGENN (ours) lessti
CLGENN (ours) [Test]
1055
15
Iteration
Iteratton
Iteration
Figure 6. O(7, 0) Convex Hull. The plots illustrate the training and test loss curves for CGENN and GLGENN across different training iterations. Subfigures correspond different training sizes: 256 (left), 512 (middle), 1024 (right), respectively.
The results, presented in Table 8, show that GLGENN consistently outperforms matches CGENN. The smaller is the training set size, the higher is the difference.
Figure compares the training and test loss for CGENN and GLGENN across different iterations. The observed behavior consistent with the results from the O(5, 0)-Cvexx Hull Experiment: CGENN tends to achieve lower training loss, while GLGENN exhibits better generalization, particularly for smaller training sets.
34

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Table 8. MSB (L) on O(7, 0)-Convex Hull Experiment on the test dataset.
MODEL
OF TRAINING SAMPLES
OF PARA-
2
2
2
METERS
GLGGEN
4.0032
3.7378
3.5343
24.1K
CGENN
4.4914
3.7756
3.5408
83.7K
CGENN Same Size as GLGENN
In all our experiments, GLGENN architecture, by construction, has fewer trainable parameters than CGENN. This is because, for each task, we first construct the best-perforning CGENN architecture and then obtain GLGENN by replacing CGENN layers with their corresponding GLGENN counterparts considered in Section which have fewer parameters. Across all experiments, GLGENN either matches outperforms CGENN, despite using fewer parameters.
In this subsection, we explore another setting: we compare GLGENN with CGENN architectures constrained to have the same number of parameters as the best-performing GLGENN. Our results show that, in almost all cases, CGENN performs worse under this constraint.
Table 9 presents results from the (55, O)-Coneex Hull Volume Estimation Experiment. In this setup, the CGENN model downsized to have approximately 25K parameters (compared 58.8K in its pest-performing version), while GLGENN has 24.1K parameters. As shown, GLGENN outperforms the size-matched CGENN across all training set sizes.
Table 9. MSE (1) on the O55, 0)- -)-nvxx Hull Experiment (K 16 Points) on the test dataset. Comparison of CGENN of different sizes with GLGENN.
MODEL
OF PARA- METERS
# OF TRAINING
SAMPLES
2
2
212
2
GLGENN
24.1K
16.94
10.40
4.46
CGENN (SAME SIZE AS GLGENN)
25K
19.79
15.94
7.69
4.23
CGENN (BEST PERFORMING)
58.8K
18.71
11.93
6.1
4.11
Table 10 reports results for the O(5, Regressson Task, where both GLGENN and CGENN are constrained to have approximately 149.1K total parameters, including about 0.6K parameters in geometric algebra-based layers. Again, GLGENN achieves superior or comparable performance.
Table MSE (L) on the (55, Re-gerrsso Task on the Test Dataset. Comparison CGENN different sizes with GLGENN, The second column shows the number of trainable parameters associated with geometric algebra-basee based layers.
MODEL
# OF GA-PARA-
# OF TRAINING SAMPLES
METERS
3 10
3 10
3 10
3 10*
GLGENN
0.6K
0.1055
0.0020
0.0031
0.0011
CGENN (SAME SIZE AS GLGENN) CGENN (BEST PERFORMING)
0.6K
0.2856
0.0076
0.0017
0.0005
1.8K
0.0791
0.0089
0.0012
0.0003
Application of Typical Activation Functions
is possible to combine geometric algebra-based layers with standard neural network layers, such as MLPs, while preserving equivariance - provided that the non-geometric algebra- layers are applied only to scalars (i.e., elements of the subspace Ct The diversity of layers in this combination may lead to better results, although in equivariant tasks, non- geometric algebra-based layers on their own generally underperform compared GLGENN layers other geometric algebra-based layers. Applying non-geometric layers to Ct" has inherent limitations compared to geometric algebra-based nonlinearities: they do not mix grades and instead isolate them, thereby hindering interactions across subspaces.
We provide an example in the case of the O(5, Regressiin Task with 300 training samples. The results presented in Table 1I and Figure show that the best performance is achieved by a combination GLGENN (applied to all grades) and ML.P (applied scalars).
35

---

GLGENN: A Novel Equivariant Neural Network Architecture Based on Clifford Geometric Algebras
Table 11. MSE (4) on comhination ML.P with GLGGENN and CGENN ( Reggrrsiion.n..
MODEL
MSE ON TEST
MLP
0.1706
GLGENN W/ MLP
0.0010
CGENN w/ MI.P
0.0066
GLGENN w/O MIP
0.0265
CGENN W/O MLP
0.0723
(5,, 0)-y-Body Experiment
We consider system of N = 5 charged particles (bodies) in 125 with given masses, initial positions, and velocities. The task is to predict the final positions of the bodies after the system evolves under Newtonian gravity for 1000 Euler integration steps. For this purpose, we embed all the data in the geometric algebra Cε5,0-
We construct graph neural network (GNN) based on the message-passing passing paradigm (Gilmer al., 2017), where bodies are treated nodes in graph, and their pairwise interactions are modeled as edges. The message and update networks are equivariant GLGENN. We use -ogno-linear, C -noomallivaainn, and --goometti product GLGENN layers combined with MVSiLU layer from CGENN (Ruhe et al., 2023). We compare against CGENN, which itself outperforms several state-of-the the-art methods, including steerable SE(3)- Transformers (Fuchs al., 2020), Tensor Field Networks (Thomas al., 2018 8), SEGNN (Brandstetter et al., 2022), Radial Field (Köhler et al., 2020), EGNN (Satorras al., 2021), and NMP (Gilmer et al., 2017). To ensure fair comparison, layers with analogous GLGENN counterparts to obtain the GLGENN architecture, which automatically has two times fewer trainable parameters. The results are presented in Table 12 and Figure 2 (right).
use the best-performing CGENN architecture and then replace its
Table 12. MSE (L) on the O(5, D)-N-Body Experiment.
MODEL
# OF TRAINING SAMPLES
# OF PARA- METERS
3-10'
3-102
3 103
GLGENN
0.007
0.0011
0.0009
103K
CGENN
0.0136
0.0015
0.0007
210K
36


Reviews:
[{"review_id": "mpSf3ebhGs", "reviewer": "Reviewer_qfnz", "summary": "This paper introduces a novel group equivariant architecture based on Clifford geometric algebra that are equivariant to pseudo orthogonal transformation and are more parameter efficient and avoids overfitting compared to prior works in equivariant networks based on Clifford algebra. The design is based on first designing a generalized Lipschitz group and then designing an equivariant network for this generalized Lipschitz group that leads to more parameter efficient networks. Experimental results on two synthetic datasets show competitive performance with respect to existing Clifford algebra equivariant networks.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We are happy to answer the questions and address the concerns as follows:\n\n**1. Q1: Relation between generalized Lipschitz groups equivariance and parameter efficiency?**\n\nApplication of the generalized Lipschitz groups (introduced in this work) instead of ordinary Lipschitz groups allows to achieve parameter efficiency, we explain it below. The generalized Lipschitz groups are important because they preserve the four fundamental subspaces of Clifford algebras under the significant operation of the twisted adjoint representation. We prove that equivariance of a mapping w.r.t. these groups, as well as the ordinary Lipschitz groups, implies its orthogonal groups equivariance. The key distinction is that the generalized Lipschitz groups contain ordinary Lipschitz groups as subgroups. As a result, the set of operations equivariant w.r.t. the generalized Lipschitz groups is a subset of the set of operations equivariant w.r.t. ordinary Lipschitz groups. This reduction in the number of ‘degrees of freedom’ encourages us to parametrize operations in layers in a more ‘economic’ way (there is a smaller number of parameters that we can place). Specifically, in all GLGENN layers, we employ such equivariant operations as projections of inputs-multivectors onto the 4 fundamental subspaces of Clifford algebras mentioned above, whereas CGENN relies on projections onto the subspaces of fixed grades. GLGENN and CGENN layers parameterize linear combinations, products, and normalizations of the corresponding projections. The number of the fixed-grades subspaces is equal to $n+1$, where $n$ is the dimension of the task’s vector space. As $n$ increases, the number of CGENN parameters grows significantly, while for GLGENN, it remains constant. To summarize, the key reason why GLGENN layers are parameter-efficient lies in how we parametrize the operations in them: using the projections onto the 4 fundamental subspaces, instead of $n+1$ subspaces as in CGENN.\n\nThe growth in the number of parameters for similar layers in GLGENN vs. CGENN as $n$ increases can be estimated by the formulas in our paper. GLGENN geometric product layer contains $4k^2+4^3k$ parameters for $k$ input channels (Sect. 4.4), while CGENN geometric product layer has $(n+1)k^2+(n+1)^3k$ parameters. Similar formulas for other GLGENN layers are in Sect. 4.3 and 4.5. The formulas for CGENN can be obtained by replacing 4 by $n+1$.\n\nWe thank the reviewer for this good question for presentation improvement, **we will add** these more detailed explanations.\n\n\n**2. Q1 (part 2): Better performance of GLGENN in experiments?**\n\nOne of the goals of our work is to design a model that performs better than CGENN in case of small training datasets, which is a common scenario in the natural sciences, where datasets are often manually derived from experimental results. GLGENN is designed to mitigate overfitting in such cases, while its expressivity does not necessarily provide an advantage when data is abundant. Specifically, GLGENN significantly outperforms CGENN in both convex hull experiments, particularly when the training set is small (e.g., $2^{8}$ or $2^{10}$ samples in $n=5$ case, Fig. 3). When the training set size is large for the task ($\\geq 2^{12}$ samples), GLGENN and CGENN are on a par (with the difference <0.4 MSE, which is, in our opinion, almost insignificant in comparison to the difference in parameters quantity – 24.1K for GLGENN vs. 58.8K for CGENN), the similar behaviour remains in the training set size $>2^{14}$ samples. In the regression task, GLGENN again achieves lower MSE than CGENN when trained on a small dataset (300 samples, 0.002 vs. 0.0089). With an extremely small training dataset (30 samples), GLGENN is on a par with all other models, except MLP and MLP+Aug, likely due to insufficient data for meaningful generalization. When the dataset is large ($3\\cdot10^3$ or $3\\cdot10^4$ samples), CGENN slightly outperforms GLGENN, but the difference diminishes as data grows.\n\n**3. Q2: More experiments done in prior works?**\n\nPlease refer to Point 6 in our response to Reviewer xqBf, where we provide details on which experiments **we will add**. \n\n**4. Q3: Number of parameters?**\n\nThe number of parameters in Table 2 (O(5,0) Regression Experiment) is as follows. All the models besides GLGENN and CGENN have ~150.3K parameters in total. In CGENN and GLGENN, the architecture contains sequentially applied Clifford algebra-based layers (applied to the subspaces of all grades) and ordinary MLP layers (applied only to the subspace of 0-grade (scalars)). The most significant layers and at the same time the most consuming for training are Clifford algebra-based ones, and CGENN possess ~1.8K of parameters associated with such layers, while GLGENN has ~0.6K of such parameters. For the MLP part, which is very fast and easy for training, both CGENN and GLGENN both have ~148.5K. **We will add** this information.", "claims_and_evidence": "While the theoretical contribution seems straightforward, the practical motivation is not very clear to me: how is equivariance to the generalized group leading to more parameter efficiency? Is it straightforward, please provide better explanation. Also, in the experiments, shouldn’t using a more general Lipschitz group lead to better performance? Why is the performance slightly worse than CGENN, i.e., is there a reason why the expressivity of the proposed method is limited compared to prior work?", "methods_and_evaluation_criteria": "The amount of experiments seem very limited compared to both the motivation of the work as well as prior works such as Geometric Algebra Transformers (GATr). Is it possible to provide more experiments done in prior works?", "theoretical_claims": "This looks good to me", "experimental_designs_or_analyses": "Soundness/validity of experiments look good.", "relation_to_broader_scientific_literature": "This paper makes progress in the area of Clifford Algebra equivariant networks, which is an established direction in the area of efficient machine learning.", "essential_references_not_discussed": "Looks good to me", "other_strengths_and_weaknesses": "Strengths:\n1. This work proposes a novel architecture based on Clifford geometric algebra for a new class of Lie groups \n2. The proposed equivariant model construction technique is light weight in terms of number of parameters compared to prior Clifford algebra networks\n3. Experimental results show that the proposed method is competitive with existing methods as well as light-weight in terms of parameters\n\nWeaknesses:\n1. While the theoretical contribution seems straightforward, the practical motivation is not very clear to me: how is equivariance to the generalized group leading to more parameter efficiency? Is it straightforward, please provide better explanation. Also, in the experiments, shouldn’t using a more general Lipschitz group lead to better performance? Why is the performance slightly worse than CGENN, i.e., is there a reason why the expressivity of the proposed method is limited compared to prior work?\n2. The amount of experiments seem very limited compared to both the motivation of the work as well as prior works such as Geometric Algebra Transformers (GATr). Is it possible to provide more experiments done in prior works?\n3. What are the number of parameters in Table 2?", "other_comments_or_suggestions": "None", "questions_for_authors": "Please check the weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nEq1EV6MFn", "reviewer": "Reviewer_xqBf", "summary": "This paper introduces a new version of Clifford Group Equivariant Neural Networks (CGENN) originally introduced by Ruhe et al., 2023 called GLGENN.  The authors develop a theory for generalized Lipschitz groups which generalize and contain Clifford groups.  Generalized Lipschitz groups preserve a subspace decomposition corresponding to involution and reversion operations on the Clifford algebra.  If I understand correctly, enforce equivariance to this larger group using a weight sharing scheme with respect to this coarser subspace decomposition as opposed to the finer grading results in fewer trainable parameters relative to CGENN.  Since the group is larger, there is still equivariance to the clifford group and thus to the orthogonal group.  The experiments show that there is no loss in performance despite the fewer parameters and the overconstraint.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We are happy to answer the questions and address the concerns as follows:\n\n**1. Q1: Higher order features such as tensors?**\n\nYes, the model can accommodate higher-order features, such as tensors. For example, it can be applied to the N-body problem, where the goal is to predict the positions of N charged particles in an n-dimensional space after a certain number of timesteps, given their initial positions and velocities. In this task, tensors represent the N initial positions and velocities (each with n coordinates). The model can process such tensors when graph neural networks are used to define the structure of connections between objects.\n\n**2. Q2: Conjugation operations layers?**\n\nThank you, this is a good question. We suggest the following method to make the optimization of conjugation operations possible, though other potentially more effective approaches may be suggested. We first apply a linear transformation of the projections of the input multivector $x_{in}$ as in (42) but with any parameters $\\phi_{c_{in}k}\\in\\mathbb{R}$. Then we can round them to -1 or 1 either by directly applying $\\phi_{c_{in}k}\\mapsto sgn(\\phi_{c_{in}k})$ or by firstly applying the sigmoid function, then scaling the value to [-1,1], and applying sign function to the result. In the experiments presented, these layers are not used in order to maintain the architecture choices of GLGENN close to the corresponding of CGENN as much as possible to ensure fair competition. We mention this in lines 1558-1560 of Appendix F and will add a comment on it in the main part of the paper.\n\n**3. Q3: Linear layers cannot mix information among $Cl^{\\overline{k}}_{p,q,r}$?**\n\nYes, you are correct. The linear layers do not mix information among the subspaces $Cl^{\\overline{k}}_{p,q,r}$, $k=0,1,2,3$. Instead, they perform linear transformations independently within each of these four subspaces. A visual representation of how a linear layer operates can be found at the following [link](https://drive.google.com/file/d/1ev594pDgVjKuMfXY5rCB4eZPK4W2jEju/view?usp=sharing).\n\n**4. Q4: Geometric products and normalization are the main source of non-linearity and mixing $Cl^{\\overline{k}}_{p,q,r}$?**\n\nYes, you are correct. GLGENN introduce non-linearity through the geometric product and normalization layers, where the subspaces $Cl^{\\overline{k}}_{p,q,r}$, $k=0,1,2,3$, are mixed.\n\n**5. Overfitting of CGENN.**\n\nOur experimental results (see, for example, Figure 3) align with the results in CGENN paper Ruhe et al., 2023 (see Figure 3 (right) and discussion of the convex hull experiment on page 9 of their work), where they also note the tendency of CGENN to overfit in scenarios with small training datasets.\n\n**6. Other experimental designs.**\n\nWe thank the reviewer for the idea on how to improve the work. We are positive regarding scaling of GLGENN as well. To strengthen our experimental section, **we will add** the experiments done in prior works: (1) real-world task in $Cl_{1,3}$ of categorizing high-energy jets produced in particle collisions by their trajectories using the data from CERN’s ATLAS detector (this task is considered in CGENN paper by Ruhe et al., 2023), (2) classic benchmark N-body experiment (the task is considered in Ruhe et al., 2023 and GATr paper by Brehmer et al., 2023), and (3) harder settings for the convex hull and regression experiments, in particular, comparing CGENN of the same size as GLGENN. We will also include a comparison of memory footprint and wall clock time between GLGENN and CGENN. We would like to note that while the current experiments are toy problems, they are considered benchmark tasks in the field and have been used in highly regarded papers on equivariant neural networks (e.g., Finzi et al., 2021; Ruhe et al., 2023; Liu et al., 2024). This is why we chose them as a proof of concept, and GLGENN successfully validated our hypothesis that structured weight sharing improves performance.", "claims_and_evidence": "- The paper notes that CGENNs are overparameterized and tend to overfit and have slow training times motivating the need for a fewer parameter version.  I didn’t specifically see evidence of overfitting in the experiments.  The slow training time for CGENN is certainly a concern, but there is not evidence in the main the GLGENNs remedy this.\n- GLGENNs are shown to have comparable performance to CGENNs even with fewer parameters. \n- Whether the larger group constraint poses a problem for expressiveness is not conclusively shown.  In the 2 synthetic tasks considered it did not.\n- The equivariance of GLGENNs is firmly established by an extensive theory section.", "methods_and_evaluation_criteria": "- The method is a reasonable evolution of CGENN as described in the summary above.  The authors provide analogs of the layers provided by Ruhe et al, 2023.  Except for the Conjugation operation, the novelty appears to be limited to generalization to the new group and grading. \n- The evaluations are similar to those in prior work, but are small scale and synthetic.  As noted above, the question of overfitting and computation speed do not appear to be directly addressed.", "theoretical_claims": "- A definitive strength of the paper. The mathematics of clifford geometric algebras, and generalized Lipschitz groups is very clearly laid out both in the background and new contributions. \n- The authors prove several results which firmly underlie their method: (1) generalized lipschitz groups contain the clifford groups and thus equivariance wrt gen. lipschitz groups implies clifford group equivariance (2) the subspace decomposition is preserved by gen. lipschitz groups, and (3) gen lipschitz equivariance implies orthogonal group equivariance, (4) several different operations are equivariant wrt gen. lipschitz groups, providing the basis for the layers in CGENN.\n- The theory section could use a bit more scaffolding/prefacing to help connect the results to their eventual use in constructing CGENNs.  I had to read forward and backward a bit to understand.", "experimental_designs_or_analyses": "- As noted above, the experiments do demonstrate a smaller GLGENN performs comparably to CGENN, but it would be good to also show that a CGENN the same size as the GLGENN underperforms the GLGENN.  If not, it undermines them main claims.\n- The experiments are very small scale and very synthetic.  The practical usefulness of the method is not demonstrated.  My guess is that it will scale, however, given how similar it is to CGENN which has ben demonstrated in practical applications.  \n- While GLGENN has fewer parameters, memory footprint and wall clock compute should also be compared.", "relation_to_broader_scientific_literature": "There is no related work section, but related work is adequately addressed in the introduction and throughout the paper in my opinion.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "320L: Equivariant misspelled", "questions_for_authors": "- Can the model accommodate higher order features such as tensors? \n- Since the parameters of the conjugation operations are discrete, how are they optimized? Are they used in experiments? It would be good to comment in the paper on this. \n- The linear layers cannot mix information among \\bar{k}, correct?\n- Are geometric products and normalization the main source of non-linearity and mixing \\bar{k}?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2sa23oExhQ", "reviewer": "Reviewer_RrP5", "summary": "This paper formally introduces neural networks that are equivariant with respect to generalised Lipschitz groups $\\tilde{\\Gamma}_{p,q,r}^{\\bar{k}}$ for $k = 0, 1, 2, 3$ that are constructed from an arbitrary degenerate or non-degenerate Clifford geometric algebra (GA). The construction is based on two fundamental conjugation operations in the GA, called grade involution and reversion, that create so-called \"subspaces of quaternion types\" for $k = 0, 1, 2, 3$ which are preserved by the generalised Lipschitz groups under the twisted adjoint representation $\\tilde{ad}$. The authors show, in particular, that for $k = 1$, all generalised Lipschitz group equivariant mappings are also equivariant to group that is restricted from a degenerate or non-degenerate orthogonal group to the radical subspace $\\Lambda_r^1$. The resulting neural network, termed GLGENN, is compared against CGENN (Ruhe et al. 2023) and other neural networks on two series of tasks, an $O(5,0)$-regression task and a $O(5,0)$- and $O(7,0)$-Convex Hull Volume estimation task. The authors claim that GLGENN achieves a reduced tendency to overfit having significantly fewer parameters than CGENN.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We are happy to answer the questions and address the concerns as follows:\n\n**1. Q1: Superposition in Clifford conjugation?**\n\nThank you. We will change the word to ‘composition’.\n\n**2. Q2: Formatting in (10),(16),(17)?**\n\nThank you. We will format definitions (10),(16),(17) so that they fit on a single line each, remove := in them, and add := in (5)-(7). \n\n**3. Q3: Restriction of $\\tilde ad$ to V?**\n\nNo, this is not a typo, we intentionally refer to V, which is the real $\\mathbb R^{p,q,r}$ or complex $\\mathbb C^{p+q,0,r}$ vector space (line 98). V is identified with the subspace $Cl^1_{p,q,r}$ of vectors. In line 235, we mean that $\\tilde{ad}^1$ is defined as the twisted adjoint representation $\\tilde ad$ (7) restricted to the entire subspace $V=Cl^1_{p,q,r}$, not just to the radical subspace $\\Lambda^1_r=Cl^1_{0,0,r}$. Namely, while $\\tilde{ad_T}$ acts on $Cl_{p,q,r}$, the restricted representation $\\tilde{ad}^1_T$ acts only on all vectors $Cl^1_{p,q,r}$ (line 237). We will add this explanation to avoid any confusion.\n\n**4. Q4: Equivariance to which of new groups is used in GLGENN?**\n\nThank you. GLGENN is equivariant w.r.t. $\\tilde\\Gamma_{p,q,r}^{\\bar1}$. We will remove k=0,2,3 in lines 641-645 and clarify in the beginning of Sect.3, that for GLGENN, we are interested in $\\tilde\\Gamma_{p,q,r}^{\\bar1}$, however other groups $\\tilde\\Gamma_{p,q,r}^{\\bar k}$, k=0,2,3, are still necessary to prove the main statements about $\\tilde\\Gamma_{p,q,r}^{\\bar1}$ (see Q7).\n\n**5. Q5: Brackets in $\\tilde{ad}_T(x)$?**\n\nIn the literature, both $\\tilde{ad}_T(x)$ or $\\tilde{ad}_T x$ are commonly used. We will put brackets in the whole paper for the sake of accuracy.\n\n**6. Q6: Behavior of test MSE when training set size increases?**\n\nThe test MSE as the training set size increases is as [follows](https://drive.google.com/file/d/1llsG1wPRNyQ4IIicZNt4Wpyxn4IJn7oZ/view?usp=sharing). We will add these results. In regression, GLGENN consistently outperforms CGENN across all training set sizes. In the convex hull task, the performance gap becomes stable, and GLGENN still is competitive. Again note that in this task, GLGENN and CGENN have 24.1K and 58.8K parameters respectively. We are positive regarding GLGENN scaling. But as we state, our goal is to reduce the risk of overfitting in case of small training data — a common scenario in natural sciences, where datasets are often manually derived from experiments. \n\n**7. Q7: Role of $\\tilde\\Gamma_{p,q,r}^{\\bar k}$, k=0,2,3?**\n\nWe share the reviewer's concern about reducing general complexity to improve readability. While equivariance w.r.t. $\\tilde\\Gamma_{p,q,r}^{\\bar1}$ is directly required for GLGENN, the other 3 groups play a crucial role in proving a key fact about this group: its elements preserve all the 4 subspaces $Cl_{p,q,r}^{\\bar k}$, k=0,1,2,3 under $\\tilde ad$ (see (28)-(29)). This implies that projections onto these 4 subspaces are $\\tilde\\Gamma_{p,q,r}^{\\bar1}$-equivariant (Theorem 3.6). We appreciate and will implement the idea of making it clearer at the beginning of Sect.3 that $\\tilde\\Gamma_{p,q,r}^{\\bar1}$ is the primary group of interest, while the others are auxiliary tools.\n\n**8. Q8: Figure of GLGENN?**\n\nFigure 1 does not introduce new information but serves as a visual representation of concepts already described in the text. We intended the following key points to be intuitively grasped by readers when looking at it: (1) GLGENN is equivariant w.r.t. orthogonal groups (we can apply an orthogonal transformation to the input or output of GLGENN and are guaranteed to get the same answer); (2) GLGENN acts in a unified manner across the subspaces $Cl_{p,q,r}^{\\bar k}$, k=0,1,2,3 defined by the grade involution $\\hat{}$ and reversion $\\tilde{}$. As a result, scalars, vectors, bivectors, etc. are formed into 4 distinct groups. Point (1) is visualized by the orange and purple arrows and the 4 main large rectangles forming a diamond shape. Point (2) is represented by the 4 smaller rectangles (yellow, green, purple, and orange) and +/- signs indicating the actions of grade involution and reversion on these groups.\n\n**9. Notation**\n\nThe notation $(Cl_{p,q,r}^{(0)\\times}\\cup Cl_{p,q,r}^{(1)\\times})\\Lambda_r^{\\times}=${$ab | a\\in Cl_{p,q,r}^{(0)\\times}\\cup Cl_{p,q,r}^{(1)\\times}, b\\in\\Lambda_r^{\\times}$} represents the product of 2 groups: the group of even or odd invertible elements (formula (3)) and the group of all invertible elements of the Grassmann subalgebra (line 104). We will add these clarifications and ensure that all other notations are properly defined.\n\n**10. Typos**\n\nWe agree with all the corrected typos, thank you.\n\n**11. Improvement of presentation**\n\nWe are very grateful to the reviewer for the comments on how to improve the presentation. We like and will surely implement the ideas of providing a running example of the main concepts and revealing more of the key ideas earlier in the paper.", "claims_and_evidence": "Let me start by saying that this is an absolute monster of a paper in terms of its conceptual difficulty, and to be able to give this review I have taken 3 full days to read/work out as many of the details of the paper as I can properly, in order to try to understand everything that it is saying and to give a fair and informed review to the authors. It seems to me that this paper follows/builds upon a sequence of previous publications (Ruhe et al, 2023; Brehmer et al, 2023; Zhdanov et al, 2024; Liu et al., 2024; Pepe et al. 2024), which I believe makes the entry into this paper very difficult if you have not read these previous works (I have not). I am convinced that for 95%+ of people reading this paper they would give up as soon as they read the Theoretical Background section (Section 2), since it is incredibly slick and technical. Despite this, having read it a few times, I felt that it was comprehensive and with a bit of work from my side I could understand what it was saying. I appreciate that the paper is already long (32 pages including Appendix) but I think to aid the reader even more I would like the authors to provide a running example (e.g on the Minkowski space $\\mathbb{R}^{1,3}$) of their constructions that appear in this section in the Appendix. From there we get into the meat of the paper in Section 3. Again this is very slick, which I understand based on the 8 page limit, but it is apparent to me that in order to even begin to understand what is going on you really need to read the Appendix first. I can't help feeling that this paper, given its technical depth, would have been better served by being written as a Journal paper instead, since in my view the logical flow would be more apparent in a journal paper, without having to make jumps between theorems (as I do in its present form). In spite of this, everything that I could understand in this section (about 80% of the section, let's say) was to me correct (and that is going line-by-line through it and the Appendix). So in my mind, on the basis of probability, I do not have doubts about the theoretical claims that were made by the authors. I would still strongly encourage them to think even more about how to present this work more cleanly - I appreciate that they have done this to a large extent already (e.g with take-away summaries after each subsection, a notation overview in the appendix etc) but there is a lot going on (many groups and algebras are introduced, some with tildas, some without, some with bars, some without) and I think at times some notation is used without it being defined anywhere. For example, I don't think, as far as I can tell, that $(Cl_{p,q,r}^{(0)\\times} \\cup Cl_{p,q,r}^{(1)\\times})\\Lambda_r^{\\times}$ is defined anywhere in the text. I would also like the authors to be consistent with whether they use $U$ or $x$ to refer to an element in $Cl_{p,q,r}$: there is a switch from $U$ to $x$ that takes place between Sections 2 and 3 that could confuse people.", "methods_and_evaluation_criteria": "For me, the methodology section (Section 4) was relatively clear, where the authors described the architecture of GLGENN. I did think the commentary about the data objects in 4.1, however, was not entirely clear - where do these data objects ($x_1, \\dots, x_l$) live? I think this could be stated directly in the text. Otherwise I was happy with what the authors wrote in this section.", "theoretical_claims": "I have already answered this above. I have checked 80% of this line by line, and where it is simply beyond my technical understanding, I have had to leave it unchecked. But I am happy with everything that I have checked.", "experimental_designs_or_analyses": "I was quite disappointed by the experiments that the authors chose, and it led me to thinking whether the theory that had been provided before was \"overkill\". Having produced their exceptionally technical theoretical results, the authors chose to demonstrate their results on tasks that are solely based on Euclidean space, with the equivariance being to a standard orthogonal group on said Euclidean space. Hence I am left wondering how their network performs on tasks that are equivariant to groups where $q > 0$. I am also not entirely convinced by the conclusions of their experiments either. In both cases the authors claim that GLGENN performs on a par with the state-of-the-art CGENN and reduces overfitting with fewer parameters.  I am not convinced that GLGENN performs on a par with CGENN. Looking at Figure 2 with Table 2, and then Figure 3 and Table 3, I see that GLGENN starts performing worse than CGENN as the number of training samples increases, and, in particular, in a divergent manner in Table 3. I therefore ask what is the behaviour of the test MSE of these two models as the number of training samples increases?", "relation_to_broader_scientific_literature": "The relation to the broader scientific literature was comprehensive, with a very nice introduction that set the work in the appropriate context.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Without repeating myself:\n\nStrengths: This is clearly a high quality technical work in the field of equivariance that demands a significant amount of the reader to understand the details. I felt that I learned a lot and in many parts enjoyed \"grokking\" the details. I also liked the mini-summaries as the work progressed, which helped me to take away the key points when I got entirely lost in the technical details (this is something I will be taking and using in my own papers!) I thought the Appendix was pretty comprehensive, and the discussion on the methodology was good too. Despite my strong doubts about whether this should be a journal paper or not, I still think there will some people at ICML who will appreciate a technical work like this, so on balance (with my score) I  recommend it (only just) for publication. However I would like the authors to take on board my comments about improving the paper's accessibility - even if it is a work in a line of works there should still be sufficient runway given to make it into enough of a standalone paper.\n\nWeaknesses: As stated above I was disappointed by the types of experiments that appeared, which to me did not show the full benefit of the theoretical construction. I also think more can be done to help facilitate the entry into this paper - whilst I appreciate the contribution is technical, the onus is still on the authors to help make things as clear as they can. I have suggested providing a running example of all key constructions (not just a summary of their meaning) in the Appendix. I would also like the message of what the authors are actually trying to show to be clearer too: for example, in the main text, lines 165-170 (2nd column) the authors say that they are looking to construct equivariant NNs with respect to $\\tilde{\\Gamma}_{p,q,r}^{\\bar{k}}$ for $k = 1$, but in Appendix B, lines 641-645, they say the NNs are equivariant to general $k$ - which one is it? I appreciate this is tough given the technical nature of the paper but again I really want the story to shine through (this doesn't always stick as you read it despite the mini summaries) - perhaps they could look to \"give up\" more of the story earlier on so that we know where you're going? An example of this appeared in Section 3.1 where the authors just suddenly introduce large families of groups (lines 184-207, second column) without telling us why/where it's leading - as readers we're already under the strain as it is!", "other_comments_or_suggestions": "Here are some comments/suggestions/typos:\n\n1) I think you should define $(Cl_{p,q,r}^{(0)\\times} \\cup Cl_{p,q,r}^{(1)\\times})\\Lambda_r^{\\times}$ somewhere before it is first used in Lemma 2.1.\n\n2) As discussed, I think you should prime the reader as to why you introduce the $Q$ families of Lie groups in lines 185 etc, second column, as well as the $Z$ centralisers (as in tell us where you're going with this much earlier than the concluding sentence.)\n\nList of (potential) typos: \n\n1. Line 320, 1st column: equivariant is spelled incorrectly\n2. Line 341, 1st column: you write $Cl$, do you not mean $Cl_{p,q,r}$?\n3. Line 362, second column: I think this should be Theorem 3.9, not Lemma 3.9\n4. Appendix F: in the title, equivariant is spelled incorrectly \n5. Appendix F, Example F1: do you not mean $T$ instead of $g$ in both (137), (138)?", "questions_for_authors": "1) Line 134, 1st column: do you really mean the word \"superposition\"? To me, superposition means add/linear combination, but here I think you mean the combination/composition of the two operations, one after the other.\n\n2) I think (10) could be formatted better (similarly (16), (17)). Also for (10) I don't think that the $:=$ that comes after $\\tilde{ad}_T(...)$ is correct, isn't it a deduction from (7)? Upon first reading I had wondered if you had meant $\\breve{ad}_T(...)$ instead.\n\n3) In line 235, second column, in the discussion, there is a switch to $V$ - just to check, is this is the same as $\\Lambda_r^1$? If so, I think you should be telling us otherwise it looks like a typo.\n\n4) (a copy from above) in the main text, lines 165-170 (2nd column) the authors say that they are looking to construct equivariant NNs with respect to $\\tilde{\\Gamma}_{p,q,r}^{\\bar{k}}$ for $k = 1$, but in Appendix B, lines 641-645, they say the NNs are equivariant to general $k$ - which one is it? If it is only $k = 1$, then I think in lines 165-170 they should tell us as they introduce everything for general $k$ and to the reader at this point it looks like a mistake (hence could be confusing as it goes along).\n\n5) In (36) do you not need brackets in there around $x$ on the RHS and around $\\langle x \\rangle_{\\bar{m}}$ on the LHS?\n\n6) (a copy from above) in both of your experiments, what is the behaviour of the test MSE of these two models as the number of training samples increases? I am not entirely convinced of the \"on par\" performance of the two models GLGENN and CGENN.\n\n7) To improve readability, is there any way to reduce the general complexity that is presented if you are only targeting equivariance for $k = 1$?\n\n8) Finally I felt that Figure 1 did nothing for me - I still don't understand what I am meant to be taking from it, even after reading the text in full multiple times. Could the authors try to explain what I should be taking from this figure (and then perhaps improve it/move it further down somewhere where it might fit better).\n\n## update after rebuttal: please see my comments in my response to the authors' rebuttal.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7xpcMOQccG", "reviewer": "Reviewer_FkP7", "summary": "The paper introduces Generalized Lipschitz Group Equivariant Neural Networks (GLGENN), a neural network architecture based on Clifford geometric algebras (GAs) that is equivariant to pseudo-orthogonal transformations of a vector space with a symmetric bilinear form. GLGENN uses a weight-sharing approach for layers calculating geometric products, resulting in a parameter-light architecture. The paper presents theoretical results on generalized Lipschitz groups, constructs the GLGENN architecture, and evaluates it empirically on toy tasks (regression of a function depending on two vectors and convex hull estimation of 16 points).\n\n## update after rebuttal\n\nPlease see comments below.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We are happy to answer the questions and address the concerns:\n\n**1. Novelty of theory**\n\nWe claim that the proposed theory of generalized Lipschitz groups (GLG) in Clifford algebras (CA) $Cl_{p,q,r}$ is new (Sect. 3). These groups are introduced in arbitrary $Cl_{p,q,r}$ for the first time in this work. All theorems in the main part are original and cannot be found in the literature. \n\n**2. How GLGENN goes beyond CGENN**\n\nGLGENN goes beyond CGENN in the following: (1) Parameter-sharing approach for CA-based NN is presented for the first time (see Point 3). (2) New layers with parameter sharing are proposed, theoretical justification is provided (Sect.4). (3) General idea is proposed that if we need orthogonal groups equivariance, then we may search for broader groups equivariance, such as new GLG, and get reasonable results (Theor.3.4 and experiments). (4) New GLG, that are interesting by themselves, are introduced and studied (Sect.3). **We will add** these details.\n\n**3. Novelty of parameter-sharing technique**\n\nTo the best of our knowledge, this work is the first to incorporate parameter-sharing techniques in CA-based NN that explicitly respect the inner structures of CA. By our approach, the weights are shared with the step size 4 in dimension of grades subspaces. This step size aligns with reversion and grade involution, the theoretical part of our work explains why this is important. \n\n**4. Limited experiments**\n\nWe thank the reviewer for the ideas and would like to make clarifications. (1) While our experiments are toy problems, they are benchmarks in the field and used in highly regarded papers on equivariant NN (e.g., Finzi et al.,2021; Ruhe et al.,2023; Liu et al.,2024). This is why we chose them as a proof of concept. (2) The convex hull task is particularly challenging in high-dimensional settings, as evidenced by the sufficient loss obtained by previous SOTA models (Fig.3). (3) Adaptation of GLGENN (or other CA-based NN) to complex domains such as molecular/protein studies or images/point clouds is a very important but extensive task that probably deserves a distinct dedicated paper (e.g. see Pepe et al. (2024) on CGENN for protein structure prediction). (4) To strengthen our experimental section, **we will add** experiments, see Point 6 of our response to Reviewer xqBf. \n\n**5. Q1: Connection of CAs to irreps of O(3)? Comparison of GLGENN and models based on irreps of O(3)?**\n\nWe do not see a direct connection between $Cl_{0,3,0}$ and the irreps of O(3). Irreps of O(3) follow the tensor approach, whereas the CA operates with multivectors. The dimensions of the subspaces of scalars, vectors, bivectors, and trivectors in CA are equal to 1,3,3,1 resp.; in contrast, for rank-2 tensors, the dimensions of the l-irreps of O(3), l=0,1,2 are 1,3,5 resp., so there is no isomorphism between them. E.g., bivectors are isomorphic to antisymmetric tensors of degree 2, which do not correspond to l=2-irrep of O(3) for rank-2 tensors.\n\nWhile equivariant models based on irreps of O(3) rely on tensor product reps, which can be complex and require additional basis reps, GLGENN bypass the need for them. This leads to 2 benefits: GLGENN directly transform data in a vector basis, avoiding the need for operating on alternative basis reps such as spherical harmonics; GLGENN involve geometrically meaningful product structure through the geometric product.\n\n**6. Q2: Typical activations to scalars?**\n\nYes, it is possible to apply standard activations to scalars and be equivariant, we explored this in our work. Our results indicate that for simple tasks, the best approach is to combine GLGENN (applied to all grades subspaces) with standard networks (e.g. MLP) with typical activations applied to scalars (0-grade subspace). E.g. in O(5,0)-Regression: (1) MLP (3 linear layers with ReLU) alone performs poorly, (2) GLGENN alone performs reasonably well but converges slower than (3) GLGENN (to all grades)+MLP (to scalars). In case of 300 train samples, we get the results [visualized here](https://drive.google.com/file/d/1VqlvFTx-SqGJo3-OSES6MMVBkuJJi1K5/view?usp=sharing) and presented in the [table](https://drive.google.com/file/d/14nemp06cX7XtjftbKHplxcPCLIk-VHcU/view?usp=sharing)). But in more complex tasks, this combination becomes less critical. In all our convex hull experiments, the best results are achieved with the GLGENN nonlinearities, w/o additional activations. The key issue is that nonlinearities applied only to specific subspaces (e.g. scalars) do not allow interactions between different grades (e.g. vectors and bivectors), isolating them; while the nonlinearities in geometric product layers mix up all grades, creating strong interactions. **We will add** these details. \n\n**7. GLGENN on large train datasets**\n\nWe are positive regarding GLGENN scalability to larger train datasets and have extended our experiments. Please see Point 6 of our response to Reviewer RrP5.", "claims_and_evidence": "The paper claims three main contributions:\n\n1. Introduction of Generalized Lipschitz Groups\n\n2. Design and Implementation of GLGENN\n\n3. Superior Performance of GLGENN compared to other equivariant architectures\n\nWith regards to claim 1, the paper describes the theory of generalized Lipschitz groups in great detail and the concept seems sound to me. However, I cannot say whether this is truly novel: I believe it is possible (and even somewhat likely) that this is an already known result in mathematics that is just not widely known in the ML community, but I cannot say for sure (I don't know the mathematical literature well enough). I recommend reaching out to mathematicians that specialise in group/ring-theory to check this.\n\nFor claim 2, the design and implementation of GLGENN is well-described, and the authors also open-source their code. With regards to the claimed novelty of the work, it is unclear to me in what respects GLGENN goes beyond the already published CGENN (Ruhe et al., 2023). Even the parameter sharing technique seems to have been proposed previously. The differences with respect to published work, in particular, which improvements are made, need to be made more clear.\n\nFinally, the claimed superior performance of GLGENN despite having significantly fewer trainable parameters seems not sufficiently well-supported by the results to me. While GLGENN outperforms MLPs, EMLP-O(5), and EMLP-SO(5) on the regression task, CGENN achieves a noticeably lower error (especially for larger training set sizes). For the convex hull estimation, performance of CGENN and GLGENN seem more similar, but the shown learning curve suggests that CGENN may start outperforming GLGENN for larger training set sizes. I think extended experiments on other (non-toy!) problems are required to support this claim.", "methods_and_evaluation_criteria": "The proposed methods (GLGENN layers) are well-motivated theoretically and make sense for the investigated problems. They are derived directly from the properties of geometric algebras and the generalized Lipschitz groups. The use of conjugation operations, adapted linear layers, geometric product layers, and normalization layers are all justified theoretically in terms of equivariance.\n\nHowever, the toy problems investigated in this work are too simplistic to allow a meaningful assessment of the proposed architecture. I recommend that the authors also apply their architecture to real world problems where the use of equivariant models is common (e.g., regression of molecular properties such as energy or forces). This is necessary to demonstrate superiority over existing models, simple toy tasks are not sufficient.", "theoretical_claims": "I did not notice any obvious mistakes in the proofs or theoretical claims while reading the paper, but I did not check carefully.", "experimental_designs_or_analyses": "The experimental design of the toy problems investigated in this work seems sound to me. However, I would recommend adding additional experiments for real world datasets (see above), and possibly also extending the investigated toy problems (e.g., significantly more points for the convex hull estimation, larger training set sizes, etc.).", "relation_to_broader_scientific_literature": "The paper clearly positions itself within the existing literature on equivariant neural networks. It primarily builds upon prior work on Clifford Group Equivariant Neural Networks (CGENN) (Ruhe et al., 2023). The paper also cites foundational work on equivariant networks (Cohen & Welling, 2016a; Weiler & Cesa, 2019; Thomas et al., 2018).\n\nAs I mentioned before, it is less clear to me in which aspects the work goes beyond existing work, and I think the authors should try to state this more clearly.", "essential_references_not_discussed": "The paper appears to cite all essential references. I'm not aware of any missing crucial publications. However, as mentioned above, I think it is possible/somewhat likely that the generalized Lipschitz groups are actually already known in the mathematical literature, and I recommend reaching out to mathematicians that specialise in group/ring-theory to check this.", "other_strengths_and_weaknesses": "**Strengths**\n\n+ The paper is mathematically rigorous, with detailed proofs and justifications provided in the appendices.\n\n+ Despite the complex mathematical concepts, the paper is generally well-written and clearly explains the proposed methods and results.\n\n+ The code is publicly available and the work is therefore easy to reproduce.\n\n**Weaknesses**\n\n- The experimental evaluation is limited to a toy regression and convex hull estimation problems. Evaluation on real world tasks (e.g., molecular property prediction, point cloud classification) and comparison to existing equivariant models on these tasks would strengthen the paper.\n\n- It is not clear in what aspects the paper goes beyond existing work, in particular, CGENN (Ruhe et al., 2023).", "other_comments_or_suggestions": "While the visualization shown in Figure 1 is nice, the labels are too small and therefore hard to read.", "questions_for_authors": "1. The case of Cl_(0,3,0) is arguably one of the most relevant for real world tasks in e.g., physics and chemistry. It seems that here, the scalars, vectors, bivectors (pseudovectors), and trivectors (pseudoscalars) of the Clifford algebra are isomorphic to irreducible representations of O(3) with degrees 0 and 1 of even and odd parity. Can the authors comment on this connection and do direct comparisons to equivariant models based on irreducible representations of O(3)?\n\n2. The authors present Linear, Geometric Product, and Normalization layers for the Clifford algebra. I am aware that the Geometric Product and Normalization layers introduce nonlinearities, but I still wonder about the possibility of using more typical activation functions. It seems like any nonlinearity could be applied to the scalar feature components, and antisymmetric functions could be applied to the trivector/pseudoscalar feature components, without affecting the equivariance of the resulting model. Have the authors considered this possibility? How does the introduction of nonlinear activation functions affect the performance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper had 3/4 reviewers recommending acceptance. Reviewers appreciated the high difficulty of the problem tackled, strong and rigorous theoretical contributions, and high quality of presentation.

Reviewers were generally disappointed by the experiments. The method was only tested on two toy datasets and improvements with respect to CGENN were limited and seem to appear only in the small data regimen. The rebuttals provided extra experiments and promised more in an updated version. 

Reviewer qfnz is the only who believes the weak experiments are reason enough for rejection, but they unfortunately did not respond to the rebuttal.

My opinion is that the strong theoretical contributions are worth being published, but I urge the authors to include the experiments listed in the rebuttal to xqBf, which should mitigate the current weak experimental section.
Decision: Accept (poster)