A Program to Build E(N)-Equivariant Steerable CNNs 
Authors: Gabriele Cesa
Leon Lang
Maurice Weiler
Venue: ICLR 2022 Poster (2022)
Keywords: equivariance
3D
geometric deep learning
isometries
steerable CNN

Abstract: Equivariance is becoming an increasingly popular design choice to build data efficient neural networks by exploiting prior knowledge about the symmetries of the problem at hand. Euclidean steerable CNNs are one of the most common classes of equivariant networks. While the constraints these architectures need to satisfy are understood, existing approaches are tailored to specific (classes of) groups. No generally applicable method that is practical for implementation has been described so far. In this work, we generalize the Wigner-Eckart theorem proposed in Lang & Weiler (2020), which characterizes general $G$-steerable kernel spaces for compact groups $G$ over their homogeneous spaces, to arbitrary $G$-spaces. This enables us to directly parameterize filters in terms of a band-limited basis on the whole space rather than on $G$'s orbits, but also to easily implement steerable CNNs equivariant to a large number of groups. To demonstrate its generality, we instantiate our method on a variety of isometry groups acting on the Euclidean space $\mathbb{R}^3$. Our framework allows us to build $E(3)$ and $SE(3)$-steerable CNNs like previous works, but also CNNs with arbitrary $G\leq O(3)$-steerable kernels. For example, we build 3D CNNs equivariant to the symmetries of platonic solids or choose $G=SO(2)$ when working with 3D data having only azimuthal symmetries. We compare these models on 3D shapes and molecular datasets, observing improved performance by matching the model's symmetries to the ones of the data.

Published as a conference paper at ICLR 2022
A PROGRAM TO BUILD E(n) EQQIVARIAAT STEERABLE CNNS
Gabriele Cesa
Leon Lang
Maurice Weiler
Qualcomm AI Research* University of Amsterdam
University Amsterdam
University of Amsterdam
1 Lalaannguaa.n.
m. weiler, ml (egaalllll.. com
gcesa@ati. munaccom.com com
ABSTRACT
Equivariance is becoming an increasingly popular design choice to build data efficient neural networks by exploiting prior knowledge about the symmetries of the problem at hand. Euclidean steerable CNNs are one of the most common classes of equivariant networks. While the constraints these architectures need to satisfy are understood, existing approaches are tailored to specific (classes of) groups. No generally applicable method that is practical for implementation has been described so far. In this work, we generalize the Vigner-Fckard theorem proposed in [ang & Weiler (2020), which characterizes general G-steerable kernel spaces for compact groups G over their homogeneous spaces, to arbitrary G-spaces. This enables us to directly parameterize filters in terms of a sand-limited basis on the whole space rather than on G's orbits, but also to easily implement steerable CNNs equivariant large number of groups. To demonstrate its generality, we instantiate our method variety of isometry groups acting on the Euclidean space IR3, Our framework allows us to build E(3) and (t(3)-steeeable CNNs like previous works, but also CNNs with arbitrary G S (t3)-rrbe kernels. For example, we build 3D CNNs equivariant to the symmetrics of platonic solids or choose G = SO) when working with 3D data having only azimuthal symmetries. We compare these models on 3D shapes and molecular datasets, observing improved performance by matching the model's symmetries the ones of the data.
INTRODUCTION
In machine learning, it is common for learning tasks to present number of symmetries. A symmetry in the data occurs, for example, when some property (e.g., the label) does not change if set of transformations applied to the data itself, e.g. translations or rotations of images. Symmetries are algebraically described by groups. If prior knowledge about the symmetries of task is available, is usually beneficial to encode them in the models used (Shawe-Tayll 2016ab. The property of such models is referred to equivariance and is obtained by introducing some equivariance constraints in the architecture (see Fq. 2q. A classical example are convolutional neural networks (CNNs), which achieve translation equivariance by constraining linear layers to be convolution operators. A wider class of equivariant models are Euclidean steerable CNNs (Cohen & Welling 60 Weiler et al. 2018al Weiler & Cesal 2019 Jenner Weiler (002), which guarantee equivariance to isometries R' x G' of Euclidean space R" i.e., to translations and a group G' of transformations, such as rotations and reflections. As proven in Weiler al. [2018a] 2021 Jenner & Weiler (2022), this requires convolutions with G-steerable (equivariant) kernels.
1989 Cohen & Welling
Our goal is developing program to parameterize with minimal requirements arbitrary G-steerable kernel spaces, with compact G. which are required to implement TR x G equivariant CNNs. Lang & Weiler (2020) provides first step in this direction by generalizing the Wigner-Eckart theorem from quantum mechanics obtain general technique to parametrize G-steerable kernel spaces over orbits of compact G. The theorem reduces the task of building steerable kernel bases to that of finding some pure representation theoretic ingredients. Since the equivariance constraint only relates points g.v In in the same orbit G. C R", a kernel can take independent values on different orbits. Fig. shows
"Qualcomm AI Research initiative of Qualcomm Technologies, Inc.

---

Published as a conference paper at ICLR 2022
examples of orbits in R² under the action of continuous planar rotations G = SO(2) (orbits are rings
of different radii) and of discrete rotations by 2. G = C1 (orbits are sets of four points). Hence, Lang
& Weiler (2200) suggest independently parameterizing orbits of G in R" (colored regions in Figl
While this guarantees a complete parameterization of G-steerable func- tions, the discretization of the filters requires them to be sufficiently band-limited" Indeed. particular care should be taken to avoid aliasing effects while discretizing filters on grid (e.g., process voxelized data). Lang & Weiler (2020) naturally support hand-limiting along each orbit of G but do not restrict the kernel across different orbits.
G.X X
x
Intuitively, this means that filter can be decomposed into a finite num- ber orbits but these should then be patched together with some smooth- ness considerations. This introduces an important requirement and sensitive design choice. First, building new G-steerable CNNs requires one to identify the orbits of G' in R", Second, the choice of orbits to con- sider affects a filter's band-width, but this relation is hard to explicitly quantify in general setting. This issue is more severe for smaller groups (particularly, finite ones), as IR" decomposes into a larger number of orbits. For example, if G S0( in Fig. Ta| band-limiting along rings is controlled by the Wigner-Fekart theorem but is not enforced along the radial component. If G C₁ in Fig. Tb each ring is partitioned into discrete orbits: larger number of orbits increases the maximum angular frequency, so aliasing is sensitive to this design choice.
(a) Orbits of G SO(
g
X
(b) Orbits of G C4
Figure I: Orbits in R Each color represents points in the same orbit under G's action.
Contributions and Outline In this work, we solve this issue and propose a general solution strategy to build arbitrary CNNs with G-steerable kernels, for any compact group G. Theorem [.1 generallizs the result of Lang & Weiler (2020) from G-homogeneous spaces to more general spaces carrying a G-action. Essentially, our theorem replaces the harmonic basis on the orbits of G in X with a G-steerable basis B for unconstrained scalar filters (Freeman & Adelson 1991) over the whole space X. Aliasing effects are thereby controlled by the choice of this initial basis. In practice, in this work we focus on Euclidean spaces X R", although the theory developed holds for any semi-direct product group X x G, and can be extended with minor changes to general homogeneous spaces X with compact stabilizers G (as argued in Remark D.15 in Lang & Weiler (2020). Since a G' -teeeerabl basis is G-steerable for any subgroup G < G', we also propose to use an initial G' O(n) Ott)-stteebble basis to support any compact group G with minimal requirements. In summary, the benefits of our method are two-fold: i) it allows direct control on band-width and aliasing via the initial basis B and ii) it completely disentangles the discretization issues from the choice of G. minimizing the requirements to implement equivariance to new groups. For example, this enables the parameterization of C2 filters without discretizing rings into a finite number of points as in Fig. 1b Algorithm| | summarizes our method and its requirements.
Since the planar case 2 was extensively studied in Weiler Cesa (2019) and since R33 has a large variety of isometry (sub)groups, in our experiments, we put particular emphasis on the 3D setting. To illustrate the generality of our method, we instantiate it on many different subgroups of O(3), the group of 3D rotations and reflections, and compare them experimentally. This results in many new 3D convolution networks, equivariant, for example, to icosahedral, axial, cylindrical or conical symme- tries. Axial symmetries tion while discrete symmetries occur in crystallography or solid state physics; see Sec. G In Sec (5.1 we discuss many design choices for 3D equivariant networks, and compare them in the experiments in Sec. [5.3] In particular, we find that designs based on different discretizations of the group and point- wise non-linearities to be beneficial when working with volumetric data. Additionally, Sec[5.2 studies the benefit of our basis in terms of equivariance after discretization and accuracy of trained models.
particularly relevant since natural scenes generally have vertical orienta-
In the Appendix, we derive some results for working with representation theoretic objects to reduce the work needed to implement new equivariant CNNs. This includes real-valued representations of compact groups, harmonic bases for induced representations, numerical irreps decomposition and representation theory of direct product groups. Finally, we implement the program described in this work as a general purpose library based on PyTorch at g github.. com QUwp//QVAA-aaaaa escnn
A band-limited signal's 's spectrum has bounded support. Nyquist-Shannon sampling theorem states that only sufficiently band-limited signals are faithfully represented by discrete samples. Otherwise, aliasing effects accur.
2

---

Published as a conference paper at ICLR 2022
2 THE THEORY OF STEERABLE CNNS AND G-STEERABLE KERNEL SPACES
In Sec. 2.1 we discuss preliminaries on steerable CNNs and the equivariance constraints on their filters. Sec. 2.2 describes our generalization of Lang & Weiler (2020) used parametrize kernels.
2.1 BUCLIDEAN STEERABLE CNNS
Equivariant neural networks are characterized by their property to commute with the action of a given symmetry group on their input, intermediate feature spaces, and output. We focus on convolutional networks on Euclidean spaces X = RR" acted on by the group (R" +) x i.e. the semidirect product of translations in (IR", +) and prigin-preserving transformations G S O(n). A general framework of equivariant convolutional models is that of steerable CNNs, which we briefly recapitulate here. For
more details, see Cohen & Welling (0166b;; Weiler ct al. (2018a 2021); Weiler & Cesal 2019).
The fundamental design choice underlying steerable CNNs is that they operate on feature vector fields, which are similar to feature maps but differ in that they are associated with well defined action of (R", +) X G. The geometrie rype of a feature field is prescribed by an orthogonal group representation p: G - Rd, which determines the transformation law of d, dd-dimensional feature vectors under the action of G. Specifically, feature field of type is map f 1 + R% which (R" +))x
transforms under (R", +) X G according to the induced representation Ind]
p:
([Tno (R) xx p] () 1) ((())) (x) == (x
EgE +).
(1)
Intuitively, the induced representation acts by moving feature vectors spatially from (I f) to and by transforming each feature fxx) according to p(g) (see Fig. 1 in Weiler & Cesa 2019)). Notable examples are scalar or tangent-vector fields, which transform according to the trivial representation p(g) or the standard representation p(g) = g. respectively. Groopp-onvvoution networks (Cohen & Welling 2016a Kondor & Trivedi 2018) are special cases with p being the regular representation of G. It is common to define the full feature fields of a network as a direct sum (concatenation)
= D fi of multiple individual feature fields fi. The full feature space transforms according the direct sum representation == + Ps. which ensures fields transform independently from each other.
Weiler et al. (2018a); Jenner & Weiler (2022) proved that, under mild assumptions, the most general linear equivariant maps between spaces of feature fields are convolutions with G-steerable kernels. If the input and output fields have types Pin G - dd xdm and Pont G - Rdoat Xdhout, G-steerable
kernels are convolution kernels K IR" - IR dont xdin satisfying the linear G-steerubility constraint
K (gg.x) = Pout
Vg G, x Z
(2)
Thus, designing steerable CNNs only requires finding a basis of the vector space of G-steerable ker- nels, which is then used to parameterize conventional Euclidean convolutions. Note that Eg. relates the kernel values on all points g.r in the orbit G.r. but leaves values on different orbits unrelated.
2.2 A WIGNER -IGNER-ECKART THEOREM FOR SUBGROUP EQUIVARIANT CONVOLUTION KERNELS
First, we need to introduce a few concepts and some notation. To keep the presentation general, in this section we will consider a general space X rather than R.. Since G is compact, we can assume all representations to be orthogonal, i.e., p(g-1) p(g) = p(g)²' To parametrize a kernel K : X + IR dous xdin satisfying the steerability constraint in Eq. 2 it is convenient to use its vectorized form x(.) = vec (K(-)) X dont The constrain becomes:
<(ggx)) = [pmm @ Pone) m(()) Vig E G,
The matrix (pin @ )ont) (g) is the Kronecker product of the two matrices (in(g) and pout (g)-
Irrep Decomposition A useful property is that there exists a set G of special representations of G, called irreducible representations or irreps, such that any representation p of G can be decomposed as a direct sum of them: p(g)= = Q" (+ter (g)) Q, where P. E G, I is an index set ranging over the elements of & (possibly, with repetition) and Q is an orthogonal matrix. The direct sum p1 (g) + P2 (g)
NNote that vee (ABC) = (CT @ A) vec (B), where vee (.) stacks matrix columns into vector. Moor precisely, irreps come in cquivalence classes and the set & contains representative for each class.
3

---

Published as a conference paper at ICLR 2022
-
Vi
de
1
0
OGJiLi)
PJ
Yi3
Kefsts Kej02 12
Figure 2: Two (unvectorized) basis kernels (j= E {1,2)) from Thm. 2.1 for G -=
of two matrices is a block diagonal matrix containing the two matrices in its blocks. w.I.o.g. we can assume Pin and Pout in the kernel constraint from Eq. to be inreps. The solutions for arbitrary Pin and Pout can be recovered from their irreps dccomposition, as explained in Weiler & Cesa (2019).
Hence, from now, we will assume irreps Pin = and Pout - PJ. where and J are indexes over G.
Tensor Product The matrix Pin @ Pout built with the Kronecker product in Eq. 3 is itself represen- tation of G, namely a tensor product representation. As such, we can decompose it in terms of G irreps: Pt @ PJ [CG")] ( ((())) p5) CG", where [j((J]) indicates the multiplicity of the irrep Pj. while CG is the change of basis matrix 4 The block diagonal structure of the direct sum allows one to distinguish blocks of rows in CG acted on by individual irreps in the direct sum. The block associated with the s-th occurrence of P3 is denoted as CG?(U) E gdyxdd)
Endomorphisms We are interested in the space of equivariant linear maps between two spaces transforming according to irreps Pj.P3 E G. If + j. this space is empty, otherwise, this is the endomorphism space of Pj. This is a vector space; assume {c/ E R4 x d, is a basi for it.
Steerable Basis To parametrize our kernels, we first need a basis for square-integrable functions on X, i.c. for (X).. A G-steerable basis for (X) is collection of orthogonal functions {Y X + R) JEG. with my a positive (possibly infinite) integer. Denoting by Y5s : X + Rd
the stack (Ym") Yjz has the defining property that Vg E G, Y5i(g=>) (;(o)\;;;))))
The vectorized constraint in Eq. 3ss equivalent to Eq. E with a scalar input field and Pin @ Pout output one. A (vectorized) kernel is a G--quivariant linear map from (x( to Idd'd,, which will be applied convolutionally over a scalar field. Hence, a basis for this kernel space is given by considering the irreps decomposition of G's action on (x) and R Rd. and, then, equivariant maps between pairs of irreps. The basis B defines an irrep decomposition of (X), CG decomposes Pin e Pout and equivariant maps are parameterized by each irrep's s endomorphism basis. This leads to:
Theorem 2.1 (Basis for G-Steerable Kernels). Let G be compact group acting on space X. Let B= (Y be G-steerable basis for L (X). Assume Pmn = pt and Pout = PJ in G. Under minor conditions, a basis for (vectorized) G-steerable kernels over X is given by K- (Fiteer Heer with:
Kijter = [CG] c. Yj(()).
(4)
The proof is in Appendix (including the case of complex valued representations); see also Fig.
Designing a new G-steerable basis B can be laborious; however, if G is a subgroup of G' (i.c., G E G"), a G'- basis can be turned into G-steerable one via group restriction:
Group Restriction Given two groups G S G', one can turn an irrep p' of G' into a representation ResE pf of G by restricting its domain to G. This representation is not irreducible and decomposes as: Re= Py = IDD' 1a ① (i'' p2) ID' where [ij'] is the multiplicity of Pj and ID' is the change basis. As earlier, the block-diagonal structure of the direct sum distinguishes blocks of rows of the matrix ID" We denote the block associated with the f-th occurrence of Pj as ID," e xdy.
Restricted Steerable Basis If = {Yyg)y by is a Gtestceralle basis for L2XX), the following set is G-steerable basis, with index i (G'i't) (see Appendix B.3
B = (Ysep (x) IDA "
(5)
'GG stands for Brebsh-Gordan since this matrix contains the so-called Clebsh-Gordan coefficients. 'In the complex case, such matrices are multiples of the identity, but this space can be larger in the real case.
4

---

Published as a conference paper at ICLR 2022
We present two examples the circle X ==S"; see Appendix B.4 for more detailed examples.
B.4
Example 1: SO@2)Steerable Kernels Fig.
3 we show a simple example the basis for a G' = (2( (Stteercee kernel space. Since X S' is orbit of G.
parameterization is equivalent to Lang Weiler (2020). An -tteersee basis on X = S is given by the circular harmonics: ()(()) cos( and ()) sin(jØ), for E N and i = 1. We con- Pt=o and output irrep Pour one of Ps=0= PJ=1 or
sider steerable kernels with input irrep Pmn
PJ=2 [ Hence, the kernels have form, respec-
tively, Kn S - IR I Ki : S + [2x1 and K₂ S + 2x Po & PJ = pJ. so we
always have j J and s = 1. Additionally, any 2-dimensional irrep P3 (j > 0) of (2) has a 2-dimensional endomorphism space spanned by d- This can be verified visually: in Fig. 3b and 3c the second column is obtained by swapping the rows of the first and by chang- ing the sign of the second row.
j=0,r=
|=,, j=1,r=
j=2,r=2
!] and d==
J-0
(b) J=1
(c) J-2
Figure Basis for G SO(2) SO(()-steerable steerable kemness from Eq. 4 Pin = Pi-o 1-dimensional. Pout = PJ for J E {0, 1,2), which are 1, and 2 dimensional. Each column is element Kjior of the basis, while each row is output channel. 1 always and, since po & ps pJ.
Example Ca-Steerable Kernels In Fig.
we show the basis for G C4 steerable kernels described by Theorem 2.1 using the Ca-steerable basis generated by Eq. from the circular harmonies basis used for G SO(2) the previous
example. Pj occurs in restricted representation Resc, S0(2) Pj' whenever j' + k Z. Indeed, basis prescribed in Eq. has infinitely many elements, but each element associated with circular harmonic of specific frequency j'. It is natural to use only band-limited finite subset of this basis: Fig. 4 shows only frequencies up to j' = Jf = 0 or j = 2 and j' = (j + 4k > 0, Pj is Res P.r contains two copies of p. and t (1,2) (see
1-dimensional but P. is 2-dimensional, j' = 2 or 4 in Fig. 4a). We use the trivial irrep / O in the input and J E (0, 1,2] in the output. p-=1 has 2-dimensional space of endomorphisms, spanned by the same basis described in the (0 example above, so r E {1,2) in this case. Note that the parametrization Fig. is continuous along the ring and does suffer from the discretization issue in Fig.
f=0,r=1 f=4.==1 /=4.r=1
/=1r=1 f=1,==
3=3.r=1
/=3.=2
/'-2,r=1 /--,1=1
t=1
[=1
t=1
(a)
J=
(c) J-)
Figure 4: Basis for G = -teerabbe kernels as in Eq Ausing stecrable basis B built from circular harmonics
using Eq. Pm Pt-u, while Pout PJ for {0, 2, which are 1, and 2 dimensional. Each column an element (r of the basis, while each row is different output channel. Since As 3 pJ = pJ. j J; for circular harmonies, 1. We require G C4 equivariance, so frequency j' -4 filter is invariant to G; analogously, the output of frequency j' 1i +4] filters (k Σ) transforms like the output of frequeney one.
IMPLEMENTATION DETAILS
Given the theoretical results described in the previous sections, we now describe the steps and inputs required to build new G-steerable kernels. See Algorithm [or summary. Theorem 2.1 relies a) the irreps Ō {e b) G-steerable basis B (Yfb) for L²( c) the decomposition pt & PJ = (T(C)T((()) jeō + (((T)) p,) CG", and d) basis { for the endomorphism space of Pj- Optionally, B can be generated via Eq. Juing e) larger group G" and its irreps G D'G' steerable basis B' = (Yyv ( hew and g) the decomposition Res Py. IID' (1) +.Ps) ID'.
The irreps of G are the main requirement: they allow us to build arbitrary representations (via
For 0 E [0, 2T), Po 0 0 the trivial irrep: if k 0, maps 0 to the 2x2 rotation matrix of i radians.
"One verify that any linear combination of these two matrices commutes with any 2 2 rotation matrix.
5

---

Published as a conference paper at ICLR 2022
Algorithm 1 Generate G-Steerable basis space X
Require: Pm oi and Pout = p.. G' -tterrable basis B = (Epebye.C. {ps}s and C = bpr)y
I: {r]r basis for endomorphism space of Pj. for all Pj
Appendix b Appendix
2: CG ([(()) + decompose(pt @ pJ)
3: for all Yar E do
4: ID" ((ii')) +- decompose(Resf) py))
Appendix
5: for all j E G: [if'] > 0. ( (c) do
6:
(areo(()) + IDŞ (ge(x))
Equation
7: Kspeeee (x) t unvec (1OGg d (x)r)(())
Theorem 2.1
8: yield Kageea
direct sum) and reduce the kernel constraint to the form in Sec. 2.2
by decomposing the input and
output representations. If B in b) is unknown, one can rely on Eq. 5 A convenient choice for G' in inle)
is a group whose irreps G', together with a steerable basis 1 as in are known. G' = (n) is always
possible X I", and B' is built via polar decomposition of RT and by combining byper-spherical harmonics with radial basis. Like Worrall et al. (2017); Weiler et al. (201 80 a). we use Gaussian radial profile: Sec. 5 this choice of G' allows easily experimenting with multiple subgroups G' < O(3), without the overhead of identifying each G's orbits or designing ad-hoc G-steerable bases. Note also that band cond-limiting is achieved by modifying B' and is independent from G. Other G orbits can also be used, combining harmonic bases (see Appendix D
with Gaussian kernel.
The irreps decomposition of pt @ PJ in c) provides the multiplicity [J([]) of each irrep Pj and the G' matrices CG? Similarly, the irreps decomposition of Resg Pyr in provides the multiplicity of each G-irrep P3 in each G'-irrep Py and the projection matrices W Knowing these decompositions a-priori for any J and j' is generally difficult, but they can be easily computed numerically; see Appendix Finally, requires a basis {cr)rr for the endomorphism space of each P3 E G. This can be computed numerically, but is often unnecessary; see Appendix C In summary, if X IR" and G' = (n), implementing new G-steerable CNNs only requires knowing the irreps G: by knowing the action of G on IR,,, one implicitly knows its embedding into (nn) and, therefore, can apply Eq.
Practical Example and Experiments In Sec. 5.3 we implement a variety of subgroups of the isometries of RR3 (see Appendix G)). To do so, we choose G' = O(3) and B' is built by combining spherical harmonies with a Gaussian radial profile. Since irreps used define the types of the feature fields, we need the irreps of SO 33), (2), S2(,). Cv. I and O. Direct product groups such as Inv (O( are built as Appendix F Since there are generally multiple subgroups G < O(3) isomorphic to the same abstract group (e.g. see O(2) in Tab. D. for each G, we explicitly define its isomorphism with an abstract group: this enables the automatic restriction of G''s irreps and the numerical computation of the IDAd matrices. The matrices CGZ(()) are also computed numerically.
Limitations Our choice of B' implies we only parameterize filters supported on a compact ball, which is slightly more restrictive than the usual parameterization of filters with support on a cube. For a fixed kernel size, an ad-hoc implementation of G-steerable filters can exploit larger initial basis, potentially leading to some performance gain (e.g. C4 allows filters supported on square rather than disk). However, if an ad-hoe implementation of G' < O(n) steerable kernels available, our method can be used to construct G < G' steerable spaces by using the G'-steerable basis rather than the O(n) (ttterrbeeealle one. Comparing the effect of different B' on the performance of G-stcerable CNN is beyond the scope of this paper, which instead focuses on building a method suitable for any G.
RELATED WORKS
Similar in spirit to our work is e2nn: library (Weiler & Cesa 2019). although limited to X = R²
and G ( G' = 0(2) recovers their solutions. Geiger et al. 2020) present library implement- ing general 3D stecrable CNNs, but limited to the choices G = SO(3) or 0()). In comparison, we currently support both 2D and 3D convolution and any compact group G < 0(3) (including discrete
*Other radial profiles are suitable as well without any substantial difference in our theory. The Gaussian profile is chosen mostly for presentation and implementation convenience.

---

Published as a conference paper at ICLR 2022
and planar subgroups acting on R and other spaces X can be potentially integrated. Finzi et al. (2021) propose a numerical method to parameterize (finite dimensional) MLPs that are equivariant to arbitrary matrix groups. Their numerical method to compute basis of equivariant linear maps resembles our irreps decomposition method; see Appendix E Bekkers (2020) and Finzi et al. (2020) implement group convolution Lie groups using finite number of samples from the continuous
group. This is similar to our (3(3) or O(3) architectures with pointwise non- oonlinearitiee; see Sec. 5.1 However, since our features and filters are explicitly parametrized on band-limited space, we can adapt the sampling set to control the equivariance error caused by this approximation.
Aronsson (2021) previously discussed bandlimited convolution operators in equivariant CNNs. Jenner
& Weiler (2022) generalize Euclidean steerable CNNs to partial differential operators. (Cohen et al.
2019b Kondor & Trivedi 2018: [eekerrs 2020) define steerable CNNs on homogeneous spaces.
G-steerable kemels are necessary to implement gauge equivariant CNNs on general manifolds (Weiler
al. 2021, Cohen et al., 2019a Kicanaoglu et al., 2019 Haan et al.) (02T). D. Li et al. (2021) use transformed filters to parameterize steerable kernels. Previously, Mallat (2012); Oyallon & Mallat
(2013);
[Sifre & Mallat (013) described similar architectures based on scattering. Brandstetter et al.
(2021
use non-linear parameterization of steerable convolution in geometric graph.
INSTANTIATION OF THE METHOD AND EXPERIMENTS
We demonstrate the generality of our method by instantiating steerable CNNs for different groups G. We will mostly focus on R and its isometries; Appendix G briefly summarizes the subgroups used. Sec. discusses existing and new design choices for 3D) steerable CNNs. We then compare these designs in Sec. 5.3 Sec. 5.2 numerically compares our steerable basis with alternative bases.
5.1 DESIGN CHOICES OF 3D STEERABLE CNNS
A G-steerable CNN design involves a choice of feature field types, i.e. G representations acting on the channels of the features, see Sec. 2.1 The chosen types constrain the non-linear operations permitted.
Many works in the literature focus designs which achieve perfect SO(3) equivariance (Weiler ctt al. 2018b Thomas et al.) 2018 (Anderson et al. (219). To ensure this, a common choice is the use of irreps feature types and gated non-linearities. This operation combines feature field fx) Rd of type P with a gate fz) E R as fp(2) + (fo(z)) ff)))) fz)), where a is a sigmoid function and fo is feature field of trivial type. This non- non-liiear is equivariant to any G but Weiler & Cesal (2019) found them to perform worse than point-wise non-linearities like those used in GCNNs. Another popular choice using the tensor product of two feature fields as a non-linear operation (Kondor et al. 2018). , i.c. (f (z), (x)) + fp. (x) @ fp, (z). However, this non non- non-line is quadratic and turns the network into a polynomial function of the input, which can lead to training difficulties (Anderson et al., 2019). Additionally, computing the tensor product @ has computational and memory cost O(dp. dpp); ); It is common to project the output features to a smaller feature field through a learnable linear map. A different approach is used in Worrall & Brostow (2018); Winkels & Cohen (218)), which use the octahedral group O on voxelized data. The finiteness of the group allows for a simple GCNN (Cohen) & Welling 2016a) architecture, corresponding to the choice of the regular representation Preg the feature type and standard pointwise non oon-lineassties (e.g. ReLU). The representation Preg of a group G acts on feature vectors in TR [G] by permutations and a vector in this space can be interpreted as a scalar function over G: see Weiler & Cesa (2019) for an intuitive description.
Because (3(3) (and 0(3)) has only a limited number of discrete subgroups, we also consider discretizations without group structure. We interpret feature vector f(x:) E R& the coefficients parameterizing a band-limited function over G in Fourier space; see Appendix D To apply a pointwise non-linearity o R + E (e.g. ELU) on f(x), we i) sample the parameterized function on a finite set G C G. ii) apply a and, then, lii) recover the coefficients f'(x) of the output via a Fourier transform. If a smooth, the output signal is still approximatively band-limited but can contain higher frequencies than the input f()). This implies that a larger set 9 is required to reconstruct f'(x) than for f(x). If a fixed budget of channels (i.e. number of samples [G) is available for a non-linear module, the input field f(x) must be strongly bandlimited. While this can reduce the expressiveness of the model's 's features, it also implies that linear layers are computationally cheaper as they operate on smaller number of channels. A similar result was leveraged in Cheng al. (2019) for G SO(2). This regular non- -ineerrr was also used in Kicanaoglu et al. (2019) for G S0(2). The same approach can be generalized to quotient fields, í.e. feature fields which parameterize functions over
7

---

Published as a conference paper at ICLR 2022
Table 1: Rotated ModelNet10 (3(3) symmetry). indicates wider models to fix the computational cost.
Description
Accuracy
{c)
Conventional CNN
82.5 1 1.4
S0(2)
Axial Symmetry
86.9 1 1.9
SO(2) x F = (
Dihedral Symmetry
87.5 87.5 1 0.7
SO(2) M = 0(2)
Conical Symmetry
88.5 0.8
Inv SO(2)
Cylindrical Symmetry
86.8
Inv x S0(2) x
Full Cylindrical Symmetry
87.0
O
Octahedral Symmetry [Winkels Cohen 2018
89.7
Icosahedral Symmetry
90.0 10.00..0
Icosahedral Symmetry (finite orbits basis)
$8.2
S0(3)
Chiral (Tensor product) (Anderson et al
2019
16.314
80(3)
Chiral (Gated non-linearity) Werler
et Weelerr
2018b
88,8 1.2
80(3)
Chiral (Regular, [g] = 96)
89.1
S0(3)
Chiral (Regular, [ = 192)*
89.4 1.4
80(3)
Chiral (Quotient S SO(3) |xl| 30)
89.5 1.0
0(3)
Achiral (Regular, [g] = 120)
89.2 0.6
((3)
Achiral (Regular, [9] 144)*
89.4 1 0.7
((3)
Achiral (Quotient Inv x 0(3) |X| 60)
88.6 1 0.9
quotient space X G/ (e.g. 8 S0(3)/S0(2)) point-wise non-linearities and their computational benefit.
(2))). See Appendix H.2 for more details
5.2
NUMERICAL COMPARISON OF STEERABLE BASES
In this section, we compare basis with one built according to Lang & Weiler (2020). We consider the example of Icosahedral steerable 3D convolution. Our basis is composed by a band-limited set of spherical harmonies per spherical shell, combined as in Theorem 2.1 The
two baselines use respectively the center of the 20 faces of the dodecahedron and the 12 faces
of the icosahedron embedded in each shell orbits and parameterize the kernel along each
orbit directly using (Lang & Weiler 2020). Fig. 5
shows the histograms of the equivariance errors of the three bases with respect to the Icosahedral group; each point represents an element the ba- sis and the errors averaged all transformations; see Appendix [.I Our basis shows significantly higher stability. Finally, we study the effect of these bases on a model's performance. In Tab. [ using
Dodecshedron (12)
Icosahedran (20)
Hormaniss (Durs)
25
20
15.
10.
compare two T-equivariant models: our harmonic basis and one using the 20 faces of the Icosahedron as orbit. Our anti-aliased basis leads to a significant improvement in accuracy.
28
56
84
112
Relative Error (76)
28 56
04 112
56 112
Figure Histograms of relative equivariance errors of three different Icosahedral I-steerable bases. The first two parameterize two different orbits of while the third uses our basis based spherical harmonies.
5.3 EXPERIMENTS
To emphasize our method limited to 3 we include simple experiment with 2D images. the rest of the section, we compare different model designs on two volumetric datasets: ModelNet10 (Wu al. 2015) (and rotated version of it) and LBA (Townshend al.. 2020). For each model, we run simple search over hyperparameters and minor variants of the designs described in Sec. 5. using validation accuracy, to ensure a fair comparison. Unless specified, within a task, models approximatively share the same width. In particular, in all tasks, the models which more closely match the symmetries of the data perform the best. See Appendix H for more details on the experiments. The cquivariance groups used (column G in the tables) described in Appendix G
Rotated MNIST As a simple 2D experiment, we train a conventional CNN and a Cs equivariant CNN rotated MNIST. The steerable kernel bases similar those in Weiler & Cesa (2019) and we use regular field types with pointwise ELU. The conventional model achieves 96.3 = 0.1 test accuracy while the C3 equivariant
96.7 + 0.4.
Rotated ModelNet10 We generate a voxelized version of ModelNet10 with resolution 33px by sampling 3 random S3(3) rotations for each object. During training, we rotations from O for
8

---

Published as a conference paper at ICLR 2022
augmentation. In Tab. 1 compare different equivariance groups and different equivariant designs. includes our implementation of some relevant related works; see Sec. [5.1 We first highlight the large margin between all equivariant networks and the conventional CNN. SO(3) and O(3) models achieve better performance than the other continuous subgroups. The best results are achieved by using the discrete groups and O or the SO(3) model with spherical quotient feature types. Recall also that the models with quotient or regular non-linearities are computationally cheaper; see Sec 5.1
ModelNet10 To experiment with data with only axial and mirror symmetries, we gen- erate a version of ModelNet10 by sampling
Table 2: ModelNet10 (0(2) symmetry)
Description
Accuracy
the random rotations only around the Z axis. T rotations and reflection augmenta-
{e}
Conventional CNN
91.2 0.5
SO(2)
Azimuthal Symmetry
91.9 4 0.8
tion in the XY plane is used for training but no rotation averaging is used for test- ing. The final accuracies are in Tab. 2 The models equivariant to 3D rotations are out- performed by the SO(2) and O(2) models, which better match the data symmetries.
SO(3) Chiral (Regular, |g] = 72)
89.8 + 0.6
(
Full Azimuthal Symmetry
92.3 + 0.4
O(3) Achiral (Regular, |0|| 120)
89.9 : 1.0
C2 Klein Group (dihedral symm symmetry)
91.0 0.6
VOXNet (Maturana & Scherer 2015)
92.0
C2 F Klein Group WWorrall & Brostow 2018
94.2
Ligand Binding Affinity (LBA) regression We also evaluate models regression task using the LBA dataset (Twwsheedd et al., 2020), containing protein-ligand complexes from the PDBBind database (Wang al.) 2004 Liu et al. 2014). In Townshend et al. (2020), an -equivaanva model with tensor product activations performed the best, despite its cost prevented training on the full dataset. We propose a volumetric S3(3) network with a computational cost equivalent conventional CNN by leveraging the benefit of regular non-linearities. Tab. 3 reports our results.
Discussion We again emphasize that the purpose of our experiments is mostly demonstrating the generality and flexibility of our program, but we do not necessarily envision applications of all subgroups G < ((3) considered. Still, some subgroups are practically relevant; indeed, models with azimuthal symmetry perform the best in Table where the data lacks full rotational symmetry. Here, enforcing S3(3) equivariance results in over-conttrained models which are not sufficiently expressive and are outperformed even by the conventional CNN. Overall, we observe models whose symmetry matches the data tend to perform the best. Some discrete subgroups, e.g. the platonic ones, constitute special exception: they approximate the full rotational symmetry but are less restrictive than a fully rotation equivariant model, sometimes better matching the approximate symmetry of voxelized data. Finally, we only found minor differences among the various SO(3) models, although regular and quotient non-linearities stood out for their reduced computational cost (see also Appendix H.7
6 CONCLUSIONS
In this work, gave general characterization of G-steerable kernel spaces over arbitrary space X and proposed general strategy to automatically parameterize them based only on G's irreps. This enabled to implement steerable CNNs equivariant to variety of new groups < 0(3), which compare in an exploratory study in our experiments. Finally, we believe that Theorem 2. suggests a new theoretical perspective on the construction of equivariant CNNs, reducing the problem to that of devising a suitable starting basis B, regardless of the choice of feature types in the model.
Table 3: Ligand Binding Affinity dataset. indicates wider models to fix the computational cost.
Description
RMSD
Pearson
Spearman
(e}
Conventional CNN
1,419 + 0.017
0.575 + 0.022
0.569 = 0.021
O
Octahedral Symmetry
1,417 + 0.032
0.589 + 0.010
0.581 = 0.011
1
Icosahedral Symmetry
1,432 + 0.020
0.569 = 0.023
0.559 = 0.023
SO(3)
Chiral (Regular, [ 72)
1.397 0.039
0.580 = 0.021
0.573 0.022
SO(3)
Chiral (Regular. g 60)*
1.380 + 0.033
0.588 + 0.015
0.578 = 0.018
S0(3)
Chiral (Quotient 8², | = 24)
1.405 + 0.028
0.582 + 0.018
0.576
0.016
SO(3)
Chiral (Quotient S | 30)*
1.385 + 0.032
0.587 + 0.019
0.576 0.019
(c)
3DCNN (Townshend al.
(0200)
1.520
0.558
0.556
(e)
GGaph-NN ( Townshend et al. 2020
1.936
0.581
0.647
80(3)
E-GNN (Townshend et al. 2020
1.429
0.541
0.532

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
Gabriele would like thank Arash Behboodi for fruitful discussions on the approximation of point-wise non-linearities with finite samples.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we include a dedicated section in Appendix to describe our experiments in more details. Moreover, we implemented all the methods described in this work in a Python library which can be found at giggtthu... com/ LUUVA-aabeeeeeeee to run our experiments.
We also plan to publish the code used
POSSIBLE NEGATIVE SOCIETAL IMPACTS
Since this work is very foundational, many different negative societal impacts are conceivable. In particular, any misuse which is eased by improved computer vision systems can profit from this and similar work. Of course, we cannot discuss them all.
Nevertheless, we want to highlight one specific misuse which seems likely to occur for the field of equivariant deep learning the future namely, adoption lethal autonomous weapons systems (LAWS). We can easily think of drones using spherical vision which would profit from equivariant CNNs. In particular, since they have a fixed vertical orientation, but arbitrary horizontal orientation, they could profit from spherical CNNs using azymuthal symmetries.
is important to realize that the mitigation of such misuses cannot happen at the level of individual research projects. Instead, what is needed is global effort to mitigate the misuse of AI systems for LAWS in general. Fortunately, the broader Al community agrees that Al should not be used in LAWS, and that international regulations necessary prevent their use in the future aut 2015)) We hope that efforts in this direction are continued.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Autonomous Weapons: An Open Letter from Al & Robotics Researchers. http:://huureeoftieooo//gppp letter-attonomous-wweeppssss 2015.
Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. Advances in neural information processing systems, 32, 2019.
Jimmy Aronsson. Homogeneous vector bundles and G-equivariant convolutional neural networks. PhD thesis, Chalmers Tekmiska Hogskola (Sweden), 2021.
Erik Bekkers. B-spline CNNs on lie groups. In International Conference on Learning Representa- tions (ICLR), 2020.
J. Michael Boardman. Real and complex representations, 2007. htRt::tt::///me jhu edu /
jmb/note/rschur. pdf
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik Bekkers, and Max Welling. Ge- ometric and physical quantities improve (3) equivariant message passing. arXiv preprint arXiv:2110.02905. 2021.
T. Bröcker and T. Dieck. Representations of Compact Lie Groups. Graduate Texts in Mathematics. Springer Berlin Heidelberg, 2003. ISBN 9783540136781. URL htttps:/bbook google
Xiuyuan Cheng, Qiang Qiu, Robert Calderbank, and Guillermo Sapiro. RotDCF: Decomposition of convolutional filters for rotation-cquivariant deep networks. In International Conference on Learning Representations (ICLR), 2019.
Taco Cohen and Max Welling, Group Equivariant Convolutional Networks. In Proceedings of The 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research, 2016a.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling, Gauge equivariant convolutional networks and the icosahedral cnn. In International Conference on Machine Learning, pp. 1321 1330. PMLR, 2019a.
Taco S. Cohen and Max Welling. Stecrable CNNs. In ICLR 2017, November 2016b.
Taco S. Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant CNNs homogeneous spaces. Advances in neural information processing systems, 32, 2019b.
Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to Lie groups arbitrary continuous data. pp. 3165-3176, 2020.
Mare Finzi, Max Welling, and Andrew Gordon Wilson. A practical method for constructing equivari- ant multilayer perceptrons for arbitrary matrix groups. In International Conference on Machine Learning, pp. 3318-3328. PMLR, 2021.
William Freeman and Edward H. Adelson. The design and use of steerable filters. IEEE Transactions on Pattern Analysis & Machine Intelligence, (9)8891-906, 1991.
Mario Geiger, Tess Smidt, Alby M., Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kos- tiantyn Lapchevskyi. Maurice Weiler, Michat Tyszkiewicz, Simon Batzner, Martin Uhrin, Jes Frellsen, Nuri Jung. Sophia Sanborn, Josh Rackers, and Michael Bailey. Euclidean neural networks: e3nn, 2020. URL htttttt://ggihubbb...tt com/ e3nn/ e3nn
Pim De Haan, Maurice Weiler, Taco Cohen, and Max Welling. Gauge equivariant mesh (cnn)s: Anisotropic convolutions geometric graphs. In International Conference Learning Represen- tations, 2021.
Erik Jenner and Maurice Weiler. Steerable partial differential operators for equivariant neural networks. In International Conference Learning Representations, 2022.
Berkay Kicanaoglu, P. D. Haan, and Taco Cohen. Gauge equivariant spherical cnns. 2019.
11

---

Published as a conference paper at ICLR 2022
Anthony Knapp. Lie Groups Beyond Introduction, Second edition, volume 140. 01 2002. ISBN 978-1-4757-2455-4 doi: 10.10079978-1475-24555-..
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning (ICML), 2018.
Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch-gordan nets: a fully fourier space spherical convolutional neural network. Advances in Neural Information Processing Systems, 31, 2018.
Leon Lang and Maurice Weiler. A Wigner-Eckart theorem for group equivariant convolution kernels. In International Conference on Learning Representations, 2020.
Bo Li, Qili Wang, and Gim Hee Lee. Filtra: Rethinking steerable cnn by filter transform. In International Conference on Machine Learning, pp. 6515-6522. PMLR, 2021.
Zhihai Liu, Yan Li, Li Han, Jie Li, Jie Liu, Zhixiong Zhao, Wei Nie, Yuchen Liu, and Renxiao Wang. PDB-wide collection of binding data: current status of the PDBbind database. Bioinformatics. 31(3):405 10 2014. ISSN 1367-4803. doi: 1t..////oiooommaiis/bbuueee URL https
//doi . orggioogg1000 /10 wttp:////woooomaaii.oooooo
Stéphane Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65
(10):1331-1398, 2012.
Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ International Conference on Intellientt Robots and Systems (IROS), pp. 922-928, 2015, doi: 10.1 00.100//R
Edouard Oyallon and Stéphane Mallat. Deep roto-translation scattering for object classification. In Conference Computer Vision and Pattern Recognition (CVPR), 2015.
Shawe- Taylor. Building symmetries into feedforward networks. In 1989 First IEE International Conference Artificial Neural Networks, (Conf. Publ. No. 313), pp. 158-162, Oct 1989.
Laurent Sifre and Stephane Mallat. Rotation, scaling and deformation invariant scattering for texture discrimination. Conference Computer Vision and Pattern Recognition (CVPR), 2013.
Keith Taylor and Eberhard Kaniutrh. Induced Representations of Locally Compact Groups. 2013. doi: 10.1017/CB09781
1 39045391.
Nathaniel Thomas, Tess Smidt, Steven M. Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation-equiivrraan neural networks for 3d point clouds. arXiv preprint arXiv: 082199 2018.
Raphael JL Townshend, Martin Vögele, Patricia Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Brandon Anderson, Stephan Eismann, Risi Kondor, et al. Atom3d: Tasks on molecules in three dimensions. 2020.
Renxiao Wang, Xueliang Fang, Yipin Lu, and Shaomeng Wang. The pdbbind database: collection of binding affinities for protein-ligand complexes with known three-dimensional structures. Journal of medicinal chemistry, 47(1)) 2980, June 2004. ISSN 0022-2623. doi: 10. 1011022/j0000 1000000000000101100011
URL wttptttpt/coooo
Maurice Weiler and Gabriele Cesa. General ((u)-Etquva Steerable CNNs. In Conference Neural Information Processing Systems (NeurIPS), 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S. Cohen. 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. In Conference on Neural Information Processing Systems (NeurIPS), 2018a.
Maurice Weiler, Fred A. Hamprecht, and Martin Storath. Learning steerable filters for rotation equivariant CNNs. In Conference Computer Vision and Pattern Recognition (CVPR), 2018b.
12

---

Published as a conference paper at ICLR 2022
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate Independent Convolu- tional Networks Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds. arXiv preprint 0tt1::200600000200 2021.
Marysia Winkels and Taco S. Cohen. 3D G-CNNs for pulmonary nodule detection, In Conference on Medical Imaging with Deep Learning (MIDL), 2018.
Daniel E. Worrall and Gabriel J. Brostow. Cubenet: Equivariance to 3D rotation and translation. In European Conference on Computer Vision (ECCV), 2018.
Daniel E. Worrall, Stephan Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow. Harmonic networks: Deep translation and rotation equivariance. In Conference Computer Vision and Pattern Recognition (CVPR), 2017.
Zhirong Wu, Shuran Song. Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912-1920, 2015. doi: 10.1 109/CR... 2015.7298801.
13

---

Published as a conference paper at ICLR 2022
APPENDIX
Preliminaries on the Representation Theory of Compact Groups
15
A.I Basics of (Unitary and Irreducible) Representations
15
A.2
Homogeneous Spaces, (quuare-Interralee Functions, and the Peter- Weyl Theorem
16
A.3
Constructing Unitary Representations from Others and their Relations
18
Compact Subgroups and Restricted Representations
19
Generalized Wigner-Eckart Theorem for Steerable Kernels
20
B.
Basis Basis-Indepen Independent Formulation of the Group Restr-eettii Wigner- Eckart Theorem
20
Steerable Kernel Bases
22
B.3
A Wigner-Eckart Theorem for Group- Restricted Steerable Kernels
25
B.4
Extended Examples Steerable Kernel Bases
25
Real Valued Representations
29
Preliminaries on the Structure of Endg.R(VV))
C.2
Orthogonality and Alignment of Matrix Coefficients
................................
32
A Convenient Choice of Basis for Real Irreps
The Computation of the Induced Representation of Compact Groups
38
D.I
Useful Definitions, Assumptions and Notations
39
D.2
Induced Representations
D.3
Harmonic Analysis of Induced Representations
42
D.4
Reconstructing the irrep decomposition: the complex case
44
D.5
Reconstructing the irrep decomposition: the real case
45
Numerical Irreps Decomposition of Real Representations
50
Real Irreducible Representations of the Direct Product of two Compact Groups
53
3D symmetries, Azimuthal symmetries and their relevance
55
H
Further Details on Network Designs and Experiments
56
H.1
Estimating the Equivariance Error of Steerable Bases
56
H.2
Regular and Quotient Nin-Liines
56
3
Architecture Details
57
H.4
MNIST Experiments
60
H.5
ModelNet10 Experiments
60
H.6
Auto-Encoder Experiments ModelNet10
61
H.7
Computational cost and Inference time of the models
61
H.8
LBA Experiments
62
14

---

Published as a conference paper at ICLR 2022
In the Appendix, we provide more detailed description and the proof of the theoretical results mentioned in the main paper. We first give a brief overview of the representation theory of compact
groups and introduce the notation will use throughout the next chapters in Section A In Section B we state more precisely and prove the main result from Sec. [.2 which we used to parametrize the steerable filters. In particular, state both complex and a real version of Theorem 2.1 In section B.4, we discuss two examples of kernel bases. Sections C D D and F theoretical results and computational methods which can ease the practical implementation of steerable CNNs, but are not necessary for understanding our main results.
provide additional
Because the implementation of steerable CNNs necessarily using real numbers, its design should rely on the real representation theory of the group considered. In Section C we review the theory of real irreducible representations and derive some useful results to deal with these representations in the implementation. Notably, we show that, for compact group G, any real irrep of G can always be expressed a particular basis which allows one recover a harmonic basis for square-integrable functions over G from the matrix coefficients of the irreps. The results shown in Section C will become useful in Section
and F
Regular, quotient and induced features types encode scalar or vector functions over a homogeneous space of G in terms of a harmonic basis. Moreover, harmonic bases provide a simple way to generate the G-stecrable bases required to define the kernel basis in Sec. 2.2 by leveraging the fact that orbits of G in X are isomorphic to homogeneous spaces, the harmonic bases for the orbits of G in X can be combined with Gaussian kernel to generate G-steerable bases over X. In Section D we derive a method to compute the (either real or complex) harmonic basis for any homogencous space of G from the harmonic basis of functions over G. The method described in Section D turns out to be more general and also supports vectors fields over homogeneous spaces, i.e. general induced representations of G.
Since the kernel parameterization in Sec. 2.2 relies on the irreps decomposition of the input and output representations, of the tensor product of irreps (to compute the CG) and of the restricted irreps of G' (to compute the ID coefficients). the ability to decompose an arbitrary representation is essential to apply our method in a fully general setting. Therefore, in Section
coefficients
D we describe numerical method to perform this decomposition for real representations.
As explained in Sec. 3 the main (and, often, only) requirement to apply our method to a new group G is the knowledge of its irreps G. A simple operation which can be used to generate new groups is the direct product of two groups. Examples of direct product groups include many subgroups of (3) that we used in Sec. 5 In Section F we describe a method to derive the real irrep of direct product group G A B from the real irreps of the two subgroups A and B.
Section H includes additional details on the experimental results reported in Sec. 5 In particular, we describe in more details the different equivariant designs, the architectures used, the datasets and the training and testing configurations.
A PRELIMINARIES ON THE REPRESENTATION THEORY OF COMPACT GROUPS
In this section, we collect many notions and concepts which we need throughout the appendix for the development of the theory. This section assumes some familiarity with the representation theory of compact groups and is meant as reminder. For readers with no background in representation theory, we can recommend the appendix of Lang & Weiler (2020).
We will use K denote any of the two fields or C, i.e., we develop the theory in such a way that covers both real and complex representations. Whenever we deviate from this assumption, we will say so explicitly.
Throughout, let furthermore G be any compact group
A.1 BASICS OF (UNITARY AND IRREDUCIBLE) REPRESENTATIONS
Linear representations are (in certain sense continuous) group homomorphisms p: G GL(V) for some topological K-vector space V. In our applications, always carries a scalar product (-1-) which induces its topology, and which we always assume to be conjugate linear in the first component,
15

---

Published as a conference paper at ICLR 2022
and linear in the second component. The pair (V, (-I-) is assumed to be Hilbert space, meaning that the scalar product makes a complete metric space. Note that, as is typical in a physics context, we freely make use of the Bra-Ket convention, and thus for example view (ol linear functional which maps Iw) == to to (o w). By the Riesz representation theorem, there are no other continuous linear functionals.
Unitary representations are linear representations p G + U(V) S GL(V). where U(V) is the unitary group of V. This is the group of unitary trunsformations on V, i.e., linear functions on V which preserve the scalar product, and thus distances. Whenever we explicitly assume K = R, we will write O(V) instead of UV). which is the group ortho transformations on V. and the representations are then called orthogonal instead of umitary
We will denote the set of isomorphism classes of all irreducible unitary representations (irreps) of G as G. Thereby, irreducible representations are representations which cannot be decomposed nontrivially into subrepresentations. We usually write indices j. 1, J E G to indicate such isomorphism classes, and then, e.g., Pj G G- U(V,) denote representative of such isomorphism class. We set dj := dim Vj as the dimension of the representation spaceDof Vz. Whenever we consider only one irrep, we may also, by abuse of notation, write p G to indicate (a representative of isomorphism class) of an irrep of G.
We now collect definitions which we use throughout.
Definition A.1 (Homomorppisns/nneerwweerr, Let pv pv: G GL(V) and pw G GL(W) be two linear representations. The space of homomorphisms, also called intertwiners, HomG.K <(V, W), consists of all linear, continuous functions f V + W which commute with the two representations, i.e,, So py (g) Pw (g) o I for all E G. We write for this space also Homg, & (Pv, Pw). ) depending on whether we want emphasize the representation spaces or the representations.
Definition A.2 (Equivalences and Isomorphisms), Let pv : G GL(V) and pw : G + GL(W) be two linear representations. An intertwiner f V - W is called an equivalence if there is an intertwiner g W - V such that g o = idy and f o gg- = idw. PV and pw, or also V and W, are then said to be equivalent.
If pv and pw are even unitary representations, and f and additionally are unitary transformations, i.e. preserve the scalar product and thus distances, then V and W or pV and pw are called isomorphic, and f and g isomorphisms. We write this as pV _ pw or V W, depending on whether we want to emphasize the representations or the representation spaces.
Definition A.3 (Endomorphismss, Endomorphism Basis). Let p: G - GL(() be linear represen- tation. The space of endomorphisms Endg, (V) consists of all linear, continuous maps e -> V that commute with p. i.e., eo p(g) p(g) o for all g E G.
For == Pj being an irrep, this space is finite-dimensional and Das a basis (the endomorphism basis) Fg (c) with F.y dim (Endg. (V))).
Note that Endg,
Homg. Vz).
By Schur's Lemma, the endomorphism spaces of complex irreps are always 1eeiyss--daie 1-dimensional, generated by the identity. For real representations, they can be 1, 2. or 4-dimensional, and the irrep is then correspondingly called of real type, complex type, or quaternionic type. See Supplementary C for more details on real representations.
A.2 HOMOGENROUS SPACES, SQUARE-INTEGRABLL FUNCTIONS, AND THE PETER-WEYL. THEOREM
Recall that homogeneous space of the compact group G is a topological (Hausdorff) space X together with a transitive continuous action G x X X, (g, + g.v gz.
As is commonly known, G carries an inversion-invarian and left-right invariant so-called Haar- measure μ, which can furthermore be assumed to be normalized: ((G) = 1. This Haar measure also
But note that in the general case of unspecified field K, we always speak of unitary representations, by which we just mean "orthogonal" in the case K = R
16

---

Published as a conference paper at ICLR 2022
induces corresponding G-action invariant measure X, which we also call faar-measure and assume to normalized: v(X) = 1.
The normalization makes both ftar-measuree the unique measures which defined on the whole Borel Sigma algebras of these spaces and satisfying the mentioned invariance properties. To reduce clutter, will mostly not mention the used measure in integral-formular but view it understood that all integrals are with respect to these unique Haar measures.
Fix from now homogeneous space X of G.
Definition A.4 (TTh Space of) Integralt Functions). A square- integrable function f X K is a measurable function with the property that the square of its absolute value integrable: [x (z) dx < 30.
We denote the space of all such square-integrable functions by (X)).. It carries scalar product given by (J g) = Jx J(x) (x)))(())e da. making it Hilbert space. also carries unitary representation of G given by A G G U(LE (x) f(g 2x).
Definition A.5 (Regular Representation). In the case that X G is the group itself, we call (G), together with the action (}(g) == ffg f(g="g") ''), the regular representation of
Theorem A.6 (Peter-Weyl Theorem). There decomposition
(x)
Vj+ jE 1=1
into irreducible subrepresentations. Thereby, Vjs is isomorphic to the representative irreducible representation Vj and generated by orthonormal harmonic basis functions Y X IK. We have my S. dj == dim Vj. Df C and X = G. then m = dj-
Note that the hat () denotes a topological closure. is a technicality which the reader can mostly ignore without problems.
Proposition A.7 (Peter-Way) Weyl Theorem: An Explicit Basis for Complex- Valued Inteerrle Functions on a Compact Group). In the complex field K. C, the orthonormal harmonic basis for (G) is given explicitly by:
(g) (e)eel))) = Vd, (9) Vd, (0;(9
In other words, the matrix coefficients of the complex irreps of form an orthogonal basis for (G). Definition A.8 (Harmonic Projections). We denote the orthogonal projection from L3 (X) ) to the i'th copy of V, by P.: (X) + Vj. On the harmonic basis, it is given by
Pjs y = 5 lim).
which just follows from the fact that the harmonic basis functions form an orthonormal basis of T2 (X). which is then mapped to the corresponding orthonormal basis of Vj. the representation space of the representative irrep Pj G U(V)).
One commonly known corollary of the Peter- Peter-Weyl theorem is the following. We make use of it the definitions of the Clebsch-Gordan coefficients and irep-deccmpossitinn coefficients which we define in the next two subsections:
Corollary 1. Any unitary representation of G, even infinite dimensional, can be decomposed into (topological closure of a) direct sum of irreps of G with a unitary isomorphism, i.e.: for all unitary representations > U(V) of G, there exists a unitary transformation M - V,, where
P: G U(V)) are unitary irreducible representations such that for all g E G the following identity holds:
(Dalco) M.
π T(g)
-
iE!
I is set indexing irreps in G, and the same irrep may occur multiple times. If such an M exists, we Die
also write V
17

---

Published as a conference paper at ICLR 2022
That M is unitary transformation means, as explained earlier, that it preserves the scalar product and thus distances. Consequently, one has M = MM, i..., the inverse is equal to the adjoin To) In the unite-dimensional case, and working with matrices, one has that M is given by the conjugate transpose of M, and if we additionally consider real-valued representations, then Mt = MM is just the transpose.
A.3 CONSTRUCTING UNITARY REPRESENTATIONS FROM OTHERS AND THEIR RELATIONS
Now we come to tensor products and Clebsch- Gordan coefficients. Recall that given two finite- dimensional unitary representations PV G + U(V) and Pw G - U(W). one can define unitary representation PV @ pw: G + U(V so W) called tensor product given by (pv @ pw) (g) = pv (g) & (w (g): @ w + [g)( (w) @ [gw(g)] (w:). Thereby, V < W is the tensor product the vector spaces V and W. If V and W have bases {UI,.... Un and {wi...., Wm. respectively, then the tensor product has a basis {u; @ W | 1,... n, = I,..., m.. Correspondingly, represented with respect to given bases, the matrix corresponding to pv (g) & pw(g) is the Kronecker product of the matrices corresponding to pv (g(g) and pw (g). The scalar product on the tensor product is given by (u o' @ w²) (v o/) (w w²) The tensor product V @) W has a universal property which relates bilinear functions on x W to linear functions on V 0 W.
Recall that for an irre P3 G - U(V)), dj denotes the dimension of V₁. Definition A.9 Gelescc--Gordar Coefficients). Let pt G - U(V) and pJ : G - U(Vs) be irreducible unitary representations of G. There a decomposition
((W)
V
+
V
jEN
of the tensor product in irreps, where [j(IJ)] is the multiplicity of Vj in Vi & Vy. This multiplicity is zero for all but finitely many j E G. We denote the coupling coefficients between basis elements coming from this decomposition by jm||nn; JM), where m dj. s & [[([))]. They are called the Clebsch-Gordar coefficients.
dr. M I d, and
Remark A. The main application of the tensor product will be in the Wigner-Fckar theorem which we state in Section B where Pj will be the irrep corresponding to harmonic basis functions on homogeneous space, Pi will be the input representation, and p. will be the output representation. Compared to the work Lang & Weiler (2020), this means that we will decompose Vi @ V, instead of V, @ Vi, which leads to a simpler implementation for real representations, the case we are concerned with, at the cost of the necessity to invoke dual representations in the case that one works with complex representations.
The definition of these coefficients can be made more explicit by invoking the corresponding projec- tions and embeddings, which we will use in the proof of our adapted Wigner- -ckart Theorem. We thereby define [Im), n S di, as the n th basis vector in V/, for a fixed basis, and similarly for ljm) and M)
Definition A.11 (Clebsch-Gordat Projections and Embeddings). We denote the projection from V/ & VJ to the s'th copy of Vú by CG)(U) & Vz. It has as matrix elements the Clebsch- Gordan coefficients for fixed s,j.
= {s.jm In; JM) = (jml (G (lmm) @ (JM)))
(6)
Dually, we denote the embedding of the s'th copy of V, into Vi & V, by 2 ((J) V2 V V,. It relates to the Clebsch-Gordan coefficients as follows:
(s, jm|ln; MM (:! (lim)))[n: JMM.
Note that this definition of real representations. Indeed, in Sec. 2.2 CG!(U) denotes the matrix of Clebsch-Gordar coefficients {s.jm][m; JM) of shape dj x (d-- d.)).
coefficient is compatible with the one in Sec. 2.2 for
10e adjoint of continuous linear operator f V + W between Hilbert spaces is defined as the unique continuous linear operator ft W V satisfying (v( w)) = (v) (1e) for all x V and W.
18

---

Published as a conference paper at ICLR 2022
For Hilbert space V. we set V as the space of continuous linear functionals V. As mentioned
above, by the Riesz representation theorem these linear functionals are given by all vf = (e) for
v E V. Vt is then also a Hilbert space with the scalar product given by (ut (wt) (w This
definition of dual can be extended to dual of unitary representations:
Definition A.12 (Dual Representation). Let p: G - U(V) be a unite-dimensional unitary represen- tation. Then we can define the dual representation pl G U(V)) as follows:
(g)((() vf p(g 1
K.
If Pr G -> U(V) is finit-ddimensional irreducible representation, then we write Vi == (V)) and (pt) = Ptt for new auxiliary label (", If [mn), n ....., di, are chosen basis elements of Vi, then we choose as the basis of Vo the vectors n) = (Im), which are the functionals defined by the property (In [m) = 6ün-
Similarly how defined tensor product representation above, one can also define Hom- representations: Let PV G + U(V) and Pw G + U(W) be umite-dimensione unitary repre- sentations. Let Homg (V, W) be defined as the vector space of linear, not necessarily equivariant, functions from V to W. Choosing bases of V and W. this space is isomorphic to IKdim V-dim W and thus carries a Fuclidean scalar product which makes a Hilbert space. The Hom-representttion then given by PHom G - U(Homy (V, W)) with [pHom (J) == (w(g) o (v Tensor product representations and Hom Noom-repreeentttion. are related as follows:
Proposition A.13. Let pV G + U(V) and pw G + U(W) be finite-dimensional unitary representations. The function ęHт v @ W - Homg (V, given by
Vur vl @ - (v").. (v') (w)) = of (v') w
an isomorphism of unitary representations. The indices HT stand for "Hom" and "Tensor".
Note that when working with real representations instead of complex representations, the map - at is linear instead of only conjugate linear, and consequently, dual representations are isomorphic to the original representation. In that case, one thus obtains V W = Homp (V, W).
A.4 COMPACT SUBGROUPS AND RESTRICTED REPRESENTATIONS
Let in this subsection G C G' be compact subgroup of the compact group G'. By restriction, one can view any unitary representation of G" as one G, which will become important both in our treatment of the adapted Wigner-Eckan Eckart Theorem in Section B and in considerations on the induced representation of a given irrep in Section D
Definition A.14 (Restricted Representation). Let p': G + U(V) be unitary representation of G". We define Res V == V' and Res p' by
Res@ G U(Res@ V') = U(V'), g (())-
It is called the restricted representation of on G.
Crucially, restrictions of irreps of G' need not be irreps of G. analogy to the Reeschh-Gordan coefficients, which emerge when decomposing tensor products, one can define ircepddecomposition coefficients, which emerge when decomposing restrictions of irreps of the group G' into irreps of G:
Definition A.15 (lrrep-Decomp Deceppoccoosiiiionn Coefficients). Let py : G' U(VG.) be an irrep of G'. Then there is a decomposition
WO's)
Res
G' Vy
Vj,
jEG
where lij'] is the multiplicity of irrep V, of G Res Vy. This multiplicity is zero for all but finitely many j E G. We denote the coupling coefficients between basis elements coming from this decomposition by (l, jmlj'm'), where m S dy. m' S dy. 1 5 [ij']. We call them the dreopdeccompoittonn coefficients.
Notice the change of order.
19

---

Published as a conference paper at ICLR 2022
Definition A.16 dlrep-Decomposition Projections). We denote the projection from Res Vy the t'th copy of V by DD Res V + Vgg Is has as matrix coefficients the irre--decomposition coefficients for fixed I.::
jm j1'm') (jml ID! 13'm').
(10)
Here, ID stands for "irrep" and "decomposition".
B A GENERALIZED VIGNER-ECKARY THEOREM FOR STEERABLE KERNELS
The kernel space solution from this work (Theorem will be based on the Wigner-Ecker Eckart theorem for stecrable kernels from Lang & Weiler (2020). We thereby adapt that theorem in two practically relevant ways, and thus have to reprove the theorem: First, rather than working on a homogeneous space X of G, we work with arbitrary space X equipped with action of G. The special case where X is a homogeneous space of G then recovers the result in Lang & Weiler (2020). Second, by decomposing certain Hom-representation instead of tensor product representation, we make the computational process easier which finds the harmonic basis functions that build the equivariant kernels.
Finally, in Section B.3 we extend the previous result by reusing G"stterable basis to parameterize a G-steerable kernel space.
In this whole section, we freely make use of the concepts and notation defined in Section
B.1 BASIS-INDEPENDDNT FORMULATION OF THE GROUP-RESTRICTER WIGNER-ECKART THEOREM
Assumptions We explain the kernel space solution for the following general setting. We assume G is a compact group. Also, is one of the two fields R or C (with our applications focused on the case R). Let X be topological Hausdorff space equipped with the Borel o-algebra, making it a measurable space. We assume X is equipped with a measure μ and a continuous action of G on X. Since we want L (X) to be unitary representation over G, we assume the action of G on X to preserve the measure μ.. All these assumptions are, for example, satisfied X is any homogeneous space of G, or if G is subgroup of O(n) and X Rn any subset of the Fuclidean space that is preserved under the action of G.
Finally, Pt G - U(V) and pJ G G UVV) are irreducible unitary input and output representationssss of G. Their dimension is denoted di and d,,, respectively.
In this setting, our goal is determine basis for the space of G-steerable kernels K X Home (Vi, Vy), which we define as any square-integrable function such that the steerability constraint is satisfied, i.e.: for all g G and E X, the following holds:
K (gx) (owlg) K(x) pin(gg
Note that, compared to Lang & Weiler (2020), the space X is in general not homogencous space of the group G. We denote the space of all these G-stecrable kernels by Homg (X, Homg (Vi, Vj)). More details and intuitions on this "abstract" definition of steerable kernels (however, only for the special case where X is a homogeneous space of G) can be found in Lang & Weiler (2200) Appendix C.I.
First, note that the space (X) of square-integrable functions on X carries an action of G as well. Similarly as in Lang & Weiler (2020), the main ingredient of the Wigner-Eckart Theorem is correspondence between G-steerable kernels on X and kernel operators, i.e., intertwiners, on L (X). For this, recall that both Homg Vi, Vy) and (X) carry a Grrepresentation.
Definition B.1 (The Kernel-Operator Correspondence). For any intertwiner K. (X +
Homg (Vi, Vy), we define its restriction X by
(x) = K(())),
where 6 is the Dirac delta function at E X
JC| is then G-steerable kernel.
Since the Dirac delta function is not actually square-integrable, formally, this is defined by performing limit over square-integrable functions that more and more resemble the Dirac delta, see Lan| & Weiler (2020)
20

---

Published as a conference paper at ICLR 2022
In the other direction, for any G-steerable kernel K X - Homg (Vi, VJ), we define the extension to (X) by
R())= = J (z))k
This is then an intertwiner.
Theorem B.2. Those operations are inverse to each other, iee, R|x K and K[x K.
Proof. This follows from the well-known fact that (X) is isomorphic to its own dual space.
,, i.. to di . d copies of the dual space of (X). Conversely, a kernel X - Home (Vi, V2) belongs to
First, note that intertwiner I. (X) + Homg (V, Vs) belongs to + dedd (X)",
+ deds (X),, i.e. to di d, copies of (X).
For < < oo and q p-I. there a natural isomorphism between the spaces (x(X)* and (2(X)
given by the map:
( s-1,(19) - J (())s())d(())
+ - f(f(() = f(6x)
-
By choosing p = we obtain the isomorphism between Lx(X) and (X)" Then, note that the operations (.) and (-)lx respectively equivalent to & and 4 applied independently each of the dj d, subspaces isomorphic to (X) or (X)). And finally, note that this isomorphism is still well-defined when restricted to the equivariant kernels and kernel operators, respectively.
Note now that, since G is acting as an isometry on X, its action on Lg (X) x) is unitary. By Corollary ! follows that this actions decomposes into direct sum of irreps of G. It follows that (X) also
decomposes into direct sum of invariant subspaces (X) = + ee Vj. where my is the multiplicity of the irrep Pj E G and each Vj = Vj is acted on by G through Pj E G. We denote basis for V with {Y:Th X + KJ Note that m, may be infinite, even uncountably large, since we do not assume X be homogeneous space of G.
Definition B.3 (Steerable Basis). The union of {YJTh X E G S my, m S dj} forms basis for (X). We call this basis steerable since the action of G' on a function (X) is realized just by recombination of the linear coefficients used expand this basis (via its action each invariant subspace through its irreps).
Note that this definition is compatible with the one in Freeman & Adelson 1991..
Similar to the harmonic projection in Def. A.8 we define an orthogonal projection on the invariant subspaces of L
Definition B.4 (Steerable Basis Projections). We denote the orthogonal projection from (X) to the i' 't copy of V, by Py: (X) Vj. On the steerable basis, it is given by
Pjs 6,55 ljm).
which just follows from the fact that the steerable basis forms an orthonormal basis of (x),X), which is then mapped to the corresponding orthonormal basis of Vj, the representation space of Pj E G.
With all this in mind, we can formulate our basis-independent Wigner-Eckart Theorem for G-steerable kernels:
Theorem B.5 (Wigner-Ecke) Theorem for Steerable Kernels). The basis-independent version of the Wigner- Wigner-Eckart theorem consists of two parts:
1. There is an isomorphism of vector spaces
my
[1 J)]
GKer
Endg,x(V) + Homg(X, Homg Vo)),
jEG
21

---

Published as a conference paper at ICLR 2022
where the hat ()) denotes a topological closure and can be ignored by the reader.
2. Explicitly, GKer given by
my bat Ji]
((ee(((
ΣΣ
Σ
VHr
oi oil 1l ojis o Ppjix.
jEG 1=1
Proof. For the first statement, we observe:
Homg(X, HoogHHooi Vy)) _ Homg. (L)(X), Homz Vs))
Home (V/, Vo @ &V)
jEG i=1
Andax&(V)
jEG
In (1), we perform the linear extension to the space of square integrable functions from Definition B.1 using Theorem B.2 In (2), we the Peter-Weyl Theorem A.6 and the isomorphism Vит which we defined in Proposition 13 Note that the decomposition from the Peter- Weyl Theorem G-equivariant,
this step valid. (3), we use the Clebsch-Gordan Gordan decomposition from Definition A.9 and Schur's Lemma which shows that Vj can only have nontrivial homomorphism Vj, and not to any other irrep. From right to left, we call the isomorphism GKer. This proves the first statement.
For the second statement, we go through the sequence of isomorphisms from bottom to top and trace back where arbitrary endomorphism tuple "comes from":
batty GKer (Cjta)iiaa
Σ
pat
S
Cjia
j
Gat Σ
VHT oif J)
Cysa Pja
JEG
i=1
==1
Gat
33
VHT oif(
Cjio Pfilx.
jEG
This finishes the proof of the second statement, and thus of the theorem.
B.2 STEERABLE KERNEL BASES
After the iesseedndeeendettt version of the theorem, we now state how a basis for the space of steerable kernels can be constructed.
The basis elements of the representation space Vi, V, and V, are denoted [In), JJM) and ljm). respectively, where the indices range in 0 S S di 0 < M & d,, and 0 < m I dz. Sometimes, we write also Y.M instead of JJM), etc.. in order to remind to the connection with harmonic basis functions.
Theorem B.6 (Steerable Basis Kernels). basis of the space of G-steerable kernels Homg X, Homg(Vi, ) is given by basis kernels { (Kjwrlj E G my, S [j(tt))) $ ET}. which are given as follows: Kjter GKer(etann where jiar is zero at every entry except at position jis, where it takes on value
Remember that cf the r'th basis endomorphism of Pj-
22

---

Published as a conference paper at ICLR 2022
Furthermore, the matrix-element of such basis kernel with respect orthonormal bases [In) of Vi and JM of V,, where S d2 and n S dy, are given by:
(JM jisr (z)[In)
(11)
(8, j.jmmlin JM)
(jilee ji (i,jmla)
where the overline denotes complex conjugation, and with (1,jmlz) = Yj (c) for steerable basis functions 3 X + K.
Proof. That these kernels form a basis follows from the fact that the jiior obviously form a basis of endomorphism tuples, and that GKer is an isomorphism by Theorem B.5 Concretely, this theorem then shows that
K Kjisr FT oig(tt J) ct prixx
We write Kjser for the extension from Definition B.1 which is just given by Kjisr Wht of off( of. o Pji- Furthermore, we expand the Dirac delta function dn for x E X the steerable basis of L (X) as
6,
6g
ym 3
jEG 7=1
m=1
This expansion can be justified with similar approximation procedure in the proof of Lang & Weiler (2020). Theorem D.13. Thereby, we will in the following write the coefficients as vi 3i
x
bm)).
which also equal to Y (z). With these tricks, we obtain:
(JM (MKKKnn (ierri
JM K Kiser (5.,) (n))
d3
(.3m (x) ( JM (HT oifll') o o o pj.) (x ln)
JEG 7=1 m=1
(JM( ( (WHT offinis o cd) (1m
(i, jm x)
(i, jm x)) Σ (jm fmmlclim) ( (Jr) WWIT ozg((())) (r,t)) (n))
JM ( (WHT (i) (mm))ln))
Σ 172m
(jmlet lim) (m/l))
We first explain the steps so far and will then deal with the last term. Step (1) uses the extension of
K from X to (X). Step (2) uses the expansion of 6, explained before. Step (3) uses the property of the projection discussed in Definition B and renaming of mn to m. Step (4) expands c, (Y,")) in terms of the basis vectors yTM with the expansion coefficients being the matrix elements of c.. Step (5) is then just reordering of the terms.
23

---

Published as a conference paper at ICLR 2022
Now we deal with the first factor in the last term:
JM
HT
o2g(()))
(Ymm)
( Le (m) ( JM v VHr (I'm;; JM)) lim)
n2; JM
M
(b)
Ti JMM (JLr) ((m1) im)
s.jm
8,jm
Li; JM < (J (177lt)) JM
8,jm ttū JM 1ont
M
ñ
(c)
{s I1; JM.
In step (a), we expand (2(")) J) (Ymm) in the basis of Vi+ @ Vy. In step (b), we use the definition of
the GebschhGGoddal coefficients using the embedding (2((' J) and of æ given in Proposition A. 13 Also, the definition of the dual basis given after Definition A.12 is used as the equality (n (I
In step (c), we use eq. 9 Step (d) and (e) are clear. Plugging this intermediate result into our earlier computation finishes the proof.
We now formulate a matrix-version of this result that is more suitable to implementation. We thereby identify K (x) as a matrix in Id, for chosen bases (J in V, and [(n) in Vi, Lee. K Kjisr(z) (z) is viewed as the matrix with coefficients JMMK,na jior jiar(<)))nn) (x)].n). Furthermore, we identify of with the matrix in Kdyxd; with coefficients (jmlellim). Additionally, with (z² we mean the column vector in IKd, with entries (i.jmlz). Finally, with CG we mean the matrix of Clebsch-Gordan coefficients (s,jm [m;;; JM) of shape dj x (dj x d,). Its conjugate transpose is
denoted as [cai((t)]] 1)]", which of shape (d, x di) x dy. When using in matrix multiplications, it is interpreted having d, di "rows" and d, columns. This is compatible with the vectorization of steerable kernels which was discussed in the main paper.
Theorem B.7 (Matrix-Version of Steerable Basis Kernels). The basis kernel evaluated at x, Kjisr (1z). is given by the matrix
Kjtor(1) (1) [CG:
"i . .....yyy. Yy(())
(12)
where each dot means conventional matrix multiplication.
Proof. We do the sanity check, i.e., test whether the resulting matrix is well-defined and of shape dj x dr. This can be verified by observing that the shape of the right- hand-side of eq. 12 is:
[(d x d:) x d;] [dl x dy] [d,] [d, x di].
The full proof of eq. 12 follows directly from eq.
Remark B.8. Note that if X is homogeneous space for G, then the steerable basis (Y jim is the harmonic basis in Theorem A.6 Additionally, if all endomorphism spaces are I-dimensional, which is for example always the case for complex representations, then cf can be chosen to be the identity matrix and thus completely omitted from the formula.
Additionally, if one is concerned with real representations, then / = I and the complex conjugation in the Sabbsch-Gordan coefficients can be omitted, which leads to the following formula which we implement in this work:
Kjter (x) = [CCq(un]] cf. Yz((x).
(13)
For the special case that G SO(3), X S and IK- R. one obtains the basis kernels
Kイ(x) [CG](U]]] Yy(a),,
24

---

Published as a conference paper at ICLR 2022
where for |1 - J
J. the Y, - - are the spherical harmonics. This result was derived in Weiler et al. (2)18a). Many other examples for homogeneous space of G, with a tensor-pen cinsor-product decomposition instead of Hom-space decomposition, can be found in Lang & Weiler (2020).
B.3 A WIGNER-ECKART THEOREM FOR GROUP-RESTRICTED STEERABLE KERNELS
In this section, we consider a special case of the previous theorem where carries an action of compact group G", with G < G". We assume this action to extend the action of G on X. This result useful reuse G"-steerable kernels parameterize new G-steerable spaces without designing a new basis for T (R").
First of all, denote by G' set of representatives of the irreps of G' and by {Ygr 1/ E G',i's my m/ I d a -teeseera basis for X. Recall also the definition of group restriction and irrep decomposition coefficients from Definition 15 In particular, [jy') is the multiplicity pj G inside Res Pyr, with E G", and ID is the projection from Res Vy to the t'th copy of V (Definition A.16).
Proposition B.9. The G'-steerable basis for X given by (Yjn 1/' E G S my,m' & dy) can be turned into a G-steerable basis defined as:
{ (Yiero
65' Yyo]
1/' e G S my.j j G.t bis'l s d
Proof. Because the space X carries an action of G', we can find the action of G (X) via group restriction:
myy Vjy
Res (x( = Res
JEEC' Myy!
8=1
Res
Vyr
jEE
s'=l
[id"]
Vg
JEE
='=1
jEG
can plug this basis in Theorem B.7 to obtain the following result:
Theorem B.10 (Matrix-Version Version of Grouu-Restricted Steerable Basis Kernels), The basis kernel evaluated at x, K Kyysttrl (x'), is given by the mairix
Kzejter (x) = [cCyjt c. ID! (x)
(14)
where each dot means conventional matrix multiplication.
B.4 EXTENDED EXAMPLES OF STEERABLE KERNEL. BASES
In this section, we provide an extended version of the two examples in Section 2.2
In the following examples we will consider the space X R² and the two groups SO(2) and C4, i.c. the group of planar rotations and the group of rotations by multiples of 7/2 radians.
SO(2) (tte))rteea Kernels For rp E SO(2):
First of all, recall the (real) irreducible representations of G SO(2).
Pa(r)) 1
[oos j sinj- -0) ()(r)) sin j cos j.j--
N+
where is a non-neen non-negative integer which can be interpreted as the rotational frequency. All irreps are dimensional but for the frequency 0, which dimensional. We consider the basis B {Yzi);
25

---

Published as a conference paper at ICLR 2022
generated by circular harmonics combined with a Gaussian radial profile, in Worral al. (2017); Weiler et al. (2018b): Weiler & Cesa (2019), defined as:
cos(i ②) sin(j o) E R
Ys,(r,)) WR, (r)
(15)
where WR, R R is a Gaussian radial kernel centered around R; E R, defining a ring of radius R. Hence, in this basis, i indexes the circular shells, while j indexes the angular frequencies along each ring. For simplicity, in the following examples we will only consider single ring, indexed by i This setting is similar the 8 examples discussed in Section 2.2 since each ring is isomorphic to a circle S One can verify that the basis elements Yj in B are G SO(2) using the irrep Pj. i.e.
(r,r, à + 0) =
To apply Theorem 2.1 we need basis for the endomorphism space of each irrep and the irreps decomposition between their tensor-products. If >> 0, Py is real irrep of complex type its endomorphism space is 2-dimensional and is spanned by T
-
0 1
-
0 11
0
If - Po pt p and if p. & = PJ. For J, O, PJ @ pt = PJJ-q @ if or Pu @ Po 2 PJ-1 otherwise. In particular, in the first case, using some trigonometric identities, one can verify that
0
0
CG
-1
V2
0
0
-1
0
1
1
satisfies P. @ pt [CG (Mlli @ pJ4)) CG In this case, CG 1
1 0
0
if
v2
0
1
0
j j= J || and CG((1) =
1
-1
if - J + Thus, for l + J > 0. Theorem 2.1
prescribes a basis containing the following elements (for a single ring at radius R₁):
cos( J - 160)
sin( - 116)
=)
sin[ (|0)
cos(I] (16)
sin(|J 11))
cos(|J J - 1())
= =)
cos(|J - -((0)
sin( J 1/-110) ((Ø)
cos((J +1)))
sin((J+1)o)
())
sin((J+1Jo)
c)s((J+())))
din((++[))
cos((J+11)
K141E12(110) = (F)
cos( [J
sin( + 1)))
Observe that these basis kernels are identical to the ones Table of Weiler & Cesa (2019), up to vectorization and minus sign for _1,_1),, and NJ+....12 Fig. 2 shows the two kernels among these four obtained with j
2| 1, in the case 1 and 2. For 0, = J and CG
"Ohserve that expressed pj in basis such that is has the form descrihed in Lemma C.13 which implies the endomorphism space is spanned by the hasis described in Section C.3
26

---

Published as a conference paper at ICLR 2022
is the identity matrix; then, for J >> 0, Theorem 2.1
prescribes the following basis elements:
cos(J sin(JØ)
6,,.1 = -wn, (r)
sin(JØ) (Jo(
(,,.1((, = (r.
which are also shown in Fig.
C Steerable Kernels We construct = Ca-steerable basis through Eq. by considering the G' S0 basis described before in Eq.
15 First, recall the representation theory of
C4. For p {0, 3), rps = (r:)² C4; C4 has the following three irreps:
Po(rpg)
Pi (T(g)) = cos p
sin p cos p
sin p
p (-1)"
which are, respectively. 1, and 1 dimensional. Again, index the irreps by E 1,2), which can be interpreted as the rotational frequency. To apply Eq we need to know the irreps decomposition of Resc. Pj¹, for each irrep Pj of SO(2). We have, for N:
S0(2)
Res Po Po
S0(2)
Resc, S0(2)
P4t Po @ Po
Cee SO(2)
(4++1 P1
Resc, Pu-+2 = +
80(2)
Resc,
80(2)
P1++3
[
-1
P1
0
1 0
-1
1 0
s0, DD"
if 4t + and ID is the identity otherwise. Thus, if B' = -Y bev is the
-1
basis described before in Eq. 15 according to Eq.
Ca-steerable basis B contains
the following elements for each E N:
Yo,rr,,.1(r, 6) = Yo,((, Φ) = WR, (r),
(r, = (r (r, Φ) = WR, (r) cos
(o, Yu, (r, Φ) wn, (r) sin (4/)),
cos((4t + 1)0)
Y1 Yuti (r. wn, (r)
+ sin[((t+1)0)
(r) = V2lt2
(T, (r, = WR, (r) cos((4t++
(1. = Y242. (r. wR, (r) sin 2))),
[1
cos( 4t 3))) - sinn + 3)
Y1,0 (1.0,,44..))11,,0
0 -1 Y1,4,, wR, (r)
The endomorphism space of pa and p2 is one dimensional and is spanned by the identity, while the endomorphism space of p1 is spanned by
c { 1 ./ = 1 -1 }
The tensor products decompose follows:
Po > Pj Pj Po = Pj
2 P2 Po
i
0
1 0
② P2 P2 P1
-1
P1
0
-1 CG)
Pr P1 (T
2 Po
02
27

---

Published as a conference paper at ICLR 2022
where
[
0
0
CG
-1
V2
0
0
-1
0
whose rows are respectively denoted CG. prescribes the following basis elements for each N and each i':
CG2
CG2 and CG2 CG2(11) Finally, Theorem 2.1
L,J 20.2):
MuC.r.ccss.)a.l11: (R. cos((4t 5)6) K,,0", wn, (r) sin( j)
where =|| || J| J (0.
|= 0 and J (or I = 1 and J= = 0):
+ + 1)6) sin( + 1)6)
41 6) = wa, (r)
sin((4/ + 1))) cos((4t+ + 1)6)
110,,,-11 = wRy (r)
1os((1++3)) 3)0) +3)
1L.0, = wHy (r)
sin((4! + 3)6) cos( + 3)6)
11004443.1 = wRy (r)
2 and (orl= and J = 2):
cos((4++1)))
ALw. = WR. (r)
sin((4++ 1)6)
sin +10))
s10.a111) 6) wR. (r)
cos( 1)Ø)
+ cos((4t+3)o))
(L WR. (r)
sin[(4t + 3)6)
sin sin((4t + 3)Ø) cos (4t 3)0)
E.0.A4..1 WR, (r)
and J
(0,( wR, (r) coos(4tg) 0
cos(4t())
(0.( wR, (r) 00 cos (4t²) cos(4tt) 0j"
(1.17.4.)).11(1.0)) Φ) WR, (r) (sin(4tg)
sin(4t())
(1. wR, (r) sin (4tp)
sin(4tØ) 0]"
N2,(V,4144.1)1111117.. wR, (r) [oos( ((4t + 2)
- cos((44 + 2)
(2.0044422))21111,6 wR, (r) [0 cos
(4t (4t
2)
cos( + 2) 0j
N2.0V.442.2)11111195 wn, (r) [ (44 2)
sin sin 2)0) T
wR, (r) sin((4 2) sin 2) 0¹
K2 (0,(0,.4442222.1111111
Fig 4 shows the basis kernels for / 0 and = 0. For simplicity, in Fig we use k E Z rather that t E I to index the basis; this difference emerges from the fact that Py is isomorphic to P1 when restricted from SO(2) to C2 for any j' |1 4k). but & > 0 and & < 0 require a different change of basis (see 4t + 1 and 4t + above). Observe also that these basis kernels span the same space the ones in Table 11 of Weiler & Cesa (2019), for N up to vectorization.
28

---

Published as a conference paper at ICLR 2022
C REAL VALUED REPRESENTATIONS
In this section, we discuss the representation theory of compact groups on the real field I.
Usually in representation theory, one assumes complex-valued signals in LG) and complex repre- sentations of G. However, in this work, we implement equivariant CNNs using real representations, and therefore must discuss general issues arising in this case.
When using real representations, the theory becomes more difficult: while the matrix coefficients of the irreducible representation span the space of real square-integrable functions L (G), they are not necessarily orthogonal each other anymore. More precisely, the matrix coefficients belonging to the same irrep might be linearly dependent, while different irreps still contain orthogonal coefficients. This is consequence of the fact that the strong version of Schur's 's Lemma stating that the endomorphism space of irreps are one dimensional does not hold for representations over IR.
As before, we freely make use of the preliminaries from Section A In particular, recall that for real representations, we talk about orthogonal representations instead of unitary representations, and such representations take values in the orthogonal group O(V) of some real Hilbert-space V.
In Section C.I we introduce some properties of real irreducible representations and characterize their endomorphism space. In Section C22 we show that any real irrep can always be expressed with respect convenient basis such that set of linearly independent matrix coefficients are contained in a subset of the columns of the irrep. This will be particularly useful to construct real harmonic bases in Section D and, in particular, in Corollary [ Additionally, we describe method to find such a basis for an arbitrary real irrep. Finally, in Section C.3 we show an example of such basis, which makes the result more intuitive. Additionally, Section F
this specific basis to simplify computations in
C.I PRELIMINARIES ON THE STRUCTURE OF Endg,
Fix an orthogonal irreducible real representation vi G O(v)) of the compact group G. Let Endg, (Vp.) be its endomorphism algebra. It is well-known that this endomorphism algebra is isomorphic (in a sense which will be made precise in the following proposition) to one of the division algebras R, C, and H., i.e., the real numbers, complex numbers, or quaternions, see BBöccker & Dieck (2003). Theorem 6.7. Let K _ Enda,>(V, (Vp)) be this division algebra. Let Egg- dim Endg. (Vp) its dimension. Let i 1, i2 has the characterizing properties 12 = j² = k2 = 1, that 1 is multiplicative identity, and that ij k: ji, jk now make the properties of the isomorphism precise:
it j and is = k with 1, k IHI the usual basis. It
Egg kjj and ki = j ik. Then K. has as orthonormal basis (ū))-1- We
Proposition C.1. There is IK E (IR, C, such that there is a function +b K - Endg, (Vo) with the following properties:
1. 4(x + v) = ((x) + ((y) for all x. E K.
2. ((Ax)= x for all X IK and K.
3. ((wy) = ((x) o (y) for all x, K.
4. +(1) =idv..
5. & is bijective.
6. (ix)) O(V.) for all k E (l...., E.
Proof. Properties to 5 are the properties of an Rkllgebra isomorphism. Such an R-algebra isomor- phism& Tb exists by Bröcker & Dieck (2003), Theorem 6.7.
We now show that this automatically already implies property 6 as well: ((i)) Ende. (V=) is a non- non-zero zero endomorphism, and thus, by Schur's Lemma for orthogonal representations (see Lang & Weiler (2020), Lemma B.29), there exists coefficient l R such that (i) E O(V,). We
29

---

Published as a conference paper at ICLR 2022
deduce:
m o ((((()) 1 - i)) (1)
=ui idv.
Together with (it) (4) E O(vg) we necessarily have 1u/ E {+1), and thus μr +1. Thus, ((;;) = E O(V),, which shows 6.
Definition C.2. Let d be real irrep. Depending on its endomorphism algebra Enda(v) t is classified one of the following three categories:
real rype: if Enda (V&) = R
complex rype: if Enda (v.) = C
quaternionic type: if Enda (V)) _ H
From now fix the isomorphism + K + Endg,a (V+) and denote 1 (
For simplifying the notation, we set dy. == dim. = dim Vg. From now on, we assume for simplicity that VP Rdv, which holds up to isomorphism. The scalar product on Vp then just given by (u w) of w, and Vy has the standard basis (ei,...,ed, with e; having at position and being zero elsewhere.
For any (Vo), we define A Endg,,(VV) as the transpose, which is also the unique matrix and linear function AT Vi - Vy such that (Av w) = (w A"w) for all v, w E Vy- That AT is also an endomorphism for the fact that v is an orthogonal representation.
We define the following inner product on (Vd).
(A (A
de Tr(BB).
(16)
where Tr is the trace of matrix, given by the sum of diagonal elements. The trace is linear and continuous and has the property Tr(AB) = Tr(BA), which results in the trace of a matrix being independent of the basis in which it is expressed. Lemma C.3. We have the following:
1. I = -I,' for 1 > 1.
2. [1]]ss = for l and E dpp..
3. (4) Ip) = uw for all
Proof.
1. This follows from I I o(ff) - - idv, for all 1 > 1.
2. From Proposition C.1 we know that I; E OV+) for all I. So, I, = ITT For 1 > 1, we
deduce / - If from 1, i.e., T + If = 0. It follows [Ti] } [ + [f = 0.
3. Note that there is some 1" E Ev such that the following holds:
(TI Ir)
Tr (ITTF)
de
1
Tr (+(1) -(iv)-1)
de
1
Tr (
de
1 Tr (#()))))
de
1 de
Tr + Iv
30

---

Published as a conference paper at ICLR 2022
Thereby, made of Iy O(V2) in step 2, and used the multiplication properties of the quaternions in the second to last step. Note that 1p" = if and only if = 1', and that in this case, we have Ip- ((vx)) idy, leading to (Ii [e) 1. If, however, I" > 1 (i.e., * ('), then have [4r]a 0 for all i by part 2, and thus we get zero trace, proving the result.
Corollary 2. The function k + Enda, is a unitary isomorphism, meaning: When considering K. RR-vector space with standard inner product (- 1) and considering EndG, (Vp)) as an Revector space with inner product given by eq. 16 then we have for all æ. E IK.
(D(x) 4 (g)) (a 9). Furthermore, {I,,. TE forms an orthonormal basis of EndG,.(Ve)
Proof. First, note that eq. 16 clearly defines a positive-defintt inner product on Endg, (Vo). Write Es E mút and y Σ v= B1 ig. Using Lemma
C.3
part we obtain:
Es
E
(+))) p(v))
Trye (Tr Tr))
1=1
V=1
E
(x y)
The last statement that the I form orthonormal basis of Endg, follows from Lemma C33 part 3 as well, together with the fact that ((r)) = I and that & is an isomorphism.
For Ein Jndy E K, the conjugation is defined by
Ev rrú-
I = I1
Furthermore, we define the norm ||W|| =
(W
W) for W Fand,
Proposition C.4. We have the following:
1. For x E K, we have ((T) 4(x)².
2. For any W E Endd,3(V) we have WW = || 2. idv.
Proof.
1. We have
E
((z) = ①
Fi
Σ
3444
1=2
E
1=2
E
r
Σ
F
T
(xx)
In the third step. we used idy, and IT - It, which was shown in Lemma C.3 together with T O(V).
31

---

Published as a conference paper at ICLR 2022
2. This follows from the well-known fact that x
||||? for K and part 1.
C.2 ORTHOGONALITY AND ALIGNMENT MATRIX COEFFICIENTS
The investigations here the orthogonality relations matrix coefficients of irreducible real representations generalize what's presented in Knapp (2002) for complex representations to real representations. The results thereby become bit more involved.
Remember that Vi = Rd has the standard basis (e1....,ea, ea. Define Esy = ejey E Rdoxdd. Furthermore, define
Oig = 1 ((g) -dg 1 ( Eig
dg
G
Remember that v/ G -> O(V)) ( C Rdoxde takes values in de dg-matrices. For j.j E (1....,dy}) define viaj - R as the function thi E GG) given by (x) = (g)aa--- Remember that we have scalar product on (G) given by (f h) = lo (g)) which means we can evaluate also scalar products of different matrix coefficients.
Lemma C.5. We have the following:
1. Oi E EEng, (Vp).
2. Oij d Ei= [Tlee Is.
3. Oij = ((iiee Vless)) ) k2=="1
Proof. We prove the three statements as follows:
1. From the integration- on G being Haar measure, i.e., left-invariant, it can easily be checked that Ogg) = vigg)) for all g' G.
2. From Oij E Enda, (Vp) and the fact that the I form an orthonormal basis of the space Endg, (V)) by Corollary we obtain Oij EEn (4 Ozy) We now determine the coefficients:
(4 O1)) = (Oi) 44)
Tr (Oizl?)
dy
= dx Tr (/c
(( 11
@ 1
de / Th (V1))),
dp [ Tr (Eng1() dg
1
dy Tr (EigTTt)
[T]ss]
dy
In (1), use that I is endomorphism, i.e., commutes with (gg). Additionally, that Tr is linear and continuous, and thus commutes with integrals. Step (2) uses that the trace satisfies Tr(AB) = Tr(BA). In the last step, we make of Eij = e,ef.
32

---

Published as a conference paper at ICLR 2022
3. We have
Ogj
((g
(1)
(
Vhi(o)
/
(vi,(9)
(d))( dg
())) de
in k1 dg
= ((tha I (has)) kg
Corollary 3. We have WPin (kaa)) matrix coefficients in the same column are orthogonal to each other, and they are normalized up to a constant factor: / Wkzll 1
6,,02 for all kn, k2,j E (1...., d.)}- other words, the de
Proof. Remember that in Lemma
C.3 we showed [Tibas = 0 for 1 and that 7 = idv. Together
with Lemma C.5 obtain:
(Whis tkast
[Ojs] k2
Es
Lilas
I
1=1
Rika
dep
5kyk2
deb
The statement about the normalization is clear.
Lemma C.6. For any p. E {1..... E for any v E Vos (Ip0l100) (pg(0))).
1/. If p q, 1 11 idv, is the identity Proof. Note that there is a such that 17' 4 matrix, hence o 1 T, x Otherwise, >> 1 and T₁ -TF skew symmetric matrix, hence of I = O. Indeed, note that, for any matrix A, holds that v" Au o² A" If A a skew symmetric matrix (A" A), it follows that v" Av O.
O
Lemma C.7. For each column j, the set of matrices {Oir)s spans the space Enda(VV) which is an Eg dimensional space.
Proof. To prove this, it is sufficient to look at the set of coefficient vectors of each of these matrices when expressed with respect the basis { of Ende If this set of coefficient vectors spans an Eg dimensional space, the proof is complete. Because O,, 2 E1 let's consider the set of vectors {(TT)s)) and stack them the rows of a matrix M E.. The set of vectors spans Eg dimensional space if and only if M has row rank equivalently, column rank Eg. Because has Es nonzero columns (namely, the l'th column of M is the j'th column of 4t, which non-zero since is invertible), then it has column rank equal to Egg if and only if all its columns are pairwise orthogonal.
The orthogonality of the columns of M follows from Lemma C.6 Indeed, note that the I-th column of M contains the j j-th column of I,. By choosing v = e; in Lemma C.6 it follows that the column j of two different basis elements It and I, are orthogonal. Thus, the columns of M are orthogonal. [T
33

---

Published as a conference paper at ICLR 2022
From Lemma C.7 and since (Vd)(Vo) contains only the zero-map and isomorphisms, it follows that, for any j. the set of vectors {Oizez 1== spans an Ev dimensional vector space. We will show that this means that there a change of basis E R4 xdo and a corresponding irrep 3 = B isomorphic to s such that, for any j. (O e contains only Es,, non-zero vectors, where O₂ Jo Eng((()) dg.
We give a constructive proof now. First, we build one such matrix B starting from the basis elements [1) F. Then, show that for ve BT BB for each j, the matrix Oig is non-zero precisely for Ey different values of i.
Lemma C.8. For any vectors v, ur E Rdv, if (1,0/w) = -Oforr all p. then (1 = 0 ffr any and q.
Proof. For any p and q. there exists I such that 1,44 = +h. Then (1p0lka0) = 1 to" IT ++(w)/)) which is zero by assumption.
To construct the matrix we use the following algorithm. Let B 0 C RR the empty set. Repeat the following process / Esg times [15]
Take a normalized vector a E Rd which is orthogonal all vectors in B. Add the vectors { Es to B. Note that all vectors {T(v)
Eg )= are orthogonal to each other because of Lemma C.6 Note that if contains the vectors {Trw) Y and we choose vector such that is orthogonal all [nw0] then, due to Lemma C.8 the updated set B still contains only pairwise orthogonal vectors.
The matrix B is built by stacking horizontally the vectors in B. Note that since all v the process normalized, and the Ii are normalized as well by Lemma C.6 the matrix is orthogonal, i.e. B = BB" id.
By construction, for any I, the left multiplication of B by I, only results in a permutation of the columns of B (and, potentially, flipping their sign). In particular, I₁ only permutes the vectors in B belonging the same group (1,r). Hence, there exists permutation matrix Pt (with, potentially, negative entries) such that B = BP,.
Let, as mentioned before, v(g)= BT BB) i.e., & is isomorphic to o via B. Also, let Ö.3 Je ((g) E.j T
Lemma C.9. Let db be real irrep and B matrix computed as above. Fix E fl....,,dy Then, the following holds:
1. Fix also i (1,...,,p) Then Ōzg # if and only if there is unique E (1,...,Fi. F.y) such that [Plla + 0. In that case, O,j d, PP.
2. The matrix Ö.g is non-zero for precisely Eg different values of
The process itself will reveal that d Fay is whole number.
34

---

Published as a conference paper at ICLR 2022
Proof.
1. Let B, and By, respectively, the i-th and the j-th columns of B.
|
Oij
B² (e
(g)) Bdg
G
(1 (5 B
BT
B3 (Twio)
Ean vlo)"dg) B
Σ
Bug Oab
a.b
Σ Σ
Bas Bbj [1]bbb
Bd TBB
de
a.b
1
Σ
LIB" 1 LBB
du
du Pi
where in the last step we used that B has the property 1B = Bd BM Pts where Pt has the structure a signed permutation matrix. Thus, if O:j * O. then there exists such that [Pilaj # O.
For the other direction, assume that there is such an I. We want to show that there cannot be another with [Palip * O. If we can do that, then it follows Oig = de [Plly Pt * and we are done.
To do so, first observe that (Me,) du. Indeed,
(Pie = Pf Pke, =T IT BBTL.Be I,Be,
=e] Bd T I Bej.
Now, note that there exists such that 14 = = In particular, if = k. 1f
T 11 = id and, therefore, e,, BT 17" Be, = e = Otherwise, e B² 17
I, =
= B Im Be, O. thanks Lemma C.6
Now, assume have [P]]ss * 0 0 [Plle Then, since Pt and Ps signed permutation matrices, all other entries in column j are zero, meaning that
0 * (Plks (Pelss = Σ Pily [Pkls = (Pie Pkez) = dgt,
meaning that ls L.
2. For 1,..., Eg. let i E (1...., d+) be the unique index such that [Plla * 0. It follows Oid * from part 1. We also know from part I that all i are pairwise different, so we found Eg such indices. Now, assume is such that Ô * O. Then, from part 1 again, we get that there exists such that [Pt]e * and thus it. That finishes the proof.
As a last classical result, we will use the Cauchy-Schward Schwartz inequality, which is well-known and therefore without proof:
Lemma C.10. Let V be any Hilbert space with scalar product ( I.), Let v, w E V. Then one has Ky wi)l S ||vil [|w]],
with equality if and only if v and linearly dependent.
35

---

Published as a conference paper at ICLR 2022
Theorem C.11. Let vp be a real irrep. Then, there exists real irrep vb, isomorphic v such that the following holds. Fix j E [l,....du}.
1. The matrix coefficients in column j of ip are orthogonal to all matrix coefficients of ib in all but precisely Ey columns i.
2. If is one of the remaining Egg columns, then the is signed permutation of wej.
Proof. Let B and v = =B"BB as before. We know from Lemma C.9 that there are precisely Enp values for such that Oij + O.
1. Assume Oi5- =
Then, since by Lemma C.5 we have Ō:ş =(Wansl ki kg= it
follows that columns and j of T are fully orthogonal to each other.
2. Assume Oig * O. Then, by Lemma C.9 there is such that [Pilst and Özg -Piley Pt, where P1 is signed permutation matrix. We have [Pi]a] =1, i.e., can wlog. assume [pt]ss = i.e., O.j = 2.
Now, let ki E (l,...,, d be arbitrary and look at the matrix coefficient viaa. Then, since 1 P ((VRi) (aai)W by by by Lemma C.5 again, and due to the structure of Pj, there de
By the preciscly one k2 E fl...., ......,,y d such that (ahkes T = = du
1.
inequality Lemma i. follows:
1
S |
dx
dub
de
In the second to last step, we thereby made use of Corollary
We obtain (Weii (ewni =
[ || and thus, by the last statement in Lemma C.10 that th and thes are linearly dependent. Since they have the same norm, it follows thk -okE Since for different k2, the value of k needs to differ as well due to Pt being signed permutation matrix, the result follows.
This result convenient to define a real harmonic basis (Y. for LG),,, which we then use in Corollary 5
C.3 A CONVENIENT CHOICE OF BASIS FOR REAL IRREPS
At the end of the previous section, we described a method that, given a real irrep 1b and a basis for Endg
(V&). Vo generates an isomorphic irrep & which satisfies the convenient properties in The- orem C.I1 To provide an intuition what structure these real irreps have and simplify some computations in Section[F) in this section we describe a choice of basis to express each real irreducible representation, satisfying the properties in Theorem C.11 Note that this particular choice of basis not necessary in most of our results, since we rely only generic orthonormal basis for End (V+)). but useful to simplify Section F
The content of this section is mostly based on Boardman (2007) and Bröcker & Dieck (2003). Recall first the classification of real irreps real, complex or quaternionic type according to Definition C.2 We will rely the following useful results:
Lemma C.12. For any real type real irrep b its complexification is isomorphic to complex irrep with 2 a. The complexification of up a representation like vb which acts on d rather than R&
and
36

---

Published as a conference paper at ICLR 2022
Lemma C.13. Any complex quaternionic type real irrep v is isomorphic the realification of complex irrep a. In other words, for any complex or quaternionic type real irrep v there exists complex irrep a such that:
Ro (a) Im (a)
Im (a) Re (o)
Additionally, it has quaternionic type, it also holds that = a.
Indeed, using Re (a) 2 (z + 2) and Im (z) -21(z I), one can verify that:
Re (a) Im (a)
Im (a)
D [g(g)
[ D
Re (o)
where
iida,
-idda, ida,
D
v2 ida,
Using these results, we define a convenient choice of basis for db for each possible irrep type. For each case, also define associated basis for EVd))Vy)).
Real type irrep Since z) a (as complex irreps), we do not need further assumptions on the basis of v. Note also that Enda(Vi contains only scalar multiples of the identity; hence, regardless of the choice of basis for th, a basis for Endg((V) is given by the identity matrix.
Complex type irrep
If th has complex type, we assume that t is expressed in basis such that Re (o) Im (a))
v = Im (a) (e) for some complex irrep a. Note that this is natural choice, for example,
for SO(2). follows that a basis for (Vv) given by the following homomorphism
c + Enda(Vo):
idd, 0
(11)
id,,
id,
(()
id,
where d and id, the identity matrix of size n.
Quaternionic type irrep Similarly, if eb has quaternionic type, we assume that e is expressed in (o) Im (a) for some complex irrep a. One can show that if has basis such that & Im (o) Re (a) quaternionic type, it can always be expressed in a basis such that it has the following block structure:
a (g)
-o2(g)
(g)
(17)
(2
(1(
and, therefore:
(1(g) - -2
(1(()) -92(g)
Re
-Im
[2(g) 01(g)
[2(()
σ1(g)
(())
(n(g)
-2((g
(1(g)
-02(g)
Im
Re
(2(() (1(g)
[02(g) σ1(g)
37

---

Published as a conference paper at ICLR 2022
A basis for Enda is given by the following homomorphism Φ H Enda(Ve))
idn
0
idn
0
(11)=
0
id,
0
0
idn
O
0
idl,
0
0
0
id,
((1)
id,
0
0
idn
0
0
idn
id,
0
(
0
ida
0
id,
0
0
id
id,
((k))
0
ich
0
0
idn
0
where n = /.
D THE COMPUTATION OF THE INDUCED REPRESENTATION OF COMPACT GROUPS
Quotient and induced feature fields, i.e. features fields where the feature type p is quotient induced representation (see Sec (.1). were studied in Weiler & Cesa (2019) for the planar groups. These feature types encode square-integrable scalar vector functions over homogeneous space X == G/H of G. If X is not finite, one restricts the consideration to subspace of bandlimited functions by using finite subset of an harmonic basis for the space of all functions. A band-limited function is then parameterized by coefficients vector, which carries an action of G as a direc-sum of irreps. Given one such vector, the function parametrizes can be sampled at any point of E X by evaluating the harmonic basis on x. For example, in Sec 5.3 we employed models with quotient features over the sphere S = (3) (0( or the space Inv xS2 = 0(3) SO(2). See also Section H.3 for more details on how this used in neural network design.
Recall also that a G-orbit isomorphic to an homogeneous space for G and, therefore, to quotient space G/ = [g. E G}. where H < G is a stabilizer subgroup of the space; e.g. (0(3).z &
Wigner-Eckart S SO(3) (0) for x E R non-zero. The Wigner- Eckart theorem from Lang & Weiler 2020) relies harmonic basis While our Theorem B.7]re on a G-steerable basis for X, simple way (although, generally more complicated band-limit and discretize) generate such basis is by embedding G-orbits inside X and use the harmonic bases of such orbits. This is visualized for = C4 in Fig. Thland used for the G baselines in Sec. 5.2
orbit G.v of the equivariance group G inside the base-space X.
In all these settings, an harmonic basis for functions on an homogeneous space is required. In this section, we describe how compute the harmonic basis for scalar and vector fields over a compact homogeneous space X G/H, directly from the matrix coefficients of the irreps of G. A vector field over
G H function f X + Rd (or Cd+) and transforms under the action of G according to an induced representation Ind f v. where vo is an irrep of H. We give precise definition of induced representation acting on such vector fields in Def. D.3 Note that scalar fields are considered a special case of vector fields. particular, scalar fields on G/ H transform according to Ind G vo. where vho H - (1} is the trivial representation of H. We occasionally call this representation acting scalar fields a quotient representation.
Similar investigations are well-documented for complex representations: for the case X = G. a complex harmonic basis is given by the matrix coefficients of its irreps, described by the Peter-
Note that in this section X with different meaning with respect to the rest of the paper.
38

---

This becomes unitary G- Published as a conference paper at ICLR 2022
Weyl theorem A.6
A similar result for compler-valued representations and scalar functions homogeneous spaces was also described Kondor & Trivedi (2018).
Outline and goal After some definitions and assumptions in Supplementary D.1 provide different equivalent definitions of the induced representation in Supplementary D.2 Then, we construct the harmonic basis of induced representations in Supplementary D.3 Finally, Corollaries 4 pndviid the final harmonic bases for the complex and real settings, respectively.
We freely make of the concepts defined in Supplementary A in this whole section.
D.1 USEFUL DEFINITIONS, ASSUMPTIONS AND NOTATIONS
In this section, we will always assume a compact group G and compact subgroup H C G. The neutral elements of these groups just denoted
H. which is the same for both groups. As before, for generality, we allow K to be any of the two fields IR or C before we will later separately consider complex and real representations. H is the set of all isomorphism classes of unitary irreps of H and v H is, by abuse of notation, meant to refer representative of such isomorphism class. For the rest of the section, we fix such an irrep v G - U(V2).
In order to generate harmonic basis for the functions in the induced representation, we only rely an harmonic basis for L (G), basis for the endomorphism space of each irrep p & and the irrep decomposition of the irreps G when restricted to H < G. Assuming these are known, in the rest of this section we will them explicitly build harmonic basis for functions transforming according to an induced representation.
D.2 INDUCED REPRESENTATIONS
We can now introduce the induced representation. In this section, we will limit to the case where <b H + U(V.)) is an irreducible representation of H. This is fixed in this whole section. Induction from non -oeerreeeeeelee representations of И can be built by combining the induced representations of the H irreps composing the reducible H -rrepresentation.
We now define the induced representation as a space of Mackey Functions, and then alternatively as space of vector fields over quotient space. The constructions are well-known and can for example be found in Taylor & Kaniutrh (2013). This book also contains more theoretical justifications, including the definition of the scalar product making the induced representation a Hilbert space and proofs that both versions of the representation are actually unitary.
Definition D.1 (Mackey Functions). A Mackey function is a viettr field f G + V which has square-integrable component-functions. i.e., in (G), and with the following equivariance-property:: for all h e H and g G one has
figh (h))((g).
(18)
We denote the space of Mackey functions by Homw Vi) representation with the action
[g-f]gg') == f'g''')
(19)
Remark D.2. One can verify that g-f still belongs to the space of Mackey functions. Indeed, for 9.g' E G. h E H and f E Homm (G, Vi+) one has
[g.fl(g'h~3) -
1(lo ((0)-1)')h~1)
-(h)f((--g'))
=hh)lg-fl(g')
Before we come to the definition as quotient vector fields, recall that for the pair of compact groups H G, one can build the space of cosets
G/H = [6) g E G).
(20)
These functions can he viewed as left-H--quivarian when using the left action h g = gh 1 of H G. which explains the notation.
39

---

Published as a conference paper at ICLR 2022
There is projection π G - G/H given by g.H which induces topology G/H, making a compact topoloicaa Hausdorff space. Together with the action g.(g' H) = (gg' H. G/ H becomes homogeneous space of G, and thus carries Haar measure.
Now, pick an arbitrary measurable section r : G/H - associating a representative element to each coset gHl G/ H 18 In other words, the section satisfies ι idg/H H. with the projection as defined above. This can also be written as
r(gH) gH, for all g G.
(21)
Since r(gH) = gH, we automatically have that
h(g) == r(gH)- H, for all g EG
(22)
Since g r(gH)- h(g), it follows that every 9 E G fully characterized by the pair (g h(g))
(G/H) x H. We fix from now on the definitions of the section G/H + G and the map h: H, with the latter implicitly depending on
Definition D.3 (ei-Vect Vector Fields over the Quotient Space). A v--vector field over the quotient space G/H is defined to be any function of the form
G
that has square integrable component-functionn with respect the Haar measure on G/H, i..., the component-functions are in HG/ The space of these vh-vector fields is denoted by Hom(G/H) H, Vy). This becomes unitary G-representation with the action
Ig ((
h ((or(( f'g
(23)
Note that the action of G on Hom G/H, V&) depends on the choice of the (not necessarily continuous) section G/H H + of the quotient space.
We now show that these two definitions are equivalent: Proposition D.4. There an isomorphism of unitary representations
Hom Vv)
Hom(G/H,Vo)
between the space of Mackey functions on the left and the space of quotient vector fields on the right, given as follows:
1. For f :: G.G G.G Vv a finctiion,,,,, we define
flgH) M((gH))
using the section defined in eq. 21
2. For a quotient vector field I G/H V4 we define
F(g) == ((
using the function h : G H defined in eq. 22
Proof. In this proof, we freely make use of eq.
21 and eq. 22
First, we show the equivariance (-):
Let G + Ve satisfy eq. [8] First, note that
gr )'H) =
g g
=r (
g g
=(gg')) r
(24)
(r(g g H(ggH))
= h r(g r(g'
1hhss section usually not be chosen be continuous, but heing measurable is enough for our purposes.
40

---

Published as a conference paper at ICLR 2022
From this, deduce:
g (o.f((((g(H)
= f(g
((too""
( H(H)
lg.f(('!11,
i.e, the desired equivariance g.ʃ g.J.
Given f' G/H + Ve. we first verify that f satisfies eq. 18 i.c. that is lives in Homg (G, V₂). First, note that
(gh- r (gh - H) gh-
= =I(HH) gh-
(25)
-h
From this, deduce:
F'ghh -(h(ggh")
-((h
(g))
We now show the equivariance of (.). We first note, using the result of eq.
24 and eq.
25
h(g')-²h (gr (0 ) h(gg)(()))))) 1o (
bb ( h(o')
(g'H)()))))))
=h
We deduce:
g-6'(( = ((b6))")b,/(6'")
((h(a-)))~)) (')
-"'g"'g')
= (9.5'(('),,
ie., the desired equivariance 9.f" g..
Finally, we need to show that f f and fr = f', ff, i.e., that the maps (-) and (-) are inverse to each other. We have
((g) (b(((()) F((g))
(hh())))))((( -(((gl))b()) f(()
41

---

Published as a conference paper at ICLR 2022
and
M(glt) M(((gl1)
- ((bloigtt)"))))(( = ((HH)
where we used r(gH)H gH, h (r(gH)) e, and (ee) = idy, in the last step. More technical details, especially on the unitarity of the isomorphism, can be found in [Taylor & Kaniutrh (2013). This finishes the proof.
Proposition D.4 justifiiss the following definition:
Definition D.5 (Induced Representation). For any irrep & of H G, we define the induced representation Ind,; as the unitary G-representation acting on Ind Vp = (o, V&) = Hom H, Vi. as in eq. 19 or eq. [3 depending on which representation space is chosen. We will always make the representation space (Mackey functions or quotient vector fields) explicit in the following.
D.3 HARMONIC ANALYSIS OF INDUCED REPRESENTATIONS
Before we come to finding harmonic bases of induced representations, we must deal with a small technicality: in the induced representation given by Mackey functions, the equivariance property (ghht) (g) is used, which suggests that we look at the following left-action of G on itself: g * g' = g' compared the usual left action g-g' = gg'.. Similarly how we solved for a basis of the space of steerable kernels K X' - Homg Vi, Vy) with our Wigner-Eckee Eckart theorem, Theorem B.5 by studying 1. (XX), we want to find basis of the space of Mackey functions f G + Vo by studying L& (G) however, due to the adapted left-action on G, we also need to consider a different action on (G).
Thus, define (G)2 == (G) together with the action (g* A()) = (g'g), which be shown to be a well-defined left-action on (G)2. This is then unitary representation. Fortunately, it turns be isomorphism the standard regular representation:
Lemma D.6. There is isomorphism of unitary representations
(G)
L2)2)
given both directions by the same function:
[o) (fg()))))))))))
Proof. Since in L (G), we integrate over an inversion-invarinn Haar measure, is unitary transformation. It is also clear that 4 is its own inverse. Thus, we are only left with checking the equivariance in one of the two directions:
[+(g)f)](o') = [o.5](o""))
=(('0)))
= [1(())
= [g+*(
follows = + i.c., the desired equivariance.
We remind the reader of the following notation introduced in Section which we use in the harmonic analysis of the induced representation: G is the set of isomorphism classes of unitary irreps of m, the multiplicity of the irrep P5 + U(V,) in the regular representation (G). For j my- Psa T2((G) Vy is the corresponding harmonic projection. [Wi]] is the multiplicity of the irrep
42

---

Published as a conference paper at ICLR 2022
G 3: H - UVvp) in Res Vj. E. dim Endg,& is the dimension of the endomorphism space
of V. The c VB + V.. Egg. form basis of endomorphisms of V. ID, Resh V, - V is an irrep decomposition projection. The yj G + K. form a harmonic basis of the regular representation (G). Y5= : G K Vj the column-vector of all the yJr- Y5a its complex conjugation.
Finally, note that after choosing bases, by abuse of notation we also refer with or and ID to the matrices corresponding to the linear maps.
Theorem D.7. A basis of Mackey functions Ind V = Homm (G, V&) is given by
{ [fur - V& =K 1
G, m,, Wr),r E
Thereby, we have
[g)r(g) = - (
Consequently, basis of vb-vector fields over the quotient space, Ind Vg Hom(G/H,VV) Vo). is given by
(Far G/H Vo k 1j E G, S my, t S [+]], & E
with Jsite (gH) = [jur(((gHH) where G/H - G is a measurable section.
Proof. We have
IRE
Hom (G, Vo)
Hom Res G (G)2,, Vo)
IR
Homæ, Res C L (G), Vi¹)
my
Hom Res G
Vy, Vo))
jeG
my [+)]
Endgg&((g))
jEG
Step (1) follows from Theorem B.2 with G replaced by H, G' replaced by G. X' replaced by G together with the left-action g * == g'g and Homg [V, Vy) replaced by V₂ Step (2) follows from Lemma D.6 Step (3) is the Peter-Weyl Theorem A.6 And finally, step (4) uses the irrep
decompositions ID Res
G # V, + Vy.
Explicitly, from right to left the isomorphism is given by:
[W]]]
(cjt) +
Σ
ID!''
Sise
1=1
my
ID
opys
jEc
my [+
Cjaa IDI' opgs +
jEc im
[+
Cja
DD opjz
Llc.
JEc
Note that in the original theorem, the structure of the ropreeeresnnnttion is not actually used, so we just replace by any finite-dimensional representation; in this case, V₂
43

---

Published as a conference paper at ICLR 2022
Thus, a basis is given by all fjar c/ ID,' dirac delta function by in the harmonic basis:
ops b/a. The result now follows by expanding the
Pjs Fla(g) Pji (D()g)
Pji (
dj
(Yjt (6-1)) -ps) (Y
=
m=1
Σ
Ym (g 2) ljm}
In the last step, identified V, = K and thus jim} with a standard basis vector cm Note that, to be precise, one would need to approximate the Dirac delta function with square-integrable functions. The details of such arguments can be found in the proof of Theorem D.13 of Lang & Weiler (2020).
The last statement about a basis of Hom(G/ GG/ H, Vi), finally, follows from Proposition D.4
Corollary 4. An harmonic basis for complex Mackey functions is given by:
{Fm [fu:G= G V 200 Ije G,i dj [0b]].
where
fsu(9)= = (d,p,(g
Proof. Using Proposition A.7 we find that -d,p Because we assume complex representations, Endg (add,cc(r e only contains scalar multiples of the identity, due to Schur's Lemma. In other words, e always the identity matrix.
Corollary 5. Assume that all real irreps are defined over a basis as Theorem C.I. basis for real Mackey functions is given by:
An harmonic
{firr : G Vo+ 1R li AjeGis m, dj/Ent S [bbi,, s E
where
[g)r(g)= -=)==; c - -DD' -TTpss 'ez
I is an index set containing the indexes of the mj d Bj columns of Pj with linearly independent matrix coefficients.
Proof. Using Proposition A.7, If p is expressed in a basis as in Theorem C.I1 the matrix coefficients of Pj in column are linearly dependent with those in E, columns of P.- Therefore, it is sufficient to consider a subset of m, d,/ E, columns of Pj which contain linearly independents matrix coefficients. Additionally, if Pj is expressed in the basis described Supplementary C.3 I'
(1,2.... (1,2,...d//E}} so I =
show that this basis is indeed harmonic and we explicitly derive the action of G on as a direc-sum of irreps.
In Section D.5
D.4 RECONSTRUCTIN THE IRREP DECOMPOSITION: THE COMPLEX CASE
The goal this and the next section to use write the action of on the bases found above explicitly as direct sum of irreps of G. We first consider the simpler case of complex valued irreps.
Recall Corollary Consider the set of all basis elements for a fixed value of j. i.e. (ffee where:
[ir(g)
1 VTppslo De
44

---

Published as a conference paper at ICLR 2022
Let be Mackey function in the span of these basis elements, i.e. there exists fwo Cha such that:
f(g) Σ (a)
Σ wn ID," Valps(o
it
ΣID Vdips(o
E wae
EnD Vispslo
where [wda]: wit and w; Ca, Now, the action of G on these functions is the usual action (²(G), Vk,g G:
[k.f((g) f(k Σ Waffaa -ID vdip;(o (k)w,)
In other words, the space of functions spanned by {ffuh ha is isomorphic ( Vj, and each lives in one such VÚ space. We call the vectors wy the Fourier Transform coefficients the function f. More generally, let f be generic Mackey function in Ind, Vg. The matrix f(p)) of shape dj x [wj] contains in its columns the vectors w, E Cd for each t. We call the collection {f(p)) the Fourier Transform of the function f. Note that if s) is the trivial representation, this definition recovers the classical definition of Fourier transform of functions over compact groups homogeneous spaces. Finally, it follows that Ind va
[y Pj.
D.5 RECONSTRUCTING THE IRREP DECOMPOSITION: THE REAL CASE
In the real case, this decomposition is less straightforward.
Recall Corollary
Consider the of all basis elements for fixed value of j. i.e. (Jsite Jitr where:
fistr c " (TpP
with S my = E, Let f be Mackey function the span of these basis elements, i.c. there exists (wier E R Jirr such that:
f(g) = Σ
Wite (itr
str
E e ID Visps(o-))e,
str
E Vappylo ')
Σ
ism;
E ID! Vippslo Wwm
where [w., = Wate if & my and [wre]; otherwise. Unfortunately, this basis is not harmonic since it does not have an explicit action of G through Pj on the coefficients vectors. To see this, consider the function above transformed by an element KE G:
[k:f(()) Σ War [k. (jar)(())))
Σ War
Σ
The vector P5 (kk)w, does not generally belong to the span of {eili m, }. i.e the function k. as written above not expressed directly in terms of the basis in Corollary [ This means that the basis found is not harmonic basis yet.
45

---

Published as a conference paper at ICLR 2022
Nevertheless, know that the space of Mackey function has compact action of G and therefore must be decomposable in a direct sum of irreps of G. Therefore, there exists a change of basis from the current one to harmonic one. In the rest of this section, will derive variation of the Gram-Schmidt process to compute such change of basis.
First of all, want to identify the invariant subspaces spanned by the basis. To do so, re-project each basis element after an action G on the basis itself, i.e. we project the function g. fifsitr. with g E G on each other basis element [akaq- First, note that:
jg-fzarl(:: = c, ID Vdzps(k (dpp)(k=))pp())e, (p){e
= Σ c ID (epps(k*))e Pzd,
ie. g.fsatr belongs to the span of the set of functions {f i S dj} (note that S d, rather than i S. m, dy/E,). Despite the functions {ffstr | my < dj} do not belong to the basis in Corollary one can easily verify that they are actually Mackey functions and, therefore, must belong
to the space spanned by the basis in Corollary (5) Additionally, we will see that (fjär S d,)
contains functions which are orthogonal to each other; before showing this, we should define an inner
product of Mackey functions:
(J1. f2)c (1(0)) (111))))))))))
/ ((19)"52())))
Note that we assumed real functions and real representations.
We compute the projection of any functions:
Sikan) = -f [ (os(s)V,,lID)) [c oy ID (o
gEG
=e, P) Lo of w "dips(o)" ex
Lemma D.8. the following statements are true:
1. Otriag E (nda((;))
2. Otr,ag (Otrsrg - TrIDIN" ID I7)
Proof. 1) By definition, Osr, is the invariant projection of a matrix with respect to left and right action of G through p. It follows that Otr,ag commutes with p(g) for any E G; hence, Ozr,gg Endo(p;).
Let (h 11 SSS Ej} be orthonormal basis for Enda (p;) as in Sec. C Then, we can project
Ozr,ay on this basis:
1 Tr(Oun.qql))
(Oer,ags I1)
dj
1
Tr( I py(h) ",(") ID (h)p,())
d, neG
1 Thf hEG (d) o TD (1)
Tr(II ct MD It)
Hence:
Osr,ag Σ (0n.sg 11) = Σ Tr "" of ID If)
46

---

Published as a conference paper at ICLR 2022
Note that the coefficients (Otr,sq It) can be easily computed numerically since all the matrices involved are known.
Corollary 6. The following statements are true:
1. (Otr,egs I1) ògtOrg
2. Owar is the identity matrix
3. (ffatr faktt) 8sk
Proof. 1) By considering 1 1, i.e. 1 id the identity, in Lemma D.8 we find
(Obr,gg: I1) T=
e ID
TT(II [ID [ ["
Leveraging the orthogonality of ID and the fact that ID and ID t/ contain the rows of IDW;
Trrssa id[c,] c
There exists a unique r' such that ct" |ct17e of and ext. is not skew-symmetric iff r' = 0, i.e. r
detrrg
2) Note that the matrix Ozrtr is symmetric; since It for * 1 is antisymmetric, (Otrger It) = 0
1 1. Using the result it follows that Otrer id is the identity matrix.
3) An immediate consequence of this is that (ffstr: fsktr) e. Oirer 8ikk-
Note that the last proposition in Corollary 6 holds also for the functions in {fjat l my=
= d
which don't belong to the basis in Corollary 5 Intuitively, for fixed value of (t,r), the basis in Corollary 5 ignores these orthogonal elements since they are linearly dependent with the basis elements associated with some other value of (t,r). This explains why the form of the basis in Corollary 5 is not convenient for us. Using the projection coefficients Otr,sq we can compute the necessary change of basis.
To do start with the over-complete basis given by:
Usitr i S dj,r S Eg, S [Íb]]]
Recall that, for fixed value of r and 4, all basis elements Br, = {Jjarh J are orthogonal. We can therefore build basis by iteratively picking set Brr and project it to the space orthogonal to the one spanned by the basis elements already chosen.
Before making this more precise, we need to introduce few more handful results. Lemma D.9. For any S. Eg, t < [Rb]] and < Eg:
1. c ID I Hom (Res Pý. p))
2. there a set of real numbers (Wirsg,))s S.f.
c ID
Σ
Σ
c
DD Waragg
q E
3. Wtr.sg.
(0 (Otr.ag It)
Proof. 1) Note that ct ID Hoomo (Res Pss +) and It E Endo(p). It follows that, for any h H < G:
c ID 4p(h)- = c.th ID p(h)d;= ( 1S' 11 = (h))c 1D It
i.e. o 1D J commutes with the action of H: hence, it belongs Hom (Res Pj: 1b).
47

---

Published as a conference paper at ICLR 2022
2) This result is an immediate consequence of 1) by using the following decomposition
[+d]
Hom (Res Pj. 1p End (+)
and the basis {c l for Endx)()).
3) Since (cy 1 fq.s is an orthonormal basis for Hom,y (Resj) Pj. v)), the coefficients (waq.tr,))s.q
are found just by projecting the matrix on each basis element using a (scaled) standard inner product, ie.:
Wtr,sqd
(c
ID e. ID I)) Tr
t ID of
dj
Tr (ID;")=[C) oy ID If)
dj (Orn.g:41) l1)
Lemma D.10. Let (b:) a set Mackey functions defined as:
b.(g) = Σ
c
ID $
(o) e,bra
r.C
parameterized by the vector b = (br Let b(g) = (bi(9)) bb(g))... E xdd the matrix obtained by stacking horizontally the vectors (b:(9)) }., i.e.
b(g) Σ
c D (d
The set of functions [b is an harmonic family, i.e. f()) = b(o)f for coefficients vector
f e Rd, then [g) f 9) = (k)) In other words, the function k.f is the function parameterized by the coefficients f.)
Proof.
9) f(k f(k-1g) = b(kk g)f
Σ brac wid' (i,p,( 9
brac DD Vdips(o)² ()()))
= b())p((k)
Finally, define few operations which we will need to combine and project harmonic functions in our final algorithm.
Lemma D.11. Let a and b be two set of harmonic functions as in Lemma D.10 respectively parame- terized by the vectors and b. Let are all harmonic families as in Lemma D.10
and M Endo then, the following sets of functions
1. the set I defined as f(g = ar(gg)
2. the set Jf defined as f(g) = b(g) a(g)
3. the set J defined f(g) a(g)
48

---

Published as a conference paper at ICLR 2022
Proof. 1) This is simple consequence of the linearity of the harmonic property.
2)
f(g)
Σ c " Vaspz(a)
b,tt Σ o IDI vdips(9)* a,
Σ
c D Vdjp((o)
(bre aa,)) =ifn
3) Let M = E, mi4. Then:
a(g)M Σ
c I' (d)p)(9)(annd an
n
2 DD
ty vdsps(9)* ar. Emih mih
r,
Σ Lrst
c ( T1) (iih)d/gg (
Σ ar
Σ
c is War.ogd) Vispslo)"
Σ
Σ
2 "D' Vdsps(9)
Wer, t0,
2-
ER
where we used Lemma D.9
Corollary 7. Given two harmonic families and b, lel Oap - " b(g)dg be the matrix containing the inner product between each pair of functions in a and b. Then, Oub E Endolps) and Oa,b Σ Wa, where Wa,b. = E Osggbe, Wsg,rrl-
Moreover, let f be set of functions defined as f(g) a0a.v Then, Oa.f is the zero matrix.
Proof. The first result follows from:
Oub
a(g) b(g)dg
G
Σ (d (e Σ c IDI' Vdges(o)= bradg
5.g
rst
bngbra / Vo & IDI" (app;(o)
8.0
r.t
br.grr,tO. sq,5r
5.9
r.t
The second result follows from
O S( alg)))(()) f(g)dg ( (gg)) [
alg)dgoO0) =O,b -i 0 0.
Additionally, note that all functions within the same harmonic family b have the same norm, i.e.
Or = Σ 62 id
||9|| id
1.r
49

---

Published as a conference paper at ICLR 2022
where used the fact Ob,6 is symmetric matrix and, therefore, Wsq.tr,1 01. [[b]] is the (squared) norm of each function in b.
We now have all the ingredients deseribe the final algorithm. Let B { the (initially empty) set of harmonic families chosen so far. other words, an element b; of B is an harmonic family of dj functions in Lemma 1.10 and is represented by vector b; (.,,.,)), Our goal now is to find M₁ = E //)lEE E vectors }o; such that Ob,,b. = 6,,x id. To do so, repeat the following steps until the set B contains Mj elements:
1. pick a vector (br, which defines the set of harmonics b
2. define 6' = b Σb.Es' b, represented by the coefficients vector b' = (b/,,)e,r. where the coefficients can be computed using Lemma D.11 and Corollary
3. normalize b' as b = b/ //v ||
4. add b to B
Note that, each iteration, we add d, orthogonal functions to the new basis and that the original basis in Corollary 5sss Mj dy dimensional. Therefore, the algorithm terminates in Mj iterations.
E NUMERICAL IRREPS DECOMPOSITION OF REAL REPRESENTATIONS
Assume a compact group G and set of (representatives the isomorphism classes of) orthogonal real irreps G = {Di}y- In this section, we only engage with real valued representations. In particular, recall that, for real-valued irrep p, its endomorphism space (ndg(p) is not necessarily 1-dimensional, see Section
As stated in Corollary T any orthogonal real representation of G can be decomposed direct sum of irreps of G 20 The goal of this section is that of explicitly computing this decomposition for an arbitrary dimensiimenionnl orthogonal real representation of G. In other words, given an input dimite-imeeniionl (real, orthogonal) representation we want to find the orthogonal transformation/matiix M and the multiplicities {[i N), such that:
M
4 GEG 1=1
+
(g(
MT
Thereby, being orthogonal means that the scalar product is preserved, which for matrices means that the matrix is orthogonal in the classical sense i.e., its columns build an orthonormal basis.
Warning: note that, with respect to Corollary and the notation used in the main paper, in this section M is transposed on the right-side rather than the left-side of the direct sum of irreps. To obtain the decomposition used in the other sections, one just need to transpose our final result. The reason why we consider this notation is simply that this the convention we used in our implementation and we wanted to keep the following algorithm consistent with our code.
In this whole section, we assume that π and all Øj are matrix representations, and correspondingly, that M is given by a matrix.
To simplify the notation, define P = +, D.p,. such that ग MPMM". Since M orthogonal, can write æ M .P,, i.e. M needs to commute with a and P. follows that M e Homg, m).
Thah corollary speaks of unitary representations. But recall that by our convention, "unitary" means "orthogonal" when the field is the real numbers.
50

---

Published as a conference paper at ICLR 2022
Recall that the block-diagonal structure of P induces block-structure in the columns of M, visually:
Mýi
P(g)
(g)) = [..IMail-.. |M (ell....-Mmml
1inll---Mill.-MMMaaaa [Mbxl....
Pz(g)
MJ
p5(9)
M [imm]
ie., we split M in a number of vertical blocks, one for each occurrence i of each irrep Pj in P. From the equation MM= M P we obtain the following: each such block Mje satisfies x Ma Mjipj. i.e. М5 E H0m )- We restrict our attention to this case. Thus, from now on, assume that is a real orthogonal irrep of G. and our goal find an orthogonal embedding in Homa(p, IT), where IT is not necessarily irreducible.
Note that Homg x) = [i=]
(pdg(p). We can easily find basis for Homo x) by solving linear system. Let dj be the dimension of the representation space of pj. and similarly, dx be the dimension of the representation space of æ. Let T the dj x dj identity matrix and I the dx x d, identity matrix; then, the matrix Mfs should satisfy:
Vig G. Vig E G,
T x M9sp((9)
x M I=M,iP,
vectorizing both sides of the equation and using vec (ABC) (C* & A) vec (B):
bg E G; Bg G,
(1, s (g)) vee (M₃²) (p)(9) x [T) vec (Mzi)
(J x()) - p;(5)² x Ix)) vec (M₃z) = 0
where + indicates the Kronecker sum here (not to be confused with the direct sum). Note that the last line above describes linear constraint on My: for each group element 9 E G. One can solve for the space of suitable Mys by stacking vertically the matrices (T₃ 3 (g) (g) @ I=) for each E G and then solving for its null space (e.g. through SVD). While this approach is feasible if G' is a finite group and |G] is sufficiently small, this can not be used for infinite groups (or large finite groups). If G is a finite group, one can easily verify that it is sufficient to consider the matrices (4, & (g) p. (g) @ Lx) associated with the set of generators of G. Indeed, note that if M5 satisfies the constraint above for two group elements a and b G, then it satisfies it also for ab and ba G. Finzi et al. (2021) follows precisely this approach to compute a basis for a general space Homg(TI, π2). when G is finite. When G is an infinite Lie group. Finzi et al. (2021) shows that it is sufficient consider the Lie algebra generators of G to solve this constraint. In our work. however, we do not want to rely on the Lie algebra of G, since would introduce additional non non-trivial requirements when implementing a new equivariance group G. Instead, we note that it is generally sufficient to consider a few random samples G C G to generate enough constraint matrices {I, @ :r(g) - P)(g)T @ I= E g) such that the null space of their vertical stack is precisely Homg (P5. T).
Note that I, @ (g) (g)) @ I==
① [g). (g)
similar method was used already in Weiler al. (2018a) to compute the Clebsh-Gordan coefficients in the decomposition of the tensor product of the irreps of S3(3). These coefficients appear in their kernel basis precisely as in our vectorized kernel constraint in Sec. 2.2 In Weiler Cesa (2019) (Appendix G), the authors discuss the cost of this approach when applied on generic input and output representation pairs, and highlight the benefits of, instead, restricting the consideration to only pairs of input and output irreducible representations, as done here.
Therefore, by following the method just described, we can generate an orthonormal basis for the space of vectorized matrices satisfying the constraint above. By un-vectorization, any element in the space spanned by this basis generates a matrix Mje such that TM, = Mjpp.. This is not sufficient yet for us, since require the whole matrix M to be orthogonal.
51

---

Published as a conference paper at ICLR 2022
From now, we assume the previous method generated an orthonormal basis Bj [B J for Homg (Pj, x), i.c. a collection of = [j E matrices of shape d, x d, which are orthogonal with respect to (scaled) standard inner product (A, B3 Jे Tr(AB") Jे vec (A)" vec (B). Recall that Fg is the dimensionality of Endg,R(P;)-
In order to build matrix M which is orthogonal, for each j. we need to choose [iv]] orthogonal such that all columns of all matrices M5; are also orthogonal to matrices ( {M [ in Homo
each other.
Lemma F.1. Any matrix A1 E Home(pp) (P:: T) has columns which are orthogonal to the columns of any matrix A2 E Homm T). for J1 + j2.
Proof. This is a direct consequence of the fact that any homomorphism in Ai E Homma(jj X) maps to a subspace V1 C Va which orthogonal to the image V22 of Ag E Homg π). This can be seen by realizing that the projection from V1 to V/ is intertwiner, and thus needs to be zero by Schur's Lemma since V1 and Vz2 are not isomorphic.
Lemma E.2. Any orthonormal basis for Home π) contains matrices which are orthogonal, i.e. such that the columns are orthogonal and normalized.
Proof. Let {E
Er be an orthonormal basis for Endg(p). We know that there exists a matrix M Therefore, there exists collection
such that 1) M is orthogonal and 2) π M + p (
M 21
( {M i=1 of orthogonal matrices in Homo x),, where Mjs the blocks of the matrix M before.
follows that there exists a basis for Homg (p, т defined as HH = (Te 1 < k S Eg, 1 S S [i<]]. This basis is made of orthogonal matrices, since both 1₁ and Mj; are orthogonal.
Note that we don't 't know M and H., but we only know they exist. Assume H = [H is a basis for Homa (p, л) made of orthogonal matrices, i.e. for each i, H. H, = T. More generally H H: = 5g
Then any orthonormal basis for this space is made of orthogonal matrices. Indeed, assume B {Bi) is a basis such that (B,, Bi) = 1. An element B₁ can be written as Br E. by H4 It follows that for any element Bt:
Σ bi.HH: )²
(I bi
B Bi =
Σ
bu bi, H H
1132
Σ
biabi
(B, BT
iee any element of an orthonormal basis for Homg, x) must contains matrices which are themselves orthogonal.
follows that we only need to focus on finding a collection [M₃c] [im] of matrices whose columns are orthogonal; the orthogonality of columns within the same matrix My: and between different matrices MA and M,2.12 with j1 * j2 is already guaranteed.
Recall that B, = {B,,), dd is the orthonormal basis for Homg л) generated by the numerical method (e.g. SVD). To find suitable {M it is sufficient to set M,0 Bjo- Let M, the subset of E, be an orthonormal basis for Endg, (P;). Let B By. Then we build the set M, through the following iterative method:
all matrices {M,e} h already set (initially containing only M,o). Let {1x {
1. create B" = [M, E Mi, E
2. project the elements in B, on the space space of matrices which is orthogonal to the span of B' and remove all zero elements (i.e. all elements which already belong to the span of 13')
This direct consequence of Lemma B.29 in Lang & Weiler 2020
52

---

Published as a conference paper at ICLR 2022
3. pick the first non-zero element of Bi, set Mys to that and add it in M,
4. repeat until the set B; is empty
Up to a scalar factor (of v dj). the matrices in My are all orthogonal. Additionally, this set contains precisely [i>] matrices.
By stacking all matrices together, one builds the matrix M.
Note also that this method can be used to find the multiplicities [j too. Indeed, because each column of Pj is related through an endomorphism precisely with Es other columns (see Theorem CCIII) once a basis Bj with elements found, it is sufficient to compute [im] = d/ Ej. since we assume the rype of Pj (and therefore Eÿ) is known.
F REAL IRREDUCIBLE REPRESENTATIONS OF THE DIRECT PRODUCT OF TWO COMPACT GROUPS
useful operation to generate new groups is the direct product. Given two groups A and B, their direct product G = A x B is group defined as:
the Cartesian product of the element of and B, i.e. {(a,b) a A
with the group law G x G + G defined (a1, b1) (a2, b2) = (a1a2, b2b2).
Many groups can be written as a direct product of two smaller groups. In particular, 0(3) = Inv x SO(3), or more generally O(n) Inv (O(n) when
is odd, where Inv {+1] is the group generated by the reflection x ト -r. In our work, we are interested in the direct product to easily build many subgroups of (n) (in particular, of 0(3) which have form G = Inv x XH, where H < (O)... This is the case for the cylindrical symmetries (H = S0(2), O(2) or their discrete subgroups) or the full isometries of the solids (H T. O I); Sec. G
In order to use these groups, it is sufficient for us to build their set of irreducible representations. In this section, we deseribe a method to generate the set of real irreducible representations of direct product of two compact groups from the irreps of the two groups.
Given two compact groups A and B and their respective set of (representatives of) complex unitary well known that any complex unitary
irreducible representations Ā = {as); and B = {5 unique "% ar @ 33 for a particular choice of and j. Thereby, (a, @ (a, b) == (a) @ (y(b) is the Kronecker product of the two matrices (a)() and (2(b). Thus, set of representatives of isomorphism classes of unitary irreps & can be constructed as
irrep of G A B is isomorphic
G = [rg a, @ 33 I a. E A, 33 E B}.
(26)
However, this results does not hold in the real field. In the rest of this section, we will derive variation of this theorem using real irreducible representations, which is more useful to describe the representations of the groups used.
In Appendix C.3 we have already shown that any real irrep e of can classified in one of the following three categories:
real-type: & 2020
complex-type: oB
with 2
quaternionic-type:
as
where a is a complex irrep of G. Moreover, for each complex irrep o,, there exists only one real irrep which contains it.
We can use this classification starting from the complex irreps of G which we have already defined in Eq. 26 First, clarify some notation. We use Greek letters like o, 3,87 to indicate complex irreps and Roman letters to indicate the corresponding real irreps a, b, c. For each complex irrep "/i,, there can be only real irrep Ct containing it. It follows that a, B3 only appears in Gjj. Moreover, if 0g # Og, i.e. as is of complex-type, index the complex irrep a; with i, i.e. og 2 04.
53

---

Published as a conference paper at ICLR 2022
We now introduce useful result for real irreps:
Definition F.1 (Frobenious Frruer Indicator). Let G be compact group and complex irreducible representation of G. Let X be the character of p. i.e. : + C, Tr(p(g)). The Frobenious- Schur indicator 'p of p is defined as:
4p = 1 x(oo)dg
This indicator can only take the following values:
is contained in real rype irrep "p- O if is contained in complex type irrep
if is contained in quaternionic type irrep
This allows us to analytically determine the type of the real irrep associated with complex one. We combine this result with the definition of complex irreps of G A B in Eq. 26 to find the type of the real irreps of G. Let Yij = a4 & 3g a complex irrep of G; then:
Los Tr((a B,)(g)))
= JAJ Tr(au(a") @ (,(())) db
Using the property of the trace and the Kronecker product:
Tr(ar(a")) (r(3) db
Tr(aa(a)))da / (r(3,
Hence, the types of the irreps and 33 of A and B fully determine the type of the irrep My of G. This can be visualized in the following table:
ai Bg real La.
real LB, = complex 13, = 0 quaternionic L3, = -1
Lyg 1
Lyg 0
Lyg --
complex to, 0
Lyg
byy
byg 0
quaternionic Lo. Lyg
Loys =
byg = 1
In the rest of this section, will use the following useful facts:
Re (7ag) Re (a: @ (3)) Re (as) Re (B,) Im (ax) Im (33)
Tm (755) = Tm (o: @ (3)) Re (os) @ Im (8,) + Tm (a:) s Re (33)
We will also assume the irreps to be expressed a (or B,) are real type irreps, we assume they are expressed with respect to basis such that they have real entries, i.c. a, a; (and By 6,). In such case, Re (a,) a, and Im (a,) If a; (or B3) is complex quaternionic type irrep, we assume a; (and by) is expressed as the realification:
the bases described in Section
In particular, if
Re (a:) Im (a:) Re (as)
a; =
Im (a:)
This enables extract Re (a:) and Im (a:) directly from the matrix coefficients of a; (or b,).
Therefore, to build the real irreps of = B we distinguish three cases, depending on the type of the irrep.
54

---

Published as a conference paper at ICLR 2022
Real type irrep Assume Yi has real rype. This implies that either both a, and 33 have real type both have quaternionic type. In the first case, Jij is simply computed as 9sj = a, @ by. In the second case, assume a; and 3y have the structure in Eq. [7 Then, one can verify that the change of basis
0
iI
if
0
1
i1
0
Q
V2
0
-1
T
0
T
0
turns Yy = as @ B3 into a matrix with only real entries, i.c. 9ij = QuaQ where / is the identity matrix of size dimyy /4.
Complex type irrep tion:
Whenever Tij has complex type, we can easily implement Pij as the realifica-
Re (11) Imm (717) 9bj Im (7k)
Re (71g)
By using the identities above, we can easily compute this matrix from entries of the matrices aj and bj. Note that, both a and 33 have complex type, the irrep "ij = a, 8 3j and 7ij - 0g & 3 generate two different real irreps Piş and 955 Since by and by are isomorphic real irreps, only one of the two belongs to the set of representatives in G chosen. However, Re (83) Re (1y) and Im (3,) = Im (3,), so 9.j can still be computed easily from the coefficients a, and bj.
Quaternionic type irrep Like for the complex types, whenever Ki has quaternionic type, we can easily implement Pij as the realification:
Re (16) Im (7t)
Im (7h3) Re (7as)
Additionally, note that this is the case only if either a or 3, of real type, while the other of quaternionic type. Hence, the formula can be simplified by noting that either Im (ax) or Im (3) is zero.
G 3D SYMMETRIES, AZIMUTHAL SYMMETRIES AND THEIR RELEVANCE
E(3) is the group of isometries in R33 and includes translations (R3, +), rotations SO(3) and in- versions through the origin Inv. The group 0(3) = SO(3) Inv the compact group of origin-preserving isometries. While continuous rotational symmetry SO(3) occurs commonly in molecular data, smaller symmetry groups are sometimes more relevant to describe scenes or objects in natural environments. For instance, when the vertical orientation of the scene is obvious, the symmetry group of the data is reduced to axial symmetries. There is only a limited number of topologically closed infinite subgroups of O(3) (up conjugation). The building blocks rotations along an axis (2)(,)) inversions (Inv), mirroring with respect to plane (M) and an out-of-plane rotation by æ (F). Together, they generate 5 continuous symmetry groups:
axial symmetry (SO(2) < S0(3)),
dihedral symmetry (O(2) S0(2) x F SO(3)).
conical symmetry (O(2) = SO(2) x M < 0(3)), cylindrical symmetry (Inv S0(2) 0(3) and
full cylindrical symmetry (Inv (SO((2) x F) < 0(3)).
In the planar case. Weiler & Cesa 2019) observed benefits from using discrete groups rather than continuous ones when the data is discretized on a grid and, therefore, the continuous symmetry of the data is lost. The only discrete subgroups of SO(3) with more than 2 rotation axes are the symmetry groups of the Platonic solids: the tetrahedral (T), the octahedral (O) and the icosahedral (T) groups which have, respectively. 12, 24, and 60 elements. T and O are perfect symmetries of voxel grid and were previously used in Worrall & Brostow (2018); Winkels & Cohen (2018). Also, note that < but O hence, an l-equivariant CNN is not equivariant all symmetries of the voxel grid.
Tnv 2 Co is group containing the identity element and the inversion
R.
55

---

Published as a conference paper at ICLR 2022
H FURTHER DETAILS ON NETWORK DESIGNS AND EXPERIMENTS
H.1 ESTIMATING THE EQUIVARIANCE ERROR OF STEERABLE BASES
In this section we provide more details on Fig. 5 In this experiment, we considered equivariance to the Icosahedral group G = I. We employ 5 x 5 x 5 filters, parameterized by two spherical shells (one at radius and one at radius 2) and the origin. In the two baselines, we use the orbits containing the vertices of either the icosahedron or the dodecahedron in the two spherical shells and the orbit containing only the origin. The kernel is parameterized independently along each orbit by directly using the Wigner-Eckart theorem from Lang & Weiler (020) and, then, the kernels are embedded in I23 by means of small Gaussian smoothing kernel. In our basis, we parameterize each shell with spherical harmonics band-limited to frequency (the origin contains only frequency 0); each shell is diffused along the radial direction with a small Gaussian kernel.
Each histogram in Fig. shows the relative equivariance error of each element of one of the bases. An histogram contains a point for each filter ls in the basis associated with each pair of input and output irreps (pin, Pout) of G I. The error of basis filter k is computed the average over all 60 rotations E I of the following error:
gx - downsample(g.XX)
+ downsample(X))
fgx t conv3d(qx, k)
fx < conv3d(x, k)
gfx < Pour (g) fx
return compare gfx)
where (and gx) have the same resolution of the filter k such that the outputs fx and fgx do not have spatial resolutions. In this way, acts on fx only by transforming its channels but no spatial interpolation necessary. To reduce the interpolation artifacts when rotating X + g.X, we perform the rotation at a higher resolution and then downsample the inputs by a factor of 3.
H.2 REGULAR AND QUOTIENT NON-LINEARIN
Quotient and regular-type features encode square-integrable scalar functions over a homogeneous space Q G/ H of G' (with H {e} and Q G for the regular case). Whenever Q G/H is infinite, one can rely on a bandlimited representation of these functions the Fourier domain, using the harmonic bases we derived in Section D More precisely, bandlimited functions are parameterized by using a finite subset of the harmonic basis in Corollary 5 considering only those j in a subset G G. Then, the action of G on function turns into the representation Po ① jeū + M, Pj acting on the coefficients vector which parameterizes it, where M, is the multiplicity of the irrep Pj- Given one such vector, the function parametrizes can be sampled at any point g E Q by evaluating the harmonic basis on
In neural network, a feature vector f(x) E Rdd of quotiens (or regular) type p is interpreted as the coefficients vector parameterizing band-limite function in (Q) = (²(GHH) on a basis, which transforms according to p.
A point-wise nonlinearity a R R (e.g. ELU) is applied on quotient feature f(x) : R by composition, obtaining a new quotient feature fz) defined as (f) = ()))))())) When Q is infinite, if fzx is band-limited and is sufficiently smooth, we can recover approximation of f((x) from a finite number of samples.
Approximate noie non-linearity Let Q Q be finite number of samples from the space Q = G/H. We define the non-linearity as
[())) FT o (f(x)
where FT is a Fourier Transform, which recovers the coefficients of band-limited function from number of samples, o is the pointwise non -on-eiicarrty and IFTT is an Inverse Fourier Transform, which just samples the function parametrized by the coefficients fxx the elements in Q. Note that IFT
56

---

Published as a conference paper at ICLR 2022
can be implemented as simple multiplication with matrix of shape Q x d, and, similarly, FT can be implemented as a multiplication with a matrix of shape dp x [2], hence
f'(x) = FT o (IFT)
where FT and IFT are interpreted as matrices. In practice, the matrix IFT is implemented as a matrix whose rows contain the Fourier transform of Dirac delta functions centered at each sample in Q, while FT is implemented as the pseudo-inverse matrix FT IFT This operation is only approximatively equivariant and the degree of equivariance strongly depends on the number of samples | as well as their distribution over the group (and the smoothness of σ). In particular, is important that the samples in Q uniformly cover the space Q = G/ H and are, therefore, as far away from each other as possible.
To generate sampling set Q, one can use the elements of discrete subgroup of G (e.g. Icosahedral group < SO(3) or optimize the location of [ points on Q by minimizing a potential energy, as done by Bekkers (2020).
Computational benefits A typical feature f in our models includes multiples fields of the same type, i.e. == +2 fi and p= DD Pi. with P; = P3 Vi Assume a sequence ConvND - a - ConvND in the model, where all layers operate
feature fields of type p. Let C = E. dp. be the size of the feature vectors (xx E RCC In our neural networks design, we ensure that the number of channels in each layer is approximately constant across different models. In particular, in models using quotient (or regular) feature types, count the number of samples Q as channels and, therefore, keep the quantity Σ: 121 ~ N approximately constant. Let a = 12) this implies that the first ConvND only has Σ, dp, = N/a output channels (and the second ConvND has this many input channels). Let be the cost of a single convolution with 1 input and output channels and m the number of pixels in the input grid. The cost of the non-linearity applied to a single feature field is (2) and there are N/1 such input fields. Then, the cost of this layer is only OkmCN/a + ( O OkkCCN/a) since mC >> dp.. For comparison, the same block in a conventional network has cost OkmmC m.N) OlmCNN, hence this layer approximatively a times cheaper.
O (kmCN/a + mNdp,) which is equal to
In Section H.7 we compare the approximate inference time of different architecture to verify the computational benefits described above.
H.3
ARCHITECTURE DETAILS
All models used in our experiments consist of a sequence of a basic inverted-bottlennckk residual block, small variations of it (see each experiment's section). The basic block for a conventional network consists of a sequence Conv3D Bat ch ELU -Conv3D, whose input and output connected with a skip connection. The block has C.n input channels and the first convolution maps to N >> Cin channels. To downsample, we used stride in the second CDnv3D and average pooling in the skip connection. We use kernel_ i ze 3 for both Conv3D. If the block has different number of channels Cout in output, the skip connection also contains a convolution with We refer the linear path in the architecture, i.e. the sequence of skip
kernel_siie size connections, as the backbone of the model. If Cj is the number of input channels of the j-th residual block, we refer to [C1, .2....] as the number of channels in the backbone features of the model. If N2 is the number of channels used by the non-linearity of the j-th residual block, we refer to N2,,N,.... as the number of channels the residual blocks of the model.
In equivariant model, we associate stack of field types {p: Ja on the input Cin channels, such that Σ. Cmn- Similarly, we associate a stack of field types to the N channels produced by the first Conv3D. The particular choice here depends on the kind of equivariant non-linearity used here instead of ELU. In general, we try to ensure that the channels used in the non- non-linearity approximatively equal to N, for a fair comparison. We now list number of non-linearities considered.
Gated Non-Linearities The input consists of a number of feature fields f(x) = D f(( of type D P:- Recall that fa()) E Rd.. For each feature field i, the input should also contain a gate De (x))) IR of trivial type. Then, each input feature field fe(x)) is mapped to f'(x) a (()
57

---

Published as a conference paper at ICLR 2022
where is non-linearity like S Sigmoid. Note that gated non-linearity inherently loses some of its input channels, i.c. the input gates {9: h- In our models, we try to ensure that Σ, (dp, + +1 = N (the +1 corresponds to the size of the gate (z))) for each input feature field f.). Note that a field type Os can be any representation. In particular, we experiment with different choices of pæ; we consider
{pi): containing:
each irrep of (3(3) of frequencies 1 to 2, in equal proportions
each irrep of (3)3) of frequencies 1 to 3, in equal proportions
with multiplicities proportional to their multiplici- ties in the regular representation of SO(3), i.c. for each trivial po, we include 3 copies of
each irrep of SO(3) of frequencies
the frequency P1 and 5 copies of P2
copies of the quotient representation Psa po @ p @ P2, band-limited up frequency 2
copies of the quotient representation Pga Po 2 p2 ps, band-limited up to frequency
copies of the regular representation Preg Pa
n3 pi
p2
band-limited up frequency
To match the N channels, we also add trivial feature fields (p: = Pu) and operate on them with simple ELU non-linearity. Note that in all cases, the first Conv3D has Cn input channels and N output channels, maintaining approximatively the same computational cost of the conventional CNN.
Tensor-Product Non-Linearity Let f(x) = ①. fa) be the input features field, with type p = D Ps- In our experiments, we always chose all {p:}s equal to unique type, i.e. Ps Pj for all í.j. The tensor-pret produutt non- con-liieyr maps fx) E R&; to f((()) = fi(x) @ fi(x) E R Note the quadratic growth in the number of channels. In the experiments. we keep the output size approximatively equal to N, i.e. E.d. & N. It is common in the literature to learn a linear projection of (x) to lower-dimensional feature field (x)) of type p/, with de. dp.- The second Conv3D originally maps Σ.
N channels the output Cout channels, while this projection would reduce the input channels to E. < N, thereby reducing the computational cost of the layer proportionally. This, however, results also in potential loss in expressiveness, since Conv3D layer taking N channels in input can learn a larger space of operations. Since we do not expect this linear projection to improve the performance of this architecture, we do not include it in our experiments. Finally, in our experiments we consider the following choices p;:
the quotient representation Psa Pu P1 P2. band-limited up frequency 2
the quotient representation P5² Po @
P1
① P2 ① Ps. band-limites up to frequency 3
93 the regular representation Preg Po @ pl
95 band-limited up to frequency @ pa
Note that a larger feature type Pt (i.e. larger dp.)) results in smaller number of independent copies of it, since constrain E, dl N. On the other hand, larger P: makes the tensor-product fz) @ (z) more expressive, since combines more different input channels. Therefore, the choice of the size of p; should be trade-off between expressiveness and computational and memory cost.
(Pointwise) Regular non-Linearities Again, we consider all input feature fields of the same type, i.e. 9; = Pj for all i,j. Here, fi()) E Rde; is interpreted as the coefficients vectors parameterizing a band-limited function in LG),,, as described in Section H.2 We consider band limiting up to frequeney 2 or 3, i.c the three following choices of p::
P = 4 P1
Pt Po b P
33
35
P: Po b P
We experiment with different sampling sets g C G:
the elements of the Octahedral group O < S0(3)
the 60 elements of the Icosahedral group I (SO(3)
58

---

Published as a conference paper at ICLR 2022
P points over G with maximal relative distance, obtained by optimizing potential energy
P P= 24 - |S] points over G with maximal relative distance, obtained by optimizing over (oeoloogle SJ. such that & has Octahedral
S G a potential energy over
symmetry: for any E O, o.g G.
In particular, the last sampling set guarantees that the resulting neural network is at least perfectly equivariant to O, i.e. the symmetries of the voxel grid. In general, we observe that at least |g] = 2dp. samples are necessary for achieving an equivariance error below 10%. For G = (()), we use similar design and just augment the sampling set with inversions, i.e. we use G = Inv x9, where G a sampling set used for SO(3..
We argued about the computational benefits of this design in Section H.2 For discrete groups such as and I, we also use regular non-linearities. In this case, however, the first Conv3D in the residual block directly use a regular feature types output types and is immediately followed by the pointwise non-linearity, with no need to perform sampling. Hence, the computational and memory cost of these models is equivalent to that of conventional network, i.e. they have no additional computational gain.
(Pointwise) Quotient non-Linearities Similarly, we define quotient non-linearities, with the only difference that fi(z) parameterizes a band-limited function over a homogeneous space Q = G/ H, for some H < G; Section H.2 For G SO(3), we consider the following quotient representations p;:
Psa Po
P2
Ps
Po
Pa
and the following sampling sets
the (center of the) 30 edges of the icosahedron; note that this set has Icosahedral symmetry
points over Q with maximal relative distance, obtained by optimizing a potential energy
24 24. |S] points over Q with maximal relative distance, obtained by optimizing over a set S C Q potential energy on Q = UpcO fo-9/q E S). such that Q has Octahedral symmetry: for any O, o
In particular, the last sampling set guarantees that the resulting neural network is at least perfectly cquivariant to O, ie. the symmetries the voxel grid. Additionally, note that for Q = S the second sampling set corresponds to the well known Thomson problem. When G O(3).
consider Q = (3)) 0(2) & S and Q = 0 S0(2) N Inv x In the first case, we can reuse the same sampling sets defined above for Q SO(3)/ S0(2) = S In the second one, augment the sampling set with inversions. The considerations about the computational cost discussed in Section H.2 apply also here.
Invariant Maps Because we consider invariant tasks, an equivariant architecture should map to invariant features before the final classification layer. To do so, it common to use some invariant map at the end of the convolutional feature extractor and then feed the invariant features in a few final culne-con layers. Here, we consider a few different invariant maps. Norm-pooling: given an input field fz(z) E Rdr, we compute its norm, i.e. f((x) + f(( = [lf((z) E R; note that this indeed an invariant quantity. We use this kind of map for models which should be perfectly G equivariant, if G is a continuous group; for instance, we use it for models which use gated non-lincarities or tensor- aasso-pprouuc non-linearities. We experiment with different choices of pt:
the quotient representation Psa Po + P1 P2. band-limiee up to frequency 2
the regular representation Preg Po @ Pi 43
band-limited up to frequency
the regular representation Preg Po @ p .3
@
P2
@5 band-limited up to frequency 2
{p:} ) contains copies of the irrep
and P2 in equal proportion
Group Pooling: if G a finite group, we can choose P; = Preg and, therefore, f((x) E R|o| In this case, compute the max fx())) + (x)))= Something similar can be done if G
59

---

Published as a conference paper at ICLR 2022
is continuous and Pj is this operation is often more unstable. To devise a more stable alternative for continuous groups, consider Average Pooling. This operation is identical to the regular non-linearity explained = FT oa o (f+ with the difference that the final Fourier Transform FT only recovers the invariant frequeney 0 component. This is equivalent averaging the output of lo o [f)) over the samples G. In our architectures, we keep the number of invariant features fed into the final fully-connected layers constant. Note that, to generate N invariant outputs, the invariant maps require Ndp, inputs. This can be quite expensive if P: is regular representation (e.g., regular representation of G S0(3) land-limited to 2 has size 35). To alleviate this, we introduce Quotient Average Pooling. This operation is similar to Average Pooling but relies on quotient representations (and samples X' from a quotient space X), which have smaller size.
regular representation by using sampling set G. However,
above
H.4 MNIST EXPERIMENTS
As a proof of concept, we train a conventional and a Cs equivariant model on rotated MNIST. A model includes a sequence of 6 residual blocks similar to the ones described in Section H.. With respect to that design, here we use Conv2D and use stride 2 in the second convolution. We downsample (with stride convolution), every two blocks. In the first layer of the first block, we use kernel_size = 7. The number of channels in the features of the backbone, connected through the skip connections, is C [1. 21, 54, 72, 108, 168). The number of channels each residual block is N = [96, 192, 288, 288, 576, 576]. Both models map to 128 final invariant features. The conventional model uses final 3 max pooling over the spatial dimension. model, we do not pad the convolution in the last residual block such that its output already has 1 x 1 resolution. However, the last block maps to 128 copies of the regular representation of G = Cs; we then apply group pooling over them to generate 128 invariant features. Finally, both models use 2 fully connected layers to perform classification.
the equivariant
We train both models using Adam. We performed simple tuning of batch size, learning rate and weight decay of each model independently by evaluating them on the validation set. The same number of syper-parameter combinations was chosen, we re-trained a model using this configuration 5 more times using different random seeds. We report the means and the standard deviations of the test performance over these runs.
evaluated for both models. Once the best set of dyper-parameet
H.5 MODELNET 10 EXPERIMENTS
To generate the two ModelNet10 datasets, we use the original mesh data. Each mesh is centered and scaled to fit in the [] cube. Then, for each mesh we apply 3 random rotations. In the normal ModelNet10 experiments, we only sample SO(2) rotations around the 2 axis. In the rotated ModelNet10 experiments, sample arbitrary SO(3) rotations. A mesh is then rendered by computing the signed distance function finction of each point in the voxel grid from the mesh. This distance is then used into a Gaussian kernel to generate smooth 3D occupancy grid of resolution 33 pixels.
The architecture used consists of a first Conv 3D with kernel_ size = 5, followed by residual blocks and final Conv3D layer. The second convolution layer of the 4 residual blocks uses stride 2. All models map to 128 final invariant features. The last convolution layer has kernel _ ze = 3 In the conventional networks, the last convolution layer has 128 output channels and uses padding so its output has resolution 3px; we apply max pooling with a window of size 3 to generate 128 translation invariant features of resolution pixel. In the equivariant networks, the last convolution uses padding 0 such that the output resolution is already 1 pixel; however, the last block maps to a larger number of channels 128- P. depending on the invariant map chosen. We choose this design to maintain the computational and memory cost of the equivariant and the conventional models comparable; indeed, the final features of the conventional network have shape 128 33 while the features of the cquivariant one have shape 128 P x 13, before applying, respectively, max pooling or an invariant map. We try to keep P x 3 = 227. In the architectures, we use N = [240, 480, 480, 960] channels in the residual blocks and C = [39, 78, 240, 480, 312] channels in the backbone. Finally, three fully connected layers, alternated with dropout, batchnorm and ELU non- non- con-aiierr,ty, use these invariant features to perform classification.
For the SO(2) model, we used regular non- con-linearities with regular representations to frequency 3 and [ 12 samples on regular grid. Similarly, for the 0(2), Inv SO(2) and
up
60

---

Published as a conference paper at ICLR 2022
Inv x (() models, we used regular representations band-limited up frequency 3 and & was built by extending a grid of 12 points on their SO(2) subgroup with reflections and flips. In all cases, as invariant map, used quotient average pooling, with X = S = SO( and maximum frequency 3.
In the S3(3) models using regular non non-lincaritics, we used features band-limited to frequency when [ 60 and frequency 3 when [g] = 120. For the O(3) model, used regular features limi-li up to frequency 2. both SO(3) and ((3) models using quotient features, we used features band-limited up to frequency 2.
For each equivariant model, the choice of feature types in the backbone (the input and output types of each residual block) is independent of the non-linearities used. Moreover, since no non-linearities are applied on these features, w.l.o.g. we can consider only irreps feature types. We consider the multiplicity of the irreps hyperparameter which we tune during hyper- parameter tuning.
We train all models using Adam. For each model, we independently performed simple tuning of batch size, learning rate and weight decay by evaluating them on the validation set. This hyper-pane syper-parameter search also includes a search over the variants of each architecture design (see the variants of each non-linear operation described in Section H.3). The same number of hyper-parameter combinations was evaluated for all models. Once the best set of hyper-paree prrpe--prameters was chosen, we re-trained model using this configuration 5 more times using different random seeds. We report the means and the standard deviations of the test performance over these runs.
H.6 AUTO-ENCODER EXPERIMENTS ON MODELNET
For completeness, here we extend the comparison between the steerable bases discussed in Section 5.2 with an auto-encoding experiment with ModelNet10 data. In this experiment, we use the rotated voxel data from Section H.S and train a model reconstruct its own input. We consider an auto-encoder architecture whose encoder and decoder comprise 5 blocks, each up/downsampling the features by factor of 2, such that the latent features have no spatial resolution. Each block residual block including sequence Conv3D BatchNorm all convolutions have kernel 1 1_si ze = and paddí ng = 2, and the last Conv 3Dt////3//r/aaaaaaaeeeaa ransossedd has stride = 2. The residual connection down/upsamples with Gaussian kernel followed by learnable x 1 x 1 convolution. Finally, the model includes an invariant map and Conv3D layer to predict single value at each voxel cell.
ELU followed by either Conv3D or
Conv33Dransposee
The outputs of the encoder's 's blocks have in order approximatively [30, 90, 240, 540, 1080] output channels, while the decoder blocks have [540, 240, 120, 90, 50] output channels. The ELU modules within each block have about [180, 360, 600, 840, 1080] and [1080, 840, 600, 360, 180) channels.
The models are trained the data from Section H.5 which include random rotations of each mesh. Testing is performed on a subset of the test meshes but each mesh is rotated by each element of the icosahedral group (i.e. 1| 60 times), since the purpose is verifying the equivariance of the model to the icosahedral group I. As in Section 5.2] we consider two versions of the same model which differ by the choice of steerable basis employed. The following table reports the test mean-squared errors (MSE) of the two models.
Description
MSE
Icosahedral Symmetry
0.785 1: 0.130
Icosahedral Symmetry (finite orbits basis)
0.948 0.004
Once again, we find the model using our anti-aliased basis outperforms the baseline using finite- orbits basis.
H.7 COMPUTATIONAL COST AND INFERENCE TIME OF THE MODELS
In Section H.2 we have argued that point- wise non- non-linearities based on discrete Fourier Transform and inverse Fourier Transform benefit from lower computational cost due the factorized linear layers. In this section, we verify this by comparing the inference time measured for different architectures. We measure inference time by feeding 100 batches of 8 random 33 33 33 inputs
61

---

Published as a conference paper at ICLR 2022
(we first feed 10 more batches for warm-up). We report mean and cundard--eviattee over these runs in Tab. [4] The architectures used are the ones described in Section H.. [nd For reference, we also report the number of parameters of each model. Note also that the equivariant models include small overhead due to the less optimized implementation of certain operations: while the T model has approximatively the same running time of conventional CNN, the gated non-linearity model and the scaled-up regular model (marked with x) are slower that the first two. Despite this overhead, the normal regular (3) model (without x) much faster than all other models.
Table 4: Approximate inference times. indicates wider models to preserve the computational cost.
G
Description
time (ms)
# params (x (00))
{e}
Conventional CNN
41.6 + 0.2
36
Icosahedral Symmetry
43.1 = 0.2
0,49
SO(3)
Chiral (Gated non linearities)
52.2
0.37
(3(3)
Chiral (Regular, [g] 96)
28.2 0.2
0.25
S3(3)
Chiral (Regular, [9] 192)*
51.6 0.2
0.24
H.8 LBA EXPERIMENTS
To perform the LBA experiments, we use the atom3d library from Townshend et al. (2020). In particular, we adapted the 3DCNN example from their repository. In the example code, each test molecule is randomly rotated at every run of the test. make the testing procedure deterministic, we generate test set by rotating each test molecule with 5 random SO(3) rotations. The occupancy grid of molecule is generated by setting to 1 the voxel cell which closest to each atom. This results in a coarser rendering of the data, with respect to the ModelNet10 datasets. To mitigate the discretization noise, we use voxels with a higher resolution of 65 pixels. Note that Townshend et al. (2020) used voxel grids with resolution 40 pixels. However, the larger resolution of the input data can drastically increase the memory cost of the models. To prevent this, all models include a first Conv3D layer with kernel size and stride 2 which immediately downsamples the input. This first convolution layer is then followed by 4 residual blocks. In each residual block, the st stride = 2 convolution is performed in the first Conv3D rather than the second. In the conventional network. the output resolution is 3px and we use max pooling to reduce it to pixel. In the equivariant models, the last residual block does not use padding such that the output resolution already pixel. However, this last equivariant block has more output channels than the conventional network since this block followed from an invariant map layer. See Section H.5 for more details about this design choice. In the architectures, we N [120, 120, 240, 480 channels in the residual blocks and C' = 266 32, 64, 128] channels in the backbone. Finally, all models produce 256 invariant features and terminate with two fully-connected layers, alternated with dropout, batchnorm and ELU, which perform regression.
In the SO(3) architecture, we use regular non-linearities with regular features band-limited up to frequency 2. Because of the lower computational cost of this architecture, we scale up the number of channels in the residual block such that the computational cost is comparable with that of a conventional CNN. This results approximatively in twice the number of channels of the original model; see the paragraph regular non- con-lineassties Section H.3 for more details about the computational cost of these operations.
To make our results comparable with Townshend et al. 2020), we use the same train, validation and test split. Moreover, during training, we augment each molecule with random SO(3) rotation, as done in Townshend al. (2020).
The training details and dyper-paramete tuning procedures are similar to those described in Sec- tion H.
62


Reviews:
[{"review_id": "zwfn0FWLrLy", "reviewer": "Reviewer_onv5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. The authors provide a generalization of the Wigner-Eckart theorem for steerable kernels in Lang and Weiler (2020). They relax the requirement for the signal domain $X$ from being a homogeneous space of the group $G$ to an arbitrary space on which $G$ can act. They also provide a simpler computational procedure to build the kernels equivariant to action of $G$.\n\n2. The theorem takes a steerable basis constructed for a larger group $G'$ and builds a basis for the group of interest $G$ by using group restriction. Group restriction takes a representation for $G'$ and converts it into a representation for $G$ using irrep decomposition projections.\n\n3. Using this theorem, the authors show that by constructing a steerable basis for a large group $G'$, like O(n), one can easily construct steerable kernels easily for $G$, a subgroup of $G'$, without having to construct a new steerable basis for every $G < G'$. \n\n4. The idea of a general recipe for constructing equivariant networks that is useful for a large number of subgroup transformations is interesting. My guess is it can be useful when the exact desired symmetry is unknown and only small modifications to the implementation are needed to test out equivariance to all the relevant subgroups of the original base group for which steerable equivariant kernels were designed. \n\n5. The experimental results for 3D symmetries are also interesting and show the usefulness of such a recipe for both shape recognition and molecular datasets. \n\n\nQuestions and suggestions for improvement:\n\n1. I am a little unclear on the practical significance of such an idea. The idea of equivariance is attractive as it has guarantees of robustness to transformations applied at the input. However, when equivariance is enforced for a larger or different symmetry group than required, performance can become worse. Usually, which equvariances should be enforced comes from domain knowledge, although this may be approximate (like full rotation equivariance for natural image recognition, even though not all rotations appear at the same rate in the dataset) or unclear. If we know which equivariance needs to be achieved for a dataset, is it much harder to devise steerable kernels for only the desired group?\n\n2. Can the authors comment on whether they are seeing this paper as a way to exhaustively search all the subgroups of a larger group like O(3) in order to find the best performing equivariant network for a specific dataset? Wouldn't this be quite a slow process?\n\n3. The paper can be made stronger with an application like omnidirectional image recognition where $X$ is a non-Euclidean manifold, rather than just \\mathbb{R}^3 and SO(3) equivariance may not be necessary but only SO(2) as the cameras usually are upright. \n\n4. The authors should also comment on how these ideas relate to papers which attempt to find the symmetries directly from data. Can we extend the ideas in the paper to find the right group restriction in an efficient manner perhaps?", "summary_of_the_paper": "The authors present a general algorithm for constructing steerable equivariant neural networks. The main idea is that by constructing a steerable basis for a large symmetry group such as O(3), using the ideas in the paper, we can readily construct equivariant networks which are equivariants to smaller subgroups, such as SO(2). Experimental results on subgroups of isometries of $\\mathbb{R}^3$ show the usefulness of the method for 3D shape recognition for voxelized data and a regression task related to molecular binding affinity.", "summary_of_the_review": "The paper presents interesting theoretical ideas that may give a practical solution to the problem of designing equivariant steerable kernels to a large number of groups and arbitrary signal domains. The experiments are done reasonably well and provide some evidence for the utility of the method. The authors should try to improve the motivation of the paper and if possible, present more possible application areas where they see these ideas being most useful. Overall, I am inclining towards acceptance of the paper.\n\nUPDATE AFTER AUTHOR RESPONSE:\n\nI thank the authors for a clear response to my questions as well as the explanations. Overall, I am not convinced about the motivation of this paper and its practical significance. If, as the authors say, only some of the subgroups are of O(3) are \"occasionally practically relevant\", then the ideas at the moment are not very practically impactful. They may be in the future, but I still don't see why designing equivariance for the subgroups of interest for an application directly is really more difficult. I am sticking to my original rating, but the paper does have theoretically interesting ideas which may be useful to the field. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hbLz235hfXs", "reviewer": "Reviewer_3dsB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper focuses on convolutional neural networks that are equivariant with respect to certain group symmetries in arbitrary spaces. It extends upon previous paper (Lang & Weiler (2020)), and the benefits are clear: (1) it allows for parametrization with a more flexible and compact basis; (2) it is easier to implement steerable CNNs that are equivariant to a composition of group symmetries.\n\nThe paper shows extensive model designs on publicly available datasets. The flexibility of the proposed method is a bonus point. As the paper aims for generality, I would not down-score the contribution for not achieving the state-of-the-art against every other method working on a specific type of symmetry. Nonetheless, very few recent works were compared in the paper, which makes it hard to have an understanding of the practical performance of the proposed method.\n\nI appreciate the extensive details provided in the supplementary materials. However, it makes the reviewing process difficult, as it is nearly impossible to check on every detail in the supplementary given such a tight review window. In addition, many concepts are well explained in the related works such as the irreducible representation, etc. ", "summary_of_the_paper": "The paper generalizes the Wigner-Eckart theorem (Lang & Weiler (2020)) from their own homogeneous space to arbitrary spaces.", "summary_of_the_review": "To summarize, the contribution of this paper is solid. However, an evaluation of the practical performance compared to recent works and a better organization of the paper structure (especially the length and the complexity) would benefit the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "gVoPJ5t5qkl", "reviewer": "Reviewer_RMPC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I was a bit torn about this paper. While I list various strengths and weaknesses below, my high level thoughts are that the idea seems quite nice, but the current exposition is very difficult to follow for someone who is not a domain expert. Perhaps that is fine; there is definitely a community that is well-versed in the group theory underlying generalized equivariant models, but this paper probably won’t be approached by people outside of this niche. Finally, while the theory is elegant and general, I don’t know that the authors did a good enough job of motivating why we need such exotic CNNs (beyond O(3), SO(3), and SO(2)). At the end of the day, I lean towards acceptance but I’m on the fence. \n\nOn the one hand:\n1. The core idea of the paper seems elegant. In particular, using restriction to generate novel kernels seems like a good idea since E(n) is itself well-studied. \n2. The general problem of building equivariant models seems important and is an area with quite a bit of active interest. \n3. The supplementary material seems very thorough.\n4. The paper itself is quite well-written.\n5. The experiments demonstrate a pretty impressive array of examples.\n6. I think it will significantly improve the impact of the paper if the authors release code to generate the equivariant kernels (as they mention). However, it would have been nice if the authors had included code along with the submission so we could vet how usable it is for novel research.\n\nAt the same time, I do think the current version of the paper has some significant issues. Many of these might arise from tension between the complexity of the theory developed by the authors and the short form of ICLR conference papers. I do think the authors might consider submitting the paper to a journal that would support the longer form factor.\n1. I found the exposition to be very unapproachable. For an incomplete list of examples where I thought the current structure was unnecessarily difficult see below.\n2. Related to 1) it really seems like one would have to read the supplementary material to get a lot out of this paper. \n3. Although I liked the examples discussed by the authors (SO(2) and C^4), I do think the paper would be better served if there was more detail in the constructions. In particular, I think the C^4 example could have explicitly constructed the kernels (perhaps in the SI).\n4. While the experiments were performed on a wide range of kernels, there was not a lot of synthesis of the results (aside from weak suggestions about which model might be appropriate for which task). This seems especially true since the accuracies all seemed pretty close to one another.\n\nAs I mention above, my main complaint is that the current structure of the paper makes it unnecessarily difficult to read. Some specific instances include:\n1. The authors talk a lot about the Wigner-Eckart theorem but they don’t describe what it is until section 2.2. It might be worth describing it qualitatively earlier in the paper.\n2. The authors introduce (R^n, +) x G in the introduction but they don’t actually define it until section 2.1.\n3. Also in section 2.1 the authors might want to state what Ind_G^{(R^n, +)xG} is (presumably the induced representation of \\rho, but why does it not depend on d_\\rho? Also, what is t supposed to be?\n4. “Band limited” is not defined (as far as I could tell) until example 2.\n5. Reference to “the constraint” in sec. 2.2. Presumably this is referring to the G-steerability constraint, why not state it by name.\n\n\n", "summary_of_the_paper": "This paper introduces a general framework for constructing equivariant steerable CNNs over arbitrary subgroups of E(3). The central idea of their approach is to use the well-known basis for steerable kernels over E(3) and then restrict them to a given subgroup. In addition to a thorough supplementary material developing the theory, the authors also provide pseudo-code for generating novel kernels. This theory is then put into practice by generating steerable CNNs for an impressively wide array of groups and evaluating their performance on rotated MNIST and ModelNet10.", "summary_of_the_review": "This paper presents an elegant and general technique for constructing equivariant kernels for subgroups of E(3). The paper is generally well-written and the authors intend on releasing code along with the paper to help researchers build their own kernels. However, the current exposition is quite difficult to approach and it seems like one would really need to read the long (40 page) appendix to get the most out of the bulk of the paper. While the authors present a wide range of experiments, they don't really provide compelling evidence that this general construction of exotic subgroups of E(3) will lead to tangible benefits. Overall, I support this paper since it seems correct and there might be researchers who will benefit from being able to construct these CNNs, but I do think the paper has issues that limit its impact as written.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KHooBggZpd8", "reviewer": "Reviewer_oCia", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The suggested method seems like an important contribution to steerable kernel designs. The shortcomings of previous approaches are well explained. Overall the paper is well written. I appreciate the detailed implementation details section. However, I do have some concerns, detailed next.\n\n*Comparison to Lang & Weiler (2020)*\n\nLang & Weiler (2020) parametrize kernels independently on each orbit. The authors identify that this could lead to non-smoothness. However, is it true to say that it is not necessary? That is, could it be possible the filters were learned to be somehow close between orbits? is there a theoretical justification for that? \nWould it be possible to design a toy experiment and visualize learned filters with the current approach versus Lang & Weiler (2020)? In some sense figure 2 and 3 tells the same story, so I would have liked to see a figure showing learned filters from some toy example instead.\n\n*Experiments*\n\nThe method is evaluated on three invariant tasks. I think that it is also important to test the method on an equivariant task. One example that comes to mind, which I think could also serve as a nice motivation for steerable networks, is learning a steerable latent space (for example with autoencoders). So for example on modelent10 (the dataset used in one of the experiments), would the method be able to generate rotated shapes by rotating in the latent space? Would the method be able to disentangle between invariant to equivariant features? \n\nWhy isn’t the method compared to other equivariant methods in the modelnet10 experiment? \n", "summary_of_the_paper": "The paper suggests an approach to design arbitrary steerable kernels. As the kernel steerable constraint only relates to the orbits, previous approaches have not yet taken into consideration that kernels should vary smoothly among close orbits. The idea is to parametrize kernels using a steerable basis. Then, it is also possible to restrict it to subgroups, alleviating the need to redesign a new subgroup equivariant. The method is evaluated on three invariant tasks.", "summary_of_the_review": "* Seems like a solid contribution to steerable network design.\n\n* Paper is well written.\n\n* I would expected to see the method tested on some equivariant task as well.\n\n* The applicability of the method to 3D classification is not clear as it is compared to other alternatives.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an approach to constructing steerable equivariant CNNs over arbitrary subgroups of E(3), by generalizing the Wigner-Eckart theorem for steerable kernels in Lang & Weiler (2020). The intuitive idea is to use a steerable basis for a large group like O(3) to build a basis for a subgroup of interest like SO(3). Reviewers were generally happy with the author response, finding the paper makes a good contribution to steerable network design, with theoretically interesting ideas. However, there were still questions after the rebuttal about the practical utility of the approach, such as the relevance of subgroups of O(3). Reviewers also felt that much of the material was not written in an accessible way, such that it could only be appreciated by experts working on group equivariant CNNs. 

In a final version, the authors should try to make a much clearer case for practical relevance, introduce the key concepts assuming less prior knowledge, and present the material in a way that makes the high-level story more clear, as detailed by reviewers.
Decision: Accept (Poster)