A P ROGRAM TO B UILD E( n )-E QUIVARIANT S TEERABLE CNN S
A BSTRACT
Equivariance is becoming an increasingly popular design choice to build data efﬁcient neural networks by exploiting prior knowledge about the symmetries of the problem at hand.
Euclidean steerable CNNs are one of the most common classes of equivariant networks.
While the constraints these architectures need to satisfy are understood, existing approaches are tailored to speciﬁc (classes of) groups. No generally applicable method that is practical for implementation has been described so far.
In this work, we generalize the Wigner-Eckart theorem proposed in Lang & Weiler (2020), which characterizes general G -steerable kernel spaces for compact groups G over their homogeneous spaces, to arbitrary G -spaces.
This enables us to directly parameterize ﬁlters in terms of a band-limited basis on the whole space rather than on G ’s orbits, but also to easily implement steerable CNNs equivariant to a large number of groups.
To demonstrate its generality, we instantiate our method on a variety of isometry groups acting on the Euclidean space R 3 .
Our framework allows us to build E(3) and SE(3) -steerable CNNs like previous works, but also CNNs with arbitrary G ≤ O(3) -steerable kernels.
For example, we build 3D CNNs equivariant to the symmetries of platonic solids or choose G = SO(2) when working with 3D data having only azimuthal symmetries.
We compare these models on 3D shapes and molecular datasets, observing improved performance by matching the model’s symmetries to the ones of the data.
I NTRODUCTION
In machine learning, it is common for learning tasks to present a number of symmetries .
A symmetry in the data occurs, for example, when some property (e.g., the label) does not change if a set of transformations is applied to the data itself, e.g. translations or rotations of images.
Symmetries are algebraically described by groups .
If prior knowledge about the symmetries of a task is available, it is usually beneﬁcial to encode them in the models used (Shawe-Taylor, 1989;
Cohen & Welling, 2016a).
The property of such models is referred to as equivariance and is obtained by introducing some equivariance constraints in the architecture (see Eq. 2).
A classical example are convolutional neural networks (CNNs), which achieve translation equivariance by constraining linear layers to be convolution operators.
A wider class of equivariant models are Euclidean steerable CNNs (Cohen &
Welling, 2016b; Weiler et al., 2018a; Weiler & Cesa, 2019; Jenner & Weiler, 2022), which guarantee equivariance to isometries R n (cid:111) G of a Euclidean space R n , i.e., to translations and a group G of origin-preserving transformations, such as rotations and reﬂections.
As proven in Weiler et al.
(2018a; 2021); Jenner & Weiler (2022), this requires convolutions with G - steerable (equivariant) kernels.
Our goal is developing a program to parameterize with minimal requirements arbitrary G -steerable kernel spaces, with compact G , which are required to implement R n (cid:111) G equivariant CNNs.
Lang &
Weiler (2020) provides a ﬁrst step in this direction by generalizing the Wigner-Eckart theorem from quantum mechanics to obtain a general technique to parametrize G -steerable kernel spaces over orbits of a compact G .
The theorem reduces the task of building steerable kernel bases to that of ﬁnding some pure representation theoretic ingredients.
Since the equivariance constraint only relates points g.x ∈ R n in the same orbit G.x ⊂ R n , a kernel can take independent values on different orbits. Fig.
1 shows examples of orbits in R 2 under the action of continuous planar rotations G = SO(2) (orbits are rings of different radii) and of discrete rotations by π 2 , G = C 4 (orbits are sets of four points).
Hence, Lang & Weiler (2020) suggest independently parameterizing orbits of G in R n (colored regions in Fig.1).
[Figure]
[Figure]
While this guarantees a complete parameterization of G -steerable functions, the discretization of the ﬁlters requires them to be sufﬁciently band-limited 1 .
Indeed, particular care should be taken to avoid aliasing effects while discretizing ﬁlters on a grid (e.g., to process voxelized data).
Lang & Weiler (2020) naturally support band-limiting along each orbit of G but do not restrict the kernel across different orbits.
Intuitively, this means that a ﬁlter can be decomposed into a ﬁnite number of orbits but these should then be patched together with some smoothness considerations.
This introduces an important requirement and a sensitive design choice.
First, building new G -steerable CNNs requires one to identify the orbits of G in R n .
Second, the choice of orbits to consider affects a ﬁlter’s band-width, but this relation is hard to explicitly quantify in a general setting.
This issue is more severe for smaller groups (particularly, ﬁnite ones), as R n decomposes into a larger number of orbits.
For example, if G = SO(2) in Fig.
1a, band-limiting along rings is controlled by the Wigner-Eckart theorem but is not enforced along the radial component.
If G = C 4 in Fig.
1b, each ring is partitioned into discrete orbits: a larger number of orbits increases the maximum angular frequency, so aliasing is sensitive to this design choice.
Contributions and Outline
In this work, we solve this issue and propose a general solution strategy to build arbitrary CNNs with G -steerable kernels, for any compact group G .
Theorem 2.1 generalizes the result of Lang & Weiler (2020) from G -homogeneous spaces to more general spaces X carrying a G -action.
Essentially, our theorem replaces the harmonic basis on the orbits of G in X with a G -steerable basis B for unconstrained scalar ﬁlters (Freeman & Adelson, 1991) over the whole space X .
Aliasing effects are thereby controlled by the choice of this initial basis.
In practice, in this work we focus on Euclidean spaces X
= R n , although the theory developed holds for any semi-direct product group X (cid:111) G , and can be extended with minor changes to general homogeneous spaces X with compact stabilizers G (as argued in Remark D.15 in Lang & Weiler (2020)).
Since a G (cid:48) -steerable basis is G -steerable for any subgroup G ≤ G (cid:48) , we also propose to use an initial G (cid:48)
=
O( n ) -steerable basis to support any compact group G with minimal requirements. In summary, the beneﬁts of our method are two-fold: i) it allows direct control on band-width and aliasing via the initial basis B and ii) it completely disentangles the discretization issues from the choice of G , minimizing the requirements to implement equivariance to new groups .
For example, this enables the parameterization of C 4 ﬁlters without discretizing rings into a ﬁnite number of points as in Fig.
1b.
Algorithm 1 summarizes our method and its requirements.
Since the planar case n =2 was extensively studied in Weiler & Cesa (2019) and since R 3 has a large variety of isometry (sub)groups, in our experiments, we put particular emphasis on the 3D setting.
To illustrate the generality of our method, we instantiate it on many different subgroups of O(3) , the group of 3D rotations and reﬂections, and compare them experimentally.
This results in many new 3D convolution networks, equivariant, for example, to icosahedral, axial, cylindrical or conical symmetries.
Axial symmetries are particularly relevant since natural scenes generally have vertical orientation while discrete symmetries occur in crystallography or solid state physics; see Sec.
G.
In Sec.5.1, we discuss many design choices for 3D equivariant networks, and compare them in the experiments in Sec.
In particular, we ﬁnd that designs based on different discretizations of the group and pointwise non-linearities to be beneﬁcial when working with volumetric data.
Additionally, Sec.5.2 studies the beneﬁt of our basis in terms of equivariance after discretization and accuracy of trained models.
In the Appendix, we derive some results for working with representation theoretic objects to reduce the work needed to implement new equivariant CNNs.
This includes real-valued representations of compact groups, harmonic bases for induced representations, numerical irreps decomposition and representation theory of direct product groups.
Finally, we implement the program described in this work as a general purpose library based on PyTorch at github.com/QUVA-Lab/escnn .
T HE T HEORY OF S TEERABLE CNN S AND G -S TEERABLE K ERNEL S PACES
In Sec. 2.1, we discuss preliminaries on steerable CNNs and the equivariance constraints on their ﬁlters.
Sec.
2.2 describes our generalization of Lang & Weiler (2020) used to parametrize kernels.
2. E UCLIDEAN STEERABLE CNN S
Equivariant neural networks are characterized by their property to commute with the action of a given symmetry group on their input, intermediate feature spaces, and output.
We focus on convolutional networks on Euclidean spaces X = R n , acted on by the group ( R n , +) (cid:111) G , i.e. the semidirect product of translations in ( R n , +) and origin-preserving transformations G ≤ O( n ) .
A general framework of equivariant convolutional models is that of steerable CNNs , which we brieﬂy recapitulate here.
For more details, see Cohen & Welling (2016b); Weiler et al.
(2018a; 2021); Weiler & Cesa (2019).
The fundamental design choice underlying steerable CNNs is that they operate on feature vector ﬁelds , which are similar to feature maps but differ in that they are associated with a well deﬁned action of ( R n , +) (cid:111) G .
The geometric type of a feature ﬁeld is prescribed by an orthogonal group representation ρ : G → R d ρ × d ρ which determines the transformation law of d ρ -dimensional feature vectors under the action of G .
Speciﬁcally, a feature ﬁeld of type ρ is a map f : R n → R d ρ which transforms under ( R n , +) (cid:111) G according to the induced representation Ind ( R n , +) (cid:111) G G ρ : (cid:16)(cid:2) Ind R n , +) (cid:111)
G G
[Equation]
Intuitively, the induced representation acts by moving feature vectors spatially from g − 1 ( x − t ) to x and by transforming each feature f ( x ) according to ρ ( g ) (see Fig.
1 in Weiler & Cesa (2019)).
Notable examples are scalar or tangent-vector ﬁelds, which transform according to the trivial representation ρ ( g )
= 1 or the standard representation ρ ( g ) =
g , respectively.
Group-convolution networks (Cohen & Welling, 2016a; Kondor & Trivedi, 2018) are special cases with ρ being the regular representation of G .
It is common to deﬁne the full feature ﬁelds of a network as a direct sum (concatenation) f :
= (cid:76) i f i of multiple individual feature ﬁelds f i .
The full feature space transforms according to the direct sum representation ρ :=
(cid:76) i ρ i , which ensures ﬁelds transform independently from each other.
Weiler et al.
(2018a)
; Jenner & Weiler (2022) proved that, under mild assumptions, the most general linear equivariant maps between spaces of feature ﬁelds are convolutions with G -steerable kernels .
If the input and output ﬁelds have types ρ in : G → R d in × d in and ρ out : G → R d out × d out , G -steerable kernels are convolution kernels K : R n → R d out × d in satisfying the linear G -steerability constraint
[Equation]
Thus, designing steerable CNNs only requires ﬁnding a basis of the vector space of G -steerable kernels, which is then used to parameterize conventional Euclidean convolutions.
Note that Eq.
2 relates the kernel values on all points g.x in the orbit G.x , but leaves values on different orbits unrelated.
2. A W IGNER -E CKART T HEOREM FOR SUBGROUP EQUIVARIANT CONVOLUTION KERNELS
First, we need to introduce a few concepts and some notation. To keep the presentation general, in this section we will consider a general space X rather than R n .
Since G is compact, we can assume all representations to be orthogonal , i.e., ρ ( g − 1 )
=
ρ ( g ) − 1 =
ρ ( g ) T .
To parametrize a kernel K : X → R d out × d in satisfying the steerability constraint in Eq. 2, it is convenient to use its vectorized form κ ( · ) = vec ( K ( · )) :
X → R d out · d in .
The constraint 2 becomes:
[Equation]
The matrix ( ρ in ⊗ ρ out )( g ) is the Kronecker product of the two matrices ρ in ( g ) and ρ out ( g ) .
Irrep Decomposition A useful property is that there exists a set 3 (cid:98) G of special representations of G , called irreducible representations or irreps , such that any representation ρ of G can be decomposed as a direct sum of them: ρ ( g ) =
Q T (cid:0)(cid:76) i ∈
I ρ i ( g ) (cid:1) Q , where ρ i ∈ (cid:98) G , I is an index set ranging over the elements of (cid:98) G (possibly, with repetition) and Q is an orthogonal matrix.
The direct sum ρ 1 ( g ) ⊕ ρ 2 ( g ) of two matrices is a block diagonal matrix containing the two matrices in its blocks.
W.l.o.g.
, we can assume ρ in and ρ out in the kernel constraint from Eq.
2 to be irreps.
The solutions for arbitrary ρ in and ρ out can be recovered from their irreps decomposition, as explained in Weiler & Cesa (2019).
Hence, from now, we will assume irreps ρ in = ρ l and ρ out = ρ J , where l and J are indexes over (cid:98) G .
Tensor Product
The matrix ρ in ⊗ ρ out built with the Kronecker product in Eq.
3 is itself a representation of G , namely a tensor product representation.
As such, we can decompose it in terms of G irreps:
[Equation] CG lJ , where [ j ( lJ )] indicates the multiplicity of the irrep ρ j , while CG lJ is the change of basis matrix.
The block diagonal structure of the direct sum allows one to distinguish blocks of rows in CG lJ , acted on by individual irreps in the direct sum.
The block associated with the s -th occurrence of ρ j is denoted as CG j ( lJ ) s ∈ R d j × d l d J .
Endomorphisms We are interested in the space of equivariant linear maps between two spaces transforming according to irreps ρ j , ρ ˜ j ∈ ˆ G .
If j (cid:54) =
˜ j , this space is empty, otherwise, this is the endomorphism space of ρ j .
This is a vector space; assume { c jr ∈ R d j × d j } r is a basis 5 for it.
Steerable Basis To parametrize our kernels, we ﬁrst need a basis for square-integrable functions on X , i.e. for L 2 ( X ) .
A G - steerable basis for L 2 ( X ) is a collection of orthogonal functions { Y mji :
X → R } j ∈ (cid:98) G,m ≤ d j ,i ≤ m j , with m j a positive (possibly inﬁnite) integer.
Denoting by Y ji : X → R d j the stack { Y mji } d j m =1 , Y ji has the deﬁning property that ∀ g ∈ G, x ∈ X, Y ji ( g.x ) =
ρ j ( g ) Y ji ( x ) .
The vectorized constraint in Eq.
3 is equivalent to Eq.
2 with a scalar input ﬁeld and a ρ in
⊗ ρ out output one.
A (vectorized) kernel κ is a G -equivariant linear map from L 2 ( X ) to R d l · d J , which will be applied convolutionally over a scalar ﬁeld.
Hence, a basis for this kernel space is given by considering the irreps decomposition of G ’s action on L 2 ( X ) and R d l · d J , and, then, equivariant maps between pairs of irreps. The basis B deﬁnes an irrep decomposition of L 2 ( X ) , CG lJ decomposes ρ in ⊗ ρ out and equivariant maps are parameterized by each irrep’s endomorphism basis. This leads to: Theorem 2.1 (Basis for G -Steerable Kernels) .
Let G be a compact group acting on a space X .
Let
B =
{ Y ji } ji be a G -steerable basis for L 2 ( X ) .
Assume ρ in = ρ l and ρ out =
ρ J in (cid:98) G .
Under minor conditions, a basis for (vectorized) G -steerable kernels over X is given by K = { κ jisr } jisr , with:
[Equation]
The proof is in Appendix B (including the case of complex valued representations); see also Fig. 2.
Designing a new G -steerable basis B can be laborious; however, if G is a subgroup of G (cid:48) (i.e.,
G ≤ G (cid:48) ), a G (cid:48) -steerable basis can be turned into a G -steerable one via group restriction : Group Restriction Given two groups G ≤ G (cid:48) , one can turn an irrep ρ (cid:48) of G (cid:48) into a representation Res G (cid:48)
G ρ (cid:48) of G by restricting its domain to G .
This representation is not irreducible and decomposes as: Res
[Equation]
where [ jj (cid:48) ] is the multiplicity of ρ j and ID j (cid:48) is the change of basis.
As earlier, the block-diagonal structure of the direct sum distinguishes blocks of rows of the matrix ID j (cid:48) .
We denote the block associated with the t -th occurrence of ρ j as ID jj (cid:48)
t
[Equation]
Restricted Steerable Basis
If B (cid:48) = { Y j (cid:48) i (cid:48) } j (cid:48) i (cid:48) is a G (cid:48)
-steerable basis for L 2 ( X ) , the following set is a G - steerable basis , with index i =
( j (cid:48) i (cid:48) t ) (see Appendix B.3):
[Equation]
We present two examples on the circle X = S 1 ; see Appendix B.4 for more detailed examples.
Example 1: SO(2)-Steerable Kernels In Fig.
3, we show a simple example of the basis for a
G = SO(2) -steerable kernel space.
Since X = S 1 is an orbit of G , our parameterization is equivalent to Lang & Weiler (2020).
An SO(2) -steerable basis on X = S 1 is given by the circular harmonics:
Y 0 ji ( θ )
= cos( jθ ) and Y 1 ji ( θ ) = sin( jθ ) , for j ∈ N and i = 1 . We consider steerable kernels with input irrep ρ in = ρ l =0 and output irrep ρ out one of ρ J =0 , ρ J =1 or j =0, r =
[Figure]
(a) J =
[Figure] j = [Figure]
[Figure]
[Figure]
=
[Figure]
ρ J =2 . Hence, the kernels have form, respectively, K 0 : S 1 → R 1 × 1 , K 1 : S 1 → R 2 ×and K 2 : S 1 → R 2 × 1 .
ρ 0 ⊗ ρ J =
ρ J , so we always have j = J and s = 1 .
Additionally, any 2 -dimensional irrep ρ j ( j > 0 ) of SO(2) has a 2 -dimensional endomorphism space spanned 7 by c jr =
= (cid:20) 1 0
(cid:21) and c jr =1 = (cid:20) {+
 [Table]+}
This can be veriﬁed visually: in Fig.
3b and 3c, the second column is obtained by swapping the rows of the ﬁrst and by changing the sign of the second row.
Example 2: C 4 -Steerable Kernels In Fig.
4, we show the basis for G = C 4 steerable kernels described by Theorem 2.1, using the C 4 -steerable basis generated by Eq. 5 from the circular harmonics basis B (cid:48) used for G (cid:48)
= SO(2) in the previous example.
ρ j occurs in a restricted representation Res SO(2)Cρ j (cid:48)
whenever j (cid:48) = | j +4 k | , k ∈ Z .
Indeed, the basis prescribed in Eq. 5 has inﬁnitely many elements, but each element is associated with a circular harmonic of a speciﬁc frequency j (cid:48) .
It is natural to use only a band-limited ﬁnite subset of this basis: Fig.
4 shows only frequencies up to j (cid:48)
= 4 .
If j = 0 or j = 2 and j (cid:48) = | j + 4 k | > 0 , ρ j is 1 -dimensional but ρ j (cid:48) is 2 -dimensional, so Res G (cid:48)
G ρ j (cid:48) contains two copies of ρ j and t ∈ { 1 , 2 } (see j (cid:48)
= 2 or 4 in Fig. 4a).
We use the trivial irrep l = 0 in the input and J ∈ { 0 , 1 , 2 } in the output.
ρ j = has a 2 -dimensional space of endomorphisms, spanned by the same basis described in the SO(2) example above, so r ∈ { 1 , 2 } in this case.
Note that the parametrization in Fig.
4 is continuous along the ring and does not suffer from the discretization issue in Fig.
1b.
[Equation]
1 t =
j 0 =1, r =2 t
= j 0 =3, r =1 t =
j 0 =3, r =2 t = (b) J =j 0 =2, r =1 t =
{+j 
+}
=2, r =1 t = (c) J =
I MPLEMENTATION D ETAILS
Given the theoretical results described in the previous sections, we now describe the steps and inputs required to build new G -steerable kernels.
See Algorithm 1 for a summary.
Theorem 2.1 relies on:
a ) the irreps (cid:98) G = { ρ j } j , b ) a G -steerable basis B =
{ Y ji } ji for L 2 ( X ) , c ) the decomposition ρ l ⊗ ρ J =
[CG lJ ] T (cid:0) (cid:76) j ∈ (cid:98) G (cid:76) [ j ( lJ )] s = ρ j (cid:1) CG lJ , and d ) a basis { c jr } r for the endomorphism space of ρ j .
Optionally , B can be generated via Eq. 5 using e ) a larger group G (cid:48) and its irreps (cid:99) G (cid:48) = { ρ j (cid:48) } j (cid:48) , f ) a G (cid:48) - steerable basis B (cid:48) = { Y j (cid:48) i (cid:48) } j (cid:48) i (cid:48) and g ) the decomposition Res G (cid:48)
G ρ j (cid:48) =
[ID j (cid:48) ]
T (cid:0) (cid:76) j ∈ (cid:98) G (cid:76) t ρ j (cid:1) ID j (cid:48) .
The irreps of G in a ) are the main requirement: they allow us to build arbitrary representations (via
Require: ρ in = ρ l and ρ out =
ρ J , G (cid:48) -steerable basis B (cid:48)
= { Y j (cid:48)
i (cid:48) } j (cid:48) i (cid:48) , (cid:98) G = { ρ j } j and (cid:99) G (cid:48)
= { ρ j (cid:48) } j (cid:48) 1: { c jr } r ← basis for endomorphism space of ρ j , for all ρ j (cid:46) Appendix C 2: CG lJ , { [ j ( lJ )] } j ← decompose ( ρ l ⊗ ρ J ) (cid:46) Appendix E 3: for all Y j (cid:48) i (cid:48)
∈ B (cid:48) do 4:
ID j (cid:48) , { [ jj (cid:48) ] } j ← decompose (Res G (cid:48)
G ρ j (cid:48) )
(cid:46) Appendix E 5: for all j ∈ ˆ G : [ jj (cid:48) ] > 0 , s ≤ [ j ( lJ )] , t ≤ [ jj (cid:48) ] , c jr ∈ { c jr } [Table] do 6:
Y j ( j (cid:48) i (cid:48) t
[Table]
( x ) ← ID jj [Table] (cid:46) Equation7: [Table]
(cid:46) [Table] 8: yield K [Table]
[Table]
(cid:48) i (cid:48) [Table] direct sum) and reduce the kernel constraint to the form in Sec. 2.2 by decomposing the input and output representations.
If B in b ) is unknown, one can rely on Eq.
A convenient choice for G (cid:48) in e ) is a group whose irreps (cid:99) G (cid:48) , together with a steerable basis B (cid:48) as in f ), are known.
G (cid:48)
=
O( n ) is always possible if X = R n , and B (cid:48) is built via polar decomposition of R n and by combining hyper-spherical harmonics with a radial basis.
Like Worrall et al.
; Weiler et al.
(2018b;a), we use Gaussian radial proﬁles 8 .
In Sec. 5, this choice of G (cid:48) allows easily experimenting with multiple subgroups G ≤ O(3) , without the overhead of identifying each G ’s orbits or designing ad-hoc G -steerable bases.
Note also that band-limiting is achieved by modifying B (cid:48) and is independent from G . Other
G (cid:48) ’s orbits can also be used, combining harmonic bases (see Appendix D) with a Gaussian kernel.
The irreps decomposition of ρ l ⊗ ρ J in c ) provides the multiplicity [ j ( lJ )] of each irrep ρ j and the matrices CG j ( lJ ) s .
Similarly, the irreps decomposition of Res G (cid:48)
G ρ j (cid:48) in g ) provides the multiplicity of each G -irrep ρ j in each G (cid:48)
-irrep ρ j (cid:48) and the projection matrices ID jj (cid:48) t .
Knowing these decompositions a-priori for any l, J and j (cid:48) is generally difﬁcult, but they can be easily computed numerically; see
Appendix E. Finally, d ) requires a basis { c jr } r for the endomorphism space of each ρ j ∈ (cid:98) G .
This can be computed numerically, but is often unnecessary; see Appendix C.
In summary, if X = R n and G (cid:48) =
O( n ) , implementing new G -steerable CNNs only requires knowing the irreps (cid:98) G : by knowing the action of G on R n , one implicitly knows its embedding into O( n ) and, therefore, can apply Eq.
Practical Example and Experiments In Sec.
5.3, we implement a variety of subgroups of the isometries of R 3 (see Appendix G).
To do so, we choose G (cid:48) = O(3) and B (cid:48) is built by combining spherical harmonics with a Gaussian radial proﬁle.
Since irreps are used to deﬁne the types of the feature ﬁelds, we need the irreps of SO(3) , O(2) , SO(2) , C N , I and O .
Direct product groups such as Inv × SO(2) are built as in Appendix F. Since there are generally multiple subgroups G < O(3) isomorphic to the same abstract group (e.g. see O(2) in Tab. 1), for each G , we explicitly deﬁne its isomorphism with an abstract group; this enables the automatic restriction of G (cid:48) ’s
irreps and the numerical computation of the ID jj (cid:48) t matrices.
The matrices CG j ( lJ ) s are also computed numerically.
Limitations Our choice of B (cid:48) implies we only parameterize ﬁlters supported on a compact ball, which is slightly more restrictive than the usual parameterization of ﬁlters with support on a cube.
For a ﬁxed kernel size, an ad-hoc implementation of G -steerable ﬁlters can exploit a larger initial basis, potentially leading to some performance gain (e.g. C 4 allows ﬁlters supported on a square rather than a disk).
However, if an ad-hoc implementation of G (cid:48) < O( n ) steerable kernels is available, our method can be used to construct G ≤ G (cid:48) steerable spaces by using the G (cid:48)
-steerable basis rather than the O( n ) -steerable one.
Comparing the effect of different B (cid:48) on the performance of a G -steerable
CNN is beyond the scope of this paper, which instead focuses on building a method suitable for any G .
R ELATED W ORKS
Similar in spirit to our work is the e2cnn library (Weiler & Cesa, 2019), although limited to X = Rand G ≤ O(2) .
G (cid:48)
= O(2) recovers their solutions.
Geiger et al.
(2020) present a library implementing general 3D steerable CNNs, but limited to the choices G = SO(3) or O(3) .
In comparison, we currently support both 2D and 3D convolution and any compact group G ≤ O(3) (including discrete and planar subgroups acting on R 3 ), and other spaces X can be potentially integrated.
Finzi et al.
(2021) propose a numerical method to parameterize (ﬁnite dimensional) MLPs that are equivariant to arbitrary matrix groups .
Their numerical method to compute a basis of equivariant linear maps resembles our irreps decomposition method; see Appendix E. Bekkers (2020) and Finzi et al.
(2020) implement group convolution on Lie groups using a ﬁnite number of samples from the continuous group.
This is similar to our SO(3) or O(3) architectures with pointwise non-linearities; see Sec. 5.1.
However, since our features and ﬁlters are explicitly parametrized on a band-limited space, we can adapt the sampling set to control the equivariance error caused by this approximation.
Aronsson (2021) previously discussed bandlimited convolution operators in equivariant CNNs.
Jenner & Weiler (2022) generalize Euclidean steerable CNNs to partial differential operators.
(Cohen et al., 2019b; Kondor & Trivedi, 2018; Bekkers, 2020) deﬁne steerable CNNs on homogeneous spaces.
G -steerable kernels are necessary to implement gauge equivariant CNNs on general manifolds (Weiler et al., 2021;
Cohen et al., 2019a; Kicanaoglu et al., 2019; Haan et al., 2021). Li et al.
(2021) use transformed ﬁlters to parameterize steerable kernels.
Previously, Mallat (2012); Oyallon & Mallat (2015); Sifre & Mallat (2013) described similar architectures based on scattering. Brandstetter et al.
(2021) use a non-linear parameterization of steerable convolution in a geometric graph.
I NSTANTIATION OF THE M ETHOD AND E XPERIMENTS
We demonstrate the generality of our method by instantiating steerable CNNs for different groups G .
We will mostly focus on R 3 and its isometries; Appendix G brieﬂy summarizes the subgroups used.
Sec. 5.1 discusses existing and new design choices for 3D steerable CNNs.
We then compare these designs in Sec.
Sec.
5.2 numerically compares our steerable basis with alternative bases.
5. D ESIGN C HOICES OF 3D S TEERABLE CNN S
A G -steerable CNN design involves a choice of feature ﬁeld types , i.e. G representations acting on the channels of the features, see Sec. 2.1.
The chosen types constrain the non-linear operations permitted.
Many works in the literature focus on designs which achieve perfect SO(3) equivariance (Weiler et al., 2018b; Thomas et al., 2018; Anderson et al., 2019) .
To ensure this, a common choice is the use of irreps feature types and gated non-linearities .
This operation combines a feature ﬁeld f ρ ( x ) ∈
R
d ρ of type ρ with a gate f 0 ( x ) ∈ R as f ρ ( x ) (cid:55)→ σ ( f 0 ( x ))
f ρ ( x ) , where σ is a sigmoid function and f 0 is feature ﬁeld of trivial type.
This non-linearity is equivariant to any G but Weiler & Cesa (2019) found them to perform worse than point-wise non-linearities like those used in GCNNs.
Another popular choice is using the tensor product of two feature ﬁelds as a non-linear operation (Kondor et al., 2018), i.e. (cid:0)
f
ρ i ( x ) , f ρ j ( x ) (cid:1) (cid:55)→ f ρ i ( x ) ⊗ f ρ j ( x ) .
However, this non-linearity is quadratic and turns the network into a polynomial function of the input, which can lead to training difﬁculties (Anderson et al., 2019).
Additionally, computing the tensor product ⊗ has computational and memory cost O
( d ρ i d ρ j ) ; it is common to project the output features to a smaller feature ﬁeld through a learnable linear map.
A different approach is used in Worrall & Brostow (2018); Winkels & Cohen (2018), which use the octahedral group O on voxelized data.
The ﬁniteness of the group allows for a simple GCNN (Cohen & Welling, 2016a) architecture, corresponding to the choice of the regular representation ρ reg as the feature type and standard pointwise non-linearities (e.g. ReLU).
The representation ρ reg of a group
G acts on feature vectors in R | G | by permutations and a vector in this space can be interpreted as a scalar function over G ; see Weiler & Cesa (2019) for an intuitive description.
Because SO(3) (and O(3) ) has only a limited number of discrete subgroups, we also consider discretizations without group structure.
We interpret a feature vector f ( x ) ∈ R B as the coefﬁcients parameterizing a band-limited function over G in Fourier space; see Appendix D. To apply a pointwise non-linearity σ : R → R (e.g. ELU) on f ( x ) , we i) sample the parameterized function on a ﬁnite set G ⊂ G , ii) apply σ and, then, iii) recover the coefﬁcients f (cid:48)
( x ) of the output via a Fourier transform.
If σ is smooth, the output signal is still approximatively band-limited but can contain higher frequencies than the input f ( x ) .
This implies that a larger set G is required to reconstruct f (cid:48)
( x ) than for f ( x ) .
If a ﬁxed budget of channels (i.e. number of samples |G| ) is available for a non-linear module, the input ﬁeld f ( x ) must be strongly bandlimited.
While this can reduce the expressiveness of the model’s features, it also implies that linear layers are computationally cheaper as they operate on smaller number of channels.
A similar result was leveraged in Cheng et al.
for G = SO(2) .
This regular non-linearity was also used in Kicanaoglu et al.
for G = SO(2) .
The same approach can be generalized to quotient ﬁelds , i.e. feature ﬁelds which parameterize functions over
[Table]
a quotient space X = G/H (e.g. S 2 = SO(3) / SO(2) ).
See Appendix H.2 for more details on
5. N UMERICAL COMPARISON OF S TEERABLE B ASES
In this section, we compare our basis with one built according to Lang & Weiler (2020).
We consider the example of Icosahedral I steerable 3D convolution.
Our basis is composed by a band-limited set of spherical harmonics per spherical shell, combined as in Theorem 2.1.
The two baselines use respectively the center of the 20 faces of the dodecahedron and the 12 faces of the icosahedron embedded in each shell as orbits and parameterize the kernel along each
[Figure]
orbit directly using (Lang & Weiler, 2020).
Fig.shows the histograms of the equivariance errors of the three bases with respect to the Icosahedral group; each point represents an element of the basis and the errors are averaged all transformations; see Appendix H.1.
Our basis shows signiﬁcantly higher stability.
Finally, we study the effect of these bases on a model’s performance.
In Tab.we compare two I -equivariant models: one using our harmonic basis and one using the 20 faces of the Icosahedron as orbit.
Our anti-aliased basis leads to a signiﬁcant improvement in accuracy.
5. E XPERIMENTS
To emphasize our method is not limited to R 3 , we include a simple experiment with 2D images.
In the rest of the section, we compare different model designs on two volumetric datasets: ModelNet10 (Wu et al., 2015) (and a rotated version of it) and LBA (Townshend et al., 2020).
For each model, we run a simple search over hyperparameters and minor variants of the designs described in Sec. 5.
using validation accuracy, to ensure a fair comparison.
Unless speciﬁed, within a task, all models approximatively share the same width.
In particular, in all tasks, the models which more closely match the symmetries of the data perform the best.
See Appendix H for more details on the experiments.
The equivariance groups used (column G in the tables) are described in Appendix G.
Rotated MNIST As a simple 2D experiment, we train a conventional CNN and a G = C 8 equivariant
CNN on rotated MNIST.
The steerable kernel bases are similar to those in Weiler & Cesa (2019) and we use regular ﬁeld types with pointwise ELU.
The conventional model achieves 96 .
3 ± 0 .
1 test accuracy while the C 8 equivariant one 96 .
7 ± 0 . 4 .
Rotated ModelNet
We generate a voxelized version of ModelNet10 with resolution 33 px by sampling 3 random SO(3) rotations for each object.
During training, we use rotations from O for augmentation.
In Tab. 1, we compare different equivariance groups and different equivariant designs.
It includes our implementation of some relevant related works; see Sec. 5.1.
We ﬁrst highlight the large margin between all equivariant networks and the conventional CNN.
SO(3) and O(3) models achieve better performance than the other continuous subgroups.
The best results are achieved by using the discrete groups I and O or the SO(3) model with spherical quotient feature types.
Recall also that the models with quotient or regular non-linearities are computationally cheaper; see Sec.5.1.
[Table]
ModelNet10 To experiment with data with only axial and mirror symmetries, we generate a version of ModelNet10 by sampling the random rotations only around the Z axis.
π 2 rotations and reﬂection augmentation in the XY plane is used for training but no rotation averaging is used for testing.
The ﬁnal accuracies are in Tab. 2.
The models equivariant to 3D rotations are outperformed by the SO(2) and O(2) models, which better match the data symmetries.
Ligand Binding Afﬁnity (LBA) regression We also evaluate our models on a regression task using the LBA dataset (Townshend et al., 2020), containing protein-ligand complexes from the PDBBind database (Wang et al., 2004;
Liu et al., 2014).
In Townshend et al.
(2020), an SO(3) -equivariant model with tensor product activations performed the best, despite its cost prevented training on the full dataset.
We propose a volumetric SO(3) network with a computational cost equivalent to a conventional CNN by leveraging the beneﬁt of regular non-linearities .
Tab.
3 reports our results.
Discussion We again emphasize that the purpose of our experiments is mostly demonstrating the generality and ﬂexibility of our program, but we do not necessarily envision applications of all subgroups G < O (3) considered.
Still, some subgroups are practically relevant; indeed, models with azimuthal symmetry perform the best in Table 2 where the data lacks full rotational symmetry.
Here, enforcing SO(3) equivariance results in over-constrained models which are not sufﬁciently expressive and are outperformed even by the conventional CNN.
Overall, we observe models whose symmetry matches the data tend to perform the best.
Some discrete subgroups, e.g. the platonic ones, constitute a special exception: they approximate the full rotational symmetry but are less restrictive than a fully rotation equivariant model, sometimes better matching the approximate symmetry of voxelized data.
Finally, we only found minor differences among the various SO(3) models, although regular and quotient non-linearities stood out for their reduced computational cost (see also Appendix H.7).
C ONCLUSIONS
In this work, we gave a general characterization of G -steerable kernel spaces over an arbitrary space
X and proposed a general strategy to automatically parameterize them based only on G ’s irreps.
This enabled us to implement steerable CNNs equivariant to a variety of new groups G ≤ O(3) , which we compare in an exploratory study in our experiments.
Finally, we believe that Theorem 2.1 suggests a new theoretical perspective on the construction of equivariant CNNs, reducing the problem to that of devising a suitable starting basis B , regardless of the choice of feature types in the model.
[Table]
A CKNOWLEDGMENTS
Gabriele would like to thank Arash Behboodi for fruitful discussions on the approximation of point-wise non-linearities with ﬁnite samples.
R EPRODUCIBILITY S TATEMENT
To ensure reproducibility, we include a dedicated section in Appendix H to describe our experiments in more details.
Moreover, we implemented all the methods described in this work in a Python library which can be found at github.com/QUVA-Lab/escnn .
We also plan to publish the code used to run our experiments.
P OSSIBLE N EGATIVE S OCIETAL I MPACTS
Since this work is very foundational, many different negative societal impacts are conceivable.
In particular, any misuse which is eased by improved computer vision systems can proﬁt from this and similar work.
Of course, we cannot discuss them all.
Nevertheless, we want to highlight one speciﬁc misuse which seems likely to occur for the ﬁeld of equivariant deep learning in the future – namely, the adoption in lethal autonomous weapons systems (LAWS).
We can easily think of drones using spherical vision which would proﬁt from equivariant
CNNs.
In particular, since they have a ﬁxed vertical orientation, but arbitrary horizontal orientation, they could proﬁt from spherical CNNs using azymuthal symmetries.
It is important to realize that the mitigation of such misuses cannot happen at the level of individual research projects.
Instead, what is needed is a global effort to mitigate the misuse of AI systems for LAWS in general.
Fortunately, the broader AI community agrees that AI should not be used in
LAWS, and that international regulations are necessary to prevent their use in the future aut (2015).
We hope that efforts in this direction are continued.
A PPENDIX A Preliminaries on the Representation Theory of Compact Groups
A.1 Basics of (Unitary and Irreducible) Representations . . . . . . . . . . . . . . . . .
A.2 Homogeneous Spaces, Square-Integrable Functions, and the Peter-Weyl Theorem .
A.3 Constructing Unitary Representations from Others and their Relations . . . . . . .
A.4 Compact Subgroups and Restricted Representations . . . . . . . . . . . . . . . . .
B A Generalized Wigner-Eckart Theorem for Steerable Kernels
B.1 Basis-Independent Formulation of the Group-Restricted Wigner-Eckart Theorem .
B.2 Steerable Kernel Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B.3 A Wigner-Eckart Theorem for Group-Restricted Steerable Kernels . . . . . . . . .
B.4 Extended Examples of Steerable Kernel Bases . . . . . . . . . . . . . . . . . . . .
C Real Valued Representations
C.1 Preliminaries the of End G, R
( V ψ ) . . . . . . . . . . . . . . . . . . . .
C.2 Orthogonality and Alignment of Matrix Coefﬁcients . . . . . . . . . . . . . . . . .
C.3 A Convenient Choice of Basis for Real Irreps . . . . . . . . . . . . . . . . . . . .
D The Computation of the Induced Representation of Compact Groups
D.1 Useful Deﬁnitions, Assumptions and Notations . . . . . . . . . . . . . . . . . . .
D.2 Induced Representations . .
D.3 Harmonic Analysis of Induced Representations . . . . . . . . . . . . . . . . . . .
D.4 Reconstructing the irrep decomposition: the complex case . . . . . . . . . . . . .
{+D.
+}
Reconstructing the irrep decomposition: the real case . . . . . . . . . . . . . . . .
E Numerical Irreps Decomposition of Real Representations
F Real Irreducible Representations of the Direct Product of two Compact Groups
G 3D symmetries, Azimuthal symmetries and their relevance
H Further Details on Network Designs and Experiments
H
1 Estimating the Equivariance Error of Steerable
Bases . . . . . . . . . . . . . . . .
H.2 Regular and Quotient Non-Linearities H.3 Architecture Details
H.4 MNIST
Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
H
5 ModelNet
Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
H.6 Auto-Encoder Experiments on ModelNet
H.7 Computational cost and Inference time of
the models
H.8 LBA
Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
In the Appendix, we provide a more detailed description and the proof of the theoretical results mentioned in the main paper.
We ﬁrst give a brief overview of the representation theory of compact groups and introduce the notation we will use throughout the next chapters in Section A.
In Section B, we state more precisely and prove the main result from Sec. 2.2, which we used to parametrize the steerable ﬁlters.
In particular, we state both a complex and a real version of Theorem 2.1. In section B.4, we discuss two examples of kernel bases.
Sections C, D, E and F provide additional theoretical results and computational methods which can ease the practical implementation of steerable CNNs, but are not necessary for understanding our main results.
Because the implementation of steerable CNNs is necessarily using real numbers, its design should rely on the real representation theory of the group considered.
In Section C, we review the theory of real irreducible representations and derive some useful results to deal with these representations in the implementation.
Notably, we show that, for a compact group G , any real irrep of G can always be expressed in a particular basis which allows one to recover a harmonic basis for square-integrable functions over G from the matrix coefﬁcients of the irreps.
The results shown in Section C will become useful in Section D, E and F.
Regular, quotient and induced features types encode scalar or vector functions over a homogeneous space of G in terms of a harmonic basis. Moreover, harmonic bases provide a simple way to generate the G -steerable bases required to deﬁne the kernel basis in Sec. 2.2: by leveraging the fact that orbits of G in X are isomorphic to homogeneous spaces, the harmonic bases for the orbits of G in X can be combined with a Gaussian kernel to generate G -steerable bases over X .[-
-]
In Section D, we derive a method to compute the (either real or complex) harmonic basis for any homogeneous space of G from the harmonic basis of functions over G .
The method described in Section D turns out to be more general and also supports vectors ﬁelds over homogeneous spaces, i.e. general induced representations of G .
Since the kernel parameterization in Sec.
2.2 relies on the irreps decomposition of the input and output representations, of the tensor product of irreps (to compute the Clebsh-Gordan coefﬁcients
CG ) and of the restricted irreps of G (cid:48) (to compute the ID coefﬁcients), the ability to decompose an arbitrary representation is essential to apply our method in a fully general setting.
Therefore, in
Section E, we describe a numerical method to perform this decomposition for real representations.
As explained in Sec. 3, the main (and, often, only) requirement to apply our method to a new group
G is the knowledge of its irreps (cid:98) G .
A simple operation which can be used to generate new groups is the direct product × of two groups.
Examples of direct product groups include many subgroups of
O(3) that we used in Sec.
In Section F, we describe a method to derive the real irrep of a direct product group G =
A × B from the real irreps of the two subgroups A and B .
Section H includes additional details on the experimental results reported in Sec. 5.
In particular, we describe in more details the different equivariant designs, the architectures used, the datasets and the training and testing conﬁgurations.
A P RELIMINARIES ON THE R EPRESENTATION T HEORY OF C OMPACT G ROUPS
In this section, we collect many notions and concepts which we need throughout the appendix for the development of the theory.
This section assumes some familiarity with the representation theory of compact groups and is meant as a reminder.
For readers with no background in representation theory, we can recommend the appendix of Lang & Weiler (2020).
We will use K to denote any of the two ﬁelds R or C , i.e., we develop the theory in such a way that it covers both real and complex representations.
Whenever we deviate from this assumption, we will say so explicitly.
Throughout, let furthermore G be any compact group.
A. B ASICS OF (U NITARY AND I RREDUCIBLE ) R EPRESENTATIONS
Linear representations are (in a certain sense continuous) group homomorphisms ρ : G → GL( V ) for some topological K -vector space V .
In our applications, V always carries a scalar product (cid:104)·|·(cid:105) which induces its topology, and which we always assume to be conjugate linear in the ﬁrst component, and linear in the second component.
The pair ( V, (cid:104)·|·(cid:105) ) is assumed to be a Hilbert space, meaning that the scalar product makes V a complete metric space.
Note that, as is typical in a physics context, we freely make use of the Bra-Ket convention, and thus for example view (cid:104) v | as a linear functional which maps | w (cid:105) :
= w to (cid:104) v | w (cid:105) .
By the Riesz representation theorem, there are no other continuous linear functionals.
Unitary representations are linear representations ρ : G → U( V ) ⊆
GL( V ) , where U( V ) is the unitary group of V .
This is the group of unitary transformations on V , i.e., linear functions on V which preserve the scalar product, and thus distances.
Whenever we explicitly assume K
=
R , we will write O( V ) instead of U
( V ) , which is the group of orthogonal transformations on V , and the representations are then called orthogonal instead of unitary .
We will denote the set of isomorphism classes of all irreducible unitary representations (irreps) of G as (cid:98) G .
Thereby, irreducible representations are representations which cannot be decomposed nontrivially into subrepresentations.
We usually write indices j, l, J ∈ (cid:98) G to indicate such isomorphism classes, and then, e.g., ρ j : G → U( V j ) to denote a representative of such an isomorphism class.
We set d j := dim V j as the dimension of the representation spaceDof V j .
Whenever we consider only one irrep, we may also, by abuse of notation, write ρ ∈ (cid:98) G to indicate (a representative of an isomorphism class) of an irrep of G .
We now collect deﬁnitions which we use throughout.
Deﬁnition A.1 (Homomorphisms/Intertwiners) .
Let ρ V : G → GL( V ) and
ρ W : G → GL( W ) be two linear representations.
The space of homomorphisms , also called intertwiners , Hom G, K ( V, W ) , consists of all linear, continuous functions f : V → W which commute with the two representations, i.e., f ◦ ρ V ( g ) =
ρ W ( g ) ◦ f for all g ∈ G .
We write for this space also Hom G, K ( ρ V , ρ W ) , depending on whether we want to emphasize the representation spaces or the representations.
Deﬁnition A.2 (Equivalences and Isomorphisms) .
Let ρ V : G → GL( V ) and
ρ W : G → GL( W ) be two linear representations.
An intertwiner f : V → W is called an equivalence if there is an intertwiner g : W → V such that g ◦ f = id V and f ◦ g = id W .
ρ V and ρ W , or also V and W , are then said to be equivalent .
If ρ V and ρ W are even unitary representations, and f and g additionally are unitary transformations, i.e., preserve the scalar product and thus distances, then V and W or ρ V and ρ W are called isomorphic , and f and g isomorphisms .
We write this as ρ V ∼ =
ρ W or V ∼ = W , depending on whether we want to emphasize the representations or the representation spaces.
Deﬁnition A.3 (Endomorphisms, Endomorphism Basis) .
Let ρ : G → GL( V ) be a linear representation.
The space of endomorphisms End G, K ( V ) consists of all linear, continuous maps c : V → V that commute with ρ , i.e., c ◦ ρ ( g ) =
ρ ( g ) ◦ c for all g ∈ G .
For ρ =
ρ j being an irrep, this space is ﬁnite-dimensional and Das a basis (the endomorphism basis ) ( c jr ) E j r =1 with E j = dim (End G, K ( V j )) .
Note that End G, K
( V j ) =
Hom G, K ( V j , V j ) .
By Schur’s Lemma, the endomorphism spaces of complex irreps are always 1 -dimensional, generated by the identity.
For real representations, they can be 1 , 2 , or 4 -dimensional, and the irrep is then correspondingly called of real type, complex type, or quaternionic type .
See Supplementary C for more details on real representations.
A. H OMOGENEOUS S PACES , S QUARE -I NTEGRABLE F UNCTIONS , AND THE P ETER -W EYL T HEOREM
Recall that a homogeneous space of the compact group G is a topological (Hausdorff) space X together with a transitive continuous action .
: G × X → X , ( g, x ) (cid:55)→ g.x = gx .
As is commonly known, G carries an inversion-invariant and left-right invariant so-called Haarmeasure µ , which can furthermore be assumed to be normalized: µ ( G ) = 1 .
This Haar measure also induces a corresponding G -action invariant measure ν on X , which we also call Haar-measure and assume to be normalized: ν ( X )
= 1 .
The normalization makes both Haar-measures the unique measures which are deﬁned on the whole
Borel Sigma algebras of these spaces and satisfying the mentioned invariance properties.
To reduce clutter, we will mostly not mention the used measure in integral-formulas, but we view it as understood that all integrals are with respect to these unique Haar measures.
Fix from now on a homogeneous space X of G .
Deﬁnition A.
(The Space of) Square-Integrable Functions) .
A square-integrable function f : X → K is a measurable function with the property that the square of its absolute value is integrable: (cid:82)
[Equation]
We denote the space of all such square-integrable functions by L 2 K ( X ) .
It carries a scalar product given by (cid:104)
f | g (cid:105) =
(cid:82) X
f
( x ) g ( x ) dx , making it a Hilbert space.
It also carries a unitary representation of G given by λ : G → U( L 2 K ( X )) , ( λ ( g ) f )( x ) :
=
f ( g − 1 x ) .
Deﬁnition A.5 (Regular Representation) .
In the case that X
=
G is the group itself, we call L 2 K ( G ) , together with the action [ λ ( g ) f ]( g (cid:48) ) :
= f ( g − 1 g (cid:48) ) , the regular representation of G .
Theorem A.6 (Peter-Weyl Theorem) .
There is a decomposition
[Equation]
into irreducible subrepresentations.
Thereby, V ji is isomorphic to the representative irreducible representation V j and generated by orthonormal harmonic basis functions Y mji : X → K .
We have m j ≤ d j := dim V j .
If D = C and X = G , then m j = d j .
Note that the hat (cid:99) ( · ) denotes a topological closure .
It is a technicality which the reader can mostly ignore without problems.
Proposition A.7 (Peter-Weyl Theorem: An Explicit Basis for Complex-Valued Square-Integrable
Functions on a Compact Group) .
In the complex ﬁeld K = C , the orthonormal harmonic basis for
L 2 C ( G ) is given explicitly by:
[Equation]
In other words, the matrix coefﬁcients of the complex irreps of G form an orthogonal basis for L 2 C ( G ) .
Deﬁnition A.8 (Harmonic Projections) .
We denote the orthogonal projection from L 2 K ( X ) to the i ’th copy of V j by p ji : L 2 K ( X ) → V j .
On the harmonic basis, it is given by
[Equation]
which just follows from the fact that the harmonic basis functions form an orthonormal basis of
L 2 K ( X ) , which is then mapped to the corresponding orthonormal basis of V j , the representation space of the representative irrep ρ j : G → U( V j ) .
One commonly known corollary of the Peter-Weyl theorem is the following.
We make use of it in the deﬁnitions of the Clebsch-Gordan coefﬁcients and irrep-decomposition coefﬁcients which we deﬁne in the next two subsections:
Corollary 1.
Any unitary representation of G , even inﬁnite dimensional, can be decomposed into a (topological closure of a) direct sum of irreps of G with a unitary isomorphism, i.e.: for all unitary representations π : G → U( V ) of G , there exists a unitary transformation M : V → (cid:99)(cid:76) i ∈ I
V i , where ρ i : G → U( V i ) are unitary irreducible representations such that for all g ∈ G the following identity holds:
[Equation]
I is a set indexing irreps in (cid:98) G , and the same irrep may occur multiple times.
If such an M exists, we also write V ∼ =
(cid:99)(cid:76) i ∈
I V i .
That M is a unitary transformation means, as explained earlier, that it preserves the scalar product and thus distances.
Consequently, one has M − 1 =
M † , i.e., the inverse is equal to the adjoint 10 .
In the ﬁnite-dimensional case, and working with matrices, one has that M † is given by the conjugate transpose of M , and if we additionally consider real-valued representations, then M † =
M T is just the transpose.
A. C ONSTRUCTING U NITARY R EPRESENTATIONS FROM O THERS AND THEIR R ELATIONS
Now we come to tensor-products and Clebsch-Gordan coefﬁcients.
Recall that given two ﬁnitedimensional unitary representations ρ V : G → U( V ) and ρ W : G → U( W ) , one can deﬁne a unitary representation ρ V ⊗
ρ W : G → U( V ⊗ W ) called tensor product given by ( ρ V ⊗ ρ W )( g ) =
ρ V ( g ) ⊗ ρ W ( g ) : v ⊗ w (cid:55)→ [ ρ V ( g )] ( v ) ⊗ [ ρ W ( g )] ( w ) .
Thereby, V ⊗ W is the tensor product of the vector spaces V and W .
If V and W have bases { v 1 , . .
. , v n } and { w 1 , . . .
, w m } , respectively, then the tensor product has a basis { v i ⊗ w j | i = 1 , . . . , n, j = 1 , . . .
, m } .
Correspondingly, represented with respect to given bases, the matrix corresponding to ρ V ( g ) ⊗ ρ W ( g ) is the Kronecker product of the matrices corresponding to ρ V ( g ) and ρ W ( g ) .
The scalar product on the tensor product is given by (cid:104) v ⊗ w | v (cid:48) ⊗ w (cid:48) (cid:105)
= (cid:104) v | v (cid:48) (cid:105) · (cid:104)
w | w (cid:48) (cid:105) .
The tensor product V ⊗ W has a universal property which relates bilinear functions on V × W to linear functions on V ⊗ W .
Recall that for an irrep ρ j : G → U( V j ) , d j denotes the dimension of V j .
Deﬁnition A.9 (Clebsch-Gordan Coefﬁcients) .
Let ρ l : G → U( V l ) and ρ J : G → U( V J ) be irreducible unitary representations of G .
There is a decomposition
[Equation]
of the tensor product in irreps, where [ j ( lJ )] is the multiplicity of V j in V l ⊗ V J .
This multiplicity is zero for all but ﬁnitely many j ∈ (cid:98) G .
We denote the coupling coefﬁcients between basis elements coming from this decomposition by (cid:104) s, jm | ln ; JM (cid:105) , where m ≤ d j , n ≤ d l , M ≤ d J and s ≤ [ j ( lJ )] .
They are called the Clebsch-Gordan coefﬁcients .
Remark A.10 .
The main application of the tensor product will be in the Wigner-Eckart theorem which we state in Section B, where ρ j will be the irrep corresponding to harmonic basis functions on a homogeneous space, ρ l will be the input representation, and ρ J will be the output representation.
Compared to the work Lang & Weiler (2020), this means that we will decompose V l ⊗ V J instead of
V j ⊗ V l , which leads to a simpler implementation for real representations, the case we are concerned with, at the cost of the necessity to invoke dual representations in the case that one works with complex representations.
The deﬁnition of these coefﬁcients can be made more explicit by invoking the corresponding projections and embeddings, which we will use in the proof of our adapted Wigner-Eckart Theorem.
We thereby deﬁne | ln (cid:105) , n ≤ d l , as the n ’th basis vector in V l , for a ﬁxed basis, and similarly for | jm (cid:105) and | JM (cid:105) .
Deﬁnition A.11 (Clebsch-Gordan Projections and Embeddings) .
We denote the projection from
V l ⊗ V J to the s ’th copy of V j by CG j ( lJ ) s : V l ⊗ V J → V j .
It has as matrix elements the ClebschGordan coefﬁcients for ﬁxed s, j, l, J :
[Equation]
Dually, we denote the embedding of the s ’th copy of V j into V l ⊗ V J by i j ( lJ ) s : V j → V l ⊗ V J .
It relates to the Clebsch-Gordan coefﬁcients as follows:
[Equation]
Note that this deﬁnition of Glebsh-Gordan coefﬁcient is compatible with the one in Sec. 2.2, for real representations.
Indeed, in Sec. 2.2, CG j ( lJ ) s denotes the matrix of Clebsch-Gordan coefﬁcients (cid:104) s, j (cid:101) m | lm
[Equation] JM (cid:105) of shape d j × ( d l · d J ) .
For a Hilbert space V , we set V † as the space of continuous linear functionals on V .
As mentioned above, by the Riesz representation theorem these linear functionals are given by all v † :
= (cid:104) v | for v ∈ V .
V † is then also a Hilbert space with the scalar product given by (cid:10) v † | w † (cid:11) =
(cid:104) w | v (cid:105) .
This deﬁnition of a dual can be extended to a dual of unitary representations:
Deﬁnition A.12 (Dual Representation) .
Let ρ : G → U( V ) be a ﬁnite-dimensional unitary representation.
Then we can deﬁne the dual representation ρ † : G → U( V † ) as follows:
[Equation]
If ρ l : G → U( V l ) is a ﬁnite-dimensional irreducible representation, then we write V l † :=
( V l ) † and ( ρ l ) † :=
ρ l † for a new auxiliary label l † .
If | ln (cid:105) , n = 1 , . .
, d l , are chosen basis elements of V l , then we choose as the basis of V l † the vectors | l † n (cid:105) :
= (cid:104) ln | , which are the functionals deﬁned by the property (cid:104) l (cid:101) n
| ln (cid:105) =
δ (cid:101) nn .
Similarly to how we deﬁned a tensor product representation above, one can also deﬁne Homrepresentations: Let ρ V : G → U( V ) and ρ W : G → U( W ) be ﬁnite-dimensional unitary representations.
Let Hom K ( V, W ) be deﬁned as the vector space of linear, not necessarily equivariant, functions from V to W .
Choosing bases of V and W , this space is isomorphic to K dim V · dim W and thus carries a Euclidean scalar product which makes it a Hilbert space.
The Hom-representation is then given by ρ Hom : G → U(Hom K ( V, W )) with [ ρ Hom ( g )]
( f ) :
= ρ W ( g ) ◦ f ◦ ρ V ( g ) − 1 .
Tensor product representations and Hom-representations are related as follows:
Proposition A.13.
Let ρ V : G → U( V ) and ρ W : G → U( W ) be ﬁnite-dimensional unitary representations.
The function Ψ HT :
V † ⊗ W → Hom K ( V, W ) given by
[Equation]
is an isomorphism of unitary representations.
The indices HT stand for “Hom” and “Tensor”.
Note that when working with real representations instead of complex representations, the map v (cid:55)→ v † is linear instead of only conjugate linear, and consequently, dual representations are isomorphic to the original representation.
In that case, one thus obtains V ⊗ W ∼ =
Hom R ( V, W ) .
A. C OMPACT S UBGROUPS AND R ESTRICTED R EPRESENTATIONS
Let in this subsection G ⊆ G (cid:48) be a compact subgroup of the compact group G (cid:48) .
By restriction, one can view any unitary representation of G (cid:48) as one of G , which will become important both in our treatment of the adapted Wigner-Eckart Theorem in Section B and in considerations on the induced representation of a given irrep in Section D:
Deﬁnition A.14 (Restricted Representation) .
Let ρ (cid:48) : G (cid:48)
→ U( V (cid:48) ) be a unitary representation of G (cid:48) .
We deﬁne Res G (cid:48)
G V (cid:48) :
= V (cid:48) and Res G (cid:48)
G ρ (cid:48) by
Res G (cid:48)
[Equation]
It is called the restricted representation of ρ on G .
Crucially, restrictions of irreps of G (cid:48) need not be irreps of G .
In analogy to the Clebsch-Gordan coefﬁcients, which emerge when decomposing tensor products, one can deﬁne irrep-decomposition coefﬁcients, which emerge when decomposing restrictions of irreps of the group G (cid:48) into irreps of G :
Deﬁnition A.15 (Irrep-Decomposition Coefﬁcients) .
Let ρ j (cid:48) : G (cid:48)
→ U( V j (cid:48) ) be an irrep of G (cid:48) .
Then there is a decomposition
[Equation]
where [ jj (cid:48) ] is the multiplicity of irrep V j of G in Res G (cid:48)
G V j (cid:48) .
This multiplicity is zero for all but ﬁnitely many j ∈ (cid:98) G .
We denote the coupling coefﬁcients between basis elements coming from this decomposition by (cid:104) t, jm | j (cid:48) m (cid:48) (cid:105) , where m ≤ d j , m (cid:48) ≤ d j (cid:48) , t ≤ [ jj (cid:48) ] .
We call them the irrep-decomposition coefﬁcients .
Deﬁnition A.16 (Irrep-Decomposition Projections) .
We denote the projection from Res G (cid:48)
G V j (cid:48) to the t ’th copy of V j by ID jj (cid:48)
t : Res G (cid:48)
G V j (cid:48)
→ V j .
Is has as matrix coefﬁcients the irrep-decomposition coefﬁcients for ﬁxed t, j, j (cid:48) :
[Equation]
Here, ID stands for “irrep” and “decomposition”.
B A G ENERALIZED W IGNER -E CKART T HEOREM FOR S TEERABLE K ERNELS
The kernel space solution from this work (Theorem 2.1) will be based on the Wigner-Eckart theorem for steerable kernels from Lang & Weiler (2020).
We thereby adapt that theorem in two practically relevant ways, and thus have to reprove the theorem: First, rather than working on a homogeneous space X of G , we work with an arbitrary space X equipped with an action of G .
The special case where X is a homogeneous space of G then recovers the result in Lang & Weiler (2020).
Second, by decomposing a certain Hom-representation instead of a tensor product representation, we make the computational process easier which ﬁnds the harmonic basis functions that build the equivariant kernels.
Finally, in Section B.3, we extend the previous result by reusing a G (cid:48)
-steerable basis to parameterize a G -steerable kernel space.
In this whole section, we freely make use of the concepts and notation deﬁned in Section A.
B. B ASIS -I NDEPENDENT F ORMULATION OF THE G ROUP -R ESTRICTED W IGNER -E CKART T HEOREM
Assumptions We explain the kernel space solution for the following general setting.
We assume G is a compact group.
Also, K is one of the two ﬁelds R or C (with our applications focused on the case R ).
Let X be a topological Hausdorff space equipped with the Borel σ -algebra, making it a measurable space.
We assume X is equipped with a measure µ and a continuous action of G on X .
Since we want L 2 K ( X ) to be a unitary representation over G , we assume the action of G on X to preserve the measure µ .
All these assumptions are, for example, satisﬁed if X is any homogeneous space of G , or if G is a subgroup of O( n ) and X ⊆ R n any subset of the Euclidean space that is preserved under the action of G .
Finally, ρ l : G → U( V l ) and ρ J : G → U( V J ) are irreducible unitary input and output representations of G .
Their dimension is denoted d l and d J , respectively.
In this setting, our goal is to determine a basis for the space of G -steerable kernels K : X →
Hom K ( V l , V J ) , which we deﬁne as any square-integrable function such that the steerability constraint is satisﬁed, i.e.: for all g ∈ G and x ∈ X , the following holds:
[Equation]
Note that, compared to Lang & Weiler (2020), the space X is in general not a homogeneous space of the group G .
We denote the space of all these G -steerable kernels by Hom G ( X, Hom K ( V l , V J )) .
More details and intuitions on this “abstract” deﬁnition of steerable kernels (however, only for the special case where X is a homogeneous space of G ) can be found in Lang & Weiler (2020), Appendix
C.1.
First, note that the space L 2 K ( X ) of square-integrable functions on X carries an action of G as well.
Similarly as in Lang & Weiler (2020), the main ingredient of the Wigner-Eckart Theorem is a correspondence between G -steerable kernels on X and kernel operators , i.e., intertwiners, on L 2 K ( X ) .
For this, recall that both Hom K ( V l , V J ) and L 2 K ( X ) carry a G -representation.
Deﬁnition B.1 (The Kernel-Operator Correspondence) .
For any intertwiner K : L 2 K ( X ) →
Hom K ( V l , V J ) , we deﬁne its restriction to X by
[Equation]
where δ x is the Dirac delta function at x ∈ X . 12 K| X is then a G -steerable kernel.
In the other direction, for any G -steerable kernel K : X → Hom K ( V l , V J ) , we deﬁne the extension to L 2 K ( X ) by
[Equation]
This is then an intertwiner.
Theorem B.2.
Those two operations are inverse to each other, i.e., (cid:98) K | X =
K and (cid:100) K| X = K .
Proof.
This follows from the well-known fact that L 2 K ( X ) is isomorphic to its own dual space .
First, note that an intertwiner K : L 2 K ( X ) →
Hom K ( V l , V J ) belongs to (cid:76)
d l · d J L 2 K ( X ) ∗ , i.e. to d l · d J copies of the dual space of L 2 K ( X ) .
Conversely, a kernel K : X → Hom K ( V l , V J ) belongs to (cid:76)
d l · d J L 2 K ( X ) , i.e. to d l · d J copies of L 2 K ( X ) .
For 1 < p < ∞ and q = pp − 1 , there is a natural isomorphism between the spaces L p ( X ) ∗ and L q ( X ) given by the map:
[Equation]
By choosing p = q = 2 we obtain the isomorphism between L 2 K ( X ) and L 2 K ( X ) ∗ .
Then, note that the operations (cid:99) ( · ) and ( · ) | X are respectively equivalent to Φ and Φ − 1 applied independently to each of the d l · d J subspaces isomorphic to L 2 K ( X ) or L 2 K ( X ) ∗ .
And ﬁnally, note that this isomorphism is still well-deﬁned when restricted to the equivariant kernels and kernel operators, respectively.
Note now that, since G is acting as an isometry on X , its action on L 2 K ( X ) is unitary.
By Corollary 1, it follows that this actions decomposes into a direct sum of irreps of G .
It follows that L 2 K ( X ) also decomposes into a direct sum of invariant subspaces
L 2 K ( X ) ∼ = (cid:99)(cid:76)
j ∈
ˆ G (cid:99)(cid:76) m j i V ij , where m j is the multiplicity of the irrep ρ j ∈ ˆ G and each V ij ∼ =
V j is acted on by G through ρ j ∈ ˆ G .
We denote a basis for V ij with { Y mji : X → K } d j m .
Note that m j may be inﬁnite, even uncountably large, since we do not assume X to be a homogeneous space of G .
Deﬁnition B.3 (Steerable Basis) .
The union of { Y mji : X → K | j ∈ ˆ G, i ≤ m j , m ≤ d j } forms a basis for L 2 K ( X ) .
We call this basis steerable since the action of G on a function L 2 K ( X ) is realized just by recombination of the linear coefﬁcients used to expand this basis (via its action on each invariant subspace through its irreps).
Note that this deﬁnition is compatible with the one in Freeman & Adelson (1991).
Similar to the harmonic projection in Def.
A.8, we deﬁne an orthogonal projection on the invariant subspaces of L 2 K ( X ) :
Deﬁnition B.4 (Steerable Basis Projections) .
We denote the orthogonal projection from L 2 K ( X ) to the i ’th copy of V j by p ji : L 2 K ( X ) → V j .
On the steerable basis, it is given by
[Equation]
which just follows from the fact that the steerable basis forms an orthonormal basis of L 2 K ( X ) , which is then mapped to the corresponding orthonormal basis of V j , the representation space of ρ j ∈ ˆ G .
With all this in mind, we can formulate our basis-independent Wigner-Eckart Theorem for G -steerable kernels:
Theorem B.5 (Wigner-Eckart Theorem for Steerable Kernels) .
The basis-independent version of the
Wigner-Eckart theorem consists of two parts:
There is an isomorphism of vector spaces GKer : (cid:91) (cid:77)
[Equation]
where the hat (cid:99) ( · ) denotes a topological closure and can be ignored by the reader.
Explicitly, GKer is given by
[Equation]
Proof.
For the ﬁrst statement, we observe:
[Equation]
[Equation] End
[Equation]
In (1) , we perform the linear extension to the space of square integrable functions from Deﬁnition B.1, using Theorem B.2.
In (2) , we use the Peter-Weyl Theorem A.6, and the isomorphism Ψ HT which we deﬁned in Proposition A.13.
Note that the decomposition from the Peter-Weyl Theorem is G -equivariant, so this step is valid.
In (3) , we use the Clebsch-Gordan decomposition from
Deﬁnition A.9, and Schur’s Lemma which shows that V j can only have a nontrivial homomorphism to V j , and not to any other irrep.
From right to left, we call the isomorphism GKer .
This proves the ﬁrst statement.
For the second statement, we go through the sequence of isomorphisms from bottom to top and trace back where an arbitrary endomorphism tuple “comes from”:
GKer : ( c jis ) jis
[Equation]
[Equation]
This ﬁnishes the proof of the second statement, and thus of the theorem.
B. S TEERABLE K ERNEL B ASES
After the basis-independent version of the theorem, we now state how a basis for the space of steerable kernels can be constructed.
The basis elements of the representation space V l , V J and V j are denoted | ln (cid:105) , | JM (cid:105) and | jm (cid:105) , respectively, where the indices range in 0 ≤ n ≤ d l , 0 ≤ M ≤ d J , and 0 ≤ m ≤ d j .
Sometimes, we write also Y MJ instead of | JM (cid:105) , etc., in order to remind to the connection with harmonic basis functions.
Theorem B.6 (Steerable Basis Kernels) .
A basis of the space of G -steerable kernels
Hom G ( X, Hom K ( V l , V J )) is given by basis kernels (cid:8) K jisr (cid:12)(cid:12)
j ∈
ˆ G, i ≤ m j , s ≤ [ j ( l † J )] , r ≤ E j (cid:9) , which are given as follows: K jisr =
GKer( c jisr ) , where c jisr is zero at every entry except at position jis , where it takes on value c jr .
Furthermore, the matrix-elements of such a basis kernel with respect to orthonormal bases | ln (cid:105) of V l and | JM (cid:105) of V J , where M ≤ d J and n ≤ d l , are given by:
[Equation]
where the overline denotes complex conjugation, and with (cid:104) i, jm | x (cid:105) :
= Y mji ( x ) for steerable basis functions Y mji : X → K .
Proof.
That these kernels form a basis follows from the fact that the c jisr obviously form a basis of endomorphism tuples, and that GKer is an isomorphism by Theorem B.5.
Concretely, this theorem then shows that
[Equation]
We write (cid:91) K jisr for the extension from Deﬁnition B.1, which is just given by (cid:91) K jisr =
Ψ
HT ◦
i j ( l † J ) s ◦ c jr ◦ p ji .
Furthermore, we expand the Dirac delta function δ x for x ∈ X in the steerable basis of
L 2 K ( X ) as
[Equation]
This expansion can be justiﬁed with a similar approximation procedure as in the proof of Lang & Weiler (2020), Theorem D.13.
Thereby
[Equation]
we will in the following write the coefﬁcients as
[Equation]
i,
[Equation]
δ x (cid:69) , which is also equal to Y (cid:101) m (cid:101) j (cid:101) i ( x ) .
With these tricks
[Equation]
we obtain:
[Equation]
We ﬁrst explain the steps so far and will then deal with the last term.
Step (1) uses the extension of
K from X to L 2 K ( X ) .
Step (2) uses the expansion of δ x explained before.
Step (3) uses the property of the projection discussed in Deﬁnition B.4 and a renaming of (cid:101) m to m .
Step (4) expands c jr ( Y mj ) in terms of the basis vectors Y (cid:101) mj , with the expansion coefﬁcients being the matrix elements of c jr .
Step (5) is then just a reordering of the terms.
[Table]
[Equation]
In step (a), we expand i j ( l † J ) s
( Y (cid:101) m j ) in the basis of V l † ⊗ V J .
In step (b), we use the deﬁnition of the Clebsch-Gordan coefﬁcients using the embedding i j ( l † J ) s and of Ψ HT given in Proposition A.13.
Also, the deﬁnition of the dual basis given after Deﬁnition A.12 is used as the equality | l † (cid:101) n (cid:105) =
(cid:104) l (cid:101) n | .
In step (c), we use eq. 9.
Step (d) and ( e ) are clear.
Plugging this intermediate result into our earlier computation ﬁnishes the proof.
We now formulate a matrix-version of this result that is more suitable to implementation.
We thereby identify K jisr ( x ) as a matrix in K d J × d l , for chosen bases | JM (cid:105) in V J and | ln (cid:105) in V l .
I.e.,
K jisr ( x ) is viewed as the matrix with coefﬁcients (cid:104) JM | K jisr ( x ) | ln (cid:105) .
Furthermore, we identify c jr with the matrix in K d j × d j with coefﬁcients (cid:10) j (cid:101) m (cid:12)(cid:12) c jr (cid:12)(cid:12) jm (cid:11) .
Additionally, with Y ji ( x ) we mean the column vector in K d j with entries (cid:104) i, jm | x (cid:105) .
Finally, with CG j ( l † J ) s we mean the matrix of
Clebsch-Gordan coefﬁcients (cid:10) s, j (cid:101) m
(cid:12)(cid:12) l † m ; JM (cid:11) of shape
d j × ( d l × d J ) .
Its conjugate transpose is denoted as (cid:104) CG j ( l † J ) s (cid:105) † , which is of shape ( d J × d l ) ×
d j .
When using it in matrix multiplications, it is interpreted as having d J · d l “rows” and d j columns.
This is compatible with the vectorization of steerable kernels which was discussed in the main paper.
Theorem B.7 (Matrix-Version of Steerable Basis Kernels) .
The basis kernel evaluated at x , K jisr ( x ) , is given by the matrix
[Equation]
where each dot means conventional matrix multiplication.
Proof.
We do the sanity check, i.e., we test whether the resulting matrix is well-deﬁned and of shape
d J × d l .
This can be veriﬁed by observing that the shape of the right-hand-side of eq.
12 is:
[Equation]
The full proof of eq.
12 follows directly from eq.
Remark B.8 .
Note that if X is a homogeneous space for G , then the steerable basis { Y mji } jim is the harmonic basis in Theorem A.6.
Additionally, if all endomorphism spaces are 1 -dimensional, which is for example always the case for complex representations, then c jr can be chosen to be the identity matrix and thus completely omitted from the formula.
Additionally, if one is concerned with real representations, then l † = l and the complex conjugation in the Clebsch-Gordan coefﬁcients can be omitted, which leads to the following formula which we implement in this work:
[Equation]
For the special case that G = SO(3) , X = S 2 and K
=
R , one obtains the basis kernels
[Equation]
where for | l − J | ≤ j ≤ l + J , the Y j : S 2 → R 2 j +1 are the spherical harmonics.
This result was derived in Weiler et al.
(2018a).
Many other examples for X homogeneous space of G , with a tensor-product decomposition instead of a Hom-space decomposition, can be found in Lang & Weiler (2020).
B. A W IGNER -E CKART T HEOREM FOR G ROUP -R ESTRICTED S TEERABLE K ERNELS
In this section, we consider a special case of the previous theorem where X carries an action of a compact group G (cid:48) , with G ≤ G (cid:48) .
We assume this action to extend the action of G on X .
This result is useful to reuse G (cid:48) -steerable kernels to parameterize new G -steerable spaces without designing a new basis for L 2 K ( R n ) .
First of all, we denote by ˆ G (cid:48) a set of representatives of the irreps of G (cid:48) and by { Y m (cid:48) j (cid:48) i (cid:48)
| j (cid:48) ∈
ˆ G (cid:48) , i (cid:48) ≤ m j (cid:48) , m (cid:48) ≤ d j (cid:48) } a G (cid:48) -steerable basis for X .
Recall also the deﬁnition of group restriction and irrep decomposition coefﬁcients from Deﬁnition A.15.
In particular, [ jj (cid:48) ] is the multiplicity of ρ j ∈ ˆ G inside Res GG (cid:48)
ρ j (cid:48) , with j (cid:
) ∈
ˆ G (cid:48) , and ID jj (cid:48) t is the projection from Res G (cid:48)
G V j (cid:48) to the t ’th copy of V j (Deﬁnition A.16).
Proposition B.9.
The G (cid:48) -steerable basis for X given by { Y m (cid:48) j (cid:48)
[Equation]
} can be turned into a G -steerable basis deﬁned as:
[Equation]
Proof.
Because the space X carries an action of G (cid:48)
[Equation]
we can ﬁnd the action of G on L 2 K ( X ) via group restriction:
[Equation]
We can plug this basis in Theorem B.7 to obtain the following result:
Theorem B.10 (Matrix-Version of Group-Restricted Steerable Basis Kernels) .
The basis kernel evaluated at x , K j [Equation] i [Equation]
jtsr ( x
[Equation]
) , is given by the matrix
[Equation]
where each dot means conventional matrix multiplication.
B. E XTENDED E XAMPLES OF S TEERABLE K ERNEL B ASES
In this section, we provide an extended version of the two examples in Section 2.2. In the following examples we will consider the space X = R 2 and the two groups SO(2) and C 4 , i.e. the group of all planar rotations and the group of rotations by multiples of π/ 2 radians.
SO(2) -Steerable
Kernels First of all, recall the (real) irreducible representations of G = SO(2) .
For r θ ∈ SO(2) :
[Equation]
where j is a non-negative integer which can be interpreted as the rotational frequency.
All irreps aredimensional but for the frequency j = 0 , which is 1 dimensional.
We consider the basis B =
{ Y ji } ji generated by circular harmonics combined with a Gaussian radial proﬁle, as in Worrall et al.
Weiler et al.
(2018b)
; Weiler & Cesa (2019), deﬁned as:
[Equation]
where ω R
i : R → R is a Gaussian radial kernel centered around R i ∈ R , deﬁning a ring of radius
R i .
Hence, in this basis, i indexes the circular shells, while j indexes the angular frequencies along each ring.
For simplicity, in the following examples we will only consider a single ring, indexed by i = 1 .
This setting is similar to the X = S 1 examples discussed in Section 2.2, since each ring is isomorphic to a circle
S 1 .
One can verify that the basis elements Y ji in B are G = SO(2) -steerable using the irrep ρ j , i.e.
[Equation]
To apply Theorem 2.1, we need a basis for the endomorphism space of each irrep and the irreps decomposition between their tensor-products.
If j > 0 , ρ j is a real irrep of complex type so its endomorphism space is 2 -dimensional and is spanned by 14 :
[Equation]
If J = 0 , ρ 0 ⊗ ρ l = ρ l and if l = 0 , ρ J ⊗ ρ 0 = ρ J .
For J, l > 0 , ρ J ⊗ ρ l ∼ =
ρ | J − l | ⊕ ρ J + l if J (cid:54) =
l or ρ 0 ⊕ ρ 0 ⊕ ρ J + l
otherwise.
In particular, in the ﬁrst case, using some trigonometric identities, one can verify that
CG lJ = 1 √   1  0 1 − 1 −0  
 satisﬁes ρ J ⊗ ρ l = [CG lJ ] T (cid:0) ρ | J − l | ⊕ ρ J + l (cid:1) CG lJ .
In this case, CG j ( lJ )= 1 √(cid:20) 1  0 1 − 1(cid:21)
if j = | J − l | and CG j ( lJ )= 1 √(cid:20) 1 0 0 −0 1 (cid:21) if j = J + l .
Thus, for l (cid:54) =
J > 0 , Theorem 2.
prescribes a basis containing the following elements (for a single ring at radius R i ):
[Equation]
r
[Equation]
·   cos( | J − l | φ ) sin( | J − l | φ ) − sin( | J − l | φ ) cos( | J − l | φ ) 
 , κ | J − l | ,i, 1 , 2 ( r, φ ) =
ω R i ( r ) ·    sin( | J − l | φ ) − cos( | J − l | φ )
cos( | J − l | φ ) sin( | J − l | φ ) 
  , κ J + l,i, 1 , 1 ( r, φ )
=
ω R i ( r ) ·   cos(( J + l ) φ ) sin(( J + l ) φ ) sin(( J + l ) φ ) − cos(( J + l ) φ ) 
 , κ J + l,i, 1 , 2 ( r, φ ) =
ω R i ( r ) ·   sin(( J + l ) φ ) − cos(( J + l ) φ ) − cos(( J + l ) φ ) − sin(( J + l ) φ ) 
 .
Observe that these 4 basis kernels are identical to the ones in Table 8 of Weiler & Cesa (2019), up to vectorization and a minus sign for κ | J − l | ,i, 1 , 2 and κ J + l,i, 1 , 2 .
Fig.
2 shows the two kernels among these four obtained with j = | J − l | = 1 , in the case l = 1 and J = 2 .
For l = 0 , j = J and CG 0 J is the identity matrix; then, for J > 0 , Theorem 2.1 prescribes the following basis elements:
[Equation]
which are also shown in Fig.
C 4 -Steerable Kernels We construct a G = C 4 -steerable basis through Eq.
5 by considering the
G (cid:48)
{+= SO(
+}
) -steerable basis described before in Eq.
First, recall the representation theory of
G = C 4 .
For p ∈ { 0 , 1 , 2 , 3 } , r p π 2 = ( r π 2 ) p ∈ C 4 ; C 4 has the following three irreps:
[Equation]
which are, respectively, 1 , 2 and 1 dimensional.
Again, we index the irreps by j ∈ { 0 , 1 , 2 } , which can be interpreted as the rotational frequency.
To apply Eq.
5, we need to know the irreps decomposition of Res SO(2)Cρ j (cid:48) , for each irrep ρ j (cid:48) of SO(2) .
We have, for t ∈ N :
[Equation]
0 −
[Equation]
−
[Equation]
so, ID j (cid:48) =
(cid:20) 0 −(cid:21) if j (cid:48)
= 4 t + 3 and ID j (cid:48) is the identity otherwise.
Thus, if B (cid:48) = { Y j (cid:48) i (cid:48) } j (cid:48) i (cid:48) is the
SO(2) -steerable basis described before in Eq. 15, according to Eq. 5, a C 4 -steerable basis B contains the following elements for each t ∈ N :
[Equation]
The endomorphism space of ρ 0 and ρ 2 is one dimensional and is spanned by the identity, while the endomorphism space of ρ 1 is spanned by (cid:26)
[Equation]
The tensor products decompose as follows:
[Equation]
where
CG 11 = 1 √   1  0 1 − 1 −0  
 whose rows are respectively denoted CG 0(11) , CG 0(11) , CG 2(11) and CG 2(11) .
Finally, Theorem 2.
prescribes the following basis elements for each t ∈ N and each i (cid:48) :
• l, J ∈ { 0 ,
[Equation]
where j = | l − J | ∈ { 0 , 2 } .
• l = 0 and J = 1 (or l = 1 and J = 0 ):
[Equation]
[Table]
• l = 2 and J = 1 (or l = 1 and J = 2 ):
[Table]
[Equation]
[Table]
• l = 1 and J = 1 : κ
[Table]
) − cos(4 tφ ) 0] T , κ 0 , ( i (cid:48) , 4 t, 2) , 1 , 1 ( r, φ ) =
ω R i (cid:48)
( r ) [sin(4 tφ ) 0 0 sin(4 tφ )]
T , κ 0 , ( i (cid:48) , 4 t, 2) , 2 , 1 ( r, φ ) =
ω R i (cid:48)
{+( r ) [0 sin(4 tφ ) − sin(4 tφ ) 
+}
T , κ 2 , ( i (cid:48) , 4 t +2 , 1) , 1 , 1 ( r, φ ) =
ω R i (cid:48)
( r ) [cos((4 t + 2) φ ) 0 0 − cos((4 t + 2) φ )] T , κ 2 , ( i (cid:48) , 4 t +2 , 1) , 2 , 1 ( r, φ ) =
ω R i (cid:48)
( r ) [0 cos((4 t + 2) φ ) cos((4 t + 2) φ ) 0] T , κ 2 , ( i (cid:48) , 4 t +2 , 2) , 1 , 1 ( r, φ ) =
ω R i (cid:48)
( r ) [sin((4 t + 2) φ ) 0 0 − sin((4 t + 2) φ )] T , κ 2 , ( i (cid:48) , 4 t +2 , 2) , 2 , 1 ( r, φ ) =
ω R i (cid:48)
( r ) [0 sin((4 t + 2) φ ) sin((4 t + 2) φ ) 0] T
Fig 4 shows the basis kernels for l = 0 and t = 0 .
For simplicity, in Fig 4, we use k ∈ Z rather that t ∈ N to index the basis; this difference emerges from the fact that ρ j (cid:48) is isomorphic to ρ 1 when restricted from SO(2) to C 4 for any j (cid:48)
=
| 1 + 4 k | , but k > 0 and k < 0 require a different change of basis (see 4 t + 1 and 4 t + 3 above).
Observe also that these basis kernels span the same space of the ones in Table 11 of Weiler & Cesa (2019), for N = 4 , up to vectorization.
C R EAL V ALUED R EPRESENTATIONS
In this section, we discuss the representation theory of compact groups on the real ﬁeld R .
Usually in representation theory, one assumes complex -valued signals in L 2 C ( G ) and complex representations of G .
However, in this work, we implement equivariant CNNs using real representations, and therefore must discuss general issues arising in this case.
When using real representations, the theory becomes more difﬁcult: while the matrix coefﬁcients of the irreducible representation span the space of real square-integrable functions L 2 R ( G ) , they are not necessarily orthogonal to each other anymore.
More precisely, the matrix coefﬁcients belonging to the same irrep might be linearly dependent, while different irreps still contain orthogonal coefﬁcients.
This is a consequence of the fact that the strong version of Schur’s Lemma stating that the endomorphism space of irreps are one dimensional does not hold for representations over R .
As before, we freely make use of the preliminaries from Section A.
In particular, recall that for real representations, we talk about orthogonal representations instead of unitary representations, and such representations take values in the orthogonal group O( V ) of some real Hilbert-space V .
In Section C.1, we introduce some properties of real irreducible representations and characterize their endomorphism space.
In Section C.2, we show that any real irrep can always be expressed with respect to a convenient basis such that a set of linearly independent matrix coefﬁcients are contained in a subset of the columns of the irrep.
This will be particularly useful to construct real harmonic bases in Section D and, in particular, in Corollary 5.
Additionally, we describe a method to ﬁnd such a basis for an arbitrary real irrep.
Finally, in Section C.3, we show an example of such basis, which makes the result more intuitive.
Additionally, we use this speciﬁc basis to simplify computations in Section F.
C. P RELIMINARIES ON THE S TRUCTURE OF End G, R ( V ψ )
Fix an orthogonal irreducible real representation ψ : G → O( V ψ ) of the compact group G .
Let
End G, R ( V ψ ) be its endomorphism algebra.
It is well-known that this endomorphism algebra is isomorphic (in a sense which will be made precise in the following proposition) to one of the division algebras R , C , and H , i.e., the real numbers, complex numbers, or quaternions, see Bröcker & Dieck (2003), Theorem 6.7.
Let K ∼
=
End G, R ( V ψ ) be this division algebra.
Let E ψ = dim End G, R ( V ψ ) its dimension.
Let i 1 = 1 , i
=
i , i 3 =
j and i 4 =
k with 1 , i, j, k ∈ H the usual basis.
It has the characterizing properties i
=
j 2 = k 2 = − 1 , that 1 is a multiplicative identity, and that ij = k
= − ji , jk = i = − kj , and ki =
j = − ik .
Then K has as an orthonormal basis { i l } E ψ l =1 .
We now make the properties of the isomorphism precise:
Proposition C.1.
There is K ∈ { R , C , H } such that there is a function Φ : K
→ End G, R
( V ψ ) with the following properties:
Φ( x + y ) =
Φ( x ) + Φ( y ) for all x, y ∈ K . 2.
Φ( λx ) =
λ Φ( x ) for all λ ∈ R and x ∈ K . 3.
Φ( xy ) =
Φ( x ) ◦ Φ( y ) for all x, y ∈ K . 4. Φ(1) = id V ψ . 5.
Φ is bijective.
Φ( i k ) ∈
O( V ψ ) for all k ∈ { 1 , . .
, E ψ } .
Proof.
Properties 1 to 5 are the properties of an R -algebra isomorphism.
Such an R -algebra isomorphism Φ exists by Bröcker & Dieck (2003), Theorem 6.7.
We now show that this automatically already implies property 6 as well: Φ( i l ) ∈
End G, R ( V ψ ) is a non-zero endomorphism, and thus, by Schur’s Lemma for orthogonal representations (see Lang & Weiler (2020), Lemma B.29), there exists a coefﬁcient µ l ∈ R such that µ l Φ( i l )
∈
O( V ψ ) .
We deduce:
[Equation]
Together with µ l Φ( i l ) ◦ µ l Φ( i l ) ∈
O( V ψ ) we necessarily have µ 2 l ∈ {± 1 } , and thus µ l = ± 1 .
Thus, Φ( i l ) = ± µ l Φ( i l ) ∈
O( V l ) , which shows 6 .
Deﬁnition C.2.
Let ψ be a real irrep.
Depending on its endomorphism algebra End G ( ψ ) , ψ is classiﬁed in one of the following three categories:
• real type : if End G ( V ψ ) ∼ = R • complex type : if End G ( V ψ ) ∼ = C • quaternionic type : if End G ( V ψ ) ∼ =
H
From now on, ﬁx the isomorphism Φ : K
→ End G, R
( V ψ )
and denote I l :=
Φ( i l ) .
For simplifying the notation, we set d ψ := dim ψ := dim V ψ .
From now on, we assume for simplicity that V ψ = R d ψ , which holds up to isomorphism.
The scalar product on V ψ is then just given by (cid:104) v | w (cid:105)
= v T w , and V ψ has the standard basis { e 1 , . . .
, e d ψ } with e i having a 1 at position i and being zero elsewhere.
For any A ∈ End G, R ( V ψ ) , we deﬁne A T ∈
End G, R ( V ψ ) as the transpose, which is also the unique matrix and linear function A T : V ψ → V ψ such that (cid:104) Av | w (cid:105) =
(cid:104) v | A T w (cid:105) for all v, w ∈ V ψ .
That
A T is also an endomorphism is for the fact that ψ is an orthogonal representation.
We deﬁne the following inner product on End G, R ( V ψ ) :
[Equation]
(16) where Tr is the trace of a matrix, given by the sum of diagonal elements.
The trace is linear and continuous and has the property
Tr( AB ) =
Tr( BA ) , which results in the trace of a matrix being independent of the basis in which it is expressed.
Lemma C.3.
We have the following:
I l = − I − 1 l for l > 1 . 2.
[ I l ] ii = 0 for l > 1 and i ∈ { 1 , . . .
, d ψ } .
(cid:104)
I l | I l (cid:48) (cid:105)
=
δ ll (cid:48) for all l .
Proof.
This follows from I l ◦ I l =
Φ( i 2 l ) = − Φ(1) = − id V ψ for all l > 1 .
From Proposition C.1 we know
that I l ∈ O( V ψ ) for all l .
So, I − 1 l = I Tl .
For l > 1 , we deduce I l = − I Tl from 1, i.e., I l + I Tl = 0 .
It follows [ I l ] ii =
[ I l + I Tl ] ii = 0 .
Note that there is some l (cid:48)(cid:48) ∈ { 1 , . . .
, E ψ } such that the following holds:
[Equation]
Thereby, we made use of I l (cid:48)
∈
O( V ψ ) in step 2 , and used the multiplication properties of the quaternions in the second to last step.
Note that i l (cid:48)(cid:48)
=
if and only if l = l (cid:48) , and that in this case, we have I l (cid:48)(cid:48) =
Φ( i l (cid:48)(cid:48) ) =
id V ψ , leading to (cid:104) I l | I l (cid:48) (cid:105)
= 1 .
If, however, l (cid:48)(cid:48) > 1 (i.e., l (cid:54) =
l (cid:48) ), then we have [ I l (cid:48)(cid:48) ] ii = 0 for all i by part 2, and thus we get a zero trace, proving the result.
Corollary 2.
The function Φ : K
→ End G, R
( V ψ ) is a unitary isomorphism, meaning: When considering K as an R -vector space with standard inner product (cid:104)· | ·(cid:105) and considering End G, R ( V ψ ) as an R -vector space with inner product given by eq.
16, then we have for all x, y ∈ K
[Equation]
Furthermore, { I 1 , . .
, I E ψ } forms an orthonormal basis of End G, R ( V ψ ) .
Proof.
First, note that eq.
16 clearly deﬁnes a positive-deﬁnite inner product on End G, R ( V ψ ) .
Write x = (cid:80) E ψ l =1 x l i l and y =
(cid:80) E ψ l (cid:48)
=1 y l (cid:48)
i l (cid:48) .
Using Lemma C.3, part 3, we obtain:
[Equation]
l =
E ψ (cid:88) l (cid:48) =
x l y l (cid:48) (cid:104) I
l | I l (cid:48) (cid:105) =
E ψ (cid:88) l = x l y l = (cid:104)
[Equation] (cid:105) The last statement that the I l form an orthonormal basis of End G, R ( V ψ ) follows from Lemma C. part 3 as well, together with the fact that Φ( i l ) =
I l and that Φ is an isomorphism.
For x = (cid:80) E ψ l =1 x l i l ∈ K , the conjugation x is deﬁned by [Equation] −
[Equation]
ψ (cid:88) l =
[Equation]
l i l . Furthermore, we deﬁne the norm (cid:107) W (cid:107) :
= (cid:112) (cid:104) W | W (cid:105) for W ∈ End G,
R ( V ψ ) .
Proposition C.4.
We have the following:
For x ∈ K , we have Φ( x ) =
Φ( x ) T .
For any W ∈ End G, R ( V ψ ) , we have WW T
= (cid:107) W (cid:107) 2 · id V ψ .
Proof.
We have Φ( x ) = Φ (cid:32) x 1 i 1 − E ψ (cid:88) l = x l i l (cid:33) = x 1 I 1 −
E ψ (cid:88) l = x l I l = x 1 I T 1 +
E ψ (cid:88) l = x l I Tl = (cid:32) E ψ (cid:88) l = x l I l (cid:33) T = (cid:32) Φ (cid:18) E ψ (cid:88) l = x l i l (cid:19)(cid:33) T = Φ( x ) T .
In the third step, we used I
= id V ψ
and I Tl = − I l , which was shown in Lemma C. together with I
l ∈ O( V l ) .
C. O RTHOGONALITY AND A LIGNMENT OF M ATRIX C OEFFICIENTS
The investigations here on the orthogonality relations of matrix coefﬁcients of irreducible real representations generalize what’s presented in Knapp (2002) for complex representations to real representations.
The results thereby become a bit more involved.
Remember that V ψ = R d ψ has the standard basis { e 1 , . . .
, e d ψ } .
Deﬁne E ij :=
e i e Tj ∈ R d ψ × d ψ .
Furthermore, deﬁne
[Equation]
Remember that ψ : G → O( V ψ ) =
O( R d ψ ) ⊆
R d ψ × d ψ takes values in d ψ × d ψ -matrices.
For k, j ∈ { 1 , . . .
, d ψ } , deﬁne ψ kj : G → R as the function ψ kj ∈ L 2 R ( G ) given by ψ kj ( g ) :
= ψ ( g ) kj .
Remember that we have a scalar product on L 2 R ( G ) given by (cid:104) f
| h (cid:105) := (cid:82) G f ( g ) h
( g )d g , which means we can evaluate also scalar products of different matrix coefﬁcients.
Lemma C.5.
We have the following:
O ij ∈
End G, R ( V ψ ) .
O ij = 1 d ψ (cid:80) E ψ l =
[ I l ] ij · I l . 3.
O ij = (cid:0) (cid:104) ψ k 1 i | ψ k 2 j (cid:105) (cid:1)
d ψ k 1 ,k 2 =1 .
Proof.
We prove the three statements as follows:
From the integration -measure on G being a Haar measure, i.e., left-invariant, it can easily be checked that O ij ψ ( g (cid:48) )
=
ψ ( g (cid:48) )
O ij for all g (cid:48) ∈ G .
From O ij ∈ End G, R ( V ψ ) and the fact that the I l form an orthonormal basis of the space
End G, R ( V ψ ) by Corollary 2, we obtain O ij =
(cid:80) E ψ l =1 (cid:104)
I l
| O ij (cid:105) I l .
We now determine the coefﬁcients:
[Equation]
In (1) , we use that I l is an endomorphism, i.e., commutes with ψ ( g ) .
Additionally, we use that Tr is linear and continuous, and thus commutes with integrals.
Step (2) uses that the trace satisﬁes Tr( AB ) =
Tr( BA ) .
In the last step, we make use of E ij =
e i e T j .
We have
[Equation]
Corollary 3.
We have (cid:104) ψ k 1 j | ψ k 2 j (cid:105) =
δ k 1 k 2 d ψ for all k 1 , k 2 , j ∈ { 1 , . . .
, d ψ } .
In other words, the matrix coefﬁcients in the same column are orthogonal to each other, and they are normalized up to a constant factor: (cid:107) (cid:112)
d ψ · ψ kj (cid:107) = 1 .
Proof.
Remember that in Lemma C.3 we showed [ I l ] jj = 0 for l > 1 and that I 1 = id V ψ .
Together with Lemma C.5, we obtain:
[Equation]
The statement about the normalization is clear.
Lemma C.6.
For any p, q ∈ { 1 , . . .
, E ψ } and for any v ∈ V ψ , (cid:104)
I p v |
I q v (cid:105) =
δ pq (cid:104) v | v (cid:105) .
Proof.
Note that there is a l such that I Tp I q = ±
I l .
If p = q , I Tp I p
= I 1 = id V ψ is the identity matrix, hence v T
I
Tp I q v = v T v .
Otherwise, l >
and I l = − I Tl is a skew symmetric matrix, hence v T I l v = 0 .
Indeed, note that, for any matrix A , it holds that v T A v = v T A T v .
If A is a skew symmetric matrix ( A T = − A ), it follows that v T A v = 0 .
Lemma C.7.
For each column j , the set of matrices { O ij } i spans the space
End G ( V ψ ) , which is an
E ψ dimensional space.
Proof.
To prove this, it is sufﬁcient to look at the set of coefﬁcient vectors of each of these matrices when expressed with respect to the basis { I l } l of End G ( V ψ ) .
If this set of coefﬁcient vectors spans an E ψ dimensional space, the proof is complete.
Because O ij = 1 d ψ (cid:80) l [ I l ] ij I l , let’s consider the set of vectors { ([ I l ] ij ) l } d ψ i =1 and stack them in the rows of a matrix M ∈ R d ψ × E ψ .
The set of vectors spans an E ψ dimensional space if and only if M has row rank or, equivalently, column rank E ψ .
Because M has E ψ nonzero columns (namely, the l ’th column of M is the j ’th column of I l , which is non-zero since I l is invertible), then it has column rank equal to E ψ if and only if all its columns are pairwise orthogonal.
The orthogonality of the columns of M follows from Lemma C.6.
Indeed, note that the l -th column of
M contains the j -th column of I l .
By choosing v = e j in Lemma C.6, it follows that the column j of two different basis elements I l 1 and I l 2 are orthogonal.
Thus, the columns of M are orthogonal.
From Lemma C.7, and since End G ( V ψ ) contains only the zero-map and isomorphisms, it follows that, for any j , the set of vectors { O ij e j } d ψ i =1 spans an E ψ dimensional vector space.
We will show that this means that there is a change of basis B ∈ R d ψ × d ψ and a corresponding irrep (cid:101) ψ = B T ψB isomorphic to ψ such that, for any j , { (cid:103)
O ij e j } d ψ i =1 contains only E ψ non-zero vectors, where (cid:103)
O ij := (cid:82) G (cid:101) ψ ( g ) E ij (cid:101) ψ
( g ) T d g .
We give a constructive proof now.
First, we build one such matrix B starting from the basis elements { I l } E ψ l =1 .
Then, we show that for (cid:101) ψ = B T ψB , for each j , the matrix (cid:103)
O ij is non-zero precisely for
E ψ different values of i .
Lemma C.8.
For any vectors v , w ∈ R d ψ , if (cid:104) I p v | w (cid:105) = 0 for all p , then (cid:104) I p v | I q w (cid:105) = 0 for any p and q .
Proof.
For any p and q , there exists l such that I p I
Tq = ±
I l .
Then (cid:104) I p v |
I q w (cid:105) =
v T
I Tp I q w = ± v T I Tl w = ±(cid:104)
I l v | w (cid:105) which is zero by assumption.
To construct the matrix B , we use the following algorithm.
Let B = ∅ ⊆
R d ψ the empty set.
Repeat the following process d ψ /E ψ times:
Take a normalized vector v ∈ R d ψ which is orthogonal to all vectors in B .
Add the vectors { I l v } E ψ l = to B .
Note that all vectors { I l v } E ψ l =1 are orthogonal to each other because of Lemma C.6.
Note that if
B contains the vectors { I l w } l and we choose a vector v such that it is orthogonal to all { I l w } l , then, due to Lemma C.8, the updated set B still contains only pairwise orthogonal vectors.
The matrix B is built by stacking horizontally the vectors in B .
Note that since all v in the process are normalized, and the I l v are normalized as well by Lemma C.6, the matrix is orthogonal, i.e.
B T B = BB T = id .
By construction, for any l , the left multiplication of B by I l only results in a permutation of the columns of B (and, potentially, ﬂipping their sign).
In particular, I l only permutes the vectors in B belonging to the same group { I l v } l .
Hence, there exists a permutation matrix P l (with, potentially, negative entries)
such that I l B = BP l .
Let, as mentioned before, (cid:101) ψ ( g ) :
= B T ψ ( g ) B , i.e., (cid:101) ψ is isomorphic to ψ via B .
Also, let (cid:101)
O ij = (cid:82)
G (cid:101) ψ ( g ) E ij (cid:101) ψ
( g ) T d g .
Lemma C.9.
Let ψ be a real irrep and B a matrix computed as above.
Fix j ∈ { 1 , . . .
, d ψ } .
Then, the following holds:
Fix also i ∈ { 1 , . . .
, d ψ } .
Then (cid:101)
O ij (cid:54) =
if and only if there is a unique l ∈ { 1 , . . .
, E ψ } such that [ P l ] ij (cid:54) = 0 .
In that case, (cid:101)
O ij = 1 d ψ [ P l ] ij P l . 2.
The matrix (cid:101) O ij is non-zero for precisely E ψ different values of i .
Proof.
Let B i and B j , respectively, the i -th and the j -th columns of B .
[Equation]
where in the last step we used that B has the property B T I l B = B T BP l = P l , where P l has the structure of a signed permutation matrix.
Thus, if (cid:101)
O ij (cid:54) = 0 , then there exists l such that [ P l ] ij (cid:54) = 0 .
For the other direction, assume that there is such an l .
We want to show that there cannot be another k with [ P k ] ij (cid:54) = 0 .
If we can do that, then it follows (cid:101)
O ij = 1 d ψ [ P l ] ij P l (cid:54) =
and we are done.
To do so, ﬁrst observe that (cid:104) P
l e j | P k e j (cid:105) =
δ lk .
Indeed, (cid:104)
[Equation]
Now, note that there exists a m such that I Tl I k
= ±
I m .
In particular, if l = k , I Tl I l =
I 1 = id and, therefore, e Tj B T I Tl I
k B e j = e Tj e j = 1 .
Otherwise, e Tj B T I Tl
I k B e j = ±
e Tj B T I m B e j = 0 , thanks to Lemma C.6.
Now, assume we have [ P l ] ij (cid:54) =
(cid:54) =
[ P k ] ij .
Then, since P l and P k are signed permutation matrices, all other entries in column j are zero, meaning that
[Equation]
meaning that k = l .
For l = 1 , . . .
, E ψ , let i l ∈ { 1 , . . .
, d ψ } be the unique index such that [ P l ] i l j (cid:54) = 0 .
It follows (cid:101) O i l j (cid:54)
=
from part 1.
We also know from part 1 that all i l are pairwise different, so we found E ψ such indices.
Now, assume i is such that (cid:101)
O ij (cid:54) = 0 .
Then, from part 1 again, we get that there exists l such that [ P l ] ij (cid:54) = 0 , and thus i = i l .
That ﬁnishes the proof.
As a last classical result, we will use the Cauchy-Schwartz inequality, which is well-known and therefore without a proof:
Lemma C.10.
Let V be any Hilbert space with scalar product (cid:104)· | ·(cid:105) .
Let v, w ∈ V .
Then one has |
[Equation]
≤
[Equation] w [Equation] , with equality if and only if v and w are linearly dependent.
Theorem C.11.
Let ψ be a real irrep.
Then, there exists a real irrep (cid:101) ψ , isomorphic to ψ , such that the following holds.
Fix j ∈ { 1 , . . .
, d ψ } :
The matrix coefﬁcients in column j of (cid:101) ψ are orthogonal to all matrix coefﬁcients of (cid:101) ψ in all but precisely E ψ columns i .
If i is one of the remaining E ψ columns, then (cid:101) ψ
e i is a signed permutation of (cid:101) ψ
e j .
Proof.
Let B and (cid:101)
ψ
= B T ψB as before.
We know from Lemma C.9 that there are precisely E ψ values for i such that (cid:101)
O ij (cid:54) = 0 .
1. Assume (cid:101)
O ij = 0 .
Then, since by Lemma C.5 we have (cid:101) O ij = (cid:16)(cid:10) (cid:101) ψ k 1 i (cid:12)(cid:12) (cid:101) ψ k 2 j (cid:11)(cid:17) d ψ k 1 ,k 2 =1 , it follows that columns i and j of (cid:101) ψ are fully orthogonal to each other.
Assume (cid:101)
O ij (cid:54) = 0 .
Then, by Lemma C.9, there is l such that [ P l ] ij (cid:54) = 0 and (cid:101)
O ij =d ψ · [ P l ] ij · P l , where P l is a signed permutation matrix.
We have [ P l ] ij = ± 1 , i.e., we can wlog.
assume [ P l ] ij = 1 , i.e., (cid:101)
O ij = 1 d ψ P l .
Now, let k 1 ∈ { 1 , . . .
, d ψ } be arbitrary and look at the matrix coefﬁcient (cid:101) ψ
ik 1 .
Then, since 1 d ψ P l = (cid:16)(cid:10) (cid:101) ψ k 1 i (cid:12)(cid:12) (cid:101) ψ k 2 j [Equation]
d ψ k 1 ,k 2 =1 by Lemma C.5 again, and due to the structure of P l , there is precisely one k 2 ∈ { 1 , . . .
, d ψ } such that (cid:104) (cid:101) ψ k 1 i | (cid:101) ψ k 2 j (cid:105) =
± 1 d ψ .
By the Cauchy-Schwartz inequality Lemma C.10 it follows:
[Equation]
In the second to last step, we thereby made use of Corollary 3.
We obtain (cid:12)(cid:12)(cid:10) (cid:101) ψ k 1 i (cid:12)(cid:12) (cid:101) ψ k 2 j (cid:11)(cid:12)(cid:12) =
(cid:13)(cid:13) (cid:101) ψ k 1 i (cid:13)(cid:13) · (cid:13)(cid:13) (cid:101) ψ k 2 j (cid:13)(cid:13) and thus, by the last statement in Lemma C.10, that (cid:101) ψ k 1 i and (cid:101) ψ k 2 j are linearly dependent.
Since they have the same norm, it follows (cid:101) ψ k 1 i = ± (cid:101)
ψ k 2 j .
Since for different k 1 , the value of k 2 needs to differ as well due to P l being a signed permutation matrix, the result follows.
This result is convenient to deﬁne a real harmonic basis { Y mij } for L 2 R ( G ) , which we then use in
Corollary 5.
C. A C ONVENIENT C HOICE OF B ASIS FOR R EAL I RREPS
At the end of the previous section, we described a method that, given a real irrep ψ and a basis for End G
( V ψ ) , generates an isomorphic irrep (cid:101) ψ which satisﬁes the convenient properties in Theorem C.11.
To provide an intuition of what structure these real irreps have and to simplify some computations in Section F, in this section we describe a choice of basis to express each real irreducible representation, satisfying the properties in Theorem C.11.
Note that this particular choice of basis is not necessary in most of our results, since we rely only on a generic orthonormal basis for End G ( V ψ ) , but is useful to simplify Section F.
The content of this section is mostly based on Boardman (2007) and Bröcker & Dieck (2003).
Recall ﬁrst the classiﬁcation of real irreps in real , complex or quaternionic type according to Deﬁnition C.2.
We will rely on the following useful results:
Lemma C.12.
For any real type real irrep ψ , its complexiﬁcation is isomorphic to a complex irrep σ , with σ ∼ = σ .
The complexiﬁcation of ψ is a representation like ψ which acts on C d ψ rather than R d ψ .
and
Lemma C.13.
Any complex or quaternionic type real irrep ψ is isomorphic to the realiﬁcation of a complex irrep σ .
In other words, for any complex or quaternionic type real irrep ψ , there exists a complex irrep σ such that:
[Equation]
Additionally, if it has quaternionic type , it also holds that σ ∼ = σ .
Indeed, using Re ( x ) = 12 ( x + x ) and Im ( x ) = − 12 i ( x − x ) , one can verify that:
[Equation]
where
[Equation]
Using these results, we deﬁne a convenient choice of basis for ψ for each possible irrep type.
For each case, we also deﬁne an associated basis for End G ( V ψ ) .
Real type irrep Since ψ ∼ = σ (as complex irreps), we do not need further assumptions on the basis of ψ .
Note also that End G ( V ψ ) contains only scalar multiples of the identity; hence, regardless of the choice of basis for ψ , a basis for End G
( V ψ ) is given by the identity matrix.
Complex type irrep If ψ has complex type, we assume that ψ is expressed in a basis such that [Equation] Re
[Equation]
(cid:21) for some complex irrep σ .
Note that this is a natural choice, for example, for G = SO(2) .
It follows that a basis for End G ( V ψ ) is given by the following homomorphism Φ
[Equation]
→ End G ( V ψ ) :
[Equation]
where n = d ψ / 2 and id n is the identity matrix of size n .
Quaternionic type irrep Similarly, if ψ has quaternionic type, we assume that ψ is expressed in a basis such that ψ = (cid:20)
[Equation]
for some complex irrep σ .
One can show that if σ has quaternionic type, it can always be expressed in a basis such that it has the following block structure:
[Equation]
and, therefore:
[Equation]
A basis for End G ( V ψ ) is given by the following homomorphism Φ : H → End G ( V ψ ) : Φ(1) =   id n    id n    id n    id n 
 Φ( i ) =
   − id n    − id n id n    id n  
 Φ( j ) =
  − id n  id n      id n − id n 
 Φ( k ) =     − id n  id n  − id n  id n   
 where n = d ψ / 4 .
D T HE C OMPUTATION OF THE I NDUCED R EPRESENTATION OF C OMPACT G ROUPS
Quotient and induced feature ﬁelds , i.e. features ﬁelds where the feature type ρ is an quotient or induced representation (see Sec 2.1), were studied in Weiler & Cesa (2019) for the planar groups.
These feature types encode square-integrable scalar or vector functions over a homogeneous space X = G/H of G .
If X is not ﬁnite, one restricts the consideration to a subspace of bandlimited functions by using a ﬁnite subset of an harmonic basis for the space of all functions.
A band-limited function is then parameterized by a coefﬁcients vector, which carries an action of G as a direc-sum of G irreps.
Given one such vector, the function it parametrizes can be sampled at any point of x ∈ X by evaluating the harmonic basis on x .
For example, in Sec.5.3, we employed models with quotient features over the sphere S 2 ∼ = SO(3) / SO(2) or the space
Inv ×S 2 ∼ = O(3) / SO(2) .
See also Section H.3 for more details on how this is used in our neural network design.
Recall also that a G -orbit is isomorphic to an homogeneous space for G and, therefore, to a quotient space G/H
=
{ g.H | g ∈ G } , where H < G is a stabilizer subgroup of the space; e.g. SO(3) .x
∼ =
S 2 ∼ = SO(3) / SO(2) for x ∈ R 3 non-zero.
The Wigner-Eckart theorem from Lang & Weiler (2020) relies on an harmonic basis over an orbit G.x of the equivariance group G inside the base-space X .
While our Theorem B.7 relies on a G - steerable basis for X , a simple way (although, generally more complicated to band-limit and discretize) to generate such basis is by embedding G -orbits inside X and use the harmonic bases of such orbits.
This is visualized for G = C 4 in Fig.
1b and used for the
G = I baselines in Sec. 5.
In all these settings, an harmonic basis for functions on an homogeneous space is required.
In this section, we describe how to compute the harmonic basis for scalar and vector ﬁelds over a compact homogeneous space 16 X ∼ = G/H , directly from the matrix coefﬁcients of the irreps of G .
A vector ﬁeld over X =
G/H is a function f : X → R d ψ (or C d ψ ) and transforms under the action of G according to an induced representation Ind GH ψ , where ψ is an irrep of H .
We give a precise deﬁnition of induced representation acting on such vector ﬁelds in Def.
D.3.
Note that scalar ﬁelds are considered a special case of vector ﬁelds.
In particular, scalar ﬁelds on G/H transform according to Ind GH ψ 0 , where ψ 0 : H → { 1 } is the trivial representation of H .
We occasionally call this representation acting on scalar ﬁelds a quotient representation .
Similar investigations are well-documented for complex representations: for the case X = G , a complex harmonic basis is given by the matrix coefﬁcients of its irreps, as described by the Peter-
Weyl theorem A.6.
A similar result for complex-valued representations and scalar functions on homogeneous spaces was also described in Kondor & Trivedi (2018).
Outline and goal After some deﬁnitions and assumptions in Supplementary D.1, we provide different equivalent deﬁnitions of the induced representation in Supplementary D.2.
Then, we construct the harmonic basis of induced representations in Supplementary D.3.
Finally, Corollariesand 5 provide the ﬁnal harmonic bases for the complex and real settings, respectively.
We freely make use of the concepts deﬁned in Supplementary A in this whole section.
D. U SEFUL D EFINITIONS , A SSUMPTIONS AND N OTATIONS
In this section, we will always assume a compact group G and a compact subgroup H ⊆ G .
The neutral elements of these groups are just denoted e ∈ H , which is the same for both groups.
As before, for generality, we allow K to be any of the two ﬁelds R or C before we will later separately consider complex and real representations.
(cid:98) H is the set of all isomorphism classes of unitary irreps of H and ψ ∈ (cid:98) H is, by abuse of notation, meant to refer to a representative of such an isomorphism class.
For the rest of the section, we ﬁx such an irrep ψ : G → U( V ψ ) .
In order to generate a harmonic basis for the functions in the induced representation, we only rely on an harmonic basis for L 2 K ( G ) , a basis for the endomorphism space of each irrep ρ ∈ ˆ G and the irrep decomposition of the irreps of G when restricted to H < G .
Assuming these are known, in the rest of this section we will use them to explicitly build a harmonic basis for functions transforming according to an induced representation.
D. I NDUCED R EPRESENTATIONS
We can now introduce the induced representation .
In this section, we will limit to the case where ψ : H → U( V ψ ) is an irreducible representation of H .
This is ﬁxed in this whole section.
Induction from non-irreducible representations of H can be built by combining the induced representations of the H irreps composing the reducible H -representation.
We now deﬁne the induced representation as a space of Mackey Functions, and then alternatively as a space of vector ﬁelds over a quotient space.
The constructions are well-known and can for example be found in Taylor & Kaniutrh (2013).
This book also contains more theoretical justiﬁcations, including the deﬁnition of the scalar product making the induced representation a Hilbert space and proofs that both versions of the representation are actually unitary.
Deﬁnition D.1 (Mackey Functions) .
A Mackey function is a ψ -vector ﬁeld f : G → V ψ which has square-integrable component-functions, i.e., in L 2 K ( G ) , and with the following equivariance-property: for all h ∈ H and g ∈ G one has
[Equation]
We denote the space of Mackey functions by Hom H ( G, V ψ ) .
This becomes a unitary Grepresentation with the action
[Equation]
Remark D.2 .
One can verify that g.f still belongs to the space of Mackey functions.
Indeed, for g, g (cid:48) ∈ G , h ∈ H and f ∈ Hom H ( G, V ψ ) one has
[Equation]
Before we come to the deﬁnition as quotient vector ﬁelds, recall that for the pair of compact groups
H ⊆ G , one can build the space of cosets
[Equation]
[Table] [Table] There is a projection π : G → G/H given by g (cid:55)→ gH which induces a topology on G/H , making it a compact topological Hausdorff space.
Together with the action g. ( g (cid:48) H ) :
= ( gg (cid:48) )
H , G/H becomes a homogeneous space of G , and thus carries a Haar measure.
Now, pick an arbitrary measurable section r : G/H → G associating a representative element to each coset gH ∈ G/H .
In other words, the section r satisﬁes π ◦ r
= id G/H , with the projection as deﬁned above.
This can also be written as
[Equation]
Since r ( gH ) H =
gH , we automatically have that
[Equation]
Since g = r( gH ) · h ( g ) , it follows that every g ∈ G is fully characterized by the pair ( gH, h( g ))
∈ ( G/H )
× H .
We ﬁx from now on the deﬁnitions of the section r : G/H → G and the map h : G → H , with the latter implicitly depending on r .
Deﬁnition D.3 ( ψ -Vector Fields over the Quotient Space) .
A ψ - vector ﬁeld over the quotient space
G/H is deﬁned to be any function of the form f (cid:48) :
G/H → V ψ that has square-integrable component-functions with respect to the Haar measure on G/H , i.e., the component-functions are in L 2 K ( G/H ) .
The space of these ψ -vector ﬁelds is denoted by
Hom( G/H, V ψ ) .
This becomes a unitary G -representation with the action
[Equation]
Note that the action of G on Hom( G/H, V ψ ) depends on the choice of the (not necessarily continuous) section r : G/H → G of the quotient space.
We now show that these two deﬁnitions are equivalent:
Proposition D.4.
There is an isomorphism of unitary representations
[Equation]
between the space of Mackey functions on the left and the space of quotient vector ﬁelds on the right, given as follows:
For f : G → V ψ a Mackey function, we deﬁne
[Equation]
using the section deﬁned in eq. 21.
For a quotient vector ﬁeld f (cid:48) : G/H → V ψ , we deﬁne
[Equation]
using the function h : G → H deﬁned in eq.
Proof.
In this proof, we freely make use of eq.
21 and eq. 22.
First, we show the equivariance of (cid:102) ( · ) :
Let f : G → V ψ satisfy eq. 18.
First, note that
[Equation]
From this, we deduce:
[Equation]
i.e., the desired equivariance (cid:102) g.f = g. (cid:101) f .
Given f (cid:48) : G/H → V ψ , we ﬁrst verify that f (cid:48) satisﬁes eq. 18, i.e. that is lives in Hom H ( G, V ψ ) .
First, note that
[Equation]
From this
[Equation]
we deduce:
[Equation]
We now show the equivariance of ( · ) .
We ﬁrst note
[Equation]
using the result of eq.
24 and eq.
[Equation]
We deduce:
[Equation]
i.e., the desired equivariance g.f (cid:48) =
g.f (cid:48) .
Finally, we need to show that (cid:101) f
= f and (cid:101) f (cid:48) =
f (cid:48) , i.e., that the maps (cid:102) ( · ) and ( · ) are inverse to each other.
We have
[Equation]
and
[Equation]
where we used r( gH ) H =
gH , h (cid:0) r( gH ) (cid:1) =
e , and ψ ( e ) = id V ψ in the last step.
More technical details, especially on the unitarity of the isomorphism, can be found in Taylor & Kaniutrh (2013).
This ﬁnishes the proof.
Proposition D.4 justiﬁes the following deﬁnition:
Deﬁnition D.5 (Induced Representation) .
For any irrep ψ of H ⊆ G , we deﬁne the induced representation Ind GH ψ as the unitary G -representation acting on Ind GH V ψ :=
Hom H ( G, V ψ ) ∼ =
Hom( G/H, V ψ ) as in eq.
19 or eq. 23, depending on which representation space is chosen.
We will always make the representation space (Mackey functions or quotient vector ﬁelds) explicit in the following.
D. H ARMONIC A NALYSIS OF I NDUCED R EPRESENTATIONS
Before we come to ﬁnding harmonic bases of induced representations, we must deal with a small technicality: in the induced representation given by Mackey functions, the equivariance property f ( gh − 1 )
=
ψ ( h ) f
( g ) is used, which suggests that we look at the following left-action of G on itself: g (cid:63) g (cid:48) :
= g (cid:48) g − 1 , compared to the usual left action g.g (cid:48) :
= gg (cid:48) .
Similarly to how we solved for a basis of the space of steerable kernels K : X (cid:48) → Hom G ( V l , V J ) with our Wigner-Eckart theorem, Theorem B.5, by studying L 2 K ( X (cid:48) ) , we want to ﬁnd a basis of the space of Mackey functions f : G → V ψ by studying L 2 K ( G ) – however, due to the adapted left-action on G , we also need to consider a different action on L 2 K ( G ) .
Thus, deﬁne L 2 K ( G ) 2 :=
L 2 K ( G ) together with the action ( g (cid:63) f )( g (cid:48) ) :
= f ( g (cid:48) g ) , which can be shown to be a well-deﬁned left-action on L 2 K ( G ) 2 .
This is then a unitary representation.
Fortunately, it turns out to be isomorphism to the standard regular representation:
Lemma D.6.
There is an isomorphism of unitary representations
[Equation]
given in both directions by the same function:
[Equation]
Proof.
Since in L 2 K ( G ) , we integrate over an inversion-invariant Haar measure, Φ is a unitary transformation.
It is also clear that Φ is its own inverse.
Thus, we are only left with checking the equivariance in one of the two directions:
[Equation]
It follows Φ( g.f ) =
g (cid:63) Φ( f ) , i.e., the desired equivariance.
We remind the reader of the following notation introduced in Section A, which we use in the harmonic analysis of the induced representation: (cid:98) G is the set of isomorphism classes of unitary irreps of G .
m j is the multiplicity of the irrep ρ j : G → U( V j ) in the regular representation L 2 K ( G ) .
For j ≤ m j , p ji : L 2 K ( G ) → V j is the corresponding harmonic projection.
[ ψj ] is the multiplicity of the irrep ψ : H → U( V ψ ) in Res GH V j .
E ψ = dim End H, K ( V ψ ) is the dimension of the endomorphism space of V ψ .
The c ψr : V ψ → V ψ , r ≤ E ψ , form a basis of endomorphisms of V ψ .
ID ψjt : Res GH V j
→ V ψ is an irrep decomposition projection.
The Y mji : G → K form a harmonic basis of the regular representation L 2 K ( G ) .
Y ji : G → K d j ∼ =
V j is the column-vector of all the Y mji .
Y ji is its complex conjugation.
Finally, note that after choosing bases, by abuse of notation we also refer with c ψr and ID ψjt to the matrices corresponding to the linear maps.
Theorem D.7.
A basis of Mackey functions Ind GH V ψ =
Hom H ( G, V ψ ) is given by
[Equation]
Thereby, we have
[Equation]
Consequently, a basis of ψ -vector ﬁelds over the quotient space, Ind GH V ψ =
Hom( G/H, V ψ ) , is given by
[Equation]
with (cid:103) f jitr ( gH ) :
= f jitr (r( gH )) , where r : G/H → G is a measurable section.
Proof.
We have
[Equation]
[Equation]
t =
[Equation]
End H, K ( V ψ ) . Step (1) follows from Theorem B.2 with G replaced by H , G (cid:48) replaced by G , X (cid:48) replaced by G together with the left-action g (cid:63) g (cid:48) :
= g (cid:48) g − 1 , and Hom K ( V l , V J ) replaced by V ψ .
19 Step (2) follows from Lemma D.6.
Step (3) is the Peter-Weyl Theorem A.6.
And ﬁnally, step (4) uses the irrep decompositions ID ψjt : Res GH V j
→ V ψ .
Explicitly, from right to left the isomorphism is given by:
[Equation]
j [Equation] c [Equation]
[Equation] (cid:55)→ (cid:88) j
[Equation]
G m [Equation] (cid:88) i = [ ψj ] (cid:88) t =
[Equation]
Φ (1) (cid:55)→ (cid:88) j ∈ (cid:98) [Equation]
Thus, a basis is given by all f jitr :
=
c ψr ◦ ID ψjt ◦ p ji ◦ Φ | G .
The result now follows by expanding the dirac delta function δ g in the harmonic basis:
[Equation]
In the last step, we identiﬁed V j ∼ = K d j , and thus | jm (cid:105) with a standard basis vector e m .
Note that, to be precise, one would need to approximate the Dirac delta function with square-integrable functions.
The details of such arguments can be found in the proof of Theorem D.13 of Lang & Weiler (2020).
The last statement about a basis of Hom( G/H, V ψ ) , ﬁnally, follows from Proposition D.4.
Corollary 4.
An harmonic basis for complex Mackey functions is given by:
[Equation]
where
[Equation]
Proof.
Using Proposition A.7, we ﬁnd that Y ji ( g − 1 ) =
(cid:112) d
j ρ j
( g − 1 ) e i .
Because we assume complex representations, End G, C ( ψ ) only contains scalar multiples of the identity, due to Schur’s Lemma.
In other words, c ψr is always the identity matrix.
Corollary 5.
Assume that all real irreps are deﬁned over a basis as in Theorem C.11.
An harmonic basis for real Mackey functions is given by:
[Equation]
where
[Equation]
I j is an index set containing the indexes of the m j = d j /E j columns of ρ j with linearly independent matrix coefﬁcients.
Proof.
Using Proposition A.7, If ρ j is expressed in a basis as in Theorem C.11, the matrix coefﬁcients of ρ j in a column i are linearly dependent with those in E j columns of ρ j .
Therefore, it is sufﬁcient to consider a subset of m j = d j /E j columns of ρ j which contain linearly independents matrix coefﬁcients.
Additionally, if ρ j is expressed in the basis described in Supplementary C.3, I j = { 1 , 2 , . . .
, d j /E j }
so I ji
= i .
In Section D.5, we show that this basis is indeed harmonic and we explicitly derive a the action of G on it as a direc-sum of irreps.
D. R ECONSTRUCTING THE IRREP DECOMPOSITION : THE COMPLEX CASE
The goal of this and the next section is to use write the action of G on the bases found above explicitly as a direct sum of irreps of G .
We ﬁrst consider the simpler case of complex valued irreps.
Recall Corollary 4.
Consider the set of all basis elements for a ﬁxed value of j , i.e. { f jit } it where:
[Equation]
Let f be a Mackey function in the span of these basis elements, i.e. there exists { w it ∈ C } it such that:
[Equation]
where [ w t ] i = w it and w t ∈ C d j .
Now, the action of G on these functions is the usual action on
L 2 ( G ) , i.e. ∀ k, g ∈ G :
[Equation]
In other words, the space of functions spanned by { f jit } it is isomorphic to (cid:76) [ ψj ] t
V j , and each w t lives in one such V j space.
We call the vectors w t the Fourier Transform coefﬁcients of the function f .
More generally, let f be a generic Mackey function in Ind GH V ψ .
The matrix ˆ f ( ρ j ) of shape d j × [ ψj ] contains in its columns the vectors w t ∈ C d j for each t .
We call the collection { ˆ f ( ρ j ) }
j the Fourier
Transform of the function f .
Note that if ψ is the trivial representation, this deﬁnition recovers the classical deﬁnition of Fourier transform of functions over compact groups or homogeneous spaces.
Finally, it follows that Ind GH ψ ∼ =
(cid:76) j (cid:76) [ ψj ] t ρ j .
D. R ECONSTRUCTING THE IRREP DECOMPOSITION
THE REAL CASE
In the real case, this decomposition is less straightforward.
Recall Corollary 5.
Consider the set of all basis elements for a ﬁxed value of j , i.e. { f jitr } itr where:
[Equation]
with i ≤ m j = d j /E j .
Let f be a Mackey function in the span of these basis elements, i.e. there exists { w itr ∈ R } itr such that:
[Equation]
where [ w rt ] i = w itr if i ≤ m j and [ w rt ] i = 0 otherwise.
Unfortunately, this basis is not harmonic since it does not have an explicit action of G through ρ j on the coefﬁcients vectors.
To see this, consider the function above transformed by an element k ∈ G :
[Equation]
The vector ρ j ( k ) w rt does not generally belong to the span of { e i | i ≤ m j } , i.e. the function k.f as written above is not expressed directly in terms of the basis in Corollary 5.
This means that the basis we found is not an harmonic basis yet.
Nevertheless, we know that the space of Mackey function has a compact action of G and therefore must be decomposable in a direct sum of irreps of G .
Therefore, there exists a change of basis from the current one to an harmonic one.
In the rest of this section, we will derive a variation of the
Gram–Schmidt process to compute such change of basis.
First of all, we want to identify the invariant subspaces spanned by the basis.
To do so, we re-project each basis element after an action of G on the basis itself, i.e. we project the function g.f jitr , with g ∈ G on each other basis element f jksq .
First, note that:
[Equation]
i.e. g.f jitr belongs to the span of the set of functions { f jitr | i ≤
d j } (note that i ≤ d j rather than i ≤ m j = d j /E j ).
Despite the functions { f jitr | m j < i ≤ d j } do not belong to the basis in
Corollary 5, one can easily verify that they are actually Mackey functions and, therefore, must belong to the space spanned by the basis in Corollary 5.
Additionally, we will see that { f jitr | i ≤ d j } contains functions which are orthogonal to each other; before showing this, we should deﬁne an inner product of Mackey functions:
[Equation]
Note that we assumed real functions and real representations.
We can now compute the projection of any two functions:
[Equation]
Lemma D.8.
the following statements are true:
O tr,sq ∈ End G ( ρ j ) 2.
O tr,sq = (cid:80) l (cid:104)
O
tr,sq , I l (cid:105) I l = (cid:80) l Tr([ID ψjt ] T [ c ψr ] T c ψq ID ψjs I Tl ) I l
Proof.
1) By deﬁnition, O tr,sq is the invariant projection of a matrix with respect to a left and right action of G through ρ .
It follows that O tr,sq commutes with ρ ( g ) for any g ∈ G ; hence,
O tr,sq ∈ End G ( ρ j ) .
2) Let { I l | 1 ≤ l ≤ E j } be an orthonormal basis for End G ( ρ j ) as in Sec.
C.
Then, we can project
O tr,sq on this basis:
[Equation]
Note that the coefﬁcients (cid:104) O tr,sq , I l (cid:105) can be easily computed numerically since all the matrices involved are known.
Corollary 6.
The following statements are true:
(cid:104) O tr,sq , I 1 (cid:105) =
δ st δ rq 2.
O tr,tr is the identity matrix 3.
(cid:104) f jitr , f jktr (cid:105) =
δ ik
Proof.
By considering l = 1 , i.e. I l = id the identity, in Lemma D.8, we ﬁnd
[Equation]
Leveraging the orthogonality of ID ψj and the fact that ID ψjt and ID ψjs contain the rows of ID ψj :
[Equation]
There exists a unique r (cid:48) such that c ψr (cid:48) =
[ c ψr ] T c ψq and c ψr (cid:48) is not skew-symmetric iff r (cid:48)
= 0 , i.e. r = q : =
[Equation]
st
[Equation]
rq 2) Note that the matrix O tr,tr is symmetric; since I l for l (cid:54) = 1 is antisymmetric, (cid:104) O tr,tr , I l (cid:105)
=
if l (cid:54) = 1 .
Using the result in 1), it follows that O tr,tr = id is the identity matrix.
An immediate consequence of this is that (cid:104) f jitr , f jktr (cid:105)
[Equation]
,tr e k = δ ik .
Note that the last proposition in Corollary 6 holds also for the functions in { f jitr | i > m j = d j /E j } which don’t belong to the basis in Corollary 5.
Intuitively, for a ﬁxed value of ( t, r ) , the basis in Corollary 5 ignores these orthogonal elements since they are linearly dependent with the basis elements associated with some other value of ( t, r ) .
This explains why the form of the basis in
Corollary 5 is not convenient for us.
Using the projection coefﬁcients O tr,sq , we can compute the necessary change of basis.
To do so, we start with the over-complete basis given by:
[Equation]
Recall that, for a ﬁxed value of r and t , all basis elements B tr = { f jitr } i are orthogonal.
We can therefore build a basis by iteratively picking a set B tr and project it to the space orthogonal to the one spanned by the basis elements already chosen.
Before making this more precise, we need to introduce a few more handful results.
Lemma D.9.
For any r ≤ E ψ , t ≤ [ ψj ] and l ≤ E j :
c ψr ID ψjt I
l ∈
Hom H (Res GH ρ j , ψ ) 2.
there is a set of real numbers ( w tr,sq,l ) s,q s.t.
[Equation]
w tr,sq,l = (cid:104) O
tr,sq , I l (cid:105)
Proof. 1) Note that c ψr ID ψjt ∈
Hom H (Res GH ρ j , ψ ) and I
l ∈ End G ( ρ ) .
It follows that, for any h ∈ H < G :
[Equation]
i.e. [Equation] ψr ID ψjt I l commutes with the action of H ; hence, it belongs to Hom H (Res GH
ρ j , ψ ) .
This result is an immediate consequence of 1) by using the following decomposition
[Equation]
and the basis { c ψq } q for End H ( ψ ) .
3) Since { c ψq ID ψjs } q,s is an orthonormal basis for Hom H (Res GH
ρ j , ψ ) , the coefﬁcients ( w sq,tr,l ) s,q are found just by projecting the matrix on each basis element using a (scaled) standard inner product, i.e.:
[Equation]
Lemma D.10.
Let { b i } d j i =1 a set of Mackey functions deﬁned as:
[Equation]
parameterized by the vector b =
( b r,t ) r,t .
Let b ( g ) =
( b 1 ( g ) | . . .
| b i ( g ) | . . . )
∈ R d ψ × d j the matrix obtained by stacking horizontally the vectors { b i ( g ) } i , i.e.
[Equation]
The set of functions { b i } d j i =1 is an harmonic family, i.e. if f ( g )
=
b ( g ) f for a coefﬁcients vector f ∈ R d j , then [ k.f ]( g ) =
f ( k − 1 g )
=
b ( g ) ρ j ( k ) f .
In other words, the function k.f is the function parameterized by the coefﬁcients ρ j ( k ) f .
Proof.
[Equation]
Finally, we deﬁne a few operations which we will need to combine and project harmonic functions in our ﬁnal algorithm.
Lemma D.11.
Let a and b be two set of harmonic functions as in Lemma D.10, respectively parameterized by the vectors a and b .
Let α ∈ R and M ∈ End G ( ρ j ) ; then, the following sets of functions are all harmonic families as in Lemma D.10:
the set f deﬁned as f ( g ) =
αa ( g ) 2.
the set f deﬁned as f ( g ) =
b ( g ) − a ( g ) 3.
the set f deﬁned as f ( g ) =
a ( g ) M
Proof.
This is a simple consequence of the linearity of the harmonic property.
[Equation]
3) Let M = (cid:80) l m l I l .
Then:
[Equation]
where we used Lemma D.9.
Corollary 7.
Given two harmonic families a and b , let O a,b = (cid:82) G a ( g ) T b ( g )d g be the matrix containing the inner product between each pair of functions in a and b .
Then, O a,b ∈ End G ( ρ j ) and
O a,b = (cid:80) l w a,b,l I l , where w a,b,l = (cid:80) sq,tr a s,q b t,r w sq,tr,l .
Moreover, let f be a set of functions deﬁned as f ( g )
=
b − a O a,b .
Then, O a,f is the zero matrix.
Proof.
The ﬁrst result follows from:
[Equation]
The second result follows from
O
[Equation]
Additionally, note that all functions within the same harmonic family b have the same norm, i.e.
[Equation]
where we used the fact O b,b is a symmetric matrix and, therefore, w sq,tr,l = δ st δ qr δ l, 1 .
(cid:107) b (cid:107) is the (squared) norm of each function in b .
We now have all the ingredients to describe the ﬁnal algorithm.
Let B = {} the (initially empty) set of harmonic families chosen so far.
In other words, an element b i of B is an harmonic family of d j functions as in Lemma D.10 and is represented by a vector b i =
( b i,t,r ) t,r .
Our goal now is to ﬁnd
M j = E ψ [ ψj ] /E
j vectors { b i } i such that O b i ,b k = δ i,k id .
To do so, we can repeat the following steps until the set B contains M j elements:
1. pick a vector b =
( b t,r ) t,r , which deﬁnes the set of harmonics b 2.
deﬁne b (cid:48)
=
b − (cid:80) b i ∈B b i O b,b (cid:48) , represented by the coefﬁcients vector b (cid:48) =
( b (cid:48) t,r ) t,r , where the coefﬁcients can be computed using Lemma D.11 and Corollary 7. 3.
normalize b (cid:48) as ˜ b
= b (cid:48) / (cid:112) (cid:107) b (cid:48)
(cid:107) 4.
add ˜ b to B
Note that, at each iteration, we add d j orthogonal functions to the new basis and that the original basis in Corollary 5 is M j · d j dimensional.
Therefore, the algorithm terminates in M j iterations.
E N UMERICAL I RREPS D ECOMPOSITION OF R EAL R EPRESENTATIONS
Assume a compact group G and a set of (representatives of the isomorphism classes of) orthogonal real irreps (cid:98) G = { ρ j } j .
In this section, we only engage with real valued representations.
In particular, recall that, for a real-valued irrep ρ , its endomorphism space
End G ( ρ ) is not necessarily 1 -dimensional, see Section C.
As stated in Corollary 1, any orthogonal real representation π of G can be decomposed as a direct sum of irreps of G .
The goal of this section is that of explicitly computing this decomposition for an arbitrary ﬁnite-dimensional orthogonal real representation π of G .
In other words, given an input ﬁnite-dimensional (real, orthogonal) representation π , we want to ﬁnd the orthogonal transformation/matrix M and the multiplicities { [ jπ ] ∈ N } j such that:
[Equation]
Thereby, being orthogonal means that the scalar product is preserved, which for matrices means that the matrix is orthogonal in the classical sense – i.e., its columns build an orthonormal basis.
Warning: note that, with respect to Corollary 1 and the notation used in the main paper, in this section
M is transposed on the right-side rather than the left-side of the direct sum of irreps.
To obtain the decomposition used in the other sections, one just need to transpose our ﬁnal result.
The reason why we consider this notation is simply that this is the convention we used in our implementation and we wanted to keep the following algorithm consistent with our code.
In this whole section, we assume that π and all ρ j are matrix representations, and correspondingly, that M is given by a matrix.
To simplify the notation, deﬁne P =
(cid:76) j (cid:76) i ρ j , such that π = MPM T .
Since M is orthogonal, we can write πM = MP , i.e. M needs to commute with π and P .
It follows that M ∈ Hom G, R ( P, π ) .
Recall that the block-diagonal structure of P induces a block-structure in the columns of M , visually: π [Equation] [Equation]
| . . . [Equation]
| . . . | [Equation]
j [ [Equation] ] | . . .
(cid:3) ·   ...
ρ ( g ) ...
ρ j ( g ) ... ρ j ( g ) ...
  ·   ...
M Tj 1 .
M Tji ...
M Tj [ jπ ] .

 i.e., we split M in a number of vertical blocks, one for each occurrence i of each irrep ρ j in P .
From the equation πM = MP we obtain the following: each such block M ji satisﬁes πM ji = M ji ρ j , i.e.
M ji ∈ Hom G ( ρ j , π ) .
We restrict our attention to this case.
Thus, from now on, assume that ρ is a real orthogonal irrep of G , and our goal is to ﬁnd an orthogonal embedding in Hom G ( ρ, π ) , where π is not necessarily irreducible.
Note that Hom G ( ρ, π ) ∼ = (cid:76) [ jπ ] i =1 End G ( ρ ) .
We can easily ﬁnd a basis for Hom G ( ρ, π ) by solving a linear system.
Let d j be the dimension of the representation space of ρ j , and similarly, d π be the dimension of the representation space of π .
Let I j the d j × d j identity matrix
and I π the d π × d π identity matrix; then, the matrix M ji should satisfy:
[Equation]
vectorizing both sides of the equation and using vec ( ABC ) = (cid:0) C T ⊗ A (cid:1) vec ( B ) :
[Equation]
Note that I j ⊗ π ( g ) − ρ j ( g ) T ⊗ I π = − ρ j ( g ) T ⊕ π ( g ) , where ⊕ indicates the Kronecker sum here (not to be confused with the direct sum ).
Note that the last line above describes a linear constraint on M ji for each group element g ∈ G .
One can solve for the space of suitable M ji by stacking vertically the matrices ( I j ⊗ π ( g ) − ρ j ( g ) T ⊗ I π ) for each g ∈ G and then solving for its null space (e.g. through SVD).
While this approach is feasible if G is a ﬁnite group and | G | is sufﬁciently small, this can not be used for inﬁnite groups (or large ﬁnite groups).
If G is a ﬁnite group, one can easily verify that it is sufﬁcient to consider the matrices ( I j ⊗ π ( g ) − ρ j ( g ) T ⊗ I π ) associated with the set of generators of G .
Indeed, note that if M ji satisﬁes the constraint above for two group elements a and b ∈ G , then it satisﬁes it also for ab and ba ∈ G .
Finzi et al.
(2021) follows precisely this approach to compute a basis for a general space Hom G ( π 1 , π 2 ) , when G is ﬁnite.
When G is an inﬁnite Lie group, Finzi et al.
(2021) shows that it is sufﬁcient to consider the Lie algebra generators of G to solve this constraint.
In our work, however, we do not want to rely on the Lie algebra of G , since it would introduce additional – non-trivial – requirements when implementing a new equivariance group G .
Instead, we note that it is generally sufﬁcient to consider a few random samples G ⊂ G to generate enough constraint matrices { I j ⊗ π ( g ) − ρ j ( g ) T ⊗ I π | g ∈ G} such that the null space of their vertical stack is precisely Hom G ( ρ j , π ) .
A similar method was used already in Weiler et al.
(2018a) to compute the Clebsh-Gordan coefﬁcients in the decomposition of the tensor product of the irreps of SO(3) .
These coefﬁcients appear in their kernel basis precisely as in our vectorized kernel constraint in Sec. 2.2.
In Weiler & Cesa (2019) (Appendix G), the authors discuss the cost of this approach when applied on generic input and output representation pairs, and highlight the beneﬁts of, instead, restricting the consideration to only pairs of input and output irreducible representations, as done here.
Therefore, by following the method just described, we can generate an orthonormal basis for the space of vectorized matrices satisfying the constraint above.
By un-vectorization, any element in the space spanned by this basis generates a matrix M ji such that πM ji = M ji ρ j .
This is not sufﬁcient yet for us, since we require the whole matrix M to be orthogonal .
From now, we assume the previous method generated an orthonormal basis B j = { B j,i } di for
Hom G ( ρ j , π ) , i.e. a collection of d = [ jπ ] · E j matrices of shape d π × d j which are orthogonal with respect to a (scaled) standard inner product (cid:104) A, B (cid:105) =
d j Tr( AB T ) = 1 d j vec ( A ) T vec ( B ) .
Recall that E j is the dimensionality of End G, R ( ρ j ) .
In order to build a matrix M which is orthogonal, for each j , we need to choose [ jπ ] orthogonal matrices { M ji } [ jπ ] i in Hom G ( ρ j , π ) , such that all columns of all matrices M ji are also orthogonal to each other.
Lemma E.1.
Any matrix A
1 ∈ Hom G ( ρ j 1 , π ) has columns which are orthogonal to the columns of any matrix A 2 ∈ Hom G (
ρ j 2 , π ) , for j 1 (cid:54)
=
j 2 .
Proof.
This is a direct consequence of the fact that any homomorphism in A 1 ∈ Hom G ( ρ j 1 , π ) maps to a subspace V 1 ⊂ V π
which is orthogonal to the image V 2 of A 2 ∈ Hom G ( ρ j 2 , π ) .
This can be seen by realizing that the projection from V 1 to V 2 is an intertwiner, and thus needs to be zero by
Schur’s Lemma since V 1 and V 2 are not isomorphic.
Lemma E.2.
Any orthonormal basis for Hom G ( ρ j , π ) contains matrices which are orthogonal, i.e., such that the columns are orthogonal and normalized.
Proof.
Let { E k } E ρ k be an orthonormal basis for End G ( ρ ) .
We know that there exists a matrix M such that 1) M is orthogonal and 2) π
= M (cid:16)(cid:76) j ρ ⊕ [ jπ ] j (cid:17) M T .
Therefore, there exists a collection { M ji } [ jπ ] i =1 of orthogonal matrices in Hom G ( ρ j , π ) , where M ji are the blocks of the matrix M before.
It follows that there exists a basis for Hom G ( ρ, π ) deﬁned as H
=
{ I k M ji | 1 ≤ k ≤ E j , 1 ≤ i ≤ [ jπ ] } .
This basis is made of orthogonal matrices, since both I k and M ji are orthogonal.
Note that we don’t know M and H , but we only know they exist.
Assume H =
{ H j } j is a basis for
Hom G ( ρ, π ) made of orthogonal matrices, i.e. for each i , H Ti H i
= I .
More generally H Ti H l = δ il I .
Then any orthonormal basis for this space is made of orthogonal matrices.
Indeed, assume B = { B l } l is a basis such that (cid:104) B l , B l (cid:105)
= 1 .
An element B l can be written as B l = (cid:80)
i b
li H i .
It follows that for any element B l :
[Equation]
i.e. any element of an orthonormal basis for Hom G, R ( ρ j , π ) must contains matrices which are themselves orthogonal.
It follows that we only need to focus on ﬁnding a collection { M ji } [ jπ ] i =1 of matrices whose columns are orthogonal; the orthogonality of columns within the same matrix M ji and between different matrices
M j 1 ,i 1 and M j 2 ,i 2 with j 1 (cid:54) =
j 2 is already guaranteed.
Recall that B j = { B j,i } di is the orthonormal basis for Hom G ( ρ j , π ) generated by the numerical method (e.g. SVD).
To ﬁnd suitable { M ji } i , it is sufﬁcient to set M j 0 =
B j 0 .
Let M j the subset of all matrices { M ji } i already set (initially containing only M j 0 ).
Let { I k } E j k =1 be an orthonormal basis for End G, R ( ρ j ) .
Let B i =
= B j .
Then we build the set M j through the following iterative method:
create B (cid:48)
=
{ M ji I k
| M i ∈ M j , k ∈ { 1 , . . .
, E j }} 2. project the elements in B i on the space space of matrices which is orthogonal to the span of B (cid:48) and remove all zero elements (i
e. all elements which already belong to the span of B (cid:48) )
3. pick the ﬁrst non-zero element of B i , set M ji to that and add it in M j 4.
repeat until the set B i is empty
Up to a scalar factor (of (cid:112) d j ), the matrices in M j are all orthogonal.
Additionally, this set contains precisely [ jπ ] matrices.
By stacking all matrices together, one builds the matrix M .
Note also that this method can be used to ﬁnd the multiplicities [ jπ ] , too.
Indeed, because each column of ρ j is related through an endomorphism precisely with E j other columns (see Theorem C.11), once a basis B j with d elements is found, it is sufﬁcient to compute [ jπ ] =
d/E j , since we assume the type of ρ j (and therefore E j ) is known.
F R EAL I RREDUCIBLE R EPRESENTATIONS OF THE D IRECT P RODUCT OF TWO C OMPACT G ROUPS
A useful operation to generate new groups is the direct product .
Given two groups A and B , their direct product G =
A × B is a group deﬁned as:
• the Cartesian product of the element of A and B , i.e. { ( a, b ) | a ∈ A, b ∈ B } • with the group law · : G × G → G deﬁned as ( a 1 , b 1 ) · ( a 2 , b 2 ) =
( a 1 a 2 , b 2 b 2 ) .
Many groups can be written as a direct product of two smaller groups.
In particular, O(3) =
Inv × SO(3) , or more generally O( n ) =
Inv × SO( n ) when n > 1 is odd, where Inv = {± 1 } is the group generated by the reﬂection x
(cid:55)→ − x .
In our work, we are interested in the direct product to easily build many subgroups of O( n ) (in particular, of O(3) ) which have form G =
Inv × H , where
H < SO( n ) .
This is the case for the cylindrical symmetries ( H = SO(2) , O(2) or their discrete subgroups) or the full isometries of the solids ( H = T , O or I ); see Sec.
G.
In order to use these groups, it is sufﬁcient for us to build their set of irreducible representations.
In this section, we describe a method to generate the set of real irreducible representations of a direct product of two compact groups from the irreps of the two groups.
Given two compact groups A and B and their respective set of (representatives of) complex unitary irreducible representations (cid:98)
A
=
{ α i } i and (cid:98) B
= { β j } j , it is well known that any complex unitary irrep of G =
A × B is isomorphic to a unique γ ij =
α
i ⊗ β j for a particular choice of i and j .
Thereby, ( α i ⊗ β j )( a, b ) :
= α
i ( a ) ⊗ β j ( b ) is the Kronecker product of the two matrices
α i ( a ) and β j ( b ) .
Thus, a set of representatives of isomorphism classes of unitary irreps (cid:98) G can be constructed as
[Equation]
However, this results does not hold in the real ﬁeld.
In the rest of this section, we will derive a variation of this theorem using real irreducible representations, which is more useful to describe the representations of the groups we used.
In Appendix C.3, we have already shown that any real irrep ψ of G can be classiﬁed in one of the following three categories:
• real-type : ψ ∼ = σ ∼ = σ • complex-type : ψ = σ R ∼ = σ ⊕ σ , with σ (cid:29) σ • quaternionic-type : ψ = σ R ∼ = σ ⊕ σ ∼ = σ ⊕ σ , as σ ∼ = σ
where σ is a complex irrep of G .
Moreover, for each complex irrep σ , there exists only one real irrep ψ which contains it.
We can use this classiﬁcation starting from the complex irreps of G which we have already deﬁned in
Eq.
First, we clarify some notation.
We use Greek letters like α, β, γ to indicate complex irreps and Roman letters to indicate the corresponding real irreps a, b, c .
For each complex irrep γ ij , there can be only one real irrep c ij containing it.
It follows that α i ⊗ β j only appears in c ij .
Moreover, if α i (cid:29) α i , i.e. a i is of complex-type, we index the complex irrep α i with i , i.e. α i ∼ =
α i .
We can now introduce a useful result for real irreps:
Deﬁnition F.1 (Frobenious-Schur Indicator) .
Let G be a compact group and ρ a complex irreducible representation of G .
Let χ be the character of ρ , i.e. χ : G → C , g (cid:55)→
Tr( ρ ( g )) .
The FrobeniousSchur indicator ι ρ of ρ is deﬁned as:
[Equation]
This indicator can only take the following values:
[Equation]
This allows us to analytically determine the type of the real irrep associated with a complex one.
We can combine this result with the deﬁnition of complex irreps of G =
A × B in Eq.
26 to ﬁnd the type of the real irreps of G .
Let γ ij = α
i ⊗ β j a complex irrep of G ; then:
[Equation]
Using the property of the trace and the Kronecker product:
[Equation]
Hence, the types of the irreps α i and β j of A and B fully determine the type of the irrep γ ij of G .
This can be visualized in the following table:
[Equation]
quaternionic
[Equation]
real ι α i =
[Equation]
complex ι α i =
[Equation]
quaternionic ι
[Equation]
In the rest of this section, we will use the following two useful facts:
[Equation]
We will also assume the irreps to be expressed on the bases described in Section C.3.
In particular, if α i (or β j ) are real type irreps, we assume they are expressed with respect to a basis such that they have real entries, i.e. α i = a i (and β j = b j ).
In such case, Re ( α i ) = a i and Im ( α i ) = 0 .
If α i (or β j ) is a complex or quaternionic type irrep, we assume a i (and b j ) is expressed as the realiﬁcation:
[Equation]
This enables us to extract Re ( α i ) and Im ( α i ) directly from the matrix coefﬁcients of a i (or b j ).
Therefore, to build the real irreps of G =
A × B we distinguish three cases, depending on the type of the irrep.
Real type irrep Assume γ ij has real type .
This implies that either both α i and β j have real type or both have quaternionic type.
In the ﬁrst case, g ij is simply computed as g ij = a i ⊗ b j .
In the second case, assume α i and β j have the structure in Eq.
Then, one can verify that the change of basis
Q = 1 √  iI iI − iI  iI − I
I
I I 
 turns γ ij =
α
i ⊗ β j into a matrix with only real entries, i.e. g ij =
Qγ ij Q † , where I is the identity matrix of size dim γ ij / 4 .
Complex type irrep Whenever γ ij has complex type, we can easily implement g ij as the realiﬁcation:
[Equation]
By using the identities above, we can easily compute this matrix from entries of the matrices of a i and b j .
Note that, if both α i and β j have complex type, the irrep γ ij = α
i ⊗ β j and γ ij = α i ⊗ β j generate two different real irreps g ij and g ij .
Since b j and b j are isomorphic real irreps, only one of the two belongs to the set of representatives in (cid:98) G chosen.
However, Re ( β j ) =
Re (cid:0) β j (cid:1) and
Im ( β j ) = − Im (cid:0) β j (cid:1) , so g ij can still be computed easily from the coefﬁcients of a i and b j .
Quaternionic type irrep Like for the complex types, whenever γ ij has quaternionic type, we can easily implement g ij as the realiﬁcation:
[Equation]
Additionally, note that this is the case only if either α i or β j is of real type, while the other is of quaternionic type.
Hence, the formula can be simpliﬁed by noting that either Im ( α i ) or Im ( β j ) is zero.
G 3D SYMMETRIES , A ZIMUTHAL SYMMETRIES AND THEIR RELEVANCE
E(3) is the group of isometries in R 3 and includes translations ( R 3 , +) , rotations SO(3) and inversions 22 through the origin Inv .
The group O(3) = SO(3) ×
Inv is the compact group of origin-preserving isometries.
While continuous rotational symmetry SO(3) occurs commonly in molecular data, smaller symmetry groups are sometimes more relevant to describe scenes or objects in natural environments.
For instance, when the vertical orientation of a the scene is obvious, the symmetry group of the data is reduced to axial symmetries.
There is only a limited number of topologically closed inﬁnite subgroups of O(3) (up to conjugation).
The building blocks are rotations along an axis ( SO(2) ), inversions ( Inv ), mirroring with respect to a plane ( M ) and an out-of-plane rotation by π ( F ).
Together, they generate 5 continuous symmetry groups:
• axial symmetry ( SO(2) < SO(3) ), • dihedral symmetry ( O(2) = SO(2) (cid:111) F < SO(3) ), • conical symmetry ( O(2) ∼ = SO(2) (cid:111)
M < O(3) ), • cylindrical symmetry ( Inv × SO(2) < O(3) ) and • full cylindrical symmetry ( Inv × (SO(2) (cid:111) F) < O(3) ).
In the planar case, Weiler & Cesa (2019) observed beneﬁts from using discrete groups rather than continuous ones when the data is discretized on a grid and, therefore, the continuous symmetry of the data is lost.
The only discrete subgroups of SO(3) with more than 2 rotation axes are the symmetry groups of the Platonic solids: the tetrahedral ( T ), the octahedral ( O ) and the icosahedral ( I ) groups which have, respectively, 12 , 24 , and 60 elements.
T and O are perfect symmetries of a voxel grid and were previously used in Worrall & Brostow (2018);
Winkels & Cohen (2018).
Also, note that
T <
I but O (cid:2)
I ; hence, an I -equivariant CNN is not equivariant to all symmetries of the voxel grid.
H F URTHER D ETAILS ON N ETWORK D ESIGNS AND E XPERIMENTS H. E STIMATING THE E QUIVARIANCE E RROR OF S TEERABLE B ASES
In this section we provide more details on Fig.
In this experiment, we considered equivariance to the Icosahedral group G
= I .
We employ 5 × 5 × 5 ﬁlters, parameterized by two spherical shells (one at radius 1 and one at radius 2 ) and the origin.
In the two baselines, we use the orbits containing the vertices of either the icosahedron or the dodecahedron in the two spherical shells and the orbit containing only the origin.
The kernel is parameterized independently along each orbit by directly using the Wigner-Eckart theorem from Lang & Weiler (2020) and, then, the kernels are embedded in Rby means of a small Gaussian smoothing kernel.
In our basis, we parameterize each shell with spherical harmonics band-limited to frequency 2 (the origin contains only frequency 0 ); each shell is diffused along the radial direction with a small Gaussian kernel.
Each histogram in Fig.
5 shows the relative equivariance error of each element of one of the bases.
An histogram contains a point for each ﬁlter k in the basis associated with each pair of input and output irreps ( ρ in , ρ out ) of G
= I .
The error of a basis ﬁlter k is computed as the average over allrotations g ∈
I of the following error:
[Equation]
return compare ( fgx , gfx ) where x (and gx ) have the same resolution of the ﬁlter k , such that the outputs fx and fgx do not have spatial resolutions.
In this way, g acts on fx only by transforming its channels but no spatial interpolation is necessary.
To reduce the interpolation artifacts when rotating X (cid:55)→ g.X , we perform the rotation at a higher resolution and then downsample the inputs by a factor of 3 .
Quotient and regular -type features encode square-integrable scalar functions over a homogeneous space Q = G/H of G (with H = { e } and Q = G for the regular case). Whenever Q = G/H is inﬁnite, one can rely on a bandlimited representation of these functions in the Fourier domain, using the harmonic bases we derived in Section D. More precisely, bandlimited functions are parameterized by using a ﬁnite subset of the harmonic basis in Corollary 5, considering only those j in a subset (cid:101) G ⊂ (cid:98) G . Then, the action of G on a function turns into the representation (cid:102)
ρ Q = (cid:76) j ∈ (cid:101) G (cid:76) M
j ρ j acting on the coefﬁcients vector which parameterizes it, where M j is the multiplicity of the irrep ρ j .
Given one such vector, the function it parametrizes can be sampled at any point q ∈ Q by evaluating the harmonic basis on q .
In a neural network, a feature vector f ( x ) ∈ R d ρ of quotient (or regular) type ρ is interpreted as the coefﬁcients vector parameterizing a band-limited function in L 2 ( Q ) =
L 2 ( G/H ) on a basis, which transforms according to ρ .
A point-wise nonlinearity σ : R → R (e.g. ELU ) is applied on a quotient feature f ( x ) : Q → R by composition, obtaining a new quotient feature f (cid:48) ( x ) , deﬁned as [ f (cid:48) ( x )]( q ) = σ ([ f ( x )]( q )) .
When Q is inﬁnite, if f ( x ) is band-limited and σ is sufﬁciently smooth, we can recover an approximation of f (cid:48)
( x ) from a ﬁnite number of samples.
Approximate point-wise non-linearity Let Q ⊂ Q be a ﬁnite number of samples from the space
Q = G/H .
We deﬁne the non-linearity as
[Equation]
where FT is a Fourier Transform , which recovers the coefﬁcients of a band-limited function from a number of samples, σ is the pointwise non-linearity and IFT is an Inverse Fourier Transform , which just samples the function parametrized by the coefﬁcients f ( x ) on the elements in Q .
Note that IFT can be implemented as a simple multiplication with a matrix of shape |Q| ×
d ρ and, similarly, FT can be implemented as a multiplication with a matrix of shape d ρ × |Q| , hence
[Equation]
where FT and IFT are interpreted as matrices.
In practice, the matrix IFT is implemented as a matrix whose rows contain the Fourier transform of Dirac delta functions centered at each sample in Q , while
FT is implemented as the pseudo-inverse matrix FT = IFT † .
This operation is only approximatively equivariant and the degree of equivariance strongly depends on the number of samples |Q| as well as their distribution over the group (and the smoothness of σ ).
In particular, it is important that the samples in Q uniformly cover the space Q = G/H and are, therefore, as far away from each other as possible.
To generate a sampling set Q , one can use the elements of a discrete subgroup of G (e.g. Icosahedral group I < SO(3) ) or optimize the location of |Q| points on Q by minimizing a potential energy, as done by Bekkers (2020).
Computational beneﬁts A typical feature f in our models includes multiples ﬁelds of the same type, i.e. f = (cid:76) i f i and ρ =
(cid:76) i ρ i , with ρ i ∼ = ρ j ∀ i, j .
Assume a sequence ConvND → σ →
ConvND in the model, where all layers operate on feature ﬁelds of type ρ .
Let C
=
(cid:80) i
d ρ i be the size of the feature vectors f ( x ) ∈ R C .
In our neural networks design, we ensure that the number of channels in each layer is approximately constant across different models.
In particular, in models using quotient (or regular) feature types, we count the number of samples Q as channels and, therefore, keep the quantity (cid:80) i |Q| ≈ N approximately constant.
Let α = |Q| /d
ρ
i ; this implies that the ﬁrst ConvND only has (cid:80) i
d ρ i ≈ N/α output channels (and the second ConvND has this many input channels).
Let k be the cost of a single convolution with 1 input and output channels and m the number of pixels in the input grid.
The cost of the non-linearity applied to a single feature ﬁeld is O
( d ρ i |Q| ) and there are N/ |Q| such input ﬁelds.
Then, the cost of this layer is only O ( kmCN/α + m ( N/ |Q| ) |Q| d ρ i ) =
O ( kmCN/α + mNd ρ i ) which is equal to
O ( kmCN/α ) , since mC (cid:29) d ρ i .
For comparison, the same block in a conventional network has cost O ( kmCN + mN )
=
O ( kmCN ) , hence this layer is approximatively α times cheaper.
In Section H.7, we compare the approximate inference time of different architecture to verify the computational beneﬁts described above.
H. A RCHITECTURE D ETAILS
All models used in our experiments consist of a sequence of a basic inverted-bottleneck residual block, or small variations of it (see each experiment’s section).
The basic block for a conventional network consists of a sequence Conv3D - BatchNorm - ELU - Conv3D , whose input and output are connected with a skip connection.
The block has C in input channels and the ﬁrst convolution maps to N > C in channels.
To downsample, we used stride 2 in the second Conv3D and average pooling in the skip connection.
We use kernel_size = 3 for both Conv3D .
If the block has a different number of channels C out in output, the skip connection also contains a convolution with kernel_size = 1 .
We refer to the linear path in the architecture, i.e. the sequence of skip connections, as the backbone of the model.
If C j is the number of input channels of the j -th residual block, we refer to [ C 1 , C 2 , . .
as the number of channels in the backbone features of the model.
If N j is the number of channels used by the non-linearity of the j -th residual block, we refer to [ N 1 , N 2 , . .
as the number of channels in the residual blocks of the model.
In an equivariant model, we associate a stack of ﬁeld types { ρ i } i on the input C in channels, such that (cid:80) i
d ρ i ≈ C in .
Similarly, we associate a stack of ﬁeld types to the N channels produced by the ﬁrst Conv3D .
The particular choice here depends on the kind of equivariant non-linearity used here instead of ELU .
In general, we try to ensure that the channels used in the non-linearity are approximatively equal to N , for a fair comparison.
We now list a number of non-linearities considered.
Gated Non-Linearities The input consists of a number of feature ﬁelds f ( x ) =
(cid:76) i f i ( x ) of type (cid:76) i ρ i .
Recall that f i ( x ) ∈ R d ρi .
For each feature ﬁeld i , the input should also contain a gate
g i ( x ) ∈ R of trivial type.
Then, each input feature ﬁeld f i ( x ) is mapped to f (cid:48)
( x ) = σ ( g i ( x ))
f i ( x ) , where σ is a non-linearity like Sigmoid .
Note that a gated non-linearity inherently loses some of its input channels, i.e. the input gates { g i } i .
In our models, we try to ensure that (cid:80) i ( d ρ i + 1)
≈ N (the +1 corresponds to the size of the gate
g i ( x ) for each input feature ﬁeld f i ).
Note that a ﬁeld type ρ i can be any representation.
In particular, we experiment with different choices of ρ i ; we consider { ρ i } i containing:
• each irrep of SO(3) of frequencies 1 to 2 , in equal proportions • each irrep of SO(3) of frequencies 1 to 3 , in equal proportions • each irrep of SO(3) of frequencies 1 to 2 , with multiplicities proportional to their multiplicities in the regular representation of SO(3) , i
e. for each trivial ρ 0 , we include 3 copies of the frequency 1 ρ 1 and 5 copies of ρ• copies of the quotient representation ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ 2 , band-limited up to frequency• copies of the quotient representation ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ 2 ⊕ ρ 3 , band-limited up to frequency • copies of the regular representation ρ reg
=
ρ 0 ⊕ ρ ⊕ 3 1 ⊕ ρ ⊕ 5 2 , band-limited up to frequency
To match the N channels, we also add trivial feature ﬁelds ( ρ i = ρ 0 ) and operate on them with a simple ELU non-linearity.
Note that in all cases, the ﬁrst Conv3D has C in input channels and N output channels, maintaining approximatively the same computational cost of the conventional CNN.
Tensor-Product Non-Linearity Let f ( x ) =
(cid:76) i f i ( x ) be the input features ﬁeld, with type ρ =
(cid:76) i ρ i .
In our experiments, we always chose all { ρ i } i equal to a unique type, i.e. ρ i = ρ j for all i, j .
The tensor-product non-linearity maps f i ( x )
∈ R d ρi to f (cid:48)
i ( x ) =
f i ( x ) ⊗ f i ( x )
∈ R d 2 ρi .
Note the quadratic growth in the number of channels.
In the experiments, we keep the output size approximatively equal to N , i.e. (cid:80) i d 2 ρ
i
≈ N .
It is common in the literature to learn a linear projection of f (cid:48)
i ( x ) to a lower-dimensional feature ﬁeld f (cid:48)(cid:48) i ( x ) of type ρ (cid:48)
i , with d ρ (cid:48) i (cid:28) d 2 ρ i .
The second Conv3D originally maps (cid:80) i
d 2 ρ i ≈ N channels to the output C out channels, while this projection would reduce the input channels to (cid:80) i
d ρ (cid:48)
i (cid:28) N , thereby reducing the computational cost of the layer proportionally.
This, however, results also in a potential loss in expressiveness, since a
Conv3D layer taking N channels in input can learn a larger space of operations.
Since we do not expect this linear projection to improve the performance of this architecture, we do not include it in our experiments.
Finally, in our experiments we consider the following choices of ρ i :
• the quotient representation ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ 2 , band-limited up to frequency• the quotient representation ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ 2 ⊕ ρ 3 , band-limited up to frequency• the regular representation ρ reg =
ρ 0 ⊕ ρ ⊕ 3 1 ⊕ ρ ⊕ 5 2 , band-limited up to frequency
Note that a larger feature type ρ i (i.e. larger d ρ i ) results in a smaller number of independent copies of it, since we constrain (cid:80) i d 2 ρ i ≈ N .
On the other hand, a larger ρ i makes the tensor-product f i ( x ) ⊗ f i ( x ) more expressive, since it combines more different input channels.
Therefore, the choice of the size of ρ i should be a trade-off between expressiveness and computational and memory cost.
(Pointwise) Regular non-Linearities
Again, we consider all input feature ﬁelds of the same type, i.e. ρ i = ρ j for all i, j .
Here, f i ( x ) ∈
R
d ρi is interpreted as the coefﬁcients vectors parameterizing a band-limited function in L 2 ( G ) , as described in Section H.2.
We consider band-limiting up to frequency 2 or 3 , i.e. the three following choices of ρ i :
• ρ i =
ρ
⊕ ρ ⊕ 3• ρ i = ρ 0 ⊕ ρ ⊕ 3 1 ⊕ ρ ⊕ 5• ρ i = ρ 0 ⊕ ρ ⊕ 3 1 ⊕ ρ ⊕ 5 2 ⊕ ρ ⊕
We experiment with different sampling sets G ⊂ G :
• the 24 elements of the Octahedral group O < SO(3) • the 60 elements of the Icosahedral group I < SO(3)
• P points over G with maximal relative distance, obtained by optimizing a potential energy • P = 24 · | S | points over G with maximal relative distance, obtained by optimizing over a set S ⊂ G a potential energy over G =
(cid:83) o ∈ O { o.g | g ∈ S } , such that G has Octahedral symmetry: for any o ∈ O , o. G = G .
In particular, the last sampling set guarantees that the resulting neural network is at least perfectly equivariant to O , i.e. the symmetries of the voxel grid.
In general, we observe that at least |G| ≈
d
ρ i samples are necessary for achieving an equivariance error below 10% .
For G = O(3) , we use a similar design and just augment the sampling set with inversions, i.e. we use G (cid:48)
=
Inv ×G , where G is a sampling set used for SO(3) .
We argued about the computational beneﬁts of this design in Section H.2.
For discrete groups such as
O and I , we also use regular non-linearities.
In this case, however, the ﬁrst Conv3D in the residual block directly use a regular feature types as output types and is immediately followed by the pointwise non-linearity, with no need to perform sampling.
Hence, the computational and memory cost of these models is equivalent to that of a conventional network, i.e. they have no additional computational gain.
(Pointwise) Quotient non-Linearities Similarly, we deﬁne quotient non-linearities, with the only difference that f i ( x ) parameterizes a band-limited function over a homogeneous space Q = G/H , for some H < G ; see Section H.2.
For G = SO(3) , we consider the following quotient representations ρ i :
• ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ• ρ S 2 = ρ 0 ⊕ ρ 1 ⊕ ρ 2 ⊕ ρ
and the following sampling sets
• the (center of the) 30 edges of the icosahedron; note that this set has Icosahedral symmetry • P points over Q with maximal relative distance, obtained by optimizing a potential energy • P = 24 · | S | points over Q with maximal relative distance, obtained by optimizing over a set S ⊂ Q a potential energy on Q =
(cid:83) o ∈ O { o.q | q ∈ S } , such that Q has Octahedral symmetry: for any o ∈ O , o. Q = Q .
In particular, the last sampling set guarantees that the resulting neural network is at least perfectly equivariant to O , i.e. the symmetries of the voxel grid.
Additionally, note that for Q = S 2 , the second sampling set corresponds to the well known Thomson problem.
When G = O(3) , we consider
Q = O(3) / O(2) ∼ = S 2 and Q = O(3) / SO(2) ∼ =
Inv ×S 2 .
In the ﬁrst case, we can reuse the same sampling sets deﬁned above for Q = SO(3) / SO(2) ∼ = S 2 .
In the second one, we augment the sampling set with inversions.
The considerations about the computational cost discussed in Section H.2 apply also here.
Invariant Maps Because we consider invariant tasks, an equivariant architecture should map to invariant features before the ﬁnal classiﬁcation layer.
To do so, it is common to use some invariant map at the end of the convolutional feature extractor and then feed the invariant features in a few ﬁnal fully-connected layers.
Here, we consider a few different invariant maps.
Norm-pooling : given an input ﬁeld f i ( x ) ∈ R d ρi , we compute its norm, i.e. f i ( x ) (cid:55)→ f (cid:48)
i ( x ) =
||
f i ( x ) || 2 ∈ R ; note that this is indeed an invariant quantity.
We use this kind of map for models which should be perfectly G equivariant, if G is a continuous group; for instance, we use it for models which use gated non-linearities or tensor-product non-linearities.
We experiment with different choices of ρ i :
• the quotient representation ρ S 2 =
ρ 0 ⊕ ρ 1 ⊕ ρ 2 , band-limited up to frequency• the regular representation ρ reg =
ρ 0 ⊕ ρ ⊕ 3 1 , band-limited up to frequency• the regular representation ρ reg =
ρ 0 ⊕ ρ ⊕ 3 1 ⊕ ρ ⊕ 5 2 , band-limited up to frequency• { ρ i } i contains copies of the irrep ρ 0 , ρ 1 and ρ 2 in equal proportion
Group Pooling : if G is a ﬁnite group, we can choose ρ i = ρ reg and, therefore, f i ( x ) ∈ R | G | .
In this case, we can compute the max
f i ( x ) (cid:55)→ max j [ f i ( x )] j .
Something similar can be done if G is continuous and ρ i is a band-limited regular representation by using a sampling set G .
However, this operation is often more unstable.
To devise a more stable alternative for continuous groups, we consider Average Pooling .
This operation is identical to the regular non-linearity explained above (i.e., f (cid:48)
i ( x ) =
[FT ◦ σ ◦ IFT]( f i ( x )) ), with the difference that the ﬁnal Fourier Transform FT only recovers the invariant frequency 0 component.
This is equivalent to averaging the output of [ σ ◦ IFT]( f i ( x )) over the samples G .
In our architectures, we keep the number of invariant features fed into the ﬁnal fully-connected layers constant.
Note that, to generate N invariant outputs, the invariant maps require Nd
ρ i inputs.
This can be quite expensive if ρ i is a regular representation (e.g., a regular representation of G = SO(3) band-limited to 2 has size 35 ).
To alleviate this, we introduce a Quotient Average Pooling .
This operation is similar to Average Pooling but relies on quotient representations (and samples X from a quotient space X ), which have smaller size.
H. MNIST E XPERIMENTS
As a proof of concept, we train a conventional and a C 8 equivariant model on rotated MNIST.
A model includes a sequence of 6 residual blocks similar to the ones described in Section H.3.
With respect to that design, here we use Conv2D and use stride 2 in the second convolution.
We downsample (with stride 2 convolution), every two blocks.
In the ﬁrst layer of the ﬁrst block, we use kernel_size = 7 .
The number of channels in the features of the backbone, connected through the skip connections, is C =
The number of channels in each residual block is N =
Both models map to 128 ﬁnal invariant features.
The conventional model uses a ﬁnal 3 × 3 max pooling over the spatial dimension.
In the equivariant model, we do not pad the convolution in the last residual block such that its output already has 1 ×resolution.
However, the last block maps to 128 copies of the regular representation of G = C 8 ; we then apply group pooling over them to generate 128 invariant features.
Finally, both models usefully connected layers to perform classiﬁcation.
We train both models using Adam.
We performed simple tuning of batch size, learning rate and weight decay of each model independently by evaluating them on the validation set.
The same number of hyper-parameter combinations was evaluated for both models.
Once the best set of hyper-parameters was chosen, we re-trained a model using this conﬁguration 5 more times using different random seeds.
We report the means and the standard deviations of the test performance over these runs.
H. M ODEL N ET 10 E XPERIMENTS
To generate the two ModelNet10 datasets, we use the original mesh data.
Each mesh is centered and scaled to ﬁt in the [ − 1 , 1] 3 cube.
Then, for each mesh we apply 3 random rotations.
In the normal ModelNet10 experiments, we only sample SO(2) rotations around the Z axis.
In the rotated ModelNet10 experiments, we sample arbitrary SO(3) rotations.
A mesh is then rendered by computing the signed distance function of each point in the voxel grid from the mesh.
This distance is then used into a Gaussian kernel to generate a smooth 3D occupancy grid of resolution 33 pixels.
The architecture used consists of a ﬁrst Conv3D with kernel_size = 5 , followed by 4 residual blocks and ﬁnal Conv3D layer.
The second convolution layer of the 4 residual blocks uses stride 2 .
All models map to 128 ﬁnal invariant features.
The last convolution layer has kernel_size =
In the conventional networks, the last convolution layer has 128 output channels and uses padding
= 1 , so its output has resolution 3 px; we apply max pooling with a window of size 3 to generatetranslation invariant features of resolution 1 pixel.
In the equivariant networks, the last convolution uses padding =
such that the output resolution is already 1 pixel; however, the last block maps to a larger number of channels 128 · P , depending on the invariant map chosen.
We choose this design to maintain the computational and memory cost of the equivariant and the conventional models comparable; indeed, the ﬁnal features of the conventional network have shape 128 × 3 3 while the features of the equivariant one have shape 128 · P × 1 3 , before applying, respectively, max pooling or an invariant map.
We try to keep P ≈ 3 3 = 27 .
In the architectures, we use N = [240 , 480 , 480 , 960] channels in the residual blocks and C = [39 , 78 , 240 , 480 , 312] channels in the backbone.
Finally, three fully connected layers, alternated with dropout, batchnorm and ELU non-linearity, use these invariant features to perform classiﬁcation.
For the SO(2) model, we used regular non-linearities with regular representations band-limited up to frequency 3 and |G| = 12 samples on a regular grid.
Similarly, for the O(2) , Inv × SO(2) and
Inv × O(2) models, we used regular representations band-limited up to frequency 3 and G was built by extending a grid of 12 points on their SO(2) subgroup with reﬂections and ﬂips.
In all cases, as invariant map, we used quotient average pooling , with X ∼ = S 1 ∼ = SO(2) and maximum frequency 3 .
In the SO(3) models using regular non-linearities, we used features band-limited to frequencywhen |G| = 60 and to frequency 3 when |G| = 120 .
For the O(3) model, we used regular features band-limited up to frequency 2 .
In both SO(3) and O(3) models using quotient features, we used features band-limited up to frequency 2 .
For each equivariant model, the choice of feature types in the backbone (the input and output types of each residual block) is independent of the non-linearities used.
Moreover, since no non-linearities are applied on these features, w.l.o.g.
we can consider only irreps feature types.
We consider the multiplicity of the irreps a hyperparameter which we tune during hyper-parameter tuning.
We train all models using Adam.
For each model, we independently performed simple tuning of batch size, learning rate and weight decay by evaluating them on the validation set.
This hyper-parameter search also includes a search over the variants of each architecture design (see the variants of each non-linear operation described in Section H.3).
The same number of hyper-parameter combinations was evaluated for all models.
Once the best set of hyper-parameters was chosen, we re-trained a model using this conﬁguration 5 more times using different random seeds.
We report the means and the standard deviations of the test performance over these runs.
H. A UTO -E NCODER E XPERIMENTS ON M ODEL N ET
For completeness, here we extend the comparison between the steerable bases discussed in Section 5. with an auto-encoding experiment with ModelNet10 data. In this experiment, we use the rotated voxel data from Section H.5 and train a model to reconstruct its own input.
We consider an auto-encoder architecture whose encoder and decoder comprise 5 blocks, each up/downsampling the features by a factor of 2 , such that the latent features have no spatial resolution.
Each block is a residual block including a sequence Conv3D - BatchNorm - ELU followed by either a Conv3D or a
Conv3DTransposed ; all convolutions have kernel_size = 5 and padding = 2 , and the last Conv3D / Conv3DTransposed has stride = 2 .
The residual connection down/upsamples with a Gaussian kernel followed by a learnable 1 × 1 × 1 convolution.
Finally, the model includes an invariant map and a Conv3D layer to predict a single value at each voxel cell.
The outputs of the encoder’s blocks have in order approximatively [30 , 90 , 240 , 540 , 1080] output channels, while the decoder blocks have [540 , 240 , 120 , 90 , 50] output channels.
The ELU modules within each block have about [180 , 360 , 600 , 840 , 1080] and [1080 , 840 , 600 , 360 , 180] channels. The models are trained on the data from Section H.5, which include 3 random rotations of each mesh. Testing is performed on a subset of the test meshes but each mesh is rotated by each element of the icosahedral group (i.e. | I | = 60 times), since the purpose is verifying the equivariance of the model to the icosahedral group I . As in Section 5.2, we consider two versions of the same model which differ by the choice of steerable basis employed.
The following table reports the test mean-squared errors (MSE) of the two models.
[Table] Once again, we ﬁnd the model using our anti-aliased basis outperforms the baseline using a ﬁniteorbits basis. H. C OMPUTATIONAL COST AND I NFERENCE TIME OF THE MODELS In Section H.2, we have argued that point-wise non-linearities based on discrete Fourier Transform and inverse Fourier Transform beneﬁt from a lower computational cost due to the factorized linear layers.
In this section, we verify this by comparing the inference time measured for different architectures.
We measure inference time by feeding 100 batches of 8 random 33 × 33 × 33 inputs (we ﬁrst feed 10 more batches for warm-up).
We report mean and standard-deviation over these runs in Tab. 4.
The architectures used are the ones described in Section H.3 and H.5.
For reference, we also report the number of parameters of each model.
Note also that the equivariant models include a small overhead due to the less optimized implementation of certain operations: while the I model has approximatively the same running time of a conventional CNN, the gated non-linearity model and the scaled-up regular model (marked with ∗ ) are slower that the ﬁrst two.
Despite this overhead, the normal regular SO(3) model (without ∗ ) is much faster than all other models.
[Table]
H. LBA E XPERIMENTS
To perform the LBA experiments
[Table]
we use the atom3d library from Townshend et al.
In particular, we adapted the 3DCNN example from their repository.
In the example code, each test molecule is randomly rotated at every run of the test.
To make the testing procedure deterministic, we generate a test set by rotating each test molecule with 5 random SO(3) rotations.
The occupancy grid of a molecule is generated by setting to 1 the voxel cell which is closest to each atom.
This results in a coarser rendering of the data, with respect to the ModelNet10 datasets.
To mitigate the discretization noise, we use voxels with a higher resolution of 65 pixels.
Note that Townshend et al.
(2020) used voxel grids with resolution 40 pixels.
However, the larger resolution of the input data can drastically increase the memory cost of the models.
To prevent this, all models include a ﬁrst Conv3D layer with kernel_size = 7 and stride = 2 which immediately downsamples the input.
This ﬁrst convolution layer is then followed by 4 residual blocks.
In each residual block, the stride =convolution is performed in the ﬁrst Conv3D rather than the second.
In the conventional network, the output resolution is 3 px and we use max pooling to reduce it to 1 pixel.
In the equivariant models, the last residual block does not use padding such that the output resolution is already 1 pixel.
However, this last equivariant block has more output channels than the conventional network since this block is followed from an invariant map layer.
See Section H.5 for more details about this design choice.
In the architectures, we use N =
[120 , 120 , 240 , 480] channels in the residual blocks and
C =
[26 , 32 , 64 , 128] channels in the backbone.
Finally, all models produce 256 invariant features and terminate with two fully-connected layers, alternated with dropout, batchnorm and ELU , which perform regression.
In the SO(3) architecture, we use regular non-linearities with regular features band-limited up to frequency 2 .
Because of the lower computational cost of this architecture, we scale up the number of channels in the residual block such that the computational cost is comparable with that of a conventional CNN.
This results approximatively in twice the number of channels of the original model; see the paragraph regular non-linearities in Section H.3 for more details about the computational cost of these operations.
To make our results comparable with Townshend et al.
(2020), we use the same train, validation and test split.
Moreover, during training, we augment each molecule with a random SO(3) rotation, as done in Townshend et al.
The training details and hyper-parameter tuning procedures are similar to those described in Section H.5.