A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
Authors: Leon Lang
Maurice Weiler
Venue: ICLR 2021 Poster (2021)
Keywords: Group Equivariant Convolution
Steerable Kernel
Quantum Mechanics
Wigner-Eckart Theorem
Representation Theory
Harmonic Analysis
Peter-Weyl Theorem

Abstract: Group equivariant convolutional networks (GCNNs) endow classical convolutional networks with additional symmetry priors, which can lead to a considerably improved performance. Recent advances in the theoretical description of GCNNs revealed that such models can generally be understood as performing convolutions with $G$-steerable kernels, that is, kernels that satisfy an equivariance constraint themselves. While the $G$-steerability constraint has been derived, it has to date only been solved for specific use cases - a general characterization of $G$-steerable kernel spaces is still missing. This work provides such a characterization for the practically relevant case of $G$ being any compact group. Our investigation is motivated by a striking analogy between the constraints underlying steerable kernels on the one hand and spherical tensor operators from quantum mechanics on the other hand. By generalizing the famous Wigner-Eckart theorem for spherical tensor operators, we prove that steerable kernel spaces are fully understood and parameterized in terms of 1) generalized reduced matrix elements, 2) Clebsch-Gordan coefficients, and 3) harmonic basis functions on homogeneous spaces.

Published as a conference paper at ICLR 2021
A WIGNER-ECKART THEOREM FOR GROUP EQUIVARIANT CONVOLUTION KERNELS
Leon Lang"
Maurice Weiler AMLab, QUVA Lab University of Amsterdam n. weiler. ml m0@@mmalll com
AMLab, CSL University of Amsterdam 1. la.anguuaa.n nl
ABSTRACT
Group equivariant convolutional networks (GCNNs) endow classical convolu- tional networks with additional symmetry priors, which can lead a considerably improved performance. Recent advances in the theoretical description of GCNNs revealed that such models can generally be understood as performing convolu- tions with G-steerable kernels, that is, kernels that satisfy an equivariance con- straint themselves. While the G-steerability constraint has been derived, has to date only been solved for specific use cases a general characterization of G- steerable kernel spaces is still missing. This work provides such a characterization for the practically relevant case of G being any compact group. Our investigation is motivated by a striking analogy between the constraints underlying steerable kernels on the one hand and spherical tensor operators from quantum mechanics on the other hand. By generalizing the famous Wigner- Eckart theorem for spher- ical tensor operators, and parameterized in terms of 1) generalized reduced matrix elements, 2) Clebsch- Gordan coefficients, and 3) harmonic basis functions on homogeneous spaces.
prove that steerable kernel spaces are fully understood
INTRODUCTION
Undoubtedly, symmetries play a central role in the formulation of physical theories. Any imposed symmetry greatly reduces the set of admissible physical laws and dynamics. Specifically in quantum mechanics, the Hilbert space of a system is equipped with group representation which specifies the transformation law of system states. Quantum mechanical operators, which map between different states, are required to respect these transformation laws. That is, any symmetry transformation of state which they act should lead to a corresponding transformation of the resulting state after their action. This requirement imposes a symmetry constraint on the operators themselves - only specific operators can map between given pair of states.
The situation in equivariant deep learning is remarkably similar to that in physics. Instead of a physical system, one considers in this case some learning task subject to symmetries. For instance, image segmentation is usually assumed to be translationally symmetric: a shift of the input image should lead corresponding shift of the predicted segmentation mask. Convolutional networks guarantee this property via their inherent translation equivariance. The role of the quantum states in equivariant deep learning taken by the features in each layer, which are due to the enforced equivariance endowed with some transformation law. The analog of quantum mechanical operators, mapping between states, is the neural connectivity, mapping between features of consecutive layers. As in the case of operators, there is a symmetry (equivariance) constraint on the neural connectivity only specific connectivity patterns guarantee a correct transformation law of the resulting features.
In this work we are considering group equivariant convolutional networks (GCNNs), which are con- volutional networks that are equivariant w.r.t. symmetries of the space on which the convolution is performed. Typical examples 2019) or spherical CNNs (Cohen et al., 2018). Many different formulations of GCNNs have been proposed, however, it has recently been shown that Il-equivariant GCNNs on homogeneous spaces
isometry equivariant CNNs on Euclidean spaces (Weiler & Cesal
"This research has heen conducted during an internship QUVA lab, University of Amsterdam.

---

Published as a conference paper at ICLR 2021
H/G can in fairly general setting be understood as performing convolutions with G-steerable ker- nels (Cohen et al. 2019b). Convolutional weight sharing hereby guarantees the equivariance under "translations" of the space while G-steerability is a constraint the convolution kemel that ensures its equivariance under the action of the stabilizer subgroup G < H. Although the space of G- steerable kernels has been characterized for specific choices of groups G and feature transformation laws, i.e., group representations p. see Section 5 no general solution was known so far. This work characterizes the solution space for arbitrary compact groups G.
Our solution is motivated by the close resemblance of the G-steerability kernel constraint to the defining constraint of spherical tensor operators (or more general representation operators (Jee- vanjee 2011)) in quantum mechanics. The famous Wigner-Eckart theorem describes the general structure of these operators by Clebsch-Gordan coefficients, with the degrees of freedom given by reduced matrix elements. By generalizing this theorem, we find general characterization and pa- rameterization of G-steerable kernel spaces. For specific examples, like G = SO(3) or compact subgroups of G ((2), our kernel space solution specializes to earlier work, e.g., Worrall et al. 2016); Thomas et al. 2018); Weiler & Cesa (2019). Our main contributions are the following:
We present generulized Wigner-Eckurl theorem for G-steerable kernels. It describes the general structure of equivariant kernels in terms of 1) endomorphism bases, which generalize reduced matrix elements, 2) Mobscch-Gordal coefficients, and 3) harmonic basis functions on a suitable homogeneous space. In contrast to the usual formulation, we cover any compact group G and both real and complex representations.
Corollary 4.2 explains how to parameterize G-steerable kernels and thus GCNNs.
We apply the theorem exemplarily to solve for the kernel spaces for the symmetry groups SO( Z/2, SO(3) and 0(3). considering both real and complex representations. Thereby, we demonstrate that the endomorphism bases, Clebsch-Gordae coefficients, and harmonic basis functions can usually be determined for practically relevant symmetry groups.
2 SYMMETRY-CONSTRAANE OPERATORS AND THEIR MATRIX ELEMENTS
To motivate our generalized Wigner-Eckart theorem, we review quantum mechanical representation operators and G-steerable kernels with an emphasis on the similarity of their underlying symme- try constraints. Due to their symmetries, the matrix elements of such operators and kernels are fully specified by a comparatively small number of reduced matrix elements or learnable parameters, re- spectively. This reduction is for representation operators described by the -igner-Eckart theorem. For clarity, we diseuss this theorem in its most popular form, i.e., for spherical tensor operators (0)33))repeeeentttien operators transforming under irreducible representations).
The Representation Operator Constraint Consider a quantum mechanical system with symme- try under the action of some group G, for instance rotations. The action of this symmetry group on quantum states is modeled by some unitary G-representatior[]]
: + U(H) on the Hilbert space H. More specifically, G acts kets according (1)) + 1b')) == (() (bb) and bras cording to (wbl + (15) == {1bl U(g)), where U Ugg)t is the adjoint of (gg). Observables of the system correspond to self-adjoint operators A = AT. The expectation value of such an observable in some quantum state (τ)) is given by (b/A4/b) e R.
The transformation behaviors of states and observables need to be consistent with each other. As an example, consider a system consisting of a single, free particle in R3, which is (among other symmetries) symmetric under rotations G SO(3). The momentum of the particle in the direc- tion of the three frame axes is measured by the three momentum operators (Pi, Ps. Ps). Since the momentum of classical partiele transforms geometrically like vector, one needs to demand the same for the momentum observable expectation values. If we denote by p. := (v)) Pi(y)) the expected momentum in i-direction, this means that the expected momentum of rotated system is given by P/- = Σ, Rayp; = Ej Ray ( (1b)) where R. E SO(3) is an element of the rota- tion group. This result should agree with the expectation values for rotated system states, that is, p = (PlP") = (R) (.U(R) v) As this argument is independent from the particular choice of state (V), and making use of the linearity of the operations, this implies consistency
Unnitary representations are explained in Section notation U of the unitary group U(H).
3
The notation U for the operator is distinct from the
2

---

Published as a conference paper at ICLR 2021
which identifies the collection P2, as vector operator. Other geometric quantities are required to satisfy similar constraints: For instance, en- ergy is a scalar (i.e.. invariant) quantity and the Hamilton operator H is a scalar operator, satisfying H = (R)) (R).. Similarly, any matrix valued classical quantity corresponds to rank (1,1) Cartesian tensor operator (Myj). subject to Σn Rsk (R = UR) M, (R). The overarching framework to study such situations is the notion of representation operator, which we define as a family of operators (AI...., AN) which are required to satisfy the constraint
constraint =
Σ j-n A = U()) AU(
VgEG,
(1)
where π G - U(C is some unitary representation of the symmetry group under consideration. The examples above correspond to specific choices of representations, namely the trivial represen- tation 'T()) 1 for scalars, the "standard" representation *(R) R for vectors and the tensor product representation π (RR R & (R-')T for matrices. Spherical tensor operators, discussed below, correspond to the irreps (irreducible representations) of SO(3).
The Steerable Kernel Constraint Convolution kernels of group equivariant CNNs are required to satisfy very similar constraint to that in Eq. (I). Before coming to such GCNNs, consider the case of conventional CNNs, processing image-like like signals on Euclidean space R Such signals are formalized as c-channel feature maps f R + K that assign a c-dimensional feature vector f(z) E K to each point E RR, where we allow for KK being either of the real or complex numbers R or C. Each CNN layer maps its input feature map fin : Rd - IK via convolution to an output feature map fau K * Jin Rd + e Since the convolution maps Gn input channels to Cout output channels, the kernel K Rd + K X is matrix-valued
Conventional CNNs are translation equivariant, however it is often desirable that the convolution is equivariant w.r.t a larger symmetry group, for instance the isometries E(d) of R (Weiler & Cesa 2019). For simplicity, we consider semidirect product groups of the form (Rd, +) x G. where G S GL is any compact group. Group elements tg E (R" +) x G are uniquely split into a translation E [Rd, +) and an element g E G. stabilizing the origin. They act on Rd according to t + (lg) x == gx + I. The equivariance of a GCNN - which is the analog to the symmetry of quantum mechanical system requires the feature spaces be endowed with group action of the symmetry group. A natural choice is to model the feature spaces as spaces of feature fields, for instance scalar, vector or tensor fields (Cohen & Welling 2016b).
Such feature fields are defined as functions f Rd + V, where the difference to conventional feature maps is that the space = K of feature vectors is equipped with a group representation G + GV) of the stabilizer G. The full symmetry group acts on feature fields according to ʃ + (Lg) == p(g)= I (Lg)-1, which is known as the induced representation of p. As proven in (Weiler et al., 2018a), the most general linear and equivariant map from an input field fin : Rd + Vin to an output field fou : R. + Vour is convolution with a G-stcerable kernel K Rd + Homg (Vi, Vour) ~ Kan Such kernels take values in the space of linear operators from Vn to Vau and are required to satisfy the G-steerability (equivariance) constraint
K(gx) = (g) a K(x)= Pmlg))
& G, Rd.
One can easily check that a convolution with a G-steerable kernel K is indeed equivariant, i.e., satisfies K x ((tg)- f) = (tg) (K * f) for any ig (Rd,+) x G. This result was later generalized to feature fields on homogeneous spaces H/G of unimodular locally compact groups H (Cohen al., 2019b) and on Riemannian manifolds with structure group G (Cohen et al., 2019a). That the equivariance of the convolutional network requires G-steerable kemnlss in any of these settings underlines the great practical relevance of our results.
The two constraints, Eq. ① and Eq. 2 are remarkably similar: the left-hand-sides are in both cases given by a G-rransformation of the operator or kernel itself while the right-hand-sides are given by pre- and postcomposition of the operator or kernel with unitary representations. More details on this comparison can be found in Appendix C.I.3
The Wigner-Eckart Theorem for Spherical Tensor Operators All information about linear operator A : H. + H is encoded by its matrix elements Anw := (u[AAp) E C relative to a given basis, where (v) H and (μll E H* denote basis elements of the Hilbert space and its
3

---

Published as a conference paper at ICLR 2021
dual. Similarly, all information about convolution kernel K is encoded by its matrix elements (x) = (a)[()))p) E IK where (p) E Vn and (H) E Vau are elements of chosen bases for the imput representation and dual output representation. Considering general operators and ker- nels, i.e. ignoring the symmetry constraints in Eqs. ( and (2), all matrix elements are inde- pendent degrees of freedom. In the case of convolution kernels, they correspond directly to the Cout Cin learnable parameters for every point of the kernel. However, if A is a representation op- erator - if K G-steerable kernel the symmetry constraints couple the matrix elements to each other such that they can not be chosen freely anymore. For representation operators, this statement is made precise by the Higner-Eckart theorem.
The Wigner-Eckart Eckart theorem is best known in its classical form, which applies specifically to spher- ical lensor operators. These operators are the representation operators for the irreps of SO(3), i.e., the Wigner D-matrices Dj S0(3) - U(C As such, spherical tensor operators of rank defined families T3 (T, ,...., T of 2j operators T that satisfy the constraint Σ - Dj (g) T7 U() T.Th (g) for any y E SO(3).
In order to express the operators 1 in terms of matrix elements, we need to fix basis of H. Due to the S0(3) (yyymmemer of Ty. natural choice are the angular momentum eigenstated [mm), where 1 E N20 and n = -l....,. For fixed quantum numbers j. L, and J, there are 2j components 77 of T,, 21 + basis kets [ln), and 2J + basis bras (JM). This implies that there are (2./ + (2j + 1)(21 + 1) different matrix elements (JM) 17 (mn) E C for these quantum numbers. According to the Wigner-Eckant theorem, all of these matrix elements are fully specified by one single number [Jeevanjee 201T):
Theorem 2.1 (Wigner-Eckart theorem for Spherical Tensor Operators). Let j. J E N20 and let T, be a spherical tensor operator of rank j. Then there is a unique complex number, the re- duced matrix element À E C (often written (J[[T, [IT) E C), that completely determines any of the (2J + 1)(2j + 1)(21 + 1) matrix elements (1 (In)
by the relation
(JMITT (In) x (JM]jm;b))
The coupling coefficients (JM]jm; In), known as Clebsch-Gordan coefficients, are given by the projection of the tensor product basis fim;; In) == lim) @ (ln) on [JM). They are purely algebraic and therefore independent of the spherical tensor operator Tj.
This result generalizes to arbitrary representation operators of the form in Eq. [ (Agrawala 1980). The similarities between representation operators and G-steerable kemels suggests that similar statement might hold for the matrix elements of G-steerable kernels as well. As proven below, this is indeed the case: our generalized Wigner-Fckart theorem separates their independent degrees of freedom from purely algebraic relations between mutually dependent matrix elements. It does therefore give explicit parametrization of the space of G-steerable kernels.
BUILDING BLOCKS OF STEERABLE KERNELS
This chapter gives a brief introduction to the mathematical concepts that are required to formulate our Wigner-Ecker Eckart theorem for G-steerable kernels. The first two of the following paragraphs ex- plain why it is w.l.o.g. possible restrict attention steerable kernels homogeneous spaces and to irreducible representations. The following three paragraphs discuss the building blocks of steer- able kernels, which are endomorphisms, harmonic basis functions described by the Peter- Weyl the- orem, and tensor product representations and their Clebsch-Gordd decomposition. An illustration of the concepts introduced in this chapter is given in Appendix A
The Restriction to Homogeneous Spaces Convolution kernels are usually defined on a Euclidean space Rd, i.c., they are functions K : Rd > Homg ( (Vn,, Vout). The G-stecrability constraint in Eq. (2) relates kernel values K (z) at x to kernel values K (gx) at all other points gr on the orbit Ga {g=| g E G} of x. To solve the constraint,
therefore w.l.o.g. sufficient to consider restrictions of kernels to the individual orbits, from which the full solution on Rd can be assem- bled (Weiler et al. 2018a). By construction, the orbits have the structure of a homogeneous space:
Thhe system could in general have further quantum numbers, which suppress here for simplicity.
4

---

Published as a conference paper at ICLR 2021
Definition 3.1 (Homogeneous Space, Transitive Action), Let GxX + X be continuous action of a compact group G on a topological space X. Then X is called a homogeneous space w.r.t. G if 0 # X and if for all y X there is a E G such that gr y. The action is then called transitive.
We will in the following w.l.o.g. consider steerable kernels K : X + Homm (Vn, Vout) on such homogeneous spaces X.
Restriction to Irreducible Unitary Representations The theorems below apply specifically to unitary representations, that is, representations for which the automorphisms p(g) preserve dis- tances (Knapp) 2002). As asserted by Theorem B.20 this is not really restriction every smite-dimensional linear representation can be considered as being unitary. Thus, we assume G + U(V), where U(V) is the unitary group, i... the group of preeservng linear functions on V. In the case of K = R we say orthogonal instead of unitary and write O(V).
Additionally, prior research has shown that it is sufficient to solve the kernel constraint in Eq. for irreducible (unitary) input- and output representations instead of arbitrary finite dinte-dimensonall representations (Weiler & Cesal 2019). This is possible due to the linearity of the constraint and the fact that any finitedimenniinal unitary representation decomposes by Proposition B.38 into an orthogonal direct sum of irreps. The solution for general representations can thus be recovered from the solutions for irreps. More details on these considerations can be found in Section D.1.3
If two unitary irreps are related by an isometric intertwiner, they are isomorphic; see Definition B. 18 The set of isomorphism classes of unitary irreps of G is denoted by G. We assume that for each isomorphism class j E G we have picked a representative irrep G - U(V). We denote by dj the dimension of Vj. so that we have V3 = K
Overall, we can w.l.o.g. replace Rd with X and Pin and Pout by P: : G + U(Vi) and PJ : U(Vs), where X is homogeneous space and iI and PJ are (representatives of isomorphism classes of) irreducible unitary representations of G. This leads to our working definition of steerable kernels, to which we restrict from on:
Definition 3.2 (Steerable Kernel on Homogeneous Space w.r.t. Unitary Irreps). Let X be homo- geneous space of G and pt - U(Vi and P. : G - U(VJ) be representatives of isomorphism classes of irreducible unitary representations of G. A G-steerable kemnel (on a homogeneous space and w.r.t. unitary irreps) is any function K : + Homk (Vo V,) such that the following G- steerability constraint holds:
K (ga) = pو(g) o K(x) o p((g)-1
(3)
Endomorphisms An important concept, underlying the reduced matrix elements in the Wigner- Eckart theorem for spherical tensor operators, is that of endomorphisms of linear representations. Definition 3.3 (Endomorphism of of Linear Representation). Let G GL(V) be linear rep- resentation. An endomorphism of p is a linear map e V + V which satisfies co p(g) = p(g) c for all g E G. The space of all endomorphisms of p written Endg,x(V).
Endomorphisms play central role in our generalized Wigner-Eckart theorem for steerable kernels. To get an insight why this is the case, consider a given stecrable kernel K X > Homg (Vi, Vy). The post-composition (c Kxxx) Endg,K (Vr) is obviously still steerable, i.e., satisfies Eq. (3) A basis of the space of steerable kernels is therefore partly explained by bases of the endomorphism spaces, and thus occurs in our general solution. In the following, we write {c, r I,..., Es} for the basis of Endg, (Vs), where Es = dim(Endg,.e) (V7)) is the dimension of the endomorphism space
(K(z) of this kernel with any endomorphism
The Peter- Weyl Theorem and Harmonic Basis Functions A cornerstone in our proof of the Wigner-Eckar Eckart theorem for steerable kernels is Theorem C.7 It states that the space of steerable kernels, which are G-equivariant maps K X - Homg Vi, Vy), is isomorphic to the space of
$ For K = C. Schur's Lemma D.8 tells us that the endomorphism spaces of irreducille representations are always 1- 1-dimensional, generated by the identity. For IK = R. however, one can show that they have either 1, 2, or 4 dimensions, see Bröcker & Dieck (2003). Theorem II...3.
5

---

Published as a conference paper at ICLR 2021
linear G-equivariant maps of the form K (X) Homg V,). We therefore interested in the representation theory of [ which is described by the Peter- Weyl theorem] Theorem 3.4 (Peter-Weyl Theorem, Existence of Harmonic Basis Functions). Let G be a compact group and X a homogeneous space. Let G be the set of isomorphism classes of irreducible repre- sentations. For j E G, let Pj :G 4++Us +V,)) be a representative with dimension d, = dim (V₂). Then there are multiplicities mj E 20 with mj S dj. and for each 1..... m, there are harmonic basis functions Yj² X + K, m = -.....,dy dj, such that the following three properties hold:
1. The yj fixed and i, are steerable (Freeman & Adelson 1991- Hel-Or & Teo 1998 i.e., transformation via 9 G can be expressed by shifting basis coefficients with Pji
Y YJ ( 1x) (Em 03 10 (g) ja (x).
2. Any square qquare-integrable function f X IK can be uniquely expanded in terms of harmonic basis functions, i.e., f = Ejee Ene Emn Ajam Yi with coefficients >zim E K.
3. The Yj² orthonormal system with respect to the scalar product given by integration: [x (x) Yf (I) dz 633' dey 8mmm
Note the similarity of these properties those encountered in usual Fourier analysis. Indeed, the Peter-Wey, Weyl theorem can be viewed as describing the harmonic analysis on arbitrary compact groups and their homogeneous spaces.
Tensor Products and Clebsch-Gordal Coefficients The last ingredients that we need to discuss are tensor product representations and Clebsch-Gordan coefficients. They appear, roughly speaking, in the following way: the kernel K can be thought of as being built from harmonic basis functions Ym which transform according to the corresponding irrep Pj- When harmonic kernel component of type Pj acts on an input feature field of type Pt. the combination will transform according to their tensor product Pj 69 Pt- If the convolution should map to output field of type pJ, not any harmonic component Y is admissible, but only those for which PJ appears as a subrepresentation in the tensor product Pj @ pt. The Saobsch-Gordaan coefficients encode whether Pj @ pi contains pJ, and, if it does, in which way and how often P. is embedded in the tensor product. For more details on the definitions in this section see Appendix D.1
Definition 3.5 (Tensor product representation). Let p G - U(V) and أ G U(İ) be unitary representations. Then their tensor product p أ G + U(V & V) is defined by: [[o @ ()() (v i) (v) @ [p(g]] (i).
The tensor product P3 @ P of two irreps is itself in general not irreducible anymore. However, as is again unitary representation, it splits by Proposition B.38 into direct sum of irreducible unitary subrepresentations. Thus, there is an equivariant isomorphism
(JO)
CG Vg & Vi +
θ
JEG
VJ.
(4)
==1
The integer [J(i(]] is the multiplicity V,, in Vg @ Ví, which is zero for all but finitely many J.
The matrix elements of CG3r are denoted as Clebsch-Gordan coefficients:
Definition 3.6 FRebsch-Gordanl Coefficients). Let Y Y," be the basis tensors in Vj @ Vi and let
the basis element Y be the copy of YM with index in b J=@ ① (GGIl V,. Then the Clebsch- Gordan coefficients are the matrix elements of CG2z relative to these bases,
jm; In) = (Y CG2z |Y;" @ Y?"), i.e., the scalar product of CG3a (Ym @ Y and Y.M.
*Usually, the Peter Weyl theorem uses G itself as the homogeneous space and is formulated for complex representations (Knapp) 2002). However, generalizations to arbitrary homogeneous spaces and real representa- tions are possible, as we explain Appendix [B.2]
Frrmm representation theoretic viewpoint, the functions Y for fixed and span an irreducible subrep-
resentation Vy2 of the unitary representation À G + ULL2 given by [X()) (x) = f(g 1x). (X) then splits into an orthogonal direct sum L (X) = D DT Vja. This viewpoint is explained in the cquivalent, more representation theoretic formulation of the Peter- Peter-Weyl theorem in Theorem B.22

---

Published as a conference paper at ICLR 2021
4 A WIGNER-ECKART THEOREM FOR G-STEERABLE KERNELS
Now that we have discussed all of the required ingredients, we are ready for stating our main the- orem. Intuitively, our Wigner-Eckan Eckart theorem identifies exactly those combinations of harmonics, Clebsch-Gordan coefficients and endomorphisms that, when being assembled together, yield a G- steerable kernel K X - Homg ( (V, V.). The kernel will thereby comprise all those harmonics yJ for which the tensor product V @ Vi contains V, as factor. The number of possible com-
binations depends therefore on the number of different isomorphism classes j E G for which V. appears factor in the tensor product, the multiplicity [J(jl] with which it occurs, and the mul- tiplicities my of harmonics yfr in the Peter-Wey decomposition that transform according to Pj- In addition, each individual combination
subsequently be composed with endomorphism in Endg, K (V)), which increases the number of combinations by a factor of Ej dim (Endg ıK (V,) total of ÁJ = E.- E [J(i(] mj. This number finite, as we explain in Remark D.18
How are such assembled steerable kemels parameterized? The learnable parameters correspond to the degrees of freedom in the individual components from which the kernel is built. While the Clebsch-Gordad coefficients and harmonic basis functions are fixed, the endomorphisms are ele- ments of the F.y-dimensional vector spaces Endg, (V,). The degrees of freedom of a G-steerable kernel are therefore identified with the choice of endomorphisms This gives a total of A param- eters which take values in K. Note that the choice of endomorphisms corresponds directly to the choice of reduced matrix elements of spherical tensor operators.
For a kernel K : X - Homg (Vi, Vy), we write (JM)K(s (x)] for the matrix elements of K (xx) HomB (Vi, V2) with indices S di and M S d., see also Definition D.9 Similarly, endomorphisms e (nd), have matrix elements (JM [clJM') with M, M. S d,.. We furthermore write jm/z) = (z). Finally, we denote the space of G-stecrable kemelss by Homa(X (X, Homg (V,, V.)).
Our main result is the following Wigner-Ecke Eckart theorem for G-steerable kernlss. Other versions at different levels of abstraction can be found in Theorems D.13 and D.16
Theorem 4.1 Wiiger-Eckaat Theorem for Steerable Kernels). There is vector space isomorphism
GKer
my
[JG
Endg.x(V)) + Homo Homg (V. V,))
(5)
jEG
general steerable kernel K (iKer((Ctte)ar with Cyts Endo, (VJ) has matrix elements
my
JOD dg (J JM]culJMY (s, JM jmmm))m
(zM)
(i, (m.mml (x)
(6)
jEG
kemeel matrix elements
endomorphisms
Clebsch-Gordan
harmonics
Proof. We shortly sketch a proof this theorem. We use the notation Homg, to denote linear equivariant maps. The space of steerable kernels can be progressively transformed as follows:
Homg (X, Homg ( (Vi, V2))
Home, K (X), Homg (V,, Vy))
HomG.K
my
Vj, HomB [1, Vz)
my
HomG.K (V,, Hom (Vi, Vz))
jeG
==1
jEN
i=1
(5)
(JG0] Vyr, V3)
HomG, K (V, 8 VG, VJ)
2
Home k (
J'G
==1 Endg,
[J(D]
HOmG. (V, V,)
my
[J(1]]
jEG
In (1), we linearize the kernels such that they become representation operators, as detailed in Step (2) applies the pirrssetaatonntthoreet version of the Peter- Peter-Weyl Theorem B.22 to decompose L (X) in harmonic basis functions. Step (3) makes use of the well-known fact that linear maps can be described on each direct summand individually. Topological details are explained
Theorem C.7
"hhi statement is made precise by the isomorphism GKer, defined in Eq. 5 in Thenrem 4.1

---

Published as a conference paper at ICLR 2021
in Lemma D.20 In (4), use the hom-tensor adjunction Proposition D.23 In (5). we the Goebsch-Gordaad decomposition Eq. ( which provides us with Clebsch- Gordan coefficients. In (6), we use that nontrivial linear equivariant maps from Vy to VJ exist by Schur's 's Lemma 8.29|only for J' = and, once again, that we can describe linear maps on each direct summand individually. Finally, in (7) note that Home, k (V,, V5) Endg, x (V2) is the space of endomorphisms. The formula of the matrix elements Eq. (6 is fully proven in Theorem D.13 by carefully tracing back all the isomorphisss above ?
Technically, step (1) is the main gap that we had to bridge: it establishes that non-linear kernels on X can be seen as linear representation operators on Lix (X). Steps (2) to (7) orient at the proof of the Wigner-Eckart theorem for representation operators by [Agrawala (1980). However, it differs non-trivially from the reference by a) allowing the operator to be non-injective, b) topological con- siderations, since 1g (X) is not simply direct sum of irreps but its topological closure, and c) the possibility to allow for real representations, which is why we end up with endomorphisms.
L
We obtain the following corollary, which clarifies how steerable kernels can be parameterized:
Corollary 4.2. The space (omo(X, Homp (Vi, V,)) of steerable kernels is spanned by basis ker- nels (K (issr= X Hlomm(V.. Vo) 1 G, is my, (J(iT), r S Eς} with matrix elements
(is E Σ (JM(cclMM') (s Ms,JM"jjm;hn jmm:t)) (1.jmlz).
(7)
A matrix-ean matrix-expression of the basis kernels from Eq. ① is given in Eq. (22). Here, Or is one of the E, basis endomorphisms of Enda, (Vv). This means that a general steerable kernel K X + HomB (Vi, V5) is of the form K = Ejee E === E₁- EzeālJ(it) mj learnable parameters Ajior E K. Overall, the kernel space can therefore be parameterized with an isomorphism GKer KAR + Homa(X, Homm(V,, Vs)).
[(1G)) EP= Afser Kiir with total of AJ =
Hiar =dyz 8pg 8y's C,.. Clearly, Proof. We simply choose Kjisr == ( (pos))))))) with Cyve
the clis basis of + je + u + [JGD] Endc,x(Vs), and since GKer is isomorphism, the Kj jisr form basis of steerable kernels.
Remark 4.3. The matrix elements (JMlejis JM') relate to the reduced matrix elements À C of spherical tensor operators as follows: in the case of spherical tensor operators one deals with complex irreps, whose endomorphism spaces are according to Schur's Lemma D.8 generated by the identity. Consequently, such endomorphisms c have matrix-elements (J (JM ce @JMM
X ÓMM
for some scaling factor À C. A is denoted as the reduced matrix element of the spherical tensor operator. The analog to À in our Wigner-Ecks Eckart theorem are the learnable parameters Ajier E K.
5 RELATED WORK
Harmonic convolution kernels date back to at least the early '80s (Hsu Arsenauli 1982 Rosen
& Shamir 1988). The term steerable filter coined in Freeman & Adelson [991] }) Hel-Or &
Teo (1998) generalized steerable filters to Lie groups. Reisert & Burkhardi (2007) proposed matrix valued steerable kernels between representation spaces, which are similar to our G-steerable kernels.
Steerable CNNs formulate GCNNs in the language of representation theory and feature fields. This design was proposed by [Cohen & Welling (2016b). who specifically considered finite groups, for which the kernel constraint can be solved numerically. /Weiler et al. (2018) Sab introduced the G- steerability constraint in the form in Eq. (2) for G = SO(3). The authors choose slightly different approach to solve the constraint in which they decompose the space HomR (Vi, Vy) = Vi2 @ V, instead of V, @ Vi via neously proposed by TThoma et al. (2018), who decomposed V,, @ Vi as in the present work, see Appendix E.5) The case of complex valued irreps of SO(2) was investigated by Worrall et al. (2016) and Wiersma et al. (2020); see Appendix E.I Weiler & Cesa 2019) solve the constraint for any,
coefficients. An essentially equivalent design was simulta-
In Eq. (6) terms (i. j.mmmpa mm/z)) which are not present in the original Wigner-Eckart Theorem 2.1 They appear through the lincarization of steerable kernels by Theorem
8

---

Published as a conference paper at ICLR 2021
not necessarily irreducible, representation of the groups (2)), (0) Dy and Cy. Their solution strategy is based on an expansion of the kernel in the Fourier basis of L (S') and solving for the Fourier coefficients satisfying the constraint. This is special case of the strategy that we employ in the proof of our Wigner-Fckart theorem. de Haan et al. (2020) solve for (tttetteee kernels by viewing them invariants of the tensor product representation [ (5*) & Vi" & Vy. As they use real valued irreps, they can use that the duals are isomorphic to their original counterparts. Our Wigner-Eckard theorem unifies all of these results in one general framework.
To which use cases does the proposed kernel space solution apply? As argued by Cohen et al. (2019b), any Heequivariant convolutional network on a homogeneous space H/G needs to satisfy a G-stecrability constraint - if H is locally compact and unimodular. While these works proved the necessity of steerable kernels, they did not solve the constraint
gap which is filled by our Wigner- Eckart theorem for compact groups G, see also Remark DD.15 This framework includes in particular the popular group convolutions on flat spaces (Cohen & Welling, 20T and homogeneous spaces of compact groups (Kondor & Trivedi 2018) and Lie groups (Bekkers) 2020), including for instance the sphere (Cohen et al. 2018)). Specifically, if Pmn and Pout are chosen to be regular representations L (G', steerable convolutions are equivalent to group convolutions (Weiler & Cesa 2019).
A related line of work are Clebsch-Gordan Networks (Kondor et al., 2018 Kondor 2018 Anderson et al.) 2019 Bogatskiy et al. 2020). They apply bilinear equivariant nonlinearities which compute the tensor products of global irrep features. A subsequent Clebsch- Gordan decomposition disen- tangles the product features back into irrep features. Note that in this network design, the Clebsch- Gordan coefficients are used in the nonlinear part, which differs from our use of these coefficients in the construction of steerable basis kernels, i.e. in the linear part of the network.
EXAMPLE APPLICATIONS
Cohen et al. (019b) showed in a fairly general setting that every GCNN is based on G-steerable kernels. In practice, a basis for the space of G-steerable kernels needs to be determined for param- eterizing GCNNs. This work determines the general structure of these basis kernels for compact (point-)symmetry groups G and their homogeneous spaces X: Corollary (42)) explains that one needs to determine 1) the irreps Pt of G, 2) harmonic basis functions Y T in (X) according to the Peter-Weyl Theorem 3.4 3) the Clebsch-Gordar decomposition of V, @ Vi. given by the Clebsch- Gordan coefficients (s, J M jjm; In), and 4) a basis of endomorphisms cr E EndG,.(V)) for any
jt
J E G. Given these ingredients, they can in fifth step be put together according to Eq. 7 to obtain a complete, A.j-dimensional basis of G-stecrable kernels Kjisr : X + Homg (Vi, (Vi, Vy).
Appendix demonstrates this for the examples of G being SO(2), SO(3). 0(3). and 2 consider- ing both real and complex irreps. In any of these cases, we derive the kernel bases following exuctly the five steps outlined above. This procedure can easily be applied to further compact groups, for instance (U) or SU)3), which play an important role in physics applications of deep learning.
7 CONCLUSIONS AND FUTURE WORK
Prior work revealed that group equivariant convolutions generally rely on G-steerable kernels. Our Vigner-Eckaan theorem for G-steerable kernels characterizes them for the practically relevant case of G being any compact group. The degrees of freedom or learnable parameters correspond thereby precisely to the choice of endomorphisms. This mirrors the situation in quantum mechanics, where the degrees of freedom of spherical tensor operators are given by reduced matrix elements.
It would be desirable to extend this result to non-compact groups, where the Peter-Weyl Theorem does not hold anymore. One alternative might be Pontryagin duality (Reiter 1968). which describes the Fourier transform on locally compact abelian groups. Furthermore, for many non-compact, non- abelian groups, one can often find a direct integral decomposition of (G). This generalization of the Peter Peter-Weyl theorem can be found in Segal (1950) and Mautner (1955). Such generalizations of our Wigner-Eccart Theorem might lead to better theoretical understanding of several recent work
(Worrall& & Welling 2019 Bekkers 2020; Sosnovik et al. 2020; Shutty & Wierzynski 2020).
Finally, we hope that the analogies between stecrable kernels and representation operators appearing in physics inspire further research in this fascinating crossdisciplinary domain. This could lead to applications of GCNNs for learning tasks with physical symmetries.
9

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGMENTS
We thank Lucas Lang for discussions on the Wigner-Eckart Theorem and observables in physies and Patrick Forré for discussions on the link between steerable kemelss and representation operators. Additionally, we are greatful for discussions with Gabriele Cesa on the connection between real and complex representations of compact groups. Furthermore, we thank Stefan Dawydiak and Terrence Tao for online discussions on aspects surrounding a real version of the Peter- Peter-Weyl theorem. Finally, we thank Roberto Bondesan, Miranda Cheng, Tom Lieberum, and Rupert McCallum for feedback on different aspects of our work.
REFERENCES
Vishnu Agrawala. Wigner-Fekart theorem for an arbitrary group or Lie algebra Journal of Mathe- matical Physics, 21, July 1980. doi: 10. 1063/1.524639.
Brandon Anderson, Truong-Son Hy, and Risi Kondor. Cormorant: Covariant Molecular Neural
Networks In Conference Neural Information Processing Systems (NeurIPS), 2019.
A.V. Arkhangel'ski and M. Tkachenko. Topological Groups and Related Structures in mathematics. Atlantis Press, Jan 2008.
Atlantis studies
Erik Bekkers. B-Spline CNNs on Lie groups In International Conference Learning Represen- tations (ICLR), 2020.
Alexander Bogatskiy, Brandon Anderson, Jan T. Offermann, Marwah Roussi, David W. Miller, and Risi Kondor. Lorentz Group Equivariant Neural Network for Particle Physics In International Conference on Machine Learning (ICML), 2020.
A. Bohm and M. Löwe. Quantum Mechanics: Foundations and Applications Springer New York, 1993.
Springer study edtion.
N. Bourbaki. General Topology: Chapters 1-4
Elements of mathematics. Springer, 1998.
T. Bröcker and T. Dieck. Representations of Compact Lie Groups Springer Berlin Heidelberg, 2003.
Graduate Texts in Mathematics.
Taco Cohen and Max Welling. Group Equivariant Convolutional Networks. In International Con- ference on Machine Learning (ICML), volume 48, pp. 2990-29909999 2999, New York, New York, USA, 20-22 Jun 2016a. PMLR.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge Equivariant Convo- lutional Networks and the Icosahedral CNN International Conference on Machine Learning (ICML), volume 97, pp. 1321-1330, Long Beach, California, USA, 09-15 Jun 2019a. PMLR.
Taco S. Cohen and Max Welling. Steerable CNNs, In International Conference on Learning Rep- resentations (ICLR), 2016b.
Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs In International Conference on Learning Representations (ICLR), 2018
Taco S Cohen, Mario Geiger, and Maurice Weiler. A General Theory of Equivariant CNNs on Homogeneous Spaces In Advances in Neural Information Processing Systems (NeuRIPS),
John Conway. A Course in Point Set Topology Jan 2014. doi:
Stefan Dawydiak. Is there Peter-Weyl-Theoreen over the real numbers? Exchange, 2020. UKL.htttp:://mah.tttaae http://matt..staaccccccccaaaaa com/ /////523559222
Mathematics Stack
Pim de Haan, Maurice Weiler, Taco Cohen, and Max Welling, Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs arXiv e-prints, art. arXiv:2006.00722 2020.
L. Debnath and Mikusinski. Introduction to Hilbert Spaces with Applications Elsevier Science, 2005.
10

---

Published as a conference paper at ICLR 2021
William Freeman and Edward Adelson. The Design and Use of Steerable Filters IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 13:891-900 906, 10 1991. doi: 10.1109/34.9 3808.
Gallier and J. Quaintance. Differential Geometry and Lie Groups: Second Course and Computing. Springer International Publishing, 2020.
Geometry
Y. Hel-Or and Patrick C. Teo. Canonical Decomposition of Steerable Functions Journal of Mathe-
matical Imaging and Vision, 9:83 95, 1998.
Roger A. Horn and Charles R. Johnson. Matrix Analysis edition, 2012.
Cambridge University Press, USA, 2nd
Yuan-Neng Hsu and H. Arsenault. Optical pattern recognition using circular harmonic expansion Applied Optics, 21(22):4016 1982.
Nadir Jeevanjee. An introduction to tensors and group theory for physicists Birkhäuser, New York, NY, 2011. doi: 10. 10.1007/988-088176447777
R.V. Kadison and J.R. Ringrose. Fundamentals of the Theory of Operator Algebras. Volume I Fundamentals of the Theory of Operator Algebras. American Mathematical Society, 1997.
I. Kaplansky, Set Theory and Metric Spaces AMS Chelsea Publishing Series. AMS Chelsea Pub- lishing. 2001.
Anthony Knapp. Lie Groups Beyond an Introduction, Second edition volume 140. Jan 2002. doi: 10. 1-4757-2453-0.
Risi Kondor. N-noor.--bodyy Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials, arXiv e-prints, art. arXiv:1803.01555 2018
Risi Kondor and Shubhendu Trivedi. On the Generalization of Equivariance and Convolution in
Neural Networks the Action of Compact Groups In International Conference Machine Learning (ICML), Feb 2018.
Risi Kondor, Zhen Lin, and Shubhendu Trivedi. CCebsch- Nets: a Fully Fourier Space Spher- ical Convolutional Neural Network In Conference on Neural Information Processing Systems (NeuriPS), 2018.
E. Kowalski. An Introduction to the Representation Theory of Groups Graduate Studies in Mathe- matics. American Mathematical Society, 2014.
S.M. Lane, S.J. Axler, Springer-Verlas (Nowy Jork)., F.W. Gehring, and P.R. Halmos. Categories For the Working Mathematician) Graduate Texts in Mathematics. Springer, 1998.
T.M. MacRobert. Spherical Harmonics: An Elementary Treatise Harmonic Functions, with Applications Methuen, 1947.
I. Mautner. Note on the Fourier inversion formula on groups Mathematical Society, 78:371-384, 1955.
Transactions of the American
L. Nachbin and I.. Bechtolsheim, The Haar integral University series in higher mathematics. Van
Nostrand, 1965.
Marco Reisert and Hans Burkhardt. Learning Equivariant Functions with Matrix Valued Kernels. Journal of Machine Learning Research, 8:385 4088 Mar 2007.
H. Reiter. Classical Harmonic Analysis and Locally Compact Groups Oxford mathematical mono- graphs. Clarendon P.,, 1968.
Joseph Rosen and Joseph Shamir. Circular harmonic phase filters for efficient rotation-invant invariannn pattern recognition Applied Optics, 27(14):895--289 1988.
I. E. Segal. An Extension of Plancherel's 's Formula to Separable Unimodular Groups Mathematies, 52(2):272292, 1950.
Annals of
11

---

Published as a conference paper at ICLR 2021
Noah Shutty and Casimir Wierzynski. Learning Irreducible Representations of Noncommutative Lie Groups arXiv e-prints, art. arXiv:2006.00724, June 2020.
Ivan Sosnovik, Michal Szmaja, and Arnold Smeulders. Scale-Equivarian Steerable Networks In International Conference Learning Representations (ICLR), 2020.
W.A. Sutherland. Introduction Metric and Topological Spaces Open university set book. Claren- don Press, 1975.
T. Tao. An Introductionn Measure Theory ical Society, 2013.
Graduate studies in mathematics, American Mathemat-
Terrence Tao. The Peter- Weyl Theorem, and non-abelian Fourier analysis compact groups 2011 1.
URL http:///wee.aaaaro..ooooooorooea orem and www.-doo--aeeeaaaneeee -
the
Nathamiel Thomas, Tess Smidt, Steven M. Keamne,, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley. Tensor Field Networks: Rotation- and cannslation--qivariaa Neural Networks for 3D
Point Clouds arXiv e-prints, art. arXiv//802.082213 2018.
Maurice Weiler and Gabriele Cesa. General E(2 Equivvaaat Steerable CNNs In Conference Neural Information Processing Systems (NeurIPS), 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3D Steerable
CNNs: Learning Rotationally Equivariant Features in Volumetrie Data In Advances in Neural Information Processing Systems (NeuRIPS). 2018a.
Maurice Weiler, Fred Hamprecht, and Martin Storath. Learning Steerable Filters for Rotation Equiv- ariant CNNs, In Conference on Computer Vision and Puttern Recognition (CVPR), pp. 849-858,
Jun 2018b.
Ruben Wiersma, Elmar Eisemann, and Klaus Hildebrandt. CNNs Surfaces using Rotation- Equivariant Features) Transactions on Graphics, 39(4), July 2020. doi: 10. 15.1145/386669.33
92437.
E.P. Wigner. Gruppentheorie und ihre Anwendung auf die Quantenmechamik der Atomspektren Die Wissenschaft. J.W. Edwards, 1944.
Dana P. Williams, The Weyl Theorem for Compact Groups 1991. URL httt::://t:: artmouth. artaaootth.eeeeeaaaaam/////ppppp
Daniel Worrall and Max Welling. Deep Scale-spaces: Equivariance Over Scale, In Conference on Neural Information Processing Systems (NeurIPS). 2019.
Daniel E. Worrall, Stephan Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow. Harmonic
Networks: Deep Translation and Rotation Equivariance In Conference on Computer Vision and Pattern Recognition (CVPR), volume abs/1612.046A4 2016.
12

---

Published as a conference paper at ICLR 2021
APPENDIX
This appendix contains detailed and rigorous treatment of the Wigner- Wigner-Eckart theorem for steerable kernels, including background knowledge, proofs, and many example applications.
In Chapter A we shortly look at the simple example SO(2) for motivating the concepts and results in Section 3
Everything afterwards, starting with Chapter B can be read independently of the main paper and a self-contained treatment of our investigations. In Chapter B we start with the foundations of the representation theory of compact groups. We formulate the Peter-Weyl Theorem [B.22] which tells us how to decompose the space of square-integrable functions on homogeneous space into irreducible representations, leading harmonic basis functions. In the second half, we include proof of the more algebraic parts of this theorem. We do this since the theorem is usually only proven for complex representations in the literature, but we need it for real representation well.
In Chapter we investigate steerable kernels and show their similarities to representation operators from physics and representation theory. In Theorem C.7 we will then proof a precise isomorphism between steerable kernels and representation operators on the space of square-integrable functions on homogeneous space. We call these kernel operators.
In Chapter D we will then formulate and prove the Wigner-Eckan theorem for steerable kernels of general compact groups D.13 The proof makes in essential parts use of the Peter-Weyl Theorem and Theorem C.7 and additionally of Schur's s Lemma the hom-tensor adjunction Proposition D.23 and the Clebsch-Gordan decomposition of tensor products.
In Chapter D we then look specific example applications of our theory. In these examples, look at specific compact transformation groups G, specific, relevant homogeneous spaces of the group and one of the fields R or C. For this combination derive basis for the space of steerable kernels between arbitrary irreducible input- and output representations of the group. Specifically, look harmonic networks (Worrall et al.) 2016). -eeeeeequivariant networks for real represen- tations (Weiler Cesa 2019), Zz-equivariant networks for real representations,
networks for both real and complex representations (Weiler al., 2018a Thomas al. 2018), and -equivvavaa networks for both real and complex representations. The investigation of Zz- equivariant CNNs will additionally show that our result consistent with group convolutional CNNs for the regular representation (Cohen & Welling, 2016a).
In Chapter F we summarize some important notions and results from the theory of topological spaces, metric spaces, normed vector spaces, and (pre-)Hilbert spaces that we use throughout this appendix.
Chapters B C. and D contain the bulk of the theoretical work. We recommend the reader to first only I
read the first halves of these chapters, Sections [BI C. [.. since they contain the formulation of the most important results and the main intuitions, whereas the second halves of these chapters, i... Sections B.2 C.2 and D.2 mainly contain detailed proofs that can be skipped when going over the material for the first time.
CONTENTS OF THE APPENDIX
A Building Blocks of Ste2)-Steerable Kernels Running Example for Section 3
19
B Representation Theory of Compact Groups
22
I Foundations of Representation Theory and the Peter- Weyl Theorem
22
B.2 A Proof of the Peter-Weyl Theorem
29
C The Correspondence between Steerable Kernels and Representation Operators
40
Fundamentals of the Correspondence
40
A Proof of the Correspondence between Steerable Kernels and Kernel Operators
47
13

---

Published as a conference paper at ICLR 2021
D A Wigner-Eckart Theorem for Steerable Kernels of General Compact Groups
51
D.1 A Wigner-Eckart Theorem for Steerable Kernels and their Kernel Bases
52
D.2 Proof of the Wigner-Eckart Theorem for Kernel Operators
63
Example Applications
67
E.1
SO(2)- (tte-rtteebbeee Kernels for Complex Representations Harmonic Networks
68
S0(2)- Kermels for Real Representations
70
Zz-Steerable Kernels for Real Representations
75
-ttee-etee Kernels for Complex Representations.
79
E.5
() -tteerale Kernels for Real Representations
81
E.6
3 -Stecrable Kernels for Complex Representations
86
E.7
Steerable Kernels for Real Representations
90
Mathematical Preliminaries
90
F1
Topological Spaces, Normed Spaces, and Metric Spaces
90
F2 Limits of nets and approximated Dirac delta functions
94
Pre-Hilbert Spaces and Hilbert Spaces
95
LIST OF SYMBOLS
GENERAL SET THEORY AND FUNCTIONS
AnB
intersection of sets A and B union of sets A and B intersection of sets A; union of sets A; union of sets A; which are disjoint from each other a subset of B
AUB
ler A
ier A
eET A
B
A C
B
A is strict subset of B set of all elements A which are not in Cartesian product of sets structures (e.g., groups) A, B empty set
A B
A B
X =
X is defined Y
often an equivalence relation equivalence class with respect to an equivalence relation indicator function of set A composition of two composable functions f and g either the inverse of function f or the preimage function restriction of a function ff to a subset
1A
fog
f
fla
NUMBERS AND COLLECTIONS OF NUMBERS
natural numbers including 0
integers Z
field real numbers
field of complex numbers skew-field of quaternions one of the fields R and C
K
n-dimensional canonical vector space over IK complex conjugate of
14

---

Published as a conference paper at ICLR 2021
GROUPS
G
a compact topological group
e
neutral element of group with multiplication operation neutral element of an additive group semidirect product of groups G and H group of planar rotations of regular N-gon
0
G M И
CN
Dy
group of planar rotations and reflections of regular N-gon
(O(d)
special orthogonal group in d real dimensions
(d)
orthogonal group in d real dimensions
O(V)
orthogonal group of a real Hilbert space V
(U(d)
special unitary group in d complex dimensions
U(d)
unitary group in d complex dimensions
U(V)
unitary group of a complex Hilbert space Euclidean motion group in d dimensions
E(d)
BASIC REPRESENTATION THEORY
linear representation of group
or
The function G - V. + p(g)
piv
matrix coefficient of the unitary representation representations of the in-field and out-field, respectively
p
pous
PHom
Hom-representation on Homg (V, V') of representations and pf tensor product representation on @ V of representations and p'
p p
Ind H
p
induced representation on H or representation P G set of isomorphism classes of unitary representations G an isomorphism class of unitary representations representative of isomorphism class
G
I
pl
Vi
vector space on which acts
of or Y
fixed chosen orthonormal basis vector of Vi
VECTOR SPACES AND HILBERY SPACES
dim(V)
dimension of K-vectorspace
V
L
W
V and W are perpendicular
V
W
V and W are isomorphic with respect to their structures
V 2 W
V and W not isomorphic with respect to their structures
(flg)
bra-ket notation of a scalar product on a Hilbert space equivalent to (yl (x)) for function, null space of image of f adjoint of the operator f identity function on V
wl wlf())
mull(f)
im())
f
idy
(HILBERT) SPACE CONSTRUCTIONS FROM OTHER SPACES
Homg (V. W) GL(V)
space of K-llnnear functions from V to W space of invertible K-linear functions from V to itself, sometimes written GL(V, K) in the literature
HomG.K (V, W) Homg X, W)
space of intertwiners from V space of G-equivariant continuous maps from X to W, for a homogeneous space space of endomorphisms of V, i.e., intertwiners from V to
Endgx(()
15

---

Published as a conference paper at ICLR 2021
V W
tensor product of two vector spaces their common field. Also denotes the tensor product of pre-Hilber, spaces (orthogonal) direct sum of all V;
Dier Vi ! ier Vi
topological closure of the (orthogonal) direct sum of all V; vector subspace of K-vector space spanned by M orthogonal complement of V eigenspace of yp for eigenvalue
spank (M) V Ex(e)
TOPOLOGICAL SPACES, METRIC SPACES, NORMED SPACES
T
topology
UR
open neighborhood of X
Uy
set of all open neighborhoods of X limit over the directed set of open neighborhoods of æ
limgeu-
lim Fk
limit of the sequence (xtx )
A
topological closure of C
|
norm of
x
absolute value of æ distance of x according metric d
d(r, r (x)
e-ball around x according to some metric d
HOMOGENEOUS SPACES AND THE PETER- WEYL THEOREM
X
homogeneous space of G arbitrary point
x² E
X
S
n-dimensional sphere in (n + dimeesiioaa space measure on a compact group G or its Homogeneous Space
X
X
integral on space X with respect to its measure Hilbert space of square-integrable functions on X and G with values IK
(X), LK)
A
unitary representation on 1 (X) or (G)
s(x)
arbitrary lift of x with respect to projection G >>
X, gz* average f K along cosets lift of functions (X) (G) Dirac delta function at point x
av(f)
8
d0
approximated Dirac delta function for nonempty open U
abbreviation for pf g'' for orthonormal basis vectors of, vi Vi
linear span of all matrix coefficients of irreducible unitary representations
&
linear span of all matrix coefficients of pt linear span of all matrix coefficients p with varying i but fixed j
1/
multiplicity of in orthogonal decomposition of (G) and L (X), respectively
n,, my
Ví
copy of Vú appearing in the Peter-Weyl decomposition of L
PG
canonical projection Pl:
(X) -> Via and pa
Dry Vy - Vı:
sinms COSm
the functions x - sin(ma) and ++ cos(ma)
Yn. TY"
complex- and real-valued version of spherical harmonic complex- and real-valued version of Wigner D-matrix
D₁, Dr
16

---

Published as a conference paper at ICLR 2021
KERNELS AND REPRESENTATION OPERATORS
K
kernel X + Homg (Vin Vout)
K*f f convolution of kernel K with input f kernel operator or (more generally) representation operator : T' + Homk (U, V)
R
kernel operator K (x) - Homg (Vin Vout) corre- sponding to kernel K
X
kernel K|xx : X - Homm(Vin: Vout) corresponding to a kernel operator K.
K
for representation operator K. T + Homm (U, V), this denotes the corresponding map K. T U V under the hom-tensor adjunction
THE WIGNER-ECKART THEOREM
PLPS Yj YP, YJ
input- and output representations on the spaces Vi and Vy fixed chosen orthonormal basis vectors of the abstract irre- ducible representations Vj. Vi, V,
(x)lk) di
matrix element K ((x) for a kernel K dimension of l'th irrep Vi as K-vector space
my
number of times V, is in the Peter-Weyl decomposition of (X)
(J(T)
number of times V, is in the direct sum decomposition of
V, @ Vi
C, Cjis
endomorphisms, mostly on Vy. Cjis endomorphisms ap- pearing the Wigner-Eekart theorem for steerable kernels basis endomorphism of pJ, indexed with index set 1...., E,
(JMICJM")
matrix element at indices M, M' for endomorphism linear equivariant isometric embeddings 1, V, - Vg 8 Vi and (jte Vs Vj @
1s ljis
Pits
projection Pjts Vj X Vi + V,, corresponding to (i.e.: ad- joint to) the embedding fjis Clebsch-Gordan coefficient corresponding to 3-dimensional matrix of Clebsch-Gorde coefficients
(s. MMjmm In) CGnghs Yja
harmonic basis function, for example, spherical harmonic. Element of Vy S (X)
fm/a)
shorthand notation for limyreu (Y (ōr) Equal to Yjm (x) row vector with entries (i, (i.jm(z)
(i.jlz)
Rep
isomorphism between tuples of endomorphisms and kernel operators
GKer
isomorphism between tuples of endomorphisms and steer- able kernels
jisr Wjter
basis kemne
learnable parameter
17

---

Published as a conference paper at ICLR 2021
LIST OF THEOREMS
B 22
Theorem (Peter- Weyl Theorem)
28
B.27
Theorem (Density of Matrix Coefhcients)
30
C.7 Theorem (Cemee-Oprratoo-Coreeppoddrr
46
C.8 Theorem (Kcemet-ppratoo-Corrsspeeddedeee Restated)
47
D.11
Theorem (Wigner-Eckart Theorem)
56
D.13
Theorem (Wigner-Eccart Theorem for Steerable Kernels)
58
D. 16 Theorem (Steerable Kernel Bases)
61
F
F224 Theorem (Heine-Borel Theorem)
93
LIST OF DEFINITIONS
Definition (Group, Abelian Group)
22
B.2
Definition (Subgroup)
23
B.3
Definition (Group Homomorphism)
23
B.4
Definition (Topological Group, Compact Group)
23
B.5
Definition (Group Action)
23
B.6
Definition (Orbit)
23
B.7
Definition (Transitive Action, Homogeneous Space)
23
B.8
Detinition (Stabilizer Subgroup)
24
10
Definition (Linear Representation)
24
B.11
Definition (Intertwiner)
24
8.12
Definition (Equivalent Representations)
24
B.13
Definition (Invariant Subspace. Subrepresentation, Closed Subrepresentation)
25
B.14
Definition (Irreducible Representation)
25
15
Definition (Unitary Group)
25
16
Definition (Unitary Transformation)
25
B.17
Definition (Unitary Representation)
25
B.18
Definition (Isomorphism of Unitary Representations)
25
B
24
Definition (Matrix Coeflicients)
29
Definition (Hom-Representaaion))
42
C,3
Definition (Stecrable Kerne))
43
C.4
Definition (Representation Operator)
44
C.6
Definition (Kernel Operator)
45
D.I
Definition (Tensor Product)
52
D.2
Definition (Tensor Product of Pre-Hil spaces)
52
D.3
Definition (Tensor Product Representation)
53
D.6
Definition (Clebsch-Gordan Coefficients)
54
D.7
Detinition (Eadomorphism)
55
18

---

Published as a conference paper at ICLR 2021
D.9 Definition (Matrix Element)
55
D.12 12
Definition (Reduced Matrix Element)
56
E. 11
Definition (Real, Complex, and Quatermionic Type Irreducible Representations)
83
12
Definition (Restriction and Extension)
84
E. 14
Definition (Real Type Complex Representation)
84
Definition (Tensor Product Representation)
87
Detinition (Topological Space, Open Sets, Closed Sets)
90
Definition (Open Neighborhood)
F3
Definition (Hausdorfi Space)
91
F4
Definition (Subspace)
F,
Detinition (Closure, Density)
91
F.6
Definition (Continuous Function, Homeomorphism)
F.
Definition (Open Cover, Compact Space)
91
F.
8,10
Definition (Product Topology)
Definition (Quotient Map, Quotient Space)
91
F.
F.13
Definition (Norm)
92
F
14
Definition (Metric)
92
F115
Definition (Convergent Sequence)
92
F
16
Definition (Continuity in Metric Spaces)
92
E.17
Detinition (Uniform Continuity)
92
F.19
Definition (Cauchy Sequence)
93
F
F20
Detinition (Complete Metric Space)
93
F21
Definition (Completion)
93
F
F23
Detinition (Boundedness)
93
H226
Definition (Partially Ordered Set, Directed Set)
94
F
F28
Definition (Net)
94
F.29
Definition (Convergence Nets)
94
F33
Definition (Approximated Dirac Delta)
94
R32
Definition (pre-Hilbed) Space, Hilbert space)
95
R355
Definition (Orthogonality)
95
F
R336
Definition (Orthogonal Complement)
95
F
F339
Definition (Orthonormal System)
96
F.40
Definition (Orthonormal Basis)
96
R442
Definition (Adjoint of an Operator)
96
A BUILDING BLOCKS OF SO(2)-STEERABLE KERNELS RUNNING EXAMPLE FOR SECTION 3
In this short chapter, we briefly explain the components of steerable kemnels at the specific exam- ple of real valued irreps of the circle group SO(2). While this example is quite simple, shows
19

---

Published as a conference paper at ICLR 2021
some non-trivial properties like 2-dimensional endomorphism spaces Endgorz),(()) for J > and a Clebsch-Gordan decomposition in which the multiplicity [J(jt) can differ from 0 or 1. To give a quick overview: Example A.T considers the circle as an orbit and homogeneous space of (0(2) while Example A.2 introduces the real valued irreps. Their endomorphisms are stated in Example A.3 As discussed in Example A.44 the Peter- Peter-Weyl theorem corresponds here to the usual Fourier series S!. The Crebsch-Gordaid decomposition of tensor products of the irreps cussed in Example A.5] With these ingredients, we are ready to instantiate the kernel spaces as described by our Wigner-Eckart Theorem 4.1 for steerable kernels, for which refer, including proofs, to Section E.2
dis-
S0(2) -steeratee kemels K R2 HOmR (Vm; Vout) ~ Routt X allow for rotation equivariant convolutions, For instance, a convolution with an (0(2)-steeaable kernel on I² is guaranteed to be SE(2) = (IR,+) x SO(2) equivariant while convolution with an (S)eesrrrerll kernel on S will be
SO(2) acts on kernel's domain R² by rotating The orbits of the action are therefore given by 1) the origin {0} and 2) circles of arbitrary radius. We know that the kernel constraint can solved on each orbit individually, and so we can restrict to looking at those. Since {0} is rather trivial, we specifically consider the circle S* more interesting homogeneous space.
Homogeneous Spaces
Example A.1. Consider the circle 5 and the rotation group SO(2). For convenience, we reparam- eterize both: we view (2) as the group of angles e E R/22Z [0, 2T] / /~~2> and S as the space R/ R/2TT as well. Then the action of (2) on Sa is given by Φ x == (ø x) mod 2m. It is easy to see that this action is transitive, which makes the circle a homogeneous space of S0(2).
Irreducible Representations As it is sufficient solve the kernel constraint for irreducible thogonal input- and output representations, we now state a classification of those up to isomorphism.
Example A.2. The irreducible orthogonal representations pt SO(2) - O(V) of SO labeled by indices ("quantum numbers") E IN 20 For = O, one has the trivial representation with V/6 = IR and (o( idg. For 1 1, one has V R and
cos((o) (1o)
sin(le) cos(le)
((())
i.., rotation matrices of "frequency 1" The isomorphism classes of irreducible orthogonal repre- sentations are then given by $0(2) NNo-
We are thus in the following considering S0(2) kernels of the form
K S¹ + Homp (Vi, Vy),
where L,J 1/220 0.
Endomorphisms Remember that if c V, V, is endomorphism, i.e., commutes with PJ, that o K is then steerable as well. Thus, we now look at a classification of the endomorphisms of the irreducible orthogonal representations:
Example A.3. Let G S2()) with the irreducible representations p. as in Example A.2 Clearly, the endomorphism space Endsor2),n (V)) is 1-dimensional, i.c., Eo 1. For all J I. the endo- morphism space is two-dimensional (E, = 2) and given by combinations of scalings and rotation on V, R²a A basis of this space is given by the following two matrices:
C idga = ( 90. c2 = (1 -1). spang spanafcr.cm Endsorr).
Another way to imagine this is to identify Vy with the complex plane C. Then an endomorphism is given by multiplication with arbitrary complex number.
20

---

Published as a conference paper at ICLR 2021
That 01 is an endomorphism of PJ for J 2 is immediately clear. That the same holds for cg is checked by the following simple calculation:
(
coss( (o))
sin(J(b) cos(Je)
(- - -cos(Jo) cos(JØ) sii((Jo)
p2))
(in(J
cos(J Jmh sin(J
sin(Jø)
(i
0) p2(
cos(Jó)
The proof that there are no other endomorphisms sketched in Proposition E.5
Peter- Weyl and Harmonic Basis Functions Another ingredient that we need to construct (2 steerable kernels S is the decomposition of (S*) into its irreducible subrepresentations Vji. which the Weyl theorems guarantees to exist. Less abstractly, we are interested in an orthonor- mal set of harmonic (steerable) basis functions on S that span L (St) which corresponds the usual Fourier series on S
Example A.4. As in Example A.1 harmonic analysis says that iquare-integrable functions f St - R, i.e. f E (S*), can be uniquely written as an infinite sum of sine and cosine terms.
we assume G SO(2) and X = S A standard result in
f(x) = au + Σ و aj cos(jæ) + by sin(ja),
where on and az, by, 2 are real-valued expansion coefficients.
How does this result relate to the harmonic basis functions in the Peter- Peter-Weyl theorem 3.49 As stated above, have isomorphism classes G SO(2) = N2o of irreps with representatives Pj- A com- parison of the Fourier series in Eq. (8) with property 2 in the Weyl theorem 3.4 suggests the following identification of harmonic basis functions y and coefficients Ajm.
Y0 cOSg = 1,
A01 0u
for
j j2
Y co8
by1 aj
for
Y? sin,
A12 by for
j 1214 1,
where we introduced the shorthand notations cos,(2) (z) = cos(jær and sin, (x) := sin(ja). Note for any j SO(2). As expected, we have indices m = 1 for j = 0 with d; dim (V)) 1 and indices m = 1.2 for 2 1 with d, = dim((V)) = 2. The orthogonality relations in property 3 of the Peter- Weyl theorem hold up to simple normalization of these basis functions and are easily checked by explicitly computing the scalar products. Property 1, i.c., the -ttc)sstcrrybility of the harmonic bases, trivial for j = O. For j 1, the standard angle summation formulas for cosines and sines lead to the following expressions for harmonics that are translated by á E SO(2):
that we dropped the index i 1.... my since m,
cos, (x o) cos, (I) cos,(-o) sin, (x) (-n) (o) cos; +p² sin;) (z) )) + siny (x) cos,(- = (o) cos, +p; (0) sin,) ) (z),
siny(x b) = cos, (I) sin, (-6)
which is just property 1 in the Peter-Weyl theorem. This is concisely summarized by
cos, sin,
(Ø 1 x)
cos, sin,
(x Φ) PT(T)T
coss sin,
(x),
which shows that the basis functions Y cosj and Y? sin, span invariant subspace V, of T (S1) under rotations. From more abstract viewpoint, the Peter- Peter-Weyl theorem just states that 122
L (S1) splits into the orthogonal direct sum + jesor2) Vj.
Tensor Products and Clebsch-Gordan Coefficients Finally, need to investigate the tensor products of irreducible representations and their decomposition via Mebscch-Gordan coefficients. They will be used correctly assemble harmonic basis functions steerable kemelss.
21

---

Published as a conference paper at ICLR 2021
Example A.5. Remember the irreducible representations p SO(2) - O(Vi given in Example
A.2
As we prove in Proposition E.4 including a description of the Clebsch-Gordan coefficients, the tensor products decompose follows:
Vo 2 Vo =
Vo. vov=v, V,, V - Vi, & V @ Vs++,
where the last isomorphism only holds if j. 2 and + j. 2 and j = I, then we obtain
Vi Ø Vi = (V6) e V21+
i.e. Vo here appears twice in the decomposition of a tensor product of irreducible representations. We therefore have multiplicities [J(i(] which are 1 for [0(00),, [i(j)),, [((O)), [li -(())
[j + [(T)] and [2((1)] while [0(tt] = 2. Any other multiplicity is zero.
Bigner-Eckarr theorem for (O2)ssteerable kernels With these ingredients one can then deter- mine all (2tts-eteeebl kemelss. This is explained in Proposition E.6
B REPRESENTATION THEORY OF COMPACT GROUPS
In this chapter, we outline the main ingredients of the representation theory of compact groups that we need for our applications to steerable CNNs. Usually, this theory is only developed for representations over the complex numbers. However, since we want to apply it also to steerable CNNs using real representations, we need to be a more careful. In particular, we need to make sure that the Peter-Wey! theorem is correctly stated and proven.
The outline is as follows: In Section B.I we start by stating all the important definitions and con- cepts from group theory and representation theory of (unitary) representations that needed for formulating the Peter- weyl theorem. After defining Haar measures both for compact groups and their homogeneous spaces and shortly discussing their square-integrable functions, Peter-Weyl Theorem B.22 In Section B.2 then, we give a proof of this version of the Peter- Weyl theorem, carefully making sure not properties that steps, mainly the density of the matrix coefficients in the regular representation, we refer to the lit- erature, since the proof clearly does not make use of C per se. While we initially only give the proof for the regular representation, i.c., the space of square-integraae end this section with a discussion of general unitary representations and, in particular, the space of square-integrable functions for an arbitrary homogeneous space.
formulate the
only over C. In some essential
functions on the group itself, we
In the whole chapter, let IK be the field of real or complex numbers.
B. FOUNDATIONS OF REPRESENTATION THEORY AND THE PETER- WEYL THEOREM
B.1.1 PRELIMINARIES OF TOPOLOGICAL GROUPS AND THEIR ACTIONS
In this section, we define preliminary concepts from topological groups and their actions. This mate- rial can, for example, be found in detail in Arkhangel skii & Tkachenko (2008). For the topological concepts that we use, we refer Chapter F.I
Definition B.1 (Group, Abelian Group). A group G (G),,(()-),e)) most often simply written G, consists of the following data:
1. A set G of group elements g E G.
2. A multiplication : :G x - G. (g. + g
3. An inversion (+)-1 G + G, + + g-
4. A distinguished unit element e E G. It also called neutral element.
They are assumed to have the following properties for all g, h, l E G:
1. The multiplication is associative: g (h k) (g h) k.
2. The unit element is neutral with respect to multiplication: e-g=g
22

---

Published as a conference paper at ICLR 2021
3. The inversion of an element multiplied with itself is the neutral element: 9.9 =g- e.
group called abelian if, additionally, the multiplication is commutative: g h h for all g. h E G. If this is the case, group is often written as G = (G, +, ()),
If we consider several groups at once, say G and H, then we often do not distinguish their multipli- cation, inversion, and neutral elements in notation. It will be clear from the context which group the operation belongs to.
Definition B.2 (Subgroup). Let G be group and H C G a subset. H is called a subgroup if:
1. For all h, h E H we have h. h E H.
2. For all h E H we have h 1 E H.
3. The neutral element E G in H.
Consequently, H is also a group with the restrictions of the multiplication and inversion of G to H. Definition B.3 (Group Homomorphism). Let G and H be groups A function f G + H is called a group homomorphism if it respects the multiplication, inversion, and neutral element, i.e., for all h E G:
1. f(g h) = f(g) f(h).
2. 1) st
3. f(e)= cc
The second and third properties automatically follow from the first and so do not need to be verified order to prove that a certain function is group homomorphism.
Definition B.4 (Topological Group, Compact Group). Let be a group and be a topology the underlying set of G. Then G T) is called lopological group (Arkhangel skii & Tkachenko 2008 if both multiplication continuous maps. Additionally, always assume the topology to be Hausdorff.
G + G, (x, y) + x y and inversion -> G, x + are
topological group is called compact the underlying topological space is compact.
From now on, all groups considered are compact topological groups. Furthermore, whenever G is a finite group, we assume that it is topological group with the discrete topology, i.e., the topology with respect to which subsets G are open.
We will need the following definition in order to define homogeneous spaces:
Definition B.5 (Group Action). Let G be a compact group and X topological space. Then group action of G on is continuous function G X + X with the following properties:
1. (g h) = g (h x) for all g, h, E G and X.
2. e x x for all X.
We will often simply write ge instead of g.x. Also, note that the multiplication within G is denoted by the same symbol as the group action on the space X.
Definition B.6 (Orbit). Let G x X + X be a group action. Let x E X. Then it's orbit, denoted G. is given by the set
Gg
Definition B.7 (Transitive Action, Homogeneous Space). Let : G X be group action. This action is called transitive if for all x, y E X there exists g E G such that gr = y. Equivalently, each orbit is equal to X, that is: For all X we have G. X.
X is called a homogeneous space (with respect to the action) if the action is transitive, X is Haus- dorff and X + 0.
23

---

Published as a conference paper at ICLR 2021
The Hausdorff condition and non-emptiness in the definition of homogeneous spaces is needed for Lemma B.21 which is necessary to even define normalized Haar measure on a homogeneous space. Some texts in the literature may define homogeneous spaces without these conditions.
Definition B.8 (Stabilizerr Subgroup). Let G x X X be group action. Let X. The stabilizer subgroup G2 is the subgroup of G given by
2 = {g E G g2 = S.
Example B.9. The multiplication of the group is group action of G on itself. a homoge- neous space with this action. Furthermore, for each G the stabilizers Gg are the trivial subgroup e.
In general, homogeneous spaces with the property that all stabilizers trivial called forsors or principal homogeneous spaces. Principal homogeneous spaces are topologically indistinguishable from the group itself.
B.1.2 LINEAR AND UNITARY REPRESENTATIONS
In this section, we define many of the foundational concepts about linear and unitary representa- tions (Knapp 2002 Kowalski 2014).
Whenever we will consider linear or unitary representations of compact groups, we want those representations to be continuous. This requires that the vector spaces on which our groups act carry themselves topology. Prototypical examples of such vector spaces are (iib-rt spaces. They are the main examples of vector spaces considered in this work. Foundational concepts about (pre- Hillert spaces can be found in Chapter F33 The most important difference between how we view pre-Hiller spaces and how it can often be found in the literature is that in this work, scalar products are antilinear the first component and linear in the second. This the convention usually chosen in physics.
For a vector space V over K let GL(V) be the group of invertible linear functions from V to V. Sometimes in the literature, this is also written GL(V, K). The multiplication is given by function composition and the neutral element by the identity function idy on
Definition B.10 (Linear Representation). Let a compact group and be a K-vector space carrying topology, for example. Hilbe-Hilbert space. Then linear representation of G on V is group homomorphism p : G + GV(V) which is continuous in the following sense: for all v E V, the function
G - g g p(g(
is continuous. From the definition obtain p(c) idy, p(g h) p(g) p(h) and p(g p(g)- for all g, h E G. For simplicity. we also just say representation or G G-representation instead of linear representation. Instead of denoting the representation by p. we often denote by V if the function is clear from the context.
Note that in this definition, V can be any abstract topological K-vector space with a topology and does need to be space I something similar. Consequently, usually do not view the functions p(g) as matrices, but as abstract linear automorphisms from V to V.
Definition B.11 (Intertwiner). p: G + GL(V) and p' - GL(V') be two representations over the same group G. An intertwiner between them linear function f V + V' that is additionally equivariant with respect to p and p' and continuous. Equivariance means that for all g E G one has f p(g) = p' (g) f. which means the following diagram commutes:
V
p(g)
('(o)
V'
Definition B.12 (Equivalent Representations), Let p: G - GL(V) and p': G > GL(V') be two representations. They are called equivalent if there is intertwiner f V V that has an inverse. That is, there exists intertwiner f V V such that f f idy and f f idy..
24

---

Published as a conference paper at ICLR 2021
In categorical terms, equivalent representations are isomorphic in the category of linear representa- tions. The reason we do not call them isomorphic is that there is a stronger notion of isomorphism between representations which we will later use, namely isomorphisms of wnitary representations.
Definition B.13 (Invariant Subspace, Subrepresentation, Closed Subrepresentation), Let : G > GL(VV be a representation. An invariant subspace W S V is a linear subspace of V such that p(g)(w) W for all g G and W. Consequently, the restriction plw G - GL((W, + p(g)lw: W + W is representation as well, called subrepresentation of p-
A subrepresentation is called closed if W is closed in the topology of V.
Definition B.14 (Irreducible Representation). A representation p G GL(V) is called irre- ducible if V + 0 and if the only closed subrepresentations of are O and V itself. An irreducible representation is also shortly called irrep.
Definition B.15 (Unitary Group). Let V be pre-Hilbert space. The unitary group U(V) of V is defined as the group of all linear invertible maps f V + V that respect the inner product, i.e., (f (v)) = (zly) for all z, y E V. It is group with respect to the usual composition and inversion of invertible linear maps.
Note that the field K the real numbers, then what we call "unitary" actually called orthogo- nal, and the group would be denoted O(V). However, the mathematical properties are essentially the same, and since the term "unitary" is more widely used (as normally, representations over the complex numbers are considered) stick with "unitary".
More generally, we have the following:
Definition B.16 (Unitary Transformation). Let V, V be two pre-Hilbert spaces. A unitary transfor- mation J V + V is a bijective linear function such that (s()))[(w)) (xll)) for all x, E V. These can be regarded as isomorphisms between pre- pre-Hilbert spaces.
Note that unitary transformations are in particular isometries, i.e., they keep the distances of vectors with respect to the metric defined by the scalar product. For the definition of this metric, see the discussion before and after Definition F.14
Definition B.17 (Unitary Representation). Let V be pre-Hilbert space and G a group. Then a representation p: G + GL.(V) is called a unitary representation if p(g) U(V) for all g E G. We then write : G + U(V).
In this whole chapter, the space V of a unitary representation is supposed to be a Hilbert space, instead of just pre-Hilbert space. Only in chapter D will we consider unitary representations on pre-Hilbert spaces. Note that all finite-dimensionl pre-Hiibert spaces are already complete by Proposition F.47 so in these cases, there is no difference. The same proposition also shows that for finiteddimensionll unitary representations, we can ignore the topological closedness condition in order to check whether it is irreducible. It will later turn out that all irreducible representations of a compact group are automatically mnite-dmeensionll anyway, see Proposition B.3l so this further simplifies our considerations.
As before with the unitary group, a unitary representation actually called "orthogonal representa- tion" when the field is the real numbers R.. U(V) is then replaced by O(V). We again stick with U(V) whenever the field K is not specified.
Definition B.18 (Isomorphism of Unitary Representations). Let p G U(V), p' : U(V') be unitary representations and V + V an intertwiner. is called isomorphism (of unitary representations) if. additionally, ff is a unitary transformation. The representations are then called isomorphic. For this, write p p' or V = V" depending on whether we want to emphasize the representations or the underlying vector spaces.
We note the following, which we will frequently use: due to the unitarity of p(g) for a unitary representation A, we have p(g)* p(g)-1 i.e., the adjoint is the inverse. Adjoints defined in Definition F.42 and this statement is proven more generally in Proposition R.44 Overall, this means that (p(g) c) ( (clp(g)~1 (w)) for all v and
In the end, it will turn out that the Peter- Weyl theorem which we aim at is exclusively a statement about unitary representations. One may then wonder whether this is too restrictive. After all, the
25

---

Published as a conference paper at ICLR 2021
representations that consider for steerable CNNs (with precise definitions given in Section are not necessarily unitary, and so it is not immediately obvious how the Peter-Wey theorem will be able to help for those. However, spaces can be considered as unitary, and so the theory applies. We will discuss this in Proposition 15.00once understand Haar measures compact groups.
it turns out, all linear representations
inite-dimensional
B.1.3 THE HAAR MEASURE, THE REGULAR REPRESENTATION AND THE PETER-WEYL THEOREM
Now that we have introduced many notions the representation theory of compact groups, we can formulate the most important result, the Peter- Weyl theorem that will use throughout this work. In the next section, we will then go through a step-by-ster proof of this theorem. The material in this section is based on Nachbin & Bechtolsheim (1965); Kowalski (2014) and Knappp (2002). We thank Stefan Dawydiak for a discussion about the Peter- Weyl theorem over the real numbers (Dawydiak) 2020).
We assume that the reader knows what a measure is TTa 2013). Let G be a compact group. A standard result is that there exists measure μ on G, called a Haar measure that, among other properties, fulfills the following:
1. S()) can be evaluated for all Borel sets S S G. Here, the Borel sets form the smallest so-called o-algebra that contains all the open sets.
2. In particular, we can evaluate y(S) for all open or closed sets S C G.
3. The Haar measure normalized: μ(G) = 1.
4. μ is left and right invariant: y²g5S) (s5) ((gg) for all g E G and S measurable.
5. μ is inversion invariant: ((S-1) = =){{) for all S measurable.
These properties then translate into properties of the associated Haar integral: let f G - K be integrable with respect to μ4, then we obtain:
fa ldg = 1 for the constant function with value 1. fighddg for all h e G.
2. fa flggddg = (g)
= ((a))
3. Sa (dg fl)dd
Example B.19 (Finite Groups). If G is a finite group with elements, then the Haar measure is just the normalized counting measure which assigns u(g) for all g G. Each function - K is then integrable, and its integral is just given by
1 [())dg- Σ f())-
DEG
In this special case, one can easily verify all properties of Haar measures and Haar integrals stated above.
With this measure defined, dimensional spaces can be considered as unitary:
already understand why all linear representations finite-
Proposition B.20. Let p
GL(V) be a linear representation on a finite dimensdimensional space V. Then there exists a scalar product (-1)); V x V - K that makes (V, (---) Hilbert space and such that becomes a unitary representation with respect to this scalar product.
Proof. Since is finite-dimensional, there an isomorphism of vector spaces to some T Con- sequently, there is some scalar product (+1-) V x V + K that makes V Hilbert space. However, this scalar product does not necessarily make p a unitary representation. However, we can define (+1)), x K by
(wlw), f (P(o)))))) dg.
That this integral exists is due to the continuity of linear representations and since also the scalar product is continuous by Proposition F38 It can easily be checked that this construction makes V
26

---

Published as a conference paper at ICLR 2021
Hilbert space. And due the right invariance of the Haar measure, check that is unitary representation with respect to this scalar product. Namely, for arbitrary g' E we have:
J. )ol)))l
(olg)(
(vlw))-
Now, for measure space Y with corresponding measure μ,, we can consider the space of square- integrable functions on Y with values in K, denoted E (Y) (the measure is omitted in the notation since there is usually no ambiguity). In these spaces, functions are identified if they coincide on a set with measure O. (Y) is clearly a vector space over K, but it turns out that it can even be considered be Hilbert space follows:
(flg) = J, Mwloll)dd
Here, the overline means complex conjugation. The Hilbert space properties are easily verified.
In particular, one can consider the space (G) of square-integrabee functions on the group G it- self. Now the claim is that (G) can actually be equipped with a prototypical structure as a unitary representation over G which makes this space, some sense, "universal among unitary representa- tions". This works with the following canonical representation, called the regular representation:
A:G - (G)), (M))(() (o') = gg)
continuity of this map is non-trivial and is, for example, shown in Knapp 2002). However, the more algebraic properties of being a unitary representation are easy to appreciate. First of all, we clearly see that Å is group homomorphism mapping each group element to linear automorphism. And finally, the unitarity of this representation can be understood as a direct consequence of the properties of the Haar measure, where we notably make only use of the left-invariance:
(o) = [M())()) (o') (h)) (g')
[lg-t MMg"'g')dg
f(gr)h(g')dd JG
= -fflh)
We saw in Example B.9 that G is a homogeneous space with respect to the action on itself. We can now ask whether these constructions can also work X is an arbitrary homogeneous space of G. This requires us to define a suitable measure on X. This is indeed possible. For a fixed element x² E X, denote the stabilizer subgroup by H G. C G. Then the Hausdorff property of X allows to write down a homeomorphism between X and G/H, which in turn will allow us to use canonical measure on G/H that we study below. We denote cosets gH E G/ H by [g]. Lemma B.21. Let X be homogeneous space of the compact group G and H the stabilizer sub- group of a fixed element x² X. Then the map
ip: G/H + X, [g] + gr"
a homeomorphism. Furthermore, H is topologically closed.
Proof. Let G gr. This map is equal to the composition of the maps G + G X, ++ (g,x") and G X + X. (g,z) x) + gr. Both these are continuous, and thus & is continuous as well. Furthermore, note that if "'g' E H. then there is h H such that g' gh, and thus
((g')= ((gh) (gh)x* g(hx") ) = gx. p(())
27

---

Published as a conference paper at ICLR 2021
which means that by Proposition F.12 the map y continuous map. It is surjective since the action is transitive by definition of a homogeneous space. Furthermore, it is injective since if gz g' x² then xº (g 2 g x²) and thus g H, which means [g] = [g']-
G/H + X. gz" is well-defined
Overall, ip is continuous bijective map from G/H X. Furthermore, G/H is compact since it the continuous image of the compact group G under the projection G - G/H, see Proposition F88 Since X is Hausdorff by definition of homogeneous spaces, yp is homeomorphism according to Proposition
Now, since X is Hausdorff and yp is homeomorphism, it follows that G/H is Hausdorff well. Then, necessarily, H is topologically closed subgroup of G, see Bourbaki (1998). Chapter III. Section 2.5, Proposition 13.
Every space G/H where H is topologically closed allows measure μ with similar properties to those of G (Nachbin & Bechtolsheim 1965). Since the stabilizer H is closed and X G/H by Lemma B.21] we can do these constructions for X well, as we outline now. The only properties that we now miss are the right-invariance and inversion-invariane since G does not naturally act on X from the right and since cannot invert elements in X. But left-invariance does hold and this means that
We simply can't ask for them
G (X), (x) 1x)
makes (X) unitary representation over G, as can be shown in the exact same way as for (
Let G be the set of isomorphism classes of irreducible unitary representations over G. Furthermore,
let pt G - Vi be fixed representative of such isomorphism class / E G. We write isomorphism classes as "(" (and later also and J) in order to bring to mind quantum numbers used in quantum mechanics. Recall from linear algebra that a countable sum of subspaces of vector space is called direct if no nontrivial subspace of any of the considered spaces is contained in the sum of all the other considered spaces Furthermore, recall that two subspaces U, C V of a Hilbert space V are called perpendicular or orthogonal if (ulw) = 0 for all u E U and w E W. We then write U W. We can now formulate the Peter- Weyl theorem. Intuitively, it says that (X) splits into an orthogonal direct sum of the irreducible unitary representations, where each irreducible unitary representation appears maximally as often as its own dimension (and may not appear at all):
Theorem B.22 (Peter-Weyl Theorem). Let G be compact group. Let X be homogeneous space. invariant subspaces Vis S (x) for all l E G and i E (1,...,m,} such that the following hold:
There are numbers my E Nu for all i G and
Vι = Vj as unitary representations for all and l.
2. my S din(V)) < oo for all I.
3. Vi 1 Vyj whenever + l' or $
4. + G 1=
Vu is lopologically dense in (X), written Lx( Die
e + mi
Vis
t=1
Now additionally consider G as homogeneous space of itself. Then the same holds for L (G) as well, with numbers nı S dim(Vi) < cc. We additionally have the following:
1. my < nt.
2. KK C, then n; dim(V))
Note that the representative Vi is not assumed to be embedded in (X). is just isomorphic, as unitary representation, to each of the V₁ S L3 (X).
Example B.23. For G (2) and K we have (SO(2)) Diez Vn and all irreducible representations Vi are 1 1-dimensional.
For a vector space V and subspaces (Ui)ee,, their sum Eiel U, is the set of sums Eier with J C finite and in U, for all i. It is itself subspace of V.
28

---

Published as a conference paper at ICLR 2021
For G (0( and K = R, we obtain (SO(2)) DI2O and all irreducible representations Vi with 2 are two-dimensional, whereas Vo is one-dimensional. Thus, here where the multiplicity of most irreducible representations in the regular representation is and therefore smaller than their dimension, which cannot happen for representations over the complex numbers.
example
Both of these results are standard results in harmonic analysis. These examples are discussed in more detail, especially with respect to their applications in deep learning, in Section E.] and E.2
B.2 A PROOF OF THE PETER-WAN WEYL THEOREM
This section presents a proof of the Peter- Peter-Weyl theorem, as formulated Theorem B.22 We mostly skip the analytical parts of the proof since they well-presented in the literature and clearly work over both the real and complex numbers. However, the more algebraic parts of the proof usually make use of the property of the complex numbers be algebraically closed, which does not hold for the real numbers. This is invoked usually both in the proof of a version of Schur's lemma, well in proving Schur's orthogonality. We therefore carefully adapt the proof of the Peter-Weyl theorem in the literature so that it also works over the real numbers, and formulate and prove versions of Schur's Lemma B.29 and Schur's orthogonality B.30 that work in general.
This section can be skipped if the interest is mainly in the applications of the Peter-Wey! theorem. In this case, the reader is advised to directly move on to Chapter
We note the following convention that applies to this section: for all unitary representations p G + U(V) that we consider here, V is a Hilbert space (instead of just a pre-Hilbert space).
B.2.1 DENSITY OF MATRIX COFFFICIENTS
An important ingredient in the construction of the spaces Vis that appear in the formulation of the Peter- Peter-Weyl Theorem B.22 are matrix coefficients, which together generate those spaces in case that one considers the regular representation on L (G).
Definition B.24 (Matrix Coefficients). Let G - U(V) be unitary representation. A matrix coefficient any function of the form
6v G g (ulp(g)())
for arbitrary u. v E V.
The term "matrix coefficient" comes from the analogy to matrix elements of linear maps between pre-Hilbert spaces of which orthonormal bases are fixed. Later, in Definition D.9 will also define the notion of "matrix elements" separately. The term "matrix coefficient" only applies to unitary representations.
Remark B.25. By definition of linear representations, the function - p(g) continuous. Thus, since scalar products of Hilbert spaces are also continuous functions on V x V. Proposition F38 every matrix coefficient pn G + IK is continuous. As a continuous function on a compact space, it is of course also square- integrrbe, i.e., n ( The Peter- Peter-Weyl theorem basically asserts that these matrix coefficients can be considered as the building blocks of all square-integrable functions.
Furthermore, one may wonder why there is a complex conjugation in the definition. The reason for this is that, otherwise, the isomorphism that we will construct in Proposition B.35 is not linear but conjugate linear. The reason why this can nevertheless be called a matrix coefficient is that this actually is the matrix coefficient (without complex conjugation) on a conjugate Hilbert space, as explained in the next Proposition, which we took from Williams (1991).
Proposition B.26. Let p: G + U(V) be a unitary representation on a Hilbert space V with scalar multiplication -v and scalar product (-)) We have the following:
1. V = V (equality as abelian groups) with v v == a v and (ulv) == (ulp) is again a Hilbert space, the so-called conjugate Hilbert space V.
101, those parts that deal with approximations of square-integrable functions by matrix elements.
29

---

Published as a conference paper at ICLR 2021
2. p: G U(V) with p(g) = p(g) is again unitary representation.
3. For the matrix coefficients, we huve pr' (²) pmv
Proof. All these assertions are easy check. As demonstration, do 3:
~ (g) (ulp(g)(v)) = (ulp(o)(v)) = pm)(gg)
That's what we wanted show.
As consequence of this proposition, the matrix coefficient pda is equal to puy (ur(g), thus being "matrix coefficient without complex conjugation above the scalar product" of the conjugate unitary representation.
Theorem B.27 (Density of Matrix Coefficients). The linear span of the mutrix-coefficeens of finite- dimensional, unitary, irreducible representations of are dense in L (G) for all compact groups G.
Proof. For K C, this is shown in Knapp (2002). The same proof, without adaptions, also works for K = R. Note that the cited proof uses a definition of matrix coefficients without the complex conjugation. However, Proposition B.26 shows those span the same space, and thus we can apply to our situation.
B.2.2 SCHUR'S S LEMMA, SCHUR'S ORTHOGONALITY AND CONSEQUENCES
In this section, we state and prove versions of Schur's lemma and Schur's Orthogonality (Knapp 2002) that are valid for both K I and K = C.
Lemma B.28. Let G U(V) and p' G + U(V") be unitary representations. Furthermore, let + an intertwiner. Then the adjoint f V' V also an intertwiner.
Proof. The adjoint f* VV' + V is the unique continuous linear function from V' to V such that, for all u V and 2' E V", we have
(f(v)le") = (vl))
This always exists according Definition F.42 Note that with f being an intertwiner and using the unitarity of the representations, we obtain for all E G. V and o' E Vt:
(vlp(o)) = (p) (o)lo)
((0)))))(()
(((o))'())(''))
(wll"''(0)(()))
from which deduce p(g)f" (')()) from Proposition F.45 for all g G. i.e., f is intertwiner.
Lemma B.29 (Schur's Lemma for unitary Representations), Assume G + U(V) and p' G U(V') are irreducible unitary representations with V fiite--imensional. Also assume that 0 there is p E R such that pf is
V Vi is intertwiner. Then either isomorphism.
Proof. For this proof, follow the exposition of Tao 2011). We thank Terrence Tao for confirming in the discussion below his blogpost that this lemma can also be proven over the real numbers.
Let V + V be the adjoint of which is also intertwiner by Lemma B.28 Now, set = ft f V + V. As a composition of intertwiners, &p is also an intertwiner. Furthermore, for arbitrary composable continuous linear functions between Hilbert spaces always has (g
30

---

Published as a conference paper at ICLR 2021
h)* ht" gg" and (g")" 9, which easily follows from the definition and uniqueness of adjoints. Consequently, we have
0 -r=))))==o
and so y is self-adjoint. Thus, (p(t))w) (vlp(w)) for all v, w E V. from which we conclude that the matrix of ip corresponding to any orthonormal basis of V is Hermitian or, if K = R. even symmetric. Such an orthonormal basis exists by Proposition F.4T From the Spectral Theorem for Hermitian or symmetric matrices (Horn & Johnson 2012) we conclude that is unitarily (or for real matrices: orthogonally) diagonalizable with only real eigenvalues. Thus, there is an orthogonal decomposition of V into eigenspaces: V
eigenvalue (x())
Let (x())) be any eigenspace. We claim that it is invariant subspace of p. Indeed, for all E G and v E Ex(e) we have since yp is an intertwiner:
p(p(g)(v) = ((g)(((()) = ((o) = (p)
Since V finite-dimensional, (x( topologically closed by Proposition F:47 and since V is irreducible, we necessarily have Ex(p) 0 or Ex(p) V. Since not all eigenspaces can be zero, we conclude that there is an eigenvalue with Ex(e) = V, mcaning & = Aidy.
Assume f + O. We now claim that À > 0. Indeed, note that for all V we have A||u||2 (1(())l)) (5* o s()))))
((())((())
IIf(())?.
Thus, if v E V is any vector with f(v) + 0, then obtain À (U )² 0.
|ell
Now define g V V as X f. is clearly still intertwiner. We can also show it is an isometry:
(a(v)lo(w)) - (f(O))((w))
x (p(u)lu))
= (olw()
(rlw))
Note that since V is irreducible and f(V) V topologically closed due V being finite- dimensional, we necessarily have that f is surjective. Thus, we have shown that uf with μ = X- is an isomorphism of unitary representations.
Proposition B.30 (Schur's Orthogonality). Let p : G - U(V) and p' : G + U(V') be non- isomorphic irreducible unitary representations of the compact group G, of which at least is finite-dimensionall Let pdd and p'''vv be matrix coefficients of them, which are functions in L (G) due to their continuity. Then they are orthogonal, i.e.,
0.
Proof. Without loss of generality, we can assume V to be finite ninie-ddmmesional.. Assume that : V - is any linear function. We can associate to it the function f V' + V given by
f(w') [ (l
For all h G' we have
p(h)Sp'(h) (1))
(hg))
[ ((g))p'(g)
31

---

Published as a conference paper at ICLR 2021
and thus p(h)f = fh'(h), which means that f is intertwiner. In this derivation, p(k) could be put insight the integral since p(h) is continuous and an integral is a limit over finite sums, which commutes with the continuous p(h). By Schur's Lemma B.29, we necessarily have f = 0. Now look at the specific linear function V - V given by [w') == (u'lw") with the fixed vectors v, vf corresponding the matrix coefficients. We obtain f= 0, for f defined as before, and thus:
(M) [ (
0 = (ulf(u))
(ulo(a))/()) dy
(u)p()) (v'') dd
(ulp(o)(v)) (0'1p'()) dg
(ulp(g)(v)) (4')p'
pd (g) (g)dg
In this derivation, the integral could be put out of the scalar product since the scalar product is continuous, see Proposition F.38 and since integrals are certain limits over finite sums, with which the scalar product commutes.
Note that there are more general Schur's orthogonality relations in the case that KK = C, see Knapp (2002). Corollary 4.10. These then engage with the matrix coefficients of one and the same rep- resentation. This, together with a version of Schur's 's lemma that only holds over C leads to the strengthening of the Peter- Peter-Weyl theorem that shows that the multiplicities n, are given by dim.
Proposition B.31. All irreducible unitary representations of compact group G are finite- dimensional.
Proof. Assume p: G + U(V) was an irreducible unitary representation on an infniitediimensional space V. Let puv be any of its matrix coefficients. By Proposition B..0 and since an infinite- dimensional representation can never be isomorphic to a finite-dimensional representation, pdv is perpendicular to all matrix coefficients of finite-dimensional irreducible unitary representations. Due to the linearity of the scalar product, pd is perpendicular to the whole linear span of these matrix coefficients and thus the topological closure of this span. The last step follows from the continuity of the scalar product, see Proposition F.38 By Theorem B.27 this closure is the whole space L (G). Therefore, nav is even perpendicular to itself, and thus p 0.
Overall, for arbitrary u, v E V and g G we obtain 0 p p²"(g) = (v) and thus (by setting " = (v)) (v) = 0 and consequently p(g) = 0. We obtain = 0, a contradiction. Thus unfnite-dimensinn irreducible unitary representations cannot exist.
As consequence, mention that the finiteness conditions in Schur's lemma and Schur's Orthogo- nality were not necessary to state since all irreducible unitary representations are finite-dimensional anyway. We obtain from this and from Schur's Lemma B.29 that isomorphism classes and equiva- lence classes of irreducible unitary representations are one and the same.
B.2.3 A PROOF OF THE PETER-WEYL THEOREM FOR THE REGULAR REPRESENTATION
In this section, we engage with the Peter- Reyl theorem for the regular representation on (G). The case of L (X) for homogeneous space X will be dealt with in Section B.2.4 The core arguments in the proofs of this section are adapted from Williams (1991).
As before, let & be the set of isomorphism classes of irreducible representations of G. For [ E â let o be representative for the isomorphism class I. Furthermore, for each Pt G U(V),,
32

---

Published as a conference paper at ICLR 2021
be arbitrary orthonormal basis, which exists due Proposition F.41 (mostly written without the superscript, i.e., v² v² ....... if the corresponding isomorphism class is clear). Denote p Di poo Remember that matrix coefficients of unitary representations are continuous by Remark B.25 and thus functions in (G). Then, let & C 1 (G) be the linear span of the matrix coefficintts of all irreducible unitary representations. In the next Lemma, we want to show that & is already spanned by the matrix coefficients corresponding representatives of isomorphism classes and their orthonormal bases:
let ut ,....... ................................ 1/
dim(Vi)
Lemma B.32. We have
3 - spank { (I fl..... dim dim(V))))
Proof. First, we show that isomorphic representations don't add distinct matrix coefficients. Thus, let p = Pt and let f V and thus, since is a unitary transformation, p(g) f* pi(g) S. for all E G, see Proposition F44 Now u, v E V be arbitrary. We obtain
be the corresponding isomorphism. Then we have f f(g
( = (ulp(g)(u)
(ulf">(l))(())
which proves the first claim. Now we want to show that we only need to consider the 6 Thus, let u Vi be arbitrary. They allow for linear combinations
E, X. v= -=
with coefficients X p' E K. We obtain:
(g) = (u)p(())(v)
= Σ. E, N'W (p)))())()))
-E,E, NTA (g).
thus showing that ppde is in the linear span of the matrix coefficients corresponding to the orthonormal basis. This concludes the proof.
For an isomorphism class E G, let & = span {pts j.j {1...., dim(V S (G) be the
linear subspace of & generated by matrix coefficients corresponding to l. Let furthermore for all the space ε & the subspace generated by all nf for E (1,..., din, (ii))V).. In the next lemma, prove that these are actually closed subrepresentations of the regular representation.
Lemma B.33. For {l...., dim E/ is a closed invariant subspace of L (G), In particular, Ej is closed invariant subspace of L3 (G).
Proof. Closedness follows immediately since this space is finie-ddimenioonll and thus complete, Proposition F.47 We need show that Xgg)pt/ & for all g E G and all i,j. We can compute
33

---

Published as a conference paper at ICLR 2021
this directly:
(M))o))((1 (0) 1g '0)
En (o(()))))))) of ((()))))
(p(l))))))) (((((())())
E (o"lo(o>") (v0"))) (s))
EE pi
where the coefficients of 1) do not depend on g'. Consequenly, Mg)p E Ef.
Lemma B.34. Let p + U(V) and p' : G + U(V') be unitary representations, p being irreducible and V + O. Furthermore, assume that f V + V is surjective intertwiner. Then V also irreducible and f an equivalence.
Assume by contradiction that V is reducible. Thus, there is nontrivial closed invariant subspace & & V'. Now the following can easily be checked:
Proof.
0 0. 5 (W) V..
2. 1 (W) is an invariant subspace of V.
3. (W) is closed subset of V.
Once we have this, we have contradiction to the fact that V is irreducible.
and can checked by the reader, and follows since V is, as an irreducible representation, finteedimennionnll by Proposition B.3T and thus every subspace is closed by Proposition F.47
Therefore, we know that V" irreducible. Now use Schur's Lemma B.29 conclude that f. being nonzero, necessarily is an equivalence.
Proposition B.35. There is equivalence of representations f/ Vi + E given on the orthonor- mal basis by (( Pi Consequently, there an isomorphism Vi = El of unitary representa- tions.
Proof. We need show that ff is equivariant. Using the result of the derivation of Lemma B.33 we compute
(((o))()) = fl (E, (0" (1(0))))))
- (CT
Ep (6-))
= =x()) (r)())).
so = for for g G. which is what wanted to show. That is intertwiner also requires it to be continuous: this follows since Vi is minessiimensional, and so all linear functions on it are continuous.
34

---

Published as a conference paper at ICLR 2021
Now, that fi is even equivalence follows from Lemma B.34 by noting that &/ 0. Indeed, if it was zero then we would have p (g) for all i, and thus p(g) would be invertible, in contrast that it is unitary automorphism.
Thus, there is even an isomorphism Vi = Ef by Schur's Lemma B.29
Lemma B.36. Let p: G + U(V) be a unitary representation. Let Vi S V be a subrepresentation. Then the orthogonal complement V12 a subrepresentation as well.
Proof. We have (wlu1) 0 for all Vi+ and all 0n Vi. Now, let G be arbitrary. From the unitarity we obtain
(p(o)())lvn) - (vlp(s ")()) 0. The last step follows from (v1)) E V1. which holds since V11 is subrepresentation. Overall, this shows (v)) Vi+ well, and this subrepresentation.
Lemma B.37. Let p G + U(V) be finite-dimensional unitary representation. Furthermore, assume that W1, W2 are irreducible subrepresentations. If they are not isomorphic, then they are perpendicular, i.e., ( 0 for all w1 W1 and W2 E W2.
Proof. Let P V + W1 be the orthogonal projection from V to W1, defined as the adjoint of the canonical inclusion i: W1 - V, i.e., defined by the property
(w) ((() (i(w1)lu) (wile)
for all e E V and w1 E W1, see also Proposition F.46 We now show that P is equivariant. For all
E G. v E V and wh E W1 we have:
(mn ((((()())) = (w)l)())(v))
= (olo ((o))))
((al))((a)))(())))
((wi))())(P()))
where we used in the third step that W, is subrepresentation. Since this holds for all W1, we obtain (v)(g)(v) (P(() by Proposition F.45 and overall that equivariant.
In particular, also the restriction P Pww, W2 + W1 is equivariant. Since W1 and W2 are not isomorphic, obtain by Schur's Lemma B. that lw: 0, i.e. for all un E W, and w2 E W2 we have (w1/w2) (wi P| (wz)) = (wi (0) = 0. Thus, W1 and W2 are perpendicular as claimed.
Proposition B.38. Let p : G + U(V be any finite- dimensional unitary representation. Then V decomposes into an orthogonal direct sum
72
V
Vi
i=1
such that V; C V are irreducible subrepresentations of
Proof. Let V1 be any irreducible subrepresentation of V: This can be obtained by noting that if V is not already irreducible (in which case V1 = =V,, then we find nontrivial subrepresentation 0 ç W Ç V. By iteratively proceeding with W, we eventually need reach irreducible representation since V is finitediieensiona.
Now, let V/- be the orthogonal complement of Vi. From Lemma B.36 we know that this is subrep- resentation of V. By induction on the dimension of V, and since V has strictly smaller dimension, we can assume that V already splits into an orthogonal direct sum of irreducible subrepresentations V
+ π i=2 V, and overall, V θ == V; is the decomposition we were looking for.
The following proposition will not be used now, but we make use of it later when showing that there only finitely many basis kernels in steerable CNN for compact group:
35

---

Published as a conference paper at ICLR 2021
Proposition B.39 Remak- Theorem). In the situation of Proposition B.38 the thogonal direct sum decomposition is essentially unique. That is, the type and multiplicities of the irreducible direct summands is always the same.
Proof. If one has decomposition of V in which irreducible representation U does appear, then it cannot appear in any decomposition since U would be perpendicular to all the irreps in the decomposition of V by Lemma B.37 and thus zero. Therefore, the types of irreducible representa- tions is always the same. That the multiplicities are always the same follows by the same argument and for dimension-reasons.
We can now finally prove The Peter- Peter-Weyl Theorem B.22 for the case that X = G:
Proof. By Proposition B.38 and Lemma B.33 there is some orthogonal decomposition & + Vi, into irreducible invariant subspaces. Now assume that there is such that V ~ Vi- By Proposition B.35 this means that Vú; ~ &/ for all j = I,..., dinn()). By Lemma B.37 we
obtain Vv2 + E/ for all j and thus, since Σ, El = Eg, we obtain Va + & and overall Vi2 = 0, contradiction.
Thus, the assumption was wrong and all Vn in the orthogonal direct sum are isomorphic to Vi- Now let * 1' and i,j be arbitrary. We have & 1. Ey. by Proposition B.30 and thus in partieu- Vil lar Va L Vrr-- Furthermore, we have S dim((V) since & -) Ef
tJ i=1 Vn. and
dim (V)) co by Proposition B.31
Moreover, we have Dtea =1 V& = DieG & = E, which is topologically dense Theorem B.27
L (G) by
Finally, that ny = dim Vi) Kf = C follows by invoking a stronger version of Schur's orthogonality
than have developed, and which works only over the complex numbers (Knapp] 2002
n
B.2.4 A PROOF OF THE PETER-WEYL THEOREM FOR GENERAL (X)
Now let X be a homogeneous space of G. Then, as mentioned in Section B.1.3, there a measure on X which is left-G-invarian (Nachbin & Bechtolsheim 1965 in the sense that we have for all g E G and all square-integrable functions f E (X):
f flg x)dr = S f (z))dr.
the projection given by g +> gx. for a fixed element x² E X. One important result is that there is Fubini-like theorem for evaluation of integrals on G using the invariant measure on X. Namely, for arbitrary x X, let g(x) E be any lift, i.e., any element in G with T(g(z)) stabilizer subgroup. For a square-integrable function f : + K, we can then construct the average av(f) X + K by
Furthermore, let G ->
x. This exists since the action is transitive. Let H == GN, C G be the
av av()(z) = -S f(g(=)))dh,
where integrate using the Haar-measure on H IT If it is hard to understand why this is called an average, note that X = G/H. ie., points in X can be interpreted as cosets of G, and then the average just averages over cosets
This construction is well-defined, i.e., does not depend on the specific choice of the lift g(x). Indeed, let g be another lift of æ. Then g(z) 9(x)h' for some h' E H, since H is the stabilizer
Such a Haar measure exists since H C G' topologically closed subgroup of a compact group by Propo- sition B.2I and thus compact itself by standard topological results (Conway) 2014). Note that this measure and thus the Here, G/H is the set of equivalence classes in G with respect to the equivalence relation g' if
fulfills ((H)
the restriction of the measure Go H.
g'gg H, which has quotient topology explained in Definition F.1 The equivalence classes are given by the cosets H for g G.
36

---

Published as a conference paper at ICLR 2021
subgroup. Consequently, using the invariance of the Haar measure,
J flo(x)h)dh - f f(o(z)h'))dt H
f(g(z)h)dh,
and thus the well well-definedness of the average av(f) X + K. Integration of f on the whole of G a "complete" average, and thus we can hope that averaging av(f) leads this complete integral. This is indeed the case, i.e., af) is gquare-integrabee on X and one has (Nachbin & Bechtolsheim, 1965)
Je f(addg - -/x av (x))
We will use this important result later in order to see that (X) embeds with good properties into [
We now want to prove the Peter- weyel theorem for [ We first present a general argument showing orthogonal decomposition of (X) into irreducible subspaces, and then use a specific argument to deduce that the multiplicities of irreducible subrepresentations are necessarily bounded by the multiplicities in T.g (G).
Proposition B.40. Let p - U(V) be any unitary representation. Then there is a dense subrep- resentation which splits as an orthogonal direct sum of irreducible subrepresentations.
Proof. We sketch the proof in Kowalski (2044). Corollary 5.4.2. In this book, the proof is done only for the complex numbers C, but it is obvious that each step carries over without any changes to arbitrary K E [R, CJ. The rough steps are as follows:
1. From one builds function p (G) Homg (V, V). given by (() = Ic ((g))(()(()))) This is analogous to our construction of kernel operators (special rep- resentation operators) from kernels, which we will handle in the next chapter, See Theorem
2. Given E V fixed, one obtains the function p" GL) + V. + (v)) One can check easily that this is an intertwiner.
3. For each Unite-dimensional subrepresentation E S LG) the image 7*(E) S V is umite-dimensione subrepresentation of V.
4. For o O, using analytical arguments and the Peter-Wey, Weyl theorem for L (G), one can prove that there is an E such that (E) C V not zero.
Having that, one can use Proposition B.38 in order to deduce that (E) contains an irreducible subrepresentation, and so does V.
With this at hand, one can proceed inductively as follows: Given an irreducible subrepresentation V S V. one can consider the orthogonal complement Vi', which is by Lemma B.36 again subrepresentation of V. Thus, this also has, by the same argument above, irreducible subrep- resentation V22 and so on. By induction (or better: using Zorn's Lemma), one can then "fill up" V with orthogonal irreducible subrepresentations, deducing the result.
Consequently, since (X) carries a unitary representation of G by [x(g(())) = - we deduce that it contains dense subrepresentation which splits orthogonal direct sum of irreducible subrepresentations. But we would like to know more details about this, in particular the multiplicities of the irreps. For this to work, we want to embed [ into (G) and thus deduce a more specific result from the decomposition of (G).
Let as before x X be an arbitrary point and let म : G + X be the projection given by ((g) gz.. Consider the function πr (X) + (G) given by T (1p) = æ. It is unclear a priori whether this is well-defined: For example, might be that an f X + IK which is zero outside a measure 0 set gets lifted to π (f) G + K which does not have this property. and thus x² would not be an actual function Thus, we need some lemmas:
Remember that functions in (X) for any measurable space X are identified if they agree outside set
of measure O.
37

---

Published as a conference paper at ICLR 2021
Lemma B.41. Let f: X IK be square-integrable. Then have av ( (f)) f.
Proof. Using Eq. 9 and that H is the stabilizer subgroup we compute:
av(x = J x (z))
(((((())))dd
[(r(s())))dh
H
J f(z)dh
[ 1dh
f(x)
{(z))(H)
[(x).
Lemma B.42. Let A C X be any measurable set. Let lA X 4 {0,1) S K be its indicator function. Then x. (1 = (A))
Proof. This can easily be checked.
Lemma B.43. Le X + K be zero outside measure zero set A. Then T (5p is zero outside (A) which is also a measure zero
Proof. If & π (A) then (g) & A and thus:
0 = p(T()) x (p)
which proves the first statement. The second is shown follows using both Lemmas B.41 and B.42 and Eq. 9
(1 ((A)
1 (n)(()
(g)g
av(T"(1A)))
M(A)
0,
thus showing what was claimed.
Thus, concern about well-definedness function is invalid and now prove embed- ding result:
Proposition B.44, x² (2)( + (G) a well-defined intertwiner and a unitary transforma- tion, ie., for all s,, 1b E (X) have (x" (p))==()))) Lg(G) ((pW)) L3(())
Proof. For well-definedness, we still need to show that π (T) ()) is again cquare-integrable for square- integrable yp X K. This is indeed the case due to Eq. Namely, let [1=) G K and
(()
38

---

Published as a conference paper at ICLR 2021
consider its average (1))3)) Clearly, have [T)) x =²(1p]?) and thus, using Lemma B.41 av([="(p)) (x)]²) = lppp. We obtain:
/ n (g))
/
= / [=(x))|d
< 00.
Thus, x² is not only well-defined but even fulfills || ( ||lll which also shows the continuity of n²" With similar arguments, show that x²" respects the whole scalar product, ie., is a uniform transformation:
( (p)]= (1))
6 x² (1) (w)) (g)dg
/
av( (s)dr
p(z)v(x)dz
The step from the second the third line follows as before by noting that n (1)) (1b) xx (p) and invoking Lemma B.41 again.
The linearity of n² is obvious, and the equivariance is done as follows: note that for arbitrary g.g' E G we have Tgg -'g- = (g- g')x=" g (g'x") = g T(g') and therefore:
(X()))) (g') (x()))
x(g"
<(<((g ''))
= = (g) =
Thus, shown everything which was to show.
Thus, x² (x)(X) - (G) is an embedding which even preserves the scalar product. We can therefore view 1 as subspace: (X) C (G) 14
We can finally complete the proof of the Peter-Wey! Theorem B.22
Proof of Theorem B.22
Assume that
mi
Vn S (X) S (G)
le
dense subspace such that the direct sum is orthogonal, where V4, = Vi for all i, This exists by Proposition B.40
Remember that denotes the multiplicity of Vi subrepresentation in (G). We now want to show that my < пy. Since Vi is perpendicular to all E, with * / by Lemma B.37 Vis must be contained in the orthogonal complement of @r Ep. This is exactly &, which show in a final lemma after this proof. So Vi, C & for all Thus, obtain the result my S m by dimension reasons. This was all there was left to show.
Lemma B.45. We have & ( Evy)
"In this notation, we suppress that this embedding depends on the specific base point x which was chosen. For another base point, the embedding differs by unitary automorphism on T.g (G) as the reader may want to check.
39

---

Published as a conference paper at ICLR 2021
Proof. We already know & S ((Fve Er) from Proposition B.30 Now, assume this inclusion not an equality. Then there is & such that v E does contain an orthonormal basis by Proposition F.4T] where the procedure of Gram-Schmidt or- thonormalization allows starting with an orthonormal basis of & and to fill it up to one of the whole space spang Ei) Thus,
I & The space spank (v, &I)
E well. Overall, v E Dr Ey) Er)+ and by taking topological closure and using that the scalar product is continuous by Proposition F.38 ob- tain e E (v ) = (T& (G))+ + by the Weter-Weyy theorem for the regular representation. This means 0 E ε a contradiction
assume
& &
Thus, our assumption wrong and such vector e cannot exist. We obtain the equality as desired.
C THE CORRESPONDENCE BETWEEN STEERABLE KERNELS AND REPRESENTATION OPERATORS
In this chapter, we formulate and prove Theorem which gives a precise one-to-one correspon- dence between steerable kernels on the one hand, and certain representation operators which we call kernel operators on the other hand. Representation operators are representation-tteoreee ab- straction of the scalar, vector and tensor operators from physics, that were explained in Section 2 The correspondence will allow us to prove Wigner-Fckart theorem for steerable kernels in Chap- ter [ and, ultimately, to obtain a complete description of stecrable kernel bases. We formulate the correspondence in Section C.T while Section C.2 gives detailed and rigorous proof of it.
As in Chapter KK is either of the two fields or
C.I FUNDAMENTALS OF THE CORRESPONDENCE
In Section C.I we formulate the correspondence between steerable kernels and special represen- tation operators that we name kernel operators. We do this by first studying steerable CNNs and the kernel constraint in Section C.I.I which progressively leads to consider steerable kernels on homogeneous spaces of general compact groups in Section C.1.2 This abstract formulation of steerable kernels will show apparent similarities the concept of representation operators in Sec-
tion C.I.3 We study them in purely representation- theoretihc terms in Section I.I. However, they importantly differ in the fact that steerable kernels are not linear, whereas representation operators are - this is difference that we need to bridge. Finally, after defining kernel operators as special representation operators, we give the formulation of the correspondence in Theorem C.7 Section
C.1.3 and shortly give some intuitions about why it is true.
C.1.1 STEERABLE KERNELS AND THE RESTRICTION TO HOMOGENEOUS SPACES
The concept of stecrable CNNs outlined here follows (Weiler et al. 2018a Weiler & Cesa 2019 In nutshell, they work as follows:
The network is supposed to process feature fields f Rd + KK with E N. i the dimension of the features themselves, i.e., the number of channels. For example, planar RGB (GB-images correspond to the case d = 2 and =
Furthermore, compact group G (Definition B.4 is considered that acts Rd. for example, the special orthogonal group (O()), the orthogonal group ( or the finite groups Cy or Dy if = 2 15 Then for each layer, the input and output features have certain type, i.e., representation, which may differ from layer to layer. That is, the input (and output as well) consists of a function f Rd - K and G acts on K with linear representation p, see Definition 1.
This action induces an action
We will study some of these groups in the Examples in Chapter
P
40

---

Published as a conference paper at ICLR 2021
of the semi-direct product (R², +) x G the space of all signals 16 where & E (R", +) and g E G:
([ [Tndl w p)((g) ()) = p(g) f(g - 1)).
Let the kernel that "maps" between the layers by convolution That is, for an imput fiu : Rd Ke,,, the output fout RR Krout is given by
be given by function
K RR + Cons Xin
fo) = [K fin = / K(y)fi(( y)dy,
where (y) E Kout X acts for any y E Rd as a linear transformation from k to Sou.
The goal now to find kernels such that convolution with these kernels commutes with the induced actions on the input and output fields. That is, for all input fields fin and for all E R and g E we want the following property:
[[nnd
[Indk*xG R
Pin (tg) fin) = [Indd Bi XG Pous] (tg) (K * fia).
It was shown in Weiler et al. (2018a) that kernel K has this equivariance property if and only if the kernel satisfies a certain constraint. We are rederiving it here for convenience.
Writing out both sides obtain the following equality that needs to hold for all fin and all x, Rd:
[x- ((w)pm(o)ffm(0
-))dtt)d = Pout (g) / Kly)fu(g - v)
Rd
Substituting y و y the left side and using dlet gf due the compactness of G, and putting Pout (g) inside the integral on the right side, which is possible due to linearity, we obtain:
J/c (1) Jin (g -xx x ------- 1 + v)dy = -fr
[ ((mue())K(g (o)) finlg - xx-xx-- - -g -g- L + y)dy-
Since this needs to hold for all fields fin. we necessarily have K K(gz)pmll = Pout (out(g) ((x)) for all Rd and all g E and obtain the kernel constraint
K(gx)
Poue (g) K(x) o Pin(g)=1
(10)
This work will create general theory for how to solve this kernel constraint, which means to find parameterization for the space of all kernels that fulfill this constraint. We now explain how to make this problem more tractable: formally, the action G on Rd is group action as in Definition B.5 However, it cannot be transitive in Definition B.7 since G' is compact and Id is not. Thus IR splits into a disjoint union of orbits (Definition B.6), of the action:
R
L X AEK
That this is a disjoint union can be explained as follows: define the relation on Rd by x' if gø x' for some g E G. This is then an equivalence relation, and so Rd splits into a disjoint union of equivalence classes. One then can show that these equivalence classes are precisely the orbits of the group action. For example, such orbits take the form of spheres 5dd if SO(d) or G = (dd and the form finite set of points if G = CN or G DN-
The idea is now that the kernel constraint 10 only constrains the behavior of the kernel at each orbit individually, and thus a solution on each orbit can be "patched together" to a solution on the whole
The semidirect product R G can be imagined as the smallest subgroup of the group of all isometries of Rd that contains both the translations IR and the transformation G. It is not important know the abstract definition a semidirect product in our context.
1hehe operation is actually so-called "correlation" but the term "convolution" is more widespread in the deep learning context and follow this convention.
41

---

Published as a conference paper at ICLR 2021
Xomn of R Indeed, assume that Kg Xg Krow Xom individually fulfill the kernel constraint, which means that for all ay E Xg and E G we have
(g(grs) - Pout (g) (x(x)) o pin (in
Then, define the patch of these orbit-kernels by K Rd Sou XCnn K KA if X This well-defined since each z is in precisely one orbit. Then clearly. K satisfies the kernel constraint TO Moreover, each kernel K which fulfills the kernel constraint emerges from such construction, since we can simply set K3 == K x. Overall, we see that we can restrict our attention to orbits. In Weiler et al. (2018)) and later Weiler et al. (2018a), discretized implementation is done where the kernel is discretized into finitely many orbits with smooth Gaussian radial profile. We will come back to these practical questions of parameterization in Remark D.19 once we have fully developed the theory of steerable CNNs.
C.1.2 AN ABSTRACT DEFINITION OF STEERABLE KERNELS
Motivated by the discussion in the last section, we now define steerable kernels in precise terms and will stick to that definition throughout this work. The definition will be more abstract than usual in the deep learning community, but we are rewarded since such an abstract definition makes it easier to apply representation-theee thhoreetc results.
Without loss of generality, we will in the rest of this work only consider kernels on orbits. Thus, let X G= be arbitrary orbit. We consider steerable kernels K X Kou X Note that the restriction of the action G x IRd + Rd to X, written G x X - X, makes X to a homogeneous space of G, see Definition B.7 Thus, instead of viewing X as subset of Rd, arbitrary homogeneous space of an arbitrary compact group G. Notably, this framework is more general than usually studied the context of steerable CNNs on Rd, since we allow also groups that are not Lie groups and homogeneous spaces which are not naturally embedded in an Rd. as well as finite homogeneous spaces of finite groups all at the same time.
view X as an
Furthermore, we replace IK and Suu by coordinate- and therefore Keout X by space linear functions from Vin to Vout written Homg Vin Vont). We assume there are linear representations Pin G GL(VV.) and Pout G GL(Vout).
K-vector spaces Vn and Vout,
Overall, this means that steerable kernels are certain maps K X - Homg (Vin, Vout). The only property they need to fulfiil is the kernel constraint K(gz) Pout (g) K (xx (i)(g) for all g E and E X. This can be viewed in representation- -ieeitttattiot---etiiccc terms by defining the Hom- representation:
Definition C.1 (Hom- Room-eepresentttion)). Let Piu G - GL(Vin) and Pout G - GL(Vout) be two finite-dimensional G-representations over the field KK The space Hom (Vin; Vout) of KKllnnear (not necessarily G-equivariant) functions from Vin to Vout also carries an induced G-reppresenaaton, with action
[pHom (g)] (f) = Pout (g) o of (1)
We call this the Homm-rerresentation.
Remark C.2. Of course, one needs check that this indeed a linear representation, Continuity follows from the continuity of Pin and Pout as follows: the topology on Homg ( Vin Vout) is just the Fuclidean topology of So X coming from a basis of Vn, and Vout- In these bases, Pin Pin(g) and Pout (g) are given by matrices. All matrix coefficients are continuous by Remark B B.25) Now, in order to show that PHom continuous, pick fixed element f E Kein Xfout One needs to show that the map
P G
Re XCont + Pout (g) pin(y
continuous. Since all matrix coefficients are continuous and since also the inversion G - G, H g continuous by the definition of a topological group, the map o is basically just a stacked linear combination of continuous functions and thus continuous itself.
42

---

Published as a conference paper at ICLR 2021
The linearity of each PHom (g) is also clear. So what needs to be checked is that PHom is group homomorphism. And indeed, it is, exploiting the corresponding property of Pmn and Pout:
[oom (gg') (s) = Pout o f Piu (aggg)))
Pout(g) (g) (Pout (g') ) Pen ( pin(g)
[oom [g) [Hom (g')] (f))
[PHom (g) o PHom (g (f),
and so the claim follows.
With this definition in mind, stecrable kernels K X Homn (Vin Vout) are just functions with the property (grr = [Pttmm (9) (K (x)). Summarizing, we have the following abstract definition of steerable kernels (different from Definition 3.2 we here allow also input- and output representations that are not irreducible and make explicit reference to the Hom-representation):
Definition C.3 (Steerable Kernel.. Let G' be any compact group and X be any homogeneous space of G. Furthermore, let Pin : + GL(Vmn) and Pout G - GL(Vout) be finite-dimensional repre- sentations of G. We assume that HOmE (Vin Vout) is equipped with the Hom-representation PHom- G-steerable kernel is an equivariant function K X - Homg (Vnn Vont), i.e., a function such that
K(gx)= = [pHom (g) (K(x))
(11)
for all g E G and a E X. We denote the vector-space of all these kernels by
Homa(X, Homg (Vm: Vout)) {K X + Homı (Von(Vmn Vout) K is steerable }
Notably, steerable kernels are not linear in a meaningful sense with respect to their input.
That the space of steerable kernels forms a vector space, claimed in this definition, can easily be checked.
1.1.3 MORE DETAILS ON THE COMPARISON OF REPRESENTATION OPERATORS AND STEERABLE KERNELS
Steerable kernels satisfy the constraint
K(gx) = Pour(g) (g) K(x)) o (in(
(12)
whereas, as we saw in Section 2 representation operators are collections (Ai,..., An) of operators A, : H + H that satisfy the constraint
Σ A()) = A,U()) Gg
Hereby, U G + U(H) and n : U(C are unitary representations. Unfortunately, these
equations still look somewhat different from each other. We can make them more similar by invert- ing g and using the unitarity of π (note the swap of j and and the complex conjugation):
Σ N =()HAA (()) Vg
(13)
In order make the analogy to steerable kernels stronger, we would like interpret a representation operator as one object A instead of separate operators A,, in the same way as kernel K is one single object and not just disjoint collection of linear functions in Homg (Vin Vout). For this, we interpret A as function that assigns to arbitrary vectors in C an operator. Namely, let {er be the standard basis of C We then define A as the unique linear map which is given on basis elements as follows:
A A:e0 + A
43

---

Published as a conference paper at ICLR 2021
We can then deduce the following, where use the linearity of A in the second step, the definition of Aj in the third and fifth step, and Eq. (13) in the fourth step:
A A(T(g)(e.) A Σ, [(o)sies)
Σ A())iA
(14)
U(AUU))))
A(e))())))
If now = =; X is an arbitrary vector in C. not necessarily standard basis vector, then from the linearity of A and Eq. (14) we obtain
(()()(()) U (())(())
(15)
This equation is essentially the starting point for the definition of representation operator as it can be found in Jeevanjee [201T].
This, finally, really looks like Eq. (2). this comparison, the action of the group G on R in deep learning is replaced by the action of G' via on the space CN. The main difference is that steerable kernels are not necessarily linear. This difference will be bridged in Theorem C.7
C.1.4 REPRESENTATION OPERATORS AND KERNEL OPERATORS
Now that we have a clear abstract idea of what steerable kernels are and saw strong analogies to representation operators,
begin to formulate precise theoretical connections. In this section, we therefore begin with formulating a purely sppesentation-tthoon and more abstract working definition of representation operators and will then formulate the main theorem of this chapter, Theorem
We come to the main definition, which is directly motivated from Eq. (15). differs from (Jeevanjee) 2011) by allowing the input- and output representations to differ. We furthermore restrict to finite- dimensional input- and output representations due our specific applications. As explained in Section C.I.3 this new definition furthermore somewhat differs from the one given in Section 2 since now we view representation operators as one object instead of viewing as a collection of several linear operators.
Definition C.4 (Representation Operator). Let Pm G GL(V)n) and Pout G + GL
be dinensionn G-representations. Let X G GL(T) be a third G-representation, not necessarily finite-dimensiona. Then represeniation operator is an intertwiner K. T Homg (V,, Vout), where the right space is equipped with the rom-reppreenntation in Definition
C.I
We denote the vector space of all these representation operators by
Homg. Homg (Vin; Vont)) [K T Homg (Vin Vout) K. is an intertwiner).
Note that representation operators are by definition linear, which is requirement that needs to be satisfied for the standard Wigner-Eckart theorem. We clearly see strong similarities between this definition and the formalization of steerable kernels in Definition C The main difference is that we assume representation operators to be linear. This in notation captured by the subscript IK that put in the corresponding Hom-space. One may think that there is another difference, namely coming from the fact that intertwiners are by definition continuous with respect to the topologies involved. Two things need be said about this:
1. First of all, one may wonder what continuity for representation operators actually means. This can be clarified as follows: By assumption, G-representations are always on vector spaces with topologies, and thus T has topology. Furthermore, in Remark C.2 we clarified the topology on Homg (in. V.nt). Then, being continuous just means, as always, to be continuous with respect the topologies of these two spaces.
44

---

Published as a conference paper at ICLR 2021
2. The second remark is that this apparent difference in the requirement of continuity for steerable kernels and representation operators is actually non-existent. This is explained by the following Proposition which says that steerable kernels are automatically continuous. Note that this is not true for steerable kernels that are defined on the domain R in that case, continuity is only guaranteed when restricting orbits.
Proposition C.5. Let K X - Homk (Vi, Vowt) be a steerable kernel. Then K is continuous.
Proof. For brevity, denote V = Homk (Vin Vout) and p == PHom- Let x E X be any point and Gz. the stabilizer corresponding to the action of G on X. Remember the homeomorphism P :/G + X, [g] + ga" from Lemma B.21 Since this is a homeomorphism, the kernel K is continuous if and only if the composition K yp is continuous, since then K (K 4) o 4 is composition of continuous functions. Thus, we evaluate K o yo:
(K o ((g)) K (+((1))) = (ggx") = (())(K(x))),
where in the last step have used the equivariance of K. Thus, if we set v² = K(*) V, then we obtain the simple relation (K' o (p)) = p(g)(+*). This is by definition just the unique map on the quotient, G/ H + V. coming from 0² G - V. g + p(g)(e)). This last map is continuous by definition of linear representation. The universal property of quotients Proposition [.12 then shows that K o is continuous as well, and so we are done. All of this is visualized in the following commutative diagram, where q: G G/H, 9 is the canonical projection:
(1)c
G/H
X
V
4
K
Koip
Thus, the only difference between steerable kernels and representation operators is indeed the lin- earity. We now look special representation operators that play the main role in this work:
Definition C.6 (Kernel Operator). Let Pin G - GL(Vin) and Pout G + GL(Vout be finite- dimensional G-repressntations. Let À + UXL)(()) be the standard unitary representation on the space of square-integrable functions of homogeneous space X, given, in Section B.1.3 by
[A(g)(}) (g') "'g"'g"); "g')-
kernel operator a representation operator I (X) + Homæ (Vi, Vont). We denote the space of these by
Homg.K (X), Homg (Vins Vout))
= ((K: (X) Hom (Vi, Vout) )
IC is an intertwiner }-
Notably, kernel operators are IK-linear in their input.
C.1.5 FORMULATION OF THE CORRESPONDENCE BETWEEN STEERABLE KERNELS AND KERNEL OPERATORS
The following Theorem lies at the heart of our investigations and establishes that steerable kernels can be considered as kernel operators, which we defined as special representation operators. More precisely, we will give an explicit isomorphism between the space of steerable kernels and the space of kernel operators.
We shortly explain why the theorem is useful. First of all, using Wigner-Eckart theorem for kernel operators that we prove in Theorem D.13] one can explicitly describe a basis B of the space of kernel operators Homg, K (X)()) Hommg Vour). Then, since we have isomorphism of vector spaces to the space of steerable kernels, one can "carry over" this basis to basis for the space of steerable kernels, namely Homg (X, Homg (Vin, Vout)). This basis will then have convenient explicit form that we establish in Theorem (neorem/D.16jamm is exactly what we need in order to parameterize an equivariant neural network layer. We come to precise formulation of the theorem:
45

---

Published as a conference paper at ICLR 2021
Theorem C.7 (Kemel-Oppertor werrrerrrrttoooorrsooeeeeeeeeee Conrespondence). Let Pm G GV and Pout G (oo)) be diiee-dimessooall G-representations and X be a homogeneous space of G. Then there is an isomorphism
Homa(X, Homm: Vout))
Homg, R (Lg()) Homg (Vin, Vont))
[-xx
between the space of steerable kernels on the left and the space of kernel operators on the right. The two maps are defined as follows:
1. For a steerable kernel K X - Homgk(Vu. Vout), the extension K. (X) 7
Homg (Vin Vour) given by
R()= / (x))d
2. For kernel operator K. : (X) Homg Vout), the restriction K|x X Homg Vout) is given by
[xx(x)= = lim K(
Ueu,
Hereby, Ug is the directed set of open neighborhoods of x. see Example F:27 óy X - K is the approximated Dirac delta function with (y(y) w(U if E U and du (v) = 0, else.
The limit is a limit of nets as in Definition F:29
This theorem requires some explanation. First of all, K is supposed to be kernel operator, i.e., map Lig (X) - Homg (Vm. Vout). Thus, (f) should be linear function Vn - Vout. The formal expression of can indeed be considered as such:
R())= = 10==[ S (x))((r)dr : Uin
J ((x) [K()) (vm)da Vout-
(16)
Due to the continuity of K proven in Proposition C.5 and the integrability of f, the function
X + Vouts f(x) [x(x]]] (Vin) is also integrable, meaning the expression in Eq. (16) can be
evaluated. This explains the meaning of the map (-) in Theorem C.7
For the map x-)1x in the other direction, want to shortly explain the intuitions in more informal way. For this, we consider Dirac delta functions & for x E X. Such a "function" 5g X - K for point is characterized by the property that fx 8z (x')dx' = f(x) for any function f E L (X). We
X can be imagined
function taking value infinity at and
elsewhere. It
think of 6, as being function in (X), even though technically, it is not in this space. This is since & IK.
Now, informally, we can think of the limit (x)(x) = limu'u, KC(bv) as being given by K((rz), the value that takes the Dirac delta function 8g. This since the limit of nets progressively "shrinks down" the open neighborhood U of x. Of course, K(O,,) is not really well-defined since 5, & (xx but we can pretend that is for gaining intuitions.
Now that we have understood the formulation of the theorem, we might wonder, why should such a theorem be true? A first intuition comes from an analogy with linear algebra: namely, assume B a basis of a IK -evetr space V and W any other vector space. Then linear maps ff V + W in one-to-one correspondence with (not assumed to be linear) functions B - W, and this isomorphism is given by restriction and linear extension:
Hom (B, W)
HomB (V, W).
I
This means that all matrix elements of ((x) for chosen bases of Vin and Vout continuous.
46

---

Published as a conference paper at ICLR 2021
Thus, think of the homogeneous space X as a "continuous basis" of the space of square- integrable functions. Sums are then replaced by integrals, and evaluations at a basis element by evaluations at Dirac delta functions of clements in X.
For the actual proof Theorem of the Dirac delta:
C.7
informally, one direction seems pretty clear from the properties
/ 6. 6, K()).
K (x) R(5x)=
But the other direction is less obvious: it seems like the space of kernel operators is considerably larger than the space of steerable kernels, since kernel operators are defined on larger space. There- fore it is hard to believe that the construction is also inverse in the other direction. However, it pays off ponder bit more over what the Dirae delta construction does: Basically, we "embed" X 8, and, as such, view X subset
into (X) ) by means of the Dirac delta functions, i.e., of (X) (albeit a subset that only approximation in that space). Steerable kernels are then "partial" kernel operators in the sense that they are only defined this subset X C (X). What then needs to be understood is why there is only one unique extension each steerable kernel to kernel operator K. on the whole of i (X): J: if this is understood, then the space of kernel operators cannot be larger than the space of steerable kernels. And indeed, if there is an extension K to K. on T.x(X) (X), it has to be unique: each f T (X) can be approximated by finite linear combinations of scaled indicator functions. Then by linearity of the kernel operator K., we can evaluate KC(f) by knowing KC(&)) for scaled indicator functions dy on small measurable U. And these approxi- mate K (xx) = K((62) for E U arbitrarily well by construction. This determines the behavior of K. The details of all of this can be found the next section.
C.2 A PROOF OF THE CORRESPONDENCE BETWEEN STEERABLE KERNELS AND KERNEL OPERATORS
The details of this investigation will not be needed later, and reader who is mainly interested in the applications to steerable CNNs can safely skip reading this section and go on reading Chapter D
Here, we give a step-by-step proof of Theorem C.7
C.2.1
REDUCTION TO UNITARY IRREDUCIBLE REPRESENTATIONS
In this section, we make the proof more manageable by reducing Homg (Vi, Vout) to an irre- ducible representation. First, remember that Proposition B.20 shows that there scalar product on Homg (Vi., Vous) such that it's Hom-reprrsenttaion becomes unitary. Since all norms on finite- dimensional spaces are equivalent, decompose Homx (Vi, Vout) into an orthogonal direct sum of irreducible unitary representations by (Vi (Vi, Vont) > O "= V. be such a decomposition. We get canonica [9
is well known, this will not change the topology. Then,
Proposition B.38 Let Homg isomorphisms
Homg (X, Homg (Vi,, Vout))
Home (X, Vi)
and
Homg, (X) Homg (Vin, Von.)) Homa, (X), V.).
Thus, we can show Theorem C.7 by showing it for irreducible unitary representations instead of Homg (Vn, Vout). Overall, we have reduced our Theorem to the following, simpler statement:
Theorem C88 Orarrppppprrr--Crrppodeeeeeeee Restated), Let p G - U(V) be irreducible unitary representation and X a homogeneous space of G. Then there an isomorphism
Homg (X, (X
Homg. CamG.R(L2.. (X)
OIx
1P"Canonical" the decompositions into irreducible representations is already chosen.
47

---

Published as a conference paper at ICLR 2021
which is given follows: for K E Homg(X,V) V) we set R(f) fx fz)) and for K. Homg..(Tig (X), V) set (x) limmeu, K(Ōv), with dy being approximated Dirac delta function as before.
From now on, we assume that X and p:G. G + U(V) is fixed as in the formulation of Theorem C.8
C.2.2 WELI-DEFNFEDNESS OF (.)
Lemma C.9. The function (.) foom@(XVV) - Homg. (X), V) is well-defined, ie.: for an equivariant function K- X + V, the function K- (X) + V is linear, equivariant and continuous.
Proof. Linearity of K is clear. Equivariance can be proven using the equivariance of K and the left invariance of the Haar measure on the homogeneous space X:
R((gg)))
(x))
f(g (x)dx
= ((x)K(g x)dr
J f(x) (o(g) (K(x)) dr
=(()) I/ [())
p(g) [R(n]-
The action by p(g) could be put out of the integral since p(g) it is linear and continuous, and since integrals can be approximated by finite sums.
Now about continuity: By Proposition F.18 we only need to show continuity in 0. Thus, let (fa)a be a sequence of functions f& & (X) with limg fl Then we obtain
[[()) / [a())(())dd)
< / [fu(z)) || (x)llvd
S max [((a')llv / (z)(
x²'
where the continuity of K proven in Proposition C.5 was used For the right expression, using the Cauchy-Schwarz inequality Proposition F.34 we obtain
[x(())dd== / [ff(())) 1dr
Klf& |1)
So, overall, if limg-soo [| 1= = 0. then limg-roc [[x]]l = 0 as well, which proves continuity.
since ||| is continuous on X, which is compact by Proposition F88 an image of the compact group G, it has maximum by Corollary F25
48

---

Published as a conference paper at ICLR 2021
C.2.3 BELL-DEFINEDNESS OF (-)x
unique if exists (Conway While it is clear that the limit limgee,, K(Ōy) from Theorem C.8
2014). it is somewhat unclear why it exists in the first place. For this, we need to better understand the properties of the (approximated) Dirac delta. The most important one is the following, which we hinted at already in the intuitions we gave before this section: basically, Dirac deltas help for evaluating continuous functions at specific points:
Lemma C.10. For each
X and Y X - KK continuous we have limu (ool (x)).
Proof. We have
-Gol))-Y(()) / 60(2')((2')d' u(U). m(U) (()
1
u(U)
Le p(U) v(d')da' 1o u(U) (())dr'
So p(U) (X(x)) -x()))dr
J. Ju u(U) (x (())-Y(xx))) dz'.
Let > O. Since Y is continuous in z, there is U, E Un such that ((²) E (Y for all x' e U or, equivalently, (x') Y()) e. Thus, for all U. 2 i.e. all U. S in Ul we obtain
(ōu 8) (x)) Lc 1 (x() (((d))Y(x)(dd) (x))de M(U)
< [ M(T) cdd'
m(U) M(U)
and consequently limueu, (BulY) Yxx).
Before we can show the well-definedness of K.|xx, x we first want to get better description of K. For this, recall from the Peter-Weyl theorem that 1 (X) we can formulate the following Lemma on the form intertwiners on (X)::
θ Died Th Vi:- With this at our disposal,
Lemma C.11. Let K. (X) - V be an intertwiner. Let E G be the unique index such that
V = Vi for all i 1,..., my Let Yi I...., di be an orthonormal basis of Vis where di = dV). Then
K()
for all f (X).
Proof. We can write f E 1 (X) according to the discussion after Definition F.40
f= -=Eve E [1] (YPII)) Y?.
Note that Kly,, : Vir V is intertwiner as well, and so by Schur's Lemma B.29 it is necessarily zero unless 1' = 1 is the unique index such that Vis V. Due to its continuity and linearity, IC commutes with infinite sums and obtain
K(f) Σrse
Dm
(Y7A)) K. (Y7)
Σ
E
[0]]
(YPl.) K V.y (Y7)
1==
Σ
di
K(Y))) ((Y?).
n=1
49

---

Published as a conference paper at ICLR 2021
Corollary C.12. We have [xx(x))= E Ex= (T))) In particular, the defining limit exists.
Proof. Since the Y are by the proof of the Peter- Weell theorem in the fiite-dimensional space & spanned by matrix coefficients of the irreducible representation Pt G - U(V) and since these matrix coefficients are continuous by Remark 6.25 the Y are as finite linear combinations of them also continuous functions. Thus, from Lemma C.10 and C.1I together we obtain:
(x)
lim K. (ōv) Vells
lim VElls
Σ
Σ (V K((())
dt
lim
(V116e)
KC(Y))
Delle
()))))))))
The complex conjugation came into play since the order in the scalar product is swapped compared to Lemma C.10
Thus, since we now know that X|xx as a function makes sense, we can finally prove the well- definedness of K + K[xx
Lemma C.13. The function (-)lx: Homg, (X), V) + Homa(X, V) is well-defined, that is: for a linear, equivariant and continuous function K. [ V, the restriction K|xx : X + V is equivariant.
Proof. We have
Xlx(gg10) (g.x)
lim K. (ōv) VEllgz
lim K. (6gu)
Uetlz
lim K. (A(g)ōv)
VEul,
lim p(g) [K (ōvr)
Ueu,
-P())
lim K. (5v)
Luel,
p(g) [x]x(=)],
where the steps are justified as follows: The first step is just the definition of K|xx The second step uses that the open neighborhood of gx are precisely the g-translated open neighborhoods of since g X + X is a homeomorphism. The third step is easy to check. The fourth step uses the cquivariance of K. The fifth step uses the continuity of p(g), which follows since p(g) is unitary transformation. The last step is again the definition of KC|x.
C.2.4 (-) AND (+)Ix ARE INVERSE TO EACH OTHER
We can now finish the proof of Theorem C.8 and consequently of Theorem C.7
Proof of Theorem C.8
After all the preparation, we only need to still show that the maps (-) and
(+)xx are inverse to each other. For K | = K, i.c., the injectivity of the function K + R and surjectivity of the function AC + Klx, we compute:
K xx)
lim K (ōv)
Ueu,,
/ 80 (o')
lim Uell,
= =K(x).
50

---

Published as a conference paper at ICLR 2021
The last step follows from Lemma C.10 by identifying V = V with K and viewing K consisting of continuous component functions K X - K, n E {1...., di}. The continuity of K was shown in Proposition C.5
For showing K|x KC we a computation using the description I. from Lemma C.11 and description of X|xx from Corollary C.12
xxx()= = / ((x)((x(x)ddz
f f(0) E de
n=1 (+))(()))) do n=1 ( [
Σn En1
E E n=1 (ill))(())
K((f).
This finally finishes the proof.
D A WIGNER-ECKART THEOREM FOR STEERABLE KERNELS OF GENERAL COMPACT GROUPS
In Chapter C we have seen the most important theoretical insight of this work: stecrable kernels on a homogencous space X correspond one-to-one to kernel operators (certain representation opera- tors) on the space of gquare-integrable functions L (X). J. In this chapter, we will develop the most important consequence of this correspondence: consequently a description of a basis for steerable kernels. This works for both fields R and C, for an arbitrary compact group G, an arbitrary homogeneous space X and arbitrary finiteddimensional input- and output fields. Additionally, covers the general theory of equivariant CNNs on homoge- neous spaces developed in (Cohen et al. 2019b).
Wigner-Eckar theorem for steerable kernels and
In Section D.I we will work towards formulating the most important thcorems. Since these will involve tensor products, will start with defining and studying tensor products of pre-Hilbert spaces and (unitary) representations. Afterward, we will define the Clebsch-Gordat coefficients, which relate a tensor product of irreducible representations to the irreducible subrepresentations of this tensor product. This will lead to formulation of the original Wigner-Eckart theorem similar it appears in quantum mechanics, including proof. The original Wigner-Fckart theorem is a statement about representation operators on irreducible representations. However, we consider irreducible. Also, different from the original Theorem, we
kernel operators on L (X) which also consider representations over the real numbers, which leads to replacement of reduced matrix elements by endomorphisms. Therefore we then formulate generalization of the original theorem. Then, using the correspondence between kernel operators and steerable kernels from Theorem C.7 we can transform this into a Wigner-Eckart theorem for steerable kernels and ultimately a statement about basis of the space of steerable kernels. We conclude with some remarks about how to use the basis kernels in practice.
Afterward, in Section D.2] give the remaining proof of the Wigner-Eckart theorem for kernel operators, which we omit in the section before. First, we reduce the statement to the dense subspace of (X) which is direct sum of all irreducible subrepresentations. We then describe correspon- dence between representation operators and intertwiners on a certain tensor product, the so-called hom-tenson adjunction. Finally, finish with the full proof of the Wigner-Eckar Eckart theorem.
As always, let K be either of the two fields R and C and G be a compact topological group. X is any homogeneous space of G.
51

---

Published as a conference paper at ICLR 2021
D.1 A WIGNER-ECKART THEOREM FOR STEERABLE KERNELS AND THEIR KERNEL BASES
D.1.1 TENSOR PRODUCTS OF PRE-HILBERT SPACES AND UNITARY REPRESENTATIONS
In order to state the Wigner-Eckart theorem, we need the notion of representations on tensor prod- ucts. This is defined similarly to Hom-representations, see Definition C.1 For this, we first need to discuss the notion of tensor product of vector spaces:
Definition D.1 (Tensor Product). Let V and V be two vector spaces over K. Then V & V', the tensor product of V and V", is a vector space over K with the following properties:
1. There is a bilinear function @ : V x +VV.. (v.)) (0.))) V V is generated by elements of the form @ v..
2. It has the following universal property: for any bilinear function 3 VXV' + P into vector space P. there is unique linear function 3 V & V - P given elements of the form 8 & of by B( @ v') B(v, v'). In other words, the following diagram commutes:
V x V 8
P
V V'
3. If V and V are finite-dimensional with bases (uee.. S V and (uf....,,mm) C V then {oe uf C V V is basis of V do V". In particular, the dimension Vf s V is n m.
Property 3 follows from and 2 and would therefore not necessarily be needed in the definition. The explicit construction of tensor products shall not matter for our purposes since the properties above characterize it up to isomorphism. The second property stated in the definition is of large importance since it tells us how we can define linear funetions on V & V if we have guess for such function : V @ V + P (of which we don't t yet know whether its "assignment rule" is well-defined), then we just need test whether the function & V x V P given by G(v, v') p(v@r' is bilinear. If it is, then is a well-defined linear function. We will use this soon in the following context: Assume f :
V are linear functions. Then we would like define a function @ g: V & V - V @ V by (f @ (v @ r') = f(v) @ g(v"). For this to work, we need to test whether the assignment (v, e') + f(v) & g(v") is a bilinear function V V - V & V". Clearly, it is, and so f & g is well-defined linear function! We use this in Definition D.3 in order to define the tensor product of representations.
V and g : V
Since we actually deal with Hilbert spaces most of the time, we would like to build tensor products of Hilbert spaces. However, their definition not completely straightforward since one cannot just take the tensor product of the underlying vector spaces but needs to additionally build the completion of the resulting space (Kadison & Ringrose 1997). Since this complicates the considcrations related to correspondence we later formulate in Proposition D.233] we go a slightly different route. Instead of describing the tensor product of Hilbert spaces, we describe the tensor product of pre-Hilbert spaces, which does not require completion step. Recall from Definition F3 that a pre-Hilbert space is basically a Hilbert space that is not necessarily complete.
Definition D.2 (Tensor Product of pre-Hilbert spaces). Let V, V be two pre-Hilbert spaces with scalar products (-l-) and (-1-Y.. Then the tensor product of vector spaces V @ V can be made into a pre-Hilbert space using the scalar product which is given on generators by
(u @ v0110 @ w²)g = (p/w)) (o'Iwe')
This is then anti-linearly extended in the first (i.e., "Bra"), ) and linearly extended in the second (i.e., "Ket") component.
One can show that this makes V @ V Hilbert space. For simplicity, will from now not notationally distinguish the different scalar products involved. With this preparation, we can come to the notion of tensor product representations:
52

---

Published as a conference paper at ICLR 2021
Definition D.3 (Tensor Product Representation). Let G - GL(V) and p' G - GL(V") be two linear representations, where V and V are pre-Hilbert spaces. Then on the tensor product V & V of piibert spaces, can define the tensor product representation p @ pf by
p p G GL.(V @ V"), 9 p(g) @ pppp p'(g);
where p(g) & (g) V X V is given on generators by
(gg) 2 p (g)) ) = ,() & ('())
Lemma D.4. The map p p G GL( V') defined above linear representation.
Proof. Clearly, each (p @ pg) linear and we have (p X ('g'))) = (p & (g) - (p & ('g)- Thus, for showing that it is linear representation, already knew continuity of all maps (p @ p')ust'' G + V & V', for linear combinations ę = Σ do (vx v/) obtain using the linearity of (p 2 pg):
need to show it is continuous. Assume
[[p @ p (g)] @ v'). Then
(o p')(() = [(o @ (G)
[(e p')( (
= Σ ==1 llll & ( (') w
= EE A p/y (g).
Now, since scalar multiplication and addition in topological vector spaces is continuous, and since pre-Hilbert spaces special topological vector spaces, the continuity of (p @ p')s follows from that of all (p p')r@r'.
What's left is proving the continuity of functions of the form (p& p') For notational simplicity, write f = pp G + and I' p'e which are both continuous since p and p' are linear represen- tations. We want show that also f & f' G + V @ V' is continuous. We can test continuity in each point Go E G separately by Definition F66 real part of a complex number:
For each g E G we then obtain, with Re being the
II( & (') ($ s (0)))
[[(c)) @ ('(g) - ((() 8
((o) @ f'(0))
|[(()) s (f'(g) ('())) + ([(g) - f(9))
-II(()) 8 (f'()) [' (()) @ (())
+ 22 (f()) [f'(g) (f'()) [f(g) - f(go)) @ f(()))
[1f())) -1f()))))))))))))))))))))))))))) ((0 (l(g) - f(go))l (
Re (ff()))(()) - ( (f - (f
All in all we see the following: If is sufficiently close to 9o. then due to the continuity of f, f', the scalar product, multiplication in KK and the real part, [l(f & f')(g) (f @ (')(l) gets arbitrarily close to O. This shows the continuity of f @ f and we are done.
Lemma D.S. Let p: G U(V) and p G + U(V') be unitary representations pre-Hilbert spaces. Then also p 0 pf U(V & V') a well-defined unitary representation.
Proof. According to Lemma D.4 we only need to check whether all p(g) & p'(g) are unitary trans- formations. This follows immediately from the unitarity of p(g) and p'(g).
53

---

Published as a conference paper at ICLR 2021
D.1.2 THE CLEBSCH-GORDAR COEFFICIENTS AND THE ORIGINAL THEOREM
In this section, we describe the (lebsch-Gordae coefficients and the original Wigner- Eckert theorem. Except for the proof, we roughly follow Jeevanjee 2011). For the proof, we follow the more general
treatment in Agrawala 1980 21
For our aims, let Pj G -> UV3)) and pr. : -> UVi) be representatives of isomorphism classes of irreducible unitary representations
Then consider their tensor product representation
Pj X pr
UV, @ Vi)
which is again a unitary representation according Lemma D.5 V, and V/ are of dimension d; and di, respectively, then V, @ Vi is of dimension dj dz. Since it finite-dimensional unitary rep- resentation, it is itself an orthogonal direct sum of finitely many irreducible unitary representations by Proposition B.38
[
V Vi
=
Vy.
s=1
Here G is, as before, the set of isomorphism classes of irreducible unitary representations and [J(]]]
the number of times that PJ G - U(V)) appears in the direct sum decomposition of V Vi.
Note that for most J we have [J(jt] O, and for some J we may have [J(jl) >> 1, see Section where it turns out that Po contained twice in pm @ Pm-
F.2
Now, choose once and for all orthonormal bases of all involved irreps, which exists according to Proposition F.4
(Y m l.....dd}
dj} S Vs
{rr n = .......di di C Vi, (YM (V......,dJ) V,
This notation is supposed to remind about spherical harmonics since they form basis for irreducible representations of the group (0)) But as mentioned in the footnote, we do not consider these basis elements to be functions here.
Furthermore, let 4 : VJ + Vj @ Vi be the linear, equivariant and isometric (i.e., sealar product preserving) embeddings that correspond to the direct sum decomposition of V, Vi into irreps, where s ranges in f1...., [JGU]]} With this in mind, we can define the Clebsch- Clebsch-Gordan Gordan coefficients:
Definition D.6 (Clebsch-Gordan Coefficients). The Clebsch-Gordan Coefficients are given by
(Mjmm (1," & Y
Note that in the literature, people usually only consider Clebsch-Gordan coefficients of the specific groups SO(3). SU(,), SU(3) or similar groups appearing physics. Also note that in the physics context, there is only one linear, equivariant, isometric embedding 4, which follows directly from Schur's Lemma D Therefore, it is sensible that the embedding usually not part of the notation of these coefficients. In our case, however, when considering real representations, there can be several such embeddings 4,, This happens if the endomorphism space VJ is nontrivial. An example is given by the two-dimensional irreducible representations of SO(2) over the real numbers which we discuss in Section E.2 Since, however, we do not want to depart too much from the notation usually considered in physics, we also omit the embedding from the notation. The index however needs to be present in order to index the possibly different appearances of VJ in V, & Vi.
With this preparation, we can explain the Wigner- Wigner-Eckart Eckart theorem the way it is usually considered in physics, as a prelude for the generalization that we consider in the next section.
It more general in that it considers arbitrary groups and the situation that the considered irreducible representation appears several times in tensor product representation instead just once. Thoos are priori not assumed to he embedded in space of square-integrable functions. For such embed- ded representations, we write Vy, instead.
54

---

Published as a conference paper at ICLR 2021
In this (and only this!) section, assume that our field is C, since this is the considered in physics. The Wigner-Eckart theorem aims to obtain a description for all possible representa- tion operators K. : V2 + Homo(Vi, Vy). This is, for example, useful for describing state transi- tions in the electrons of hydrogen atoms. To motivate the generalization in the next section, we shortly explain the derivation: we can consider the equivalent function K. VV & Ví V, given by K(v; v1) == [K[);)] (0)) on the tensor product. As one can compute, and as we will see in more generality in Proposition D.23 K. Vj @ Vi - V, is an intertwiner, where on the left we consider the tensor product representation. We assume, is the case for G SO(3) or G= SU(2) for usual applications in physics, that Vy is exactly once a direct summand of V Ø V4. Then, since by Schur's Lemma B.29 there cannot be nontrivial equivariant linear maps between nonisomorphic irreps, K restricted to each direct summand of V3 Vi vanishes, except the one isomorphic to Vj- More precisely, assume that
V +
Vi
a decomposition of Vj @ V2 into copies of irreducible representations, where each Vp is noniso- morphic to Vy. Then the information contained in I. is equal to the information contained in the restriction Kl|v, VJ + Vy. Since it is an intertwiner from a representation to itself, it deserves special name. We state the following definition for arbitrary IK E {IR, C}, since it will be of crucial importance in our generalization of the Wigner-Eckard theorem:
Definition D.7 (Endomorphism), Let p : G + GL((V be a linear representation.
intertwiner from V to V is called endomorphism. The vector space of endomorphisms is written as
Endg,K(V) = Homg,K(V, V).
version of Schur's lemma gives a simple description for endomorphisms irreducible represen- tations in the case that the underlying field is the complex numbers C. It makes use of the property of the complex numbers to be algebraically closed:
Lemma D.8 (Schur's Lemma). Let p G - GL.(V) be an irreducible representation. the underlying field is the complex numbers C, then the set of endomorphisms, iee. intertwiners from V V, only consists of the complex multiples of the identity:
Enda, {c idy CCEC)2C.
Proof. See Jeevanjee (2011)
This means that Ælv, idy, for some complex number E C. Now if we let p: VG @ Vi + Vy be the projection corresponding to the direct sum decomposition of V, @ Vi, then we obtain
K = Klv, op- (c-idy,) op =
That is, we have just found out that one complex number, c, is able to completely characterize IC and consequently K!! This is basically already the Wigner-Eckart theorem. However, it is useful to find a formulation that describes K. with respect to bases of the different irreducible representations. For this, we define matrix elements of representation operators. Before we come to the definition, we introduce some notation: f + V' is a linear continuous map between Hilbert spaces, we set
(vlsl>) = (vls(x))
for each E V and y E V'. The symmetry in this notation is supposed to remind about the fact that has an adjoint, see Definition F.42, and thus can be applied to just as well as to x, but we will not make use of this fact.
Definition D.9 (Matrix Element), Let T, Vi and Vy be unitary representations with orthonormal bases } C T (with,, possibly also varying), [Y7"] C Vi and (YJ) C V,. respectively. Let K. T Homg(Vi Vy) be representation operator. Then it's matrix elements given by the scalars
(JM [m) (()"))
In the same way, ff Vi V, any linear (not necessarily equivariant) map, then its matrix elements are given by the scalars
(VM (|
55

---

Published as a conference paper at ICLR 2021
Remark D.10. We shortly explain this term. Usually, in linear algebra, has to do with linear functions f V + V' between vector spaces carrying bases {u,} S V and {u² C V. For each basis element Dj V one can then find coefficients Aij E IK such that
f(0))= --)==,- E,
The Ag are called the matrix elements of and characterize completely. Now if the bases orthonormal bases as in Definition F.40 then the coefficients are given by
Ag -x(1(())) = (v/15l1g)-
In a similar way we can understand the matrix elements of a representation operator, only that the linear function itself depends on a chosen basis vector of Vy. As for linear functions, the matrix elements of a representation operator completely characterize it.
One last remark: since in this section, Vy appears only once direct summand in V, & V4, omit the additional "quantum number" in notation for the Clebsch-Gordan coefficients. With this preparation,
can formulate and prove the original version of the Wigner-Eckar theorem. Remember that there is unique complex number such that K. is given by K. e.p for projection p: V3 & Vi - VJ. We now denote this by (J (C]li) =
Theorem D.11 (Wigner-Ecke Eckart Theorem). The matrix elements of the representation operator K. V + Home V)) are given by
K2 (J[[CC]C)) (JM[jmm;))
with the (JM jmm In) being the Clebsch-Gordan coefficients (which are independent from the rep- resentation operator K.).
Proof. Let V, + V3 @ be the embedding corresponding the direct sum decomposition of V, @ Vi, It is an adjoint of the projection p V2 @ Vi - V, according to the proof of Proposition
F46 By what we've argued above, there exists some E C such that:
(JM A [n) (YJM (((""))))) (YSM @ Y?"))
(YYM p(Y," @ Y?")
=c. p(V]" @ Y?")
= - & Y
= (JI[AC)/)) [JM]jm;;n)
As a short explanation: the fifth step it was used that and are adjoint to each other, and consequently, we
from considering the tensor product in V, to that in V, & V. In the last step, the definition of the Clebsch-Gordan coefficients was used, and additionally, the notation (J])/C] e that we mentioned before the theorem. The index everywhere missing since V appears only once in V, & V, This finishes the proof.
Definition D.12 (Reduced Matrix Element). The unique number (J[i C in this theo- rem is called the reduced matrix element. To reiterate, it characterizes the representation operator completely.
D.1.3 REDUCTION TO IRREDUCIBLE UNITARY REPRESENTATIONS
Let G be any compact group and X any homogeneous space of G. Before we state the Wigner- Eckart Theorem for steerable kernels in the next section, we first want explain why restrict to the case of irreducible unitary input- and output representations. Our explanations are adapted from Weiler & Cesal (2019)
Thus, let Pin G + GL(V))) and Pout : G + GL(Vont) be general finiteddimensional input- and output representations. We consider the task of finding basis for the space of steerable kemelss
56

---

Published as a conference paper at ICLR 2021
Homg (X, Homp (Viy Vout)). By Theorem B.20 and Proposition B.38 there are equivalences of representations (i.e., linear isomorphisms that intertwine between the representations)
Qm
Vnn
① N
Vi
Qont
Vont
V
WE Yout
where Pu G + U(Vx)) and Pa : ((2) are irreducible unitary representations. Both for the imput- and the output representation, the same irrep can appear several times, e.g., there can be a p' such that Pp. = Put- Now, notice that the map
):))
Qun
Home
x, HomR
-
Vuz
Homo (X, Houg (Vin: Vout))
PElm
VElout
given for all E X by
PQnn Qin (K) (xx) = Qout o K(x) Qin
clearly an isomorphism. Thus, once a hasis for the first kernel space is known, we just need to postcompose and precompose each basis kernel with Q and Qin respectively, in order get basis for the space we actually care about. Furthermore, the map
Vi) v.))
V
A
Homa (X, HomB (Vo Vv)) - Home X, HoHK
① pe I
Vu
A vE
We Yout pe Im
given by
(K (a)))
Σ (t)(w)))))) uE
- vElous
Vis
where X and (2n)a OpELn V are arbitrary, is also clearly isomorphism. It expresses that can take collection of steerable kernels (K"")),, and build with it block-matrix, which is steerable again, as can easily be checked. Accordingly, if we have basis kernels for a space Home (X, Homm (,,., V.)) for some μ, v, then can, by applying which are zero outside the block with indices and μ.. Overall, by doing this for all μ,, v, we thus
map it to block basis kernels
recover a full basis for the space Homg [X, Homg (DuE Vis DuETOn Vo)). By applying the base change & from above, we thus get a basis for Homg (X, Homm (Vi,, (in: Vout)). sum- mary, knowing basis of steerable kernels for irreducible unitary input- and output representations gives us one for all finite-dimensional input- and output representations. Finally, note that the trans- formation of basis kemels using @Q.. and V can be done in the network initialization process and does not need to be performed in each forward pass.
D.1.4 THE -ECKRT THEOREM FOR STEERABLE KERNELS
Now that we have seen the Wigner- Eckart theorem in a version similar to how it usually appears in physics, it is time to state the version which will need in this work for applications in deep learn- ing. The treatment is similar to the formulation in AArawalaaa 1980), which presents a generalization of the Wigner-Eckart theorem to the case that Vy may appear several times direct summand in the direct sum decomposition of the tensor product. However, this paper still only considers the Wigner-Eckart theorem for the case of the complex numbers C. If allow the real numbers as well, we cannot be sure that endomorphisms of irreducible representations are just given by one number. This is complication we will deal with below by allowing matrix elements of general endomorphisms. Furthermore, we will deal with topological considerations that did not play role in Agrawala (1980). And lastly, we transport the theorem over into the nonlinear realm of steerable kernels.
As discussed in the last section, we can restrict the considerations to (representatives of isomor- phism classes of) irreducible unitary input- and output representations. Thus, assume the input- representation to be the irrep Pt G + U(Vi) and the output-represennation to be the irrep p. G + U(V)). The idea is described on each direct summand of the domain individually, and that on each of these summands, arguments similar those for the original Wigner-Fckart theorem apply.
that kemel operators K. L (X) Homg Vi, Vy) can be
57

---

Published as a conference paper at ICLR 2021
Peter-Weyl According to the Peter- Weyl Theorem B.22 the space (X) has dense subset which is direct sum of irreducible unitary representations:
y je8
(X)
Vjc-
im
Each Vj is, as a subrepresentation of L (X), isomorphic Vj. Vj is itself not assumed to be embedded in (X).
For arbitrary E G, fix once and for all orthonormal bases ( S V corresponding to the basis
{Y}"*} of Vg Furthermore, assume that for all = ., [J(it], Pyte V₁ 09 V/ - Vj is projection which is an adjoint of the linear equivariant isometric embedding İis- VJ - V3a Vi. This is assumed to be aligned with the embeddings V,, + Vj @ Vi with respect to the isomorphisms
V, ~ V5 that underlie the correspondence basis elements Y Yj² What this means that
the Websch-Gordaan coefficients with respect to all of these embeddings, for all i, are equal:
(Y)" Y Y?")
(s, (jnn
Now state and prove the Wigner- Eckart theorem, which gives explicit description of rep- resentation operators K. (X) + Homæ (V, Vú) in terms of endomorphisms of V, and then transfers this statement
statement about steerable kernels K X Homk (Vi, Vy)- Before we state the theorem, we want to shortly explain what to expect: in the derivation of the original Wigner-Eckart theorem in Section D.1.2 we saw that kernel operator could be expressed K: V Vi + Vy, This the projection p corresponding to the appearance of V, in the direct sum decomposition of V, @ Vi. This time, however, V, can be found often in 1 (X) @ Vi, namely:
in turn equal IC co for endomorphism e V, V, and
1. For each isomorphism class of irreps j E G
2. For each appearance =l...., my of the irrep V, in (X) and
3. For each appearance s 1,.... [J(]]] of the irrep V, in the tensor product representation
V, @ Vi. [JGi]] can be zero, which means that j does not contribute.
We therefore expect Ê to be whole sum of compositions of endomorphisms with projections, for each combination of valid j, i and s. Furthermore, the specific structure of (X) will be exploited as well by using orthogonal projections from L (X) to summands Vjs. Overall, we hope this sufficiently motivates the theorem:
Theorem D.13 Wiiner-EEcart Theorem for Steerable Kernels). We state the theorem in three parts:
(Basis-independent Nigner-Eckari for Kernel Operators) There is an isomorphism of vec- tor spaces
my [J(0]]
Rep
Endg, + Homg.. (X)) Homg Vy))
jE 1=1
which is given by
my [JGD] d,
[Rep((cua) (vi) = ΣΣΣ (X Cyta (Pae(Y @ vi))
(17)
jEN 1=1
m=1
where (cjis)jis is tuple of endomorphisms, & X - IK is any square-integruble functtin and Vi E Vi is any element.
2.
Wigner-Eckart for Steerable Kernels) There is an isomorphism of vec-
tor spaces
(J(D))
GKer
Endg,((V)) + Homa Homg (Vi, Vy))
je
1=1
s=1
23, is like additional quantum number in physics.
58

---

Published as a conference paper at ICLR 2021
which is given by
my [JGD) 4
(GKer((Giss)ja)(oo) (vi) = ΣΣΣ> Σ
(i.jmla) Cjas (Psea(yz) & vi)
jEG am m=1
where (cjss)jis is tuple of endomorphisms, E X is any point and Vi is any element. Here, (i.jm|a) = limmeu, (Yz((ov), which is according to Proposition C.10 equal to (x).
3. (Basis- dependede Wigner-Eckart for Steerable Kernels) Let K GKer (Cgis) be the steerable kernel corresponding to the tuple of endomorphisms (Cfta) jãa according to the isomorphism above. Then the matrix elements of K(x) Homg (Vi, VJ) are explicitly given by
(JM)K()))m)) = my [160]
d2
ΣΣΣΣX M'=1
(JMcaalJM" JM, (i.jmm/)).
(18)
jEŽ
Remark D.14. Before we come to the proof, we have some remarks to make about this theorem:
1. In line with the usual convention, we call the (JM Citae (JM') the generalized reduced matrix elements of the representation operator K. Different from the situation in physics, these can depend nontrivially on the specific basis indices M and M'. If the space of endomorphisms 1-dimensional, as is the case when considering representations over C, then each Cjis is a diagonal matrix, meaning that it is characterized by only one complex number, for simplicity with the same name Cjis. Then one has (JM [Cjis J M') = das Cjis and the sum over M' disappears. What this means for the matrix form of basis kernels of steerable CNNs will be discussed in Corollary D.17
2. The coefficients (s, JMM jm; In) are as before the Clebsch-Gordde coefficients. Note that the input æ of K appears only in (t,jmm (zr). Those two parts of the right-hand side of the formula are always the same, independent of the kernel K.
3. The Clebsch- Gordan coefficients are traditionally defined with respect to isometric embed- dings ljts V, + Vj 8 V/ since this makes them less ambiguous. However, we mention that the property of being isometric no requirement for the construction of Clebsch- Gordan coefficients or the proof of the Wigner-Eckart theorem, being equivariant and linear is suffi- cient. This then means that the copies (YYM) ) do not anymore form an orthonormal basis. We will use this relaxation in the example in Section E.2 where we do not want to be bothered with obtaining isometric embeddings.
4. The names for the isomorphisms in the theorem are meant as follows: Rep is the map that maps a tuple of endomorphisms to kernel operator, which is special representation operator. GKer maps a tuple of endomorphisms to a G-steerable kernel. is not meant as notation for kernel in the sense of nullspace in linear algebra.
5. Furthermore, reader with a background in abstract algebra may wonder why build the direct sum of spaces of endomorphisms instead of the direct product. The reason is that a posteriori, turns out that only finitely many j contribute nontrivially, and so the direct sum is equal to the direct product. For proof of the finiteness, see Remark D.18 below.
6. As last remark, we want to mention that part 1 of the theorem is not the most gen- eral version we could do. We chose to formulate the Wigner-Eckaa theorem for L (X) specifically since this is the space we use it for. However, an appropriate isomorphism can probably be formulated for any unitary representation instead of L (X), only that we then need to take care that we replace direct sums by direct products if the index sets on the left side are infinite. Additionally, Vi and V, could be replaced by arbitrary Emite-dimensional representations, and an appropriate adaptation of the theorem would apply. Whether Vi and V, could also be replaced by dimenti-ndmeniinnll unitary representations would need to be explored, but extension to such case seems possible.
59

---

Published as a conference paper at ICLR 2021
Proof of Theorem D.13 The proof of will be done in Section D.2 since it requires some work. However, the proofs of 2 and 3 are relatively straightforward once we believe and so we do them here:
From 1 we know that Rep is an isomorphism. Furthermore, from Theorem C.7
wc know that
OIx: : Homa, & (X), Homk (Vi, V2)) + Homa. (X, HomR (Vi, V)))
an isomorphism as well, and this is given by X (z) limyeu. K((ov), where we take the limit over the directed set of open neighborhoods of z. We define the isomorphism GKer now simply as the composition, i.e., GKer (+)xx Rep. This isomorphism is then explicitly given by:
(GKer(((())) (v))
(ox)
lim (eep((Cn)))s))0 (e))
Ueus
my [JO0] d,
lim
(Y bo) Cjsa (P)ro @ vv))
UeM2z
jec
[TG]]
lim Veuzz
[o>) Cjio (pjis @ @m))
jEG t=1
m=l
[JG9]
(i.mm)a) ojts (Pjis(Y]7 @ v1)).
JEG
This already proves 2. Now, in the following computation, we will use that Cjis Pjis Cjis idy, opjs and that, inspired by notation in physics, we can write the identity on Vs as idv, = Σ Ed (Y" (Y3M". For 3, we then compute
(I)
= (YF {<>)())))>"))
- [CKKe(((C)))))))aa))))] (X,")
my (600] d Σ
(i.mm/))
o VJT @ Y?")
JEG
m. [JO0] d,
(t.jm/2) (Y3" (YM Prin Y @ YTr)
jec i=1
==1 m=1 ='
[G0) d
(JMGAsIJMM (s, JM (m]:)mn jm/lm))).
jec
m=1
=1
In the last step, used the Clebsch-Gorde coefficients, see Definition D.6 and, mentioned before, that Pjis is adjoint to the embedding Liis V, - Vya @ Vi.
Remark D.15. Here, we want to argue that our kernel space solution also covers that of general equivariant CNNs on homogeneous spaces (Cohen al. 2019b). One definition of the kernel space in that setting is
Homa x Gomt (H, Homa (Vins Vout))
(19)
[K: + Homg(Vin Vout) (gout = Pout (gout) K(h) o (in()
where H loccally compact group and Gin Gout C H are subgroups with input- and output representations Pin Gmn GL.( and Pout Gout GL((Vuut.. For compact groups Gi, and Gouts this is covered by our setting as follows: we define G = Gout x G and g = (gout. I.n) We can define the left action of on H by g = Jout hgg Furthermore, we can reformulate the representations of Gi and Gout representations of the group G by setting Pin G GL.(V..)
60

---

Published as a conference paper at ICLR 2021
with (i)(g) (in() and similarly for Pout We furthermore notice that in Eq. (9) could also have inverted Iin since that constraint needs to apply to all elements of Gin- Thus, we then see that the kernel space can be equivalently defined by
HomG, Gout (H, Homk (Vin, Vout))
(20)
= { H - Homg (Vi, Vout) (g h) = Pout (g)
K(hh) Pin(g)
which precisely is the kernel constraint of steerable CNNs in Eq. Thus, if we restrict to a homogeneous space of the action of G on H, recover steerable kernels in Definition 3.2 and can apply Theorem 4.1
D.1.5 GENERAL STEERABLE KERNEL BASES
Now that we have a Wigner-Eckate theorem for steerable kernels, which gives a one-to-one corre- spondence between steerable kernels and tuples of endomorphisms, can finally describe what basis the space of steerable kernels looks like. For this, additionally to the notation in the last section, we assume that {c, r 1..... E } is basis of Endg,x(Vj)
Theorem D.16 (Steerable Kernel Bases), A basis of the space of steerable kernels Home (,, Houn (Vi, VJ)) is given by
{K : X + HomB (V,,V) 1j E where the basis kernels K Ksser huve matrix elements
s (500),,
(JM] (z)
Σ
(JMc+ Σ M'
Js. jm; - (t.jm/a).
(21)
Now, for each M' E ......,dJJ,, let CG be the dj x di-matrix of Clebsch-Gordar coefficients (s, JM' (jm; In). with only and n varying. Furthermore, let (i.j(z) be the vector with entries (i. jmlz) for m = 1,...,d, dy- 1 In matrix-notation with respect to the bases (Y3 V, and then express the basis kernel (x)) Vi - V, follows:
{Y7) C VL
(1.jkz) CG)
(x) Cr
(22)
(1. CG
In this formula, all "dots" mean conventional matrix multiplication and is by abuse of notation the matrix of the endomorphism Gr.
Proof. For the first statement, note that basis for De + my + ++=1 Fadg..(()) is given by all
the tuples 'jisr = (O,..., .r...., 0) that have c, at position jis, for all combinations of j, i, and r. Thus, from the isomorphism GKer in the second part of Theorem D.13 we obtain that all K Kjter == GKer(tjter) together form basis for the space of steerable kernels Homg Homg (Vi, V-))- When applying the basis-dependees form in part 3 of that theorem to Kjter the first three sums in Eq. (88) just disappear since hjisr is zero almost everywhere. Furthermore, Cjts replaced by the basis endomorphism
We obtain the claimed result.
For the final statement on the matrix representation, note that
(JM) Kijer (x) = Σ dy
IN (JM|clJM") (s,, JM' jim,m)) (i)))) Σ M- (JM(aJJM') Σ dy
Σ do
m=1 (i.jm(z) JM' (im;
do
M (It ( (i.jmlz) (s, JM' JM') In) M²=1
de
c. ..M ((i.jla)
CGM'-1) M JGD)s M==1
Here, cM is the M'th row of the matrix cy. The result follows by dropping the indices M and 1
61

---

Published as a conference paper at ICLR 2021
The next corollary means that endomorphisms be ignored if the space of endomorphisms is 1-dimensional, which is in particular the case if K = C.
Corollary D.17. Assume that dim(EndG.xt (V,)) = 1. Then basis of steerable kernels K X
Homg(Vi V5) is given by all Kis with matrices
(i.5kz) CG
Kzt(())= =
(23)
(1,510) CG
Jign
In particular, this is the case if =
Proof. In this case, basis for the space of endomorphisms is given by the single endomorphism idv,- Postcomposition with the identity does not change the matrix, and so the result follows.
For K C we have dim (im(End,,c(Vo)) 1 by Schur's Lemma D.8 and thus the result follows.
C
We end with two remarks regarding the parameterization of steerable CNNs. The first remark con- siders the case of stecrable CNNs of the form X - Homg (V,, Vj) on a homogencous space X. The second remark connects this back to the case that X is an orbit embedded in IR.
Remark D.18 (Parameterization in the abstract). First of all, we want to understand that there are only finitely many basis kernels Kjisr- To this end, note that the index sets for i, 8, and are necessarily finite for all j. and thus need to understand the finite range of j. A priori, j can over the whole set G, which can be infinite. But, can have VJ in a direct sum decomposition of Vj & Vi, which rescues the finiteness:
argue now, for only finitely many G
Namely, Vy is in the direct Homg.x(V @ Vi, VJ) is nonzero by Schur's Lemma B.29 By the hom-tensor adjunction that we will show Proposition [D233 in more generality, this is the case if an only HOmG. (V,, Hom VJ)) is nonzero. And finally, this is the case if and only if V3 is in a di- rect sum decomposition of the representation Homg (V, Vy), again by Schur's lemma. Now, since Homg Vi, V)) is dintee-diensinna,, this can only be the case for finitely many j. and so we are done
decomposition of V, Vi if and only if the vector space
Overall, this means the following: To parameterize equivariant neural network, one needs arbi- trary parameters Wjier K for all combinations of j E G. {L...., my J. E {l...., [.....,[([, [JOO]] and r = 1,...,,,. E,...,,E... A general steerable Kernel K X Homk V,) then takes the form
K Σ JEG
|JG0]
r=1 Wjise K jissr
Ea
s=1
with the basis kernels Kjter as in Theorem D.I6
Remark D.19 (Paramcterization in practice). Remember that our original motivation for the use of homogeneous spaces in Section 1.1 1. was that IR splits disjoint union of homogeneous spaces, on which the kernel constraint acts separately. For simplicity, we assume that the compact group acting on Rd is either G (O(d) or G O(d), but the general ideas hold also for the finite trans- formation groups in R - the only difference is that in these finite cases, the set of representatives of orbits becomes larger.
Thus, Rd splits into orbits Rd = Ll.20 Sm (r), where 5T-1(() is the sphere of radius (with S(0) = {0) being single point).
We'll discuss the orbit X₂ {0), the origin, separately below. But note that all other orbits are necessarily homeomorphic to each other and thus can be treated on equal footing. Therefore, let S be the standard sphere with radius and Kjtar 5n + Homg (Vi, Vy) be basis kernels for this choice. Then for general steerable kernel K Rd -> Homg (Vi, Vy) there are arbitrary functions Wjjer: R500 IK such that, for all x E Rd {0}, we have:
K(x) = Σ JEŌ Σ D
(JGD) Σ
Es r=1 Wjisr ([|=]]) Kjter
œ T>o|
'f course, for this argument, we need the uniqueness of direct sum decompositions. But this follows if assume the Hom-representaation to be unitary, which works by Proposition B.20 and then using the Krull- Remak-Schmide Theorem, Proposition B.39
62

---

Published as a conference paper at ICLR 2021
For O, we might heavy theory to solve the kernel constraint, but it is more illuminating to do from scratch since this case is so simple: we have ((0)- Vi - VJ, and the kernel constraint takes the form
(())= (gg = ( pi(g)
for all g G, which is equivalent to K (()) pr(g) o (0) for all g G. This just means that (0) Vi + VJ is an intertwiner, and by Schur's Lemma B.29 it is either O if l * J or an arbitrary endomorphism Vy V, if J. Thus, assuming c, Vy + V,, there are coefficients w, E IK such that
and choosing dissi-endomorphisms
K(0) Σ Eg
Wr
Cr. Cr.
The reader may find it interesting check that this solution is precisely what is also predicted by our theory using that L (10)) = K is just isomorphic to the trivial representation of G.
All in all, we now know what the most general steerable kernels look like. In practice, one needs to choose the functions Wjser : R>0 + IK. For representations over the real numbers, i.e.. with K R. one choice is to only consider finitely many radii and Gaussian radial profiles around them. Then instead of learning the whole function Wijarr one learns finitely many real parameters that choose "how activated" a basis kernel K jisr is for certain radius. This is, for example, the route taken in Weiler et al. (201 88 a): Weiler & Cesa (2099) If one deals with complex representations, one usually goes the same route, only that the parameters that choose how "activated" the basis kernels are will then be complex numbers. One either parameterize them as + ib with real part a and a complex part b. This intuitively means that a activates the standard version of the kernel K jisr. whereas activates the kernel iK jisr, which can be imagined a version of the kernel turned by 90% One other possibility is to parameterize complex number as et0 with scaling factor a > O and a phase shift B. This is the route chosen in Worrall et al. (016)).
In Chapter E we will look at examples of determining the basis kernels Kjisrs which will hopefully further illuminate the theorem. In the next section, we go back to the theory and prove the remaining parts of the Wigner-Eckart theorem.
D.2 PROOF OF THE WIGNER-ECKAN THEOREM FOR KERNEL OPERATORS
In this section, we prove the first part of Theorem D.I3 the Wigner-Eckart theorem for Kernel Operators, since we have skipped this in the last section. It is not necessary to read this section and the reader may wish to directly go the chapter examples E We will make frequent use of topological concepts from Chapter F. in this section.
The strategy is the following: in Section D.2.1 show that
my ① JEG
Vjs, Homg V2)).
Homg, K (X), Homg (Vi V,)) Homg,
which basically means that we can ignore the "topological closure" of the direct sum which is dense in (X). This works, intuitively, since kernel operators are continuous, and so they are determined by what they do on a dense subset. Then, in section D.2.2 we show that
HomG.K .K
M+@ jEG t=1
θ
Vja, Home (oo Vi)) HomG.K
++eV jea 1=1
which is the main step that we need in order be able to make use of the Clebsch-Gordae coeffi- cients, namely when we decompose the tensor product. Finally, in D.2.3 we finish the proof
of Theorem D.13
D.2.1 REDUCTION TO DENSE SUBSPACE OF (X)
In this section, we reduce the statement to representation operators on jEa plicity, we write the double direct sum from +
my i=1 Vja. For sim-
63

---

Published as a conference paper at ICLR 2021
Furthermore, remember that Vi and V, are dimen-dimensional, and thus Homg (Vi V5) can be iden- tified with matrices in K This space is a Euclidean space and thus has a scalar product and consequently also a norm, see Chapter F.1 Consequently, each kernel operator is continuous map between normed vector spaces, which we use in the following.
A short terminological note: kernel operators are just representation operators on [ (X) and only have their name due to the relation to steerable kernels. Thus, the terminological difference to representation operators in the following reduction result has no further meaning:
Lemma D.20. The restriction map
Homg, K (X), Homg Vy)) + Homg K
(+),
jz Vje, Homg (Vi, Vi))
given by K. K Vi+ between kernel operators the left and representation operators on the right an isomorphism.
Proof. First of all, the kernel operators on the left are actually uniformly continuous by Proposition
F.18 Thus, by Lemma F.22 the restriction map is an injection into uniformly continuous representa-
tion operators on + Gz- The set of all these maps equal to the set of all representation operators
by Proposition F.18 again.
Thus, in order to be finished, we only need to see that the unique extension of a representation operator K: Dyx j Vj+ (VmmgVi, Vy) continuous function K. (X) + Hom (Vi, Vy) is
kernel operator, which means is linear and equivariant.
For linearity, let a E K and f E (X). Let (Jx)x: be sequence in Djs Vjz that converges to f- Using the continuity of K. and the linearity of K. we obtain:
K(a- f) RT( lim (a fi))
11+00
lim K(a fk)
lim K(a fk)
k-+00
lim a K((fk)
k-+00
a lim K(fxx)
1-000
K lim fi)
k0000
K(f).
Linearity with respect to addition can be shown similarly. For the equivariance argue in the same way, only that we additionally need to use the continuity of the representations : G (X)) and PHom G - GL(Homg (Vi, Vy)).
D.2.2 THE HOM-TENT TENSOR ADJUNCTION
Lemma D.21. Let K. Dr Vn + V be linear and equivariant, where V is an irrep. Then K. continuous.
Proof. By Schur's Lemma D.8 we know that XC factors through the irreducible representations that are isomorphic to V. That is, V, be that irrep and P5i D Vi, -> Vj the canonical projections. Then there are intertwiners G; Vj - V such that K = Σ, Pji- Each is continuous since a linear function between finit-dimensional normed vector spaces. Since also summation normed vector spaces is continuous, only need to show that the projections Pj are continuous.
S'hhhur's lemma applies since it is statement about irreducible representations which are necessarily finite- dimensional. This means that the continuity condition in the definition of intertwiners is vacuous and thus don't 't need worry about K. not being continuous priori.
64

---

Published as a conference paper at ICLR 2021
This follows from the following fact how the norm D₂ Vu is composed from the norms each Vi:: For an element f = Eu fis E Du Vu with fis E Vis, we have:
|I Σ llfill
The reason for this is that the Vi perpendicular to each other. Consequently, if (f*)a with f E Dn Vi converges to O, then also (P){(f")) = (ff)) converges to 0, which shows the continuity of Pja in and thus general continuity by Proposition F.18
Remark D.22. Note the curious fact that we cannot get rid of the equivariance condition in the preceding Lemma. I.e., if we have linear function ① 1/2 V, then we cannot deduce that K. is continuous. We omit the index for simplicity. If equivariance is requirement, then only deal with vector spaces, which are in general isomorphic to spaces of (maybe infinite) tuples of elements in K. Thus, let the function K. DIEN IK + IK given by
(at), + E, L.j.
This is linear but not continuous in O. The latter can be seen by considering the sequence (a*), )} with at (O...., 0, 1.4 0....) that has value 1/ on position k and otherwise only zeros. This sequence converges to the O-sequence in norm. However, we have K(aa)) = for thus the images do not converge to 0 = K())).
From the preceding lemma, we are able to obtain the following alternative description of represen- tation operators:
Proposition D.23 (Hom-tensor Adjunction). The map
()) HomG.K
ji Vji, HV. Vi) Homa, (+) Vxx @ v.
given by
K(u, () = [K(r;)] (wr)
is an isomorphism.
Proof. For continuity, note the following: by straightforward extensions of Lemma D.21 all linear and equivariant maps Dyc Vja Homm (Vi, Vy) and (+,s Vús Vi V necessarily contin- uous, and thus we can ignore continuity altogether. The rest of the proof can be done as in Agrawala 1980). For illustrating the most important part, we show that is actually equivariant:
&((Kes @ pi)(g) (u, @ vz)) -K((os())) (wz) [p(())] (m1)
[K(e;(e)(;)))) (wi)
(ps(g) p() (v)())()))
=p(())((1,)(())) = (R(())(((v), 8 8.)).
Remark D.24. Some readers may wonder why this is called an adjunction. With removing of the notation in the Proposition, one has
Homg. x (T, Homg(U, V)) = Homg (T e U.V.
Now, for notational clarity, set F == Homg(U; and H = (.) @ and remove the subscripts. Then the formula can be written
Hom F(V)) = (T)) V)-
With replacing the notation if the Hom-spaces with scalar product, and the isomorphism sign with equality, this reads as follows:
(T))(V)) (H()))
Similar to adjoints in Hilbert spaces, we can then view F and H as adjoint to each other. In categor- ical terms, they are pair of adjoint functors, see Lane al. (199)).
65

---

Published as a conference paper at ICLR 2021
D.2.3 PROOF OF THEOREM[D.I
After the work done in the prior sections, we are ready to complete the proof of Theorem D.13)
Proof of Theorem D.13 Only the first part of that theorem still needs to be proven. We have the following string of isomorphisms, which we will explain below:
Home, k (L (X), Homg (V, V2)) = HomG,K
Vi,, HV, vo))
HomG,K
Vis) s Vi, V3)
HomG,K
(Vis @ Vi), v.)
Homa, (Viz @ Ví, Vy)
[JO0]]
HomG, . (V,
[JG)]
Endg, (V,).
jeN
The steps justified as follows:
1. For the first step, use Lemma D.20
2. For the second step, use Proposition D.23
3. For the third step, use that there is natural isomorphism
( &V = (V
4. For the fourth step, use that linear equivariant maps can be described on each direct sum- mand individually (and that we do not need to worry about continuity due to Lemma D.2I}
5. For the fifth step. precompose with the linear equivariant isometric embeddings Lyss V + Vjx & Vi and use, again, that linear equivariant maps can be described each direct sum- mand individually. Furthermore, use Schur's Lemma B.29 in order to see that the other summands disappear.
6. The last step is just reformulation.
Now, we call the string of isomorphisms from right to left
(JGD)
Rep
Endg.x(V)) + Homg. (L)((), Hom, (Vi, Vz))
i=1
and are only left with understanding that it is actually given by Eq. 117 For this, we take a tuple (cjus)jis of endomorphisms and explicitly trace back "where comes from". As in Lemma D.21 let Pyt + j'y Vyz + Vjs be the canonical projection, which is by Proposition /.46 given by P)((p) = Enp (Y (p) Y Furthermore, let Pjte Vya & Vi + V, be the projections corre- sponding to the embeddings 1ji.- Then from bottom to top, (Cjts)jts gets transformed as follows:
(J60)
(Cyis)sis
Σ
Cjts Pjss
ji
my (J00)
Cyts Pfice (pjs @ idy;
i=1
Rep(((cis)jia
66

---

Published as a conference paper at ICLR 2021
In the last step, the hom-tensor adjunction Proposition D.23 is used, but in the other direction. As an illustration, the composition functions over which we sum can be shown in the following commutative diagram:
Dvy Vys @ Vi Prcboidy
Vis @ Vi
Poña
V
Cyéz
V
Oyea (pp.s).
We obtain:
my [IG]]
[Rep(((ga)sss((p)) (0i)
Σ
Σ
[Ctts o Pyie (ppe (
jec
my [GGD]
(ofa o (pis)) & vz)
[
de Σ m=1
Σ (crs Pjis)
(x)
jEC
[GG]] d
(Y (0) Cjis (pits (Y₁7 & mn)
jEG m=1
That, finally, finishes the proof.
E EXAMPLE APPLICATIONS
In this chapter, we develop some relevant examples of the theory outlined in prior chapters. All of these examples are applications of Theorem D.16 and Corollary D.17 These examples are concerned with the following question: Given a specific field IK (R, CJ. compact transforma- tion group G and homogeneous space X of G, how can basis of steerable kernels K X > Homg (Vi, Vy) for given irreducible representations Pt - U(Vi and pJ G U(V) be determined? The theorems give outline for what needs to be done in order to succeed in this task, and the steps are always as follows:
1. For each E G, a representative for the isomorphism class of irreducible representations / needs be determined. That is, needs determine Pt G U(V,) and an orthonor- mal basis (Y," Usually, we have V/2 = Kdi and the orthonormal basis is just the standard basis.
E {1...., de }. We omit the index n if there is only one basis element.
2. The Peter- Weyl Theorem B. 22 gives the existence-statement for decomposition of L (X) into irreducible subrepresentations. We need an explicit such decomposition, i.e.: need to find multiplicities my. irreducible subrepresentations Vjs Vj for {l....,m, m, and basis functions Yff VJa C (X) corresponding to the Y,m such that (X) =
JEG Vy+-
3. For each combination of J.l I and in G one needs to find the number of times [JGJ]] that V, appears in direct decomposition of V2 8 Vi. Then, for each s (1,..., (J()). and for all basis-indices M, m and n, one needs to determine the Clebsch-Gordan coef- ficients (s, J Mjm;; In). We omit the index if V, appears only once in the direct decomposition of Vj @ Vi.
4. For each J one needs to determine basis {cr r r-= 1,...,E,1 Ey} the space of endomor- phisms of V,. namely Endg,
Once all of this is done, one can then simply write down the basis kernels according to Eq. 22 or, in that the space of endomorphisms 1-dimensional, Eq. (23). The ingredients determined
67

---

Published as a conference paper at ICLR 2021
above are purely representation thhooretic information about the situation at hand, which hopefully makes the reader appreciate the results even more: we do not simply determine basis kernels; we understand in detail, along the way, the representation theory of the group and homogeneous space.
Note that we are not concerned with practical considerations related to how fine-grained to do this in practice (for example, if the space which the kernels operate splits into infinitely many orbits). For such questions, we refer back to Remark D.19
In the following sections, discuss harmonic networks (SO(2) -qqiivaraa CNNs with com- plex representationns, (2(()--quivarian) CNNs with real representations, reflection-equivarian net- works, (0e))-qquvvarian CNNs with both complex and real representations, and -(3)-eqquvariant CNNs with both complex and real representations. For each of these examples, we go through the four steps outlined above. We recommend looking at the first example in detail: we conduct the greatest detail and it is the easiest to understand and thus serves as nice introduction.
E.1 ATTEE-BTE KERNELS FOR COMPLEX REPRESENTATION HARMONIC NETWORKS
Here, explain how the kernel constraint for harmonic networks (Worrall al. 2016 can be solved using our theory. In the case harmonic networks, we have IK C, S0 x S As in most examples that follow, ignore the solution of the kernel constraint in the origin, since it is usually easy to solve. For simplifying the formulas, we employ the isomorphism
a b
-b aa)
SO(2)
+
and always write U(1) instead of SO(2). Here, U(1) the group of rotations of C, i.e., the group of elements in C with absolute value 1. It is also called the circle group since the group elements lie on a circle in the complex plane. Note that the change from SO(2) to the isomorphic group U(1) done purely for convenience reasons, and S2( could be used just as well.
We now go through the four steps outlined above. Our statements about the representation theory of the circle group can be found in Kowalski (2014), chapter 5.
E.1.1 CONSTRUCTION OF THE IRREDUCIBLE REPRESENTATIONS OF U(1)
We have U(1) = Z. and for / E Z we can construct representative pr U(1) - U(V) as follows: Vi C is just the canonical 1-dimensional C-vector space, and Pt is given by
[pr(g)] (z) = g'
where g is regarded as element in C. One easily check that this is an irreducible representa- tion. The orthonormal basis element for each such representation is just given by E C = V/. This already answers step of the outline above.
E.1.2 THE PETER- WEYL THEOREM FOR (S!(
For step 2, need to determine the Peter- decomposition of L3 where regard S as subset of C. Let Yι 8" + C be given by (=) z-1 Let Vn C (S just be given by its span: Vn = (Yan) We want to see that this is a subrepresentation this, remember that the unitary representation on (X) is given by X U(1) - U(L)(S)) with [X(g)x] (2) = ((g '2). We have
(St).. To see
(X()) = o --= of.
(g' Y11) (=)
(24)
and thus X(g)V YYYn E Vni. which is what we claimed. Since the Vn are 1-dimensional, they are necessarily irreducible for dimension reasons. Now, an important result from Fourier analysis is that the Yn for / E Z actually form orthonormal basis of (21)) and that, consequently, the Peter- Weyl decomposition of (5') looks as follows:
(S
Vn.
Jez
68

---

Published as a conference paper at ICLR 2021
From this that the multiplicities my all given by 1. What is missing is the connection to the irreps Pt U(1) + U(V).. but we have already indicated this in the notation. Namely, the map fi Vi + Vn given by - Yn is clearly isomorphism of vector spaces, and due to Eq. (24 even an isomorphism of representations:
fi(p((o((2)) = filg' 22)
(g' Yι
z. (g'-Yrt = =2. (X(()))) M())( Yn))
(fg)(f(
Thus, fi pi(g) X(g) for all U(1) and, as claimed, fi turns be isomorphism. This finishes step 2 of the outline above.
E.1.3 THE GORDAN DECOMPOSITION For step 3, we proceed as follows: The map
f:V 23
clearly well-defined and linear by the universal property of tensor products, Definition D.I Furthermore, it is an isometry: namely, since the scalar product in just the usual multiplication (with the left entry being complex conjugated), obtain
(f(z) @ f2) @ =1) = [
zjdyy Fd5
(2>12/) (2112)
= (2) & 2//6 2)
In the last step, we have used the definition of the scalar product on the tensor product, Definition D.2 Thus, f is an isomorphism of Hilbert spaces. Finally, it also respects the representations since
s(les & (i) (zy @ x)) = S (()s()) (2z) @ (x((a)
(g'2 @ g'
g6 24 -'zz
gitl (zzz)
[ (f(z) & 2))
and thus o (e, & pi) = ( o of for all g E U(1). Finally, the basis vectors correspond in the simplest possible way since f(1 1) 1.
Overall, what we've shown is the following: V, is direct summand of V, & Vi if and only if J= I. If this the case, we have [J(jt) and can thus omit the index The only Clebsch-
Gordan coefficient is then given by (J1151/1) = 1 since the basis elements directly correspond.
E. ENDOMORPHISSSS OF V,
This is the simplest part: Since we are considering representations over C, Schur's Lemma D.8 tells us that Endum)e (V5) I-dimensional for each irrep J, and thus can ignore the endomorphisms altogether.
E. BRINGING EVERYTHING TOGETHER
We now show that basis steerable kernels : S - Elomo(Vi V,) of the group U(1) is given, when expressed 1-matrix parameterized by 51, by the basis function Y-j S1 C. We
69

---

Published as a conference paper at ICLR 2021
remove the index "I" at the basis function to remove clutter. How can see this result, using Eq. 23
Note that Vj only appear direct summand of V, Vi j J I by what we ve shown above. The "matrix" of Glebsch-Gordan coefficients CG ((J-())) is then just the number 1. We can omit the vacuous indices and and obtain that the only basis kernel is given by
K)-((z) = (Y)-1()) ())(())
(J-11
(1-J)
Yi-J(x).
This result is precisely equal to the one obtained in the original paper (Worrall et al. 2016). This concludes our investigations of harmonic networks.
E.2 -TEERABEE KERNELS FOR REAL REPRESENTATIONS
In this section, we look the case K R, G SO(2) and X St the following sections, we again step by step determine the representation-theerriii ingredients that we need for the application of our theorem. Compared to Chapter A which focuses more on the components themselves and how they relate to the general situation, this section has stronger focus actually determining the final kemeels, which also involves the task determining the eebsch--Gordan coefficients explicitly. We remark that the resulting kernels not new, since Weiler & Cesal (2019) have solved for this kernel basis already. However, we want to emphasize again that with our method, we learn more about the representation theory of SO(2) and thus get overall better conceptual understanding of how the kernels arise.
Since will help the presentation of our results, we set SO(2) R. R./27Z, i.c. view SO(2) group of angles. We also set S = R/2TZZ i.c., we take the interval [0, 2Tj as the space where our functions are defined. Consequently, since we want our Haar measure be normalized, we have to put the fraction 1 before all of our integrals, different from what we did in our treatment of SO(2) over
Note that since we now consider representations over the real numbers, unitary representations be- come orthogonal and we write O(V) instead U(V).
E.2.1 CONSTRUCTION OF IRREDUCIBLE REPRESENTATIONS SO(2)
The irreps of (0))) over R are given by pr SO(2) - O(Vi), / N2o. For = 0, we have V/ = RR and the action is trivial. For / > 1. Vi R² as vector space. The action is given by
(os( sin(lo)
sin(lo) cos(lo)
(p) (v)
for SO(2) R/27'. The orthonormal basis is in both cases just given by standard basis vectors.
E.2.2 THE PETER- WEYL THEOREM FOR (S²)
Now we look at square integrabl functions (S") that we now assume to take real values. As before, SO(2) acts on this space by (z) = f(x - Φ) 26 For notational simplicity, we write
cos; for the function that maps æ cos() and analogously for sin, One then can show the following, which is a standard result in Fourier analysis:
Proposition E.1. The functions cos,, sing, 2 1 span an irreducible invariant subspace of (2 of dimension 2, explicitly given by
spann (cost, sin)) {a cost +.3 a E R}
Note that we have subtraction now instead of multiplicative inversion. This is because view group as additive.
70

---

Published as a conference paper at ICLR 2021
which is isomorphic as an orthogonal representation to Vi by V2cos; cos: + ()) and V2 H
)7 Furthermore, sino and coso = are constant functions and their span is 1-dimensional and equivariantly isomorphic Vo by cosu +
Finally, the functions v2 cOs1, v2 sing form orthonormal basis of I (S*), i.e, every function can be written uniquely as a (possibly infinite) linear combination of these basis functions.
When setting V11 = spanp (coSt, sin)), we thus obtain a decomposition
(+t
Vn-
120
Thus, have my for all I IN. All in all, we know everything there is to know about the Peter-War Weyl theorem in our situation.
E.2.3 THE CLEBSCH-GORDAN DECOMPOSITION
We now do the explicit decomposition of V, Ví into irreps, which will give the Clebsch-Gordan coefficients that we need. Instead of doing the decomposition in terms of V3 and Vi themselves, in the proofs we actually use the isomorphic images Vn and Vn in & For doing so, we first need some trigonometric formulas in our disposal:
Lemma F.2. The sine and cosine functions fulfill the following rules:
1. cosj+1 = cos, cos; - sing sin,
2. siny+1 = sin, cos: + cos, sing.
3. 008j-1 = cos, Cos: + sin, sinj.
4. sing-r = sin, cos: cos, simg.
Proof. The first two well-known and the last two follow directly from the first two using sin-, siny and cos_, = cosy.
We will need the following general lemma: Lemma F.3, Let f V V' an intertwiner between representations p G - GL((V) and
p' G - GLV."). Then mull(f) = {o V ((v)= 0} is an invariant linear subspace of V.
Proof. This can easily be checked by the reader.
As remark notation for the following proposition: We write the Clebsch-Gordan coefficients CGJ of irreps V,, V, and Vi with dimensions d,,, dy and dj as a d, x (dj x di)-tensor. That is, it consists of d "rows", each of which is dj di-matrix. If V, appears only once in the tensor product, we omit the index as before.
Proposition E.4. We have the following decomposition results:
1. For j 0 have Vo Vo = Vo and Clebsch-Gordan coefficients CGo (11)).
2. For = 0, 1 0 we have Vo Vi = Vi and Clebsch-Gordan coefficients CG
11 0
0
1
3. For
> 0, I 0. we get V, @ Vo ~ V, and Clebsch-Gordan coefficients CG CG;
2 acts normalization.
71

---

Published as a conference paper at ICLR 2021
4. For j > 1> 0 we get V
@ 2
Vj-+
VJ+t. The Clebsch- Gorddan coefficients given
1
0
1
0
0
by CG)-1,01) =
and CG
0
0
1
+ Vs+i. The (lebsch-Gordlee coefficients are given 5. For 1> > 0 we ger V3 @ Vi = Vi-j
0
1
0
by (G(I-)00) =
and CG =
1
0
0
6. For j
0, get an isomorphism Vi @ Vi = Vi
@
V2
We obtain the
Clebsch-Gordan coefficients CGuguj = ((b ! 19) CGuauja
0 T1
0
and
+1
1
CG))( =
the last one being the same as the Clebsch-Gorlan coefficients
0
(G from above. In CGouy and CGuuyy- fourth index is present, namely 1 and 2, respectively. This is the index "s" that was missing in all the prior examples, since this is the first time an irrep appears more than once in a tensor product decomposition. Note that for CGuuyz, we have exactly one positive and one negative entry present, but both are equally valid and mirror the lower halves in CGj-1151) from part 4 and CGI-j.Gi) from part 5.
Proof. In the proof, instead of working directly with the irreps Pj SO(2) + O(V2), we use the isomorphic copies Vn in (S') given Proposition E.1 Since we think that does not help understanding to carry the index "I" in all computations, we omit this index.
The proof of 1, 2, and 3 is clear.
For and 5, consider the (unnormalized) basis (cos, & cost, cosj 2 siny, sin, @ cost, sing @ sin; } of V, @ Vi. Our goal is to express these basis elements with respect basis elements of invariant subspaces. We do this by explicitly constructing an isomorphism to decomposition of irreps. To that end, let p Vj @ Vi2 - (S+) be given by I 8 + f g. which is clearly well-defined intertwiner. We get image of the set
im(p) = spang (p(cos, @ cosi), p(cos @ sinr) p(sin, & cost), p(siny @ sin:))
spang ( (cosy coss, cosy. sin, siny coss, siny sin)).
=
From Lemma
we obtain:
(a) p( p(cosy @ cost) - p(sin, @ sins) = cOsj+1+
(b) p(cos, sin)) p(sin, @ coss) siny+1.
(25)
(c) p(cos, @ cost) + p(sin, & siny) c085-11
(d) p(sin, @ cost) p(cos, sin)) sin,-I-
Since the right hand sides linearly independent basis functions of (S*),, obtain:
(p) = spall.. (cOSj41, sinj41, c08;-1, sinz-1) Vii-u @ Vj+t-
Note for the last step that due to symmetry, cosj cos and sin -1 = sin
We now specialize to the case of 4, i.e. > /> 0. In this case, Vi--i| V₁- and the basis is given by cosy-/ and siny-1, as in the right hand sides of Eq. (25) (c) and (d). Consequently, Eq. (25) already the expansion of the new basis elements with the old, and the coefficients are consequently the Clebsch-Gore Gordda coefficients ES More precisely, if we want to compute, for example,
Note that for two orthonormal bases in Hilbert space, when expressing one basis {b,) with respect to another basis (ct), then the expansion coefficients are given by the scalar products (c, (c)) Since work over
72

---

Published as a conference paper at ICLR 2021
then observe from (c) that
+1 p[cos; @ cosi) 0- p(cos; & sin))
cosj-1 +0 p(sing cost) 1 p(ssin sin))
from which we can already read the upper half of CG)-I,(3) the coefficients in this equation (which we conveniently visually arranged in the right way). For the lower half, we do proceed the same for sim,-t, using (d). Then, for CG)+100 we proceed exactly the same, using parts (a) and (b). That proves 4.
For 5, we have > j O. In this case, Viz-11 Vi-5. i... the basis is given by cost-j cosy-1 and simy-ş = - siny-t. The latter means that in part (d) of Eq. (25). we need to replace siny-r by sim and thus change the signs on the left hand side. This change same as 4, the upper half of CG will remain the same as the upper half of CG)-1,9)) from
that CG will remain the
part 4 since the cosine in part (c) of Eq. 25 is symmetric, and the lower part will flip the signs. This fully proves 5.
Finally, we prove 6. We have j and still consider the same function p. Note that p(cosj @ cosi) + p(sing @ sin)) = 1 and p(siny @ cOs)) p(cos @ sin)) = 0 are constant functions that span the 1- dimensional trivial representation. Thus, we see that p is surjection
p Vi Vi Vo V2
with null space spanned by sin, @ cos: cos, @ siny. Such null space is automatically an invariant subspace as well, and since is oie-dimensional, it also must be isomorphic to the trivial represen- tation. Overall, this gives us an isomorphism
Vi & V V2 + VZ--
From this, we can as before read off the Clebsch- Gorddnn coefficients. The only thing that changes
is that parts (c) and (d) of Eq. (25) now correspond to two different copies of Vo, which means that the Sebsch--Goddad coefficients CGogr now split up in two parts CGaan and CGouay2 Note that in the trivial representation, the isomorphism that sends the basis vector to its negative is clearly equivariant, which means that both combinations of signs that we give in the final formula for CGouuy2 valid.
E.2.4 ENDOMORPHISMS OF V
We now describe the endomorphisms of the irreducible representations, our last ingredient: Proposition E.5. We have Endso12). z R, iee. multiplications with all real numbers are valid endomorphisms of Vo. For I 2 1, we gel
Endso(2) (Vi) { -
which is the set of all scaled rotations of R2. When identifying IR² transformations as arbitrary multiplications with complex number.
C, we can also view these
{
1) (i })} a basis for
As a consequence, idn is a basis for Endsove; and RV)) 1.
(a 2) that commutes with all rotation
Sketch. For 2 1 and an arbitrary matrix E
matrices (r()), i.e., Eo pr()) = (() o E, one can easily show the constraints a = d and -c from which the result follows.
the real numbers, the scalar product is symmetric, and these coefficients thus also the expansion coefficients when expressing {c.) } with (6,). This is why we do not have rearrange the expressions Eq. (25). it simply doesn't 't matter which of the two bases is expanded. Note, however, that our bases are not normalized, and so the Clebsch-Gordan coefficients differ by constant if the equation is rearranged. This constant does not matter for us since only interested in basis of the space of steerable kernels, and constant multiples of bases still bases.
73

---

Published as a conference paper at ICLR 2021
E.2.5 BRINGING EVERYTHING TOGETHER
Now have done all needed preparation and can solve the kernel constraint explicitly, using the matrix-form of the Wigner- Eckart theorem for steerable kernels, Theorem D.16 This is, as men- tioned before, a new derivation of the results in Weiler Cesa (2019). One can compare with table 8 in their appendix which only differs by (irrelevant) constants.
Proposition E.6. We consider steerable kernels K S + Homa(Vi V5). where Vi and V, are irreducible representations of (O)2). Then the following holds:
1. For 1 = = O. we get (xx) = a (1) (f)for every a E S and an arbitrary real number R. independent of x.
CosJ
sin,
2. For I 0, J> 0, a basis for steerable kernels given by sin.
and
cos.J
3. For > 0 and 0, basis steerable kernels is given by (cost
sinr), (sin:
-cost).
CO8J
- -sin.
4. For I,J > 0, a basis for steerable kernels is given by
sin,J1
COS
-in.j- -COSJ-1
COSJ+1 sins+l -coss+1)
sins+1
) and sin COSJ+
COSJ+1 sin
-sin
CO6.J-1
Proof. The proof of 1 is clear.
For 2, note that Vy can only appear & The relevant Glebsch-Gordann coefficients are
by Proposition E.4thhreforn CG)(J0)
Furthermore, the orthonormal basis of Vf1 V
is given by Proposition E. to constants by {cosJ, sinJ). which we have to write as a row-vector according Theorem D.16 Thereby, we can ignore the complex conjugation since work over the real numbers. Our final ingredient is the endomorphism basis of V,, which is by Proposition E.5
- Overall, the basis kernels are given by
given by C1 = idga and C2
[cos,)
sinu]
0
COsSJ sin.
c.
0
=
[cosJ
siny]
The result follows.
For 3, we find Vo only in V, 8 V1 if 1, and even twice The relevant Clebsch-Gordan
coefficients therefore by Proposition E.4 given by CGuguji 0 1]
0
and CGougy
(l (2-))= d)
The basis-functions in V₁1 Vn by Proposition E.1 up constants (cost, sin:),
again written as a row-vector. Finally, V, = Vn has only idR as a basis-endomorphism by Propo- sition E..l this be ignored altogether by Corollary D.17 We obtain the following basis for steerable kernels:
0
[cost
sin]]
(1 cost
sin):
0
0
-1
[cos]
sing]
(1 siny
-1 cos:)
1
0
For 4, we consider only the case << J. By Proposition
E.4 we have
Vz-1 09 Vi & V VJ, Vi+J s Vi Vj V2t+,
74

---

Published as a conference paper at ICLR 2021
Le., j = J -
J- and = 1 1 leads to tensor product decomposition containing VJ, but other j does. Thus, the relevant Clebsch-Gordan coefficients are by Proposition E.4 the matrices (,I,(J- and J.I+
We now consider the first case, i.c., j = - The Clebsch-Gord Gorddn coefficients are (G),(J-.))
1
0
CG(+560 =
The basis functions of V(J-n) 1 are by Proposition E.1 furthermore
0
1
given by (cos J Finally, Vy has again the two basis endomorphisms 6 idg and 02 from above. Thus, we obtain the following basis kernel for c1:
0
[cosJ-I
sin,-1)
-1
COS.-1
- -sin,-1)
c1
1
sin.-!
COSJ-1
(26)
[cosJ-1
sin.--]
0
Consequently, for as the basis endomorphism we need to postcompose with and get:
(1 -))
CO8J-1 sins
-Sرj-- coss_1
sin -cosJ-i) coss-I - -inj-1
(27)
These are half of the basis kernels. For the other half, need to look the case + J. The Clebsch-Gordan Gordan coefficients are by part Proposition E.4 given by CG = CG)-1.50) =
1
0
0
1
The basis functions of V(+Jja are by Proposition E.1
furthermore given by
0
(008s+1, sin For the basis endomorphism ei thus get the basis kernel
1
[cosJ+1
sins+1]
1
/coSJ+1
sins+1 -C0JJ41
c1
0
-1
sin
(28)
[008]+]
sins-1]
[
0
Consequently, for 0o as the basis endomorphism we need to postcompose with eg and get:
C -11 ))
COSJ+I sin
sin c0SS+1 )
(1 sin CosJ+1
cos.141 sim
Overall, for the case < J we have determined all four basis kernels in Eqs. 26).. 27 and (29). The cases = J and / J can be considered analogously, and in every case the correct coefficients have to picked. By using cos; = cos and sin = sin this will, in the end, always lead to the same final formulas. This result consistent with Table 8 in Weiler & Cesal (2019).
Clebsch-Gorde
E.3 Z2 ZSESTEERBBLE KERNELS FOR REAL REPRESENTATIONS
In this section, we discuss steerable CNNs that use the finite group Z2, which we identify with (1-1,+1) +), for their symmetries. We let this group on the plane R² by vertical reflections. though other choices are possible as well:
za
This example is simple and one may see it as contrived to apply our relatively heavy theory to it. We include it mainly as demonstration that our results can also be applied to non-smooth finite groups instances of compact groups. Furthermore, original group convolutional CNNs from Cohen & Welling (2016a) and thereby demonstrate that all the different developed theories are consistent with each other.
will fully recover the relationship to the
75

---

Published as a conference paper at ICLR 2021
E.3.1 THE IRREDUCIBLE REPRESENTATIONS OF Zg OVER THE REAL NUMBERS
Let p: L2 + GL(V) be irreducible real representation. Note that
p ( = p((-1) (-1)) -
p(1) (-1))= idy:
and thus p(-1) an involution satisfying the equation p(-1)² idv 0. It is well-known from linear algebra that involutions are diagonalizable, and thus pl p(-1) leaves 1 1-dimensional subspaces invariant. By irreducibility of p this means that V itself needs be 1-dimensional. Consequently, we can assume V = I without loss of generality. Note that the computations above mean that we have
(p(-1) idR) ) (p(-1)+ idR)
and thus we need to have p(-1) - idR = 0 or p(-1) IdR O. It follows p(-1) IdR or p(-1) = - idg. Overall, all these investigations mean that we have precisely two irreducible representations of Ze up equivalence. We call them p+ Zg O(V+) and p- : Zz + O(V-), where p+ (-1) )]= = idn and (-1) idR and V+ = =V= R.
E.3.2 THE PETER-WER THEOREM FOR LR (X)
Here we do the Peter- Peter-Weyl Weyl decomposition for [x( where is one of the two homogeneous spaces X = } and X {O} with the obvious actions coming from the groups Zg. This time, we also discuss orbits with only one point since we later want to get a description of kernels on the whole of R² for comparisons with group convolutional CNNs.
We start with X = {-1,1}. Note that the measure on is just the normalized counting measure, and thus all functions f x + IR are square square-integrable integrable. We define the two functions
f+ : X R, (x) = 1 for all x E X = {-1,1}, for all X [-1,1}.
f- X - R, f-(z)
We then define V+1 = spans (f+) and V-1 = span (f-). This gives a decomposition
(X) = V+1 + V-1
since we have for all f (X)
ʃ f(1) f(1)+ S+ f(1) 1(-1))
2
2
Furthermore, the maps + f+ and + f- give isomorphisms of representations V+ V+1 and V_ = V-1, respectively.
Now, assume that = (0) with the trivial action coming from Zz. Then (x) = V+1 generated from the function f+ X + R, f0) As before, 1 f+ gives an isomorphism V4 _ V41. This concludes the investigations of the Peter-Weyl theorem.
E.3.3 THE CLEBSCH-GORDAN DECOMPOSITION
We have the following four isomorphisms of representations:
V+ & V+ = V4, V_ & V4 V_, V- Lo V++
each time simply given by a & Section E.6.3 the reader can find a proof for similar, sign-dependent isomorphisms for the case that the group is 0(3). For each such isomorphism, there precisely one Clebsch-Gordan coefficient and it is just given by 1. Thus, as in the case of harmonic networks in Section E.1.5) we can just ignore the Clebsch-Gordan coefficients altogether the final formulas for our basis kernels.
ab. It can easily be checked that these isomorphisms. In
E.3.4 ENDOMORPHISMS OF V, AND V-
Since V+ and V_ are themselves only 1-dimensional, the endomorphism spaces necessarily 1- dimensional as well and just given by arbitrary 1 x 1-matrices, i.c., arbitrary stretchings. As in the example of harmonic networks, we therefore ignore the endomorphisms well.
76

---

Published as a conference paper at ICLR 2021
E.3.5 BRINGING EVERYTHING TOGETHER
Different from the other examples, we will in this section not only engage with the final steerable kernels on homogeneous spaces but also discuss how these assemble to kernels defined on the whole plane R In the end, we will then also discuss how kernels for the regular representation would look like.
But first, we engage with the homogeneous spaces. We start with X = [-1, and consider steer- able kernels for the input and output representations:
- Homp (Vin, Vout) for irreducible Vn and Vout- There are four possibilities
STEERABLE KERNELS K X + HomR V+):
V+ can only in a tensor product V @ V. the sign V is positive as well. Such a space appears precisely once in Li (X) according to Section E.3.2 Since endomorphisms and Clebsch-Gordan coefficients do not appear by what we've shown before, and since complex conjugation doesn't do anything over the real numbers, basis for steerable kernels is just given by the one kernel K f+ itself. Here, we identify HomR(V (V+, V+) with R since only consists of x 1-matrices.
STEERABLE KERNELS K + (V,
By the same arguments, basis is given by the one kernel K_ S-.
STEERABLE KERNELS K X Homg (V_, V+):
Again, a basis for steerable kemells is given by K_ = f-.
STEERABLE KERNELS K:
(V-VV
basis is given by K+ f+-
Finally, we also need to engage with the case that X = {0} consists only of a single point. Similarly above, in the "even" case that the signs of input- and output representations agree, basis given by K4 = f+ with (0) 1. If. however, the signs do not agree, then only K = 0 fulfills the constrained and the basis empty.
Now, we assemble this to kernels on the whole of TR2. We saw above that we only need to distinguish two cases, namely (a) the case that the signs of input and output representation agree and (b) that they do not.
For case (a), let K R² + R be steerable kernel, where is isomorphic to the Hom-space
between equal-sign representations. R splits disjointly into orbits, namely ())))) for all
R>0 and E R. If a = 0, then the orbit is just single point, which means that we have vertical line of single-point orbits. The solution above showed that on each orbit, the kernel needs to be constant (since f+ is constant) and overall this just translates to
K
a b
= K
for all a 2 0 and b E R. Consequently, K is just an arbitrary left-right symmetric kernel.
In the case that the input- and output representations do not share their sign, by the same arguments we see that K : R2 + IR is arbitrary left-right anti-symmetric kernel which is zero on the vertical for arbitrary E R.
line :
Other than these left-right restrictions, the kernel be freely learned. Overall, this means that we leam one "half" the kernel and can recover the other half by the symmetry property derived above.
77

---

Published as a conference paper at ICLR 2021
E.3.6 GROUP CONVOLUTIONAL CNNS FOR Z₂
We now investigate what all this means if consider regular representations instead of irreducible representations, thus corresponding to group convolutional kernels as in (Cohen & Welling 201 6a) In this case, we will see an interesting "twist" in the kemnl,, which makes this example more inter- esting than one might initially think. The twist emerges as follows: For regular representations, we consider steerable kernels
K : - Homa [ (Z2). (Z2))
Now, there are two relatively canonical bases we can choose in the left and the right space. We already know from above that f- is the basis to choose we want to express steerable kernels corresponding to irreducible representations. However, for vanilla group convolutional CNNs, the basis usually chosen is {e+1, e-1) where e+1(x) 6+1,r and 1-1 = --1,.-- We then obtain the following four base change relations:
f+ = c+1 +c-
f- = e+1 - 6-1,
1
1 e-1 J
1
e+1 - 5 + t
Thus, the base change matrices given by
B = 1 -1).
B
1
Now, assume that K R - HomB (I (Zz), 12 (Z2)) = Rx2 is expressed with respect to the basis (f+,f-}. }. If we write K as matrix
K1 K21
K12 K22
K
then we know that K11 and K22 map between equal-sign representations and K12 and K21 between unequal-sign representations. Consequently, from what we' ve found above, Kn and K22 are sym- metric, whereas K12 and K21 are antisymmetric. What we now want to figure out how exactly this translates to a property of the kernel expressed in the basis (_+.
Thus, let K' be this corresponding kernel. Then using the base change matrices above we obtain
Kj 21
K1p
K'
K22)
B. K B-
1
-1)
Kn1
K12
K21
K22
[K1 + K12 + K21 + K22l
[K K12 + K21 K22]
[Kn K12
K21
K22)
[K1
K12 K21 K22
What symmetry properties does this kernel obey? In order to understand this, use the following convention: for y E R we set -y =
V1 i.c., the vertically flipped image of Then we have,
3/2
using the symmetry and antiisymmetry of the entries of the original kernel K:
1
Kb2(-1) = [Kn K12(-y) - (-i(-y) + K22(- v)]
2
1
(nV(v) + -ız(v) + K21(y) + Krz(v))
2
= K(i(y),
1
K1 (-v)
[Kn((-)) + +K12(-y) (21(-2) K22(-v)] v)]
2
1
[KI(v) -K1zly) + Kz)(v) Kr2(v))
2
-K)((v).
78

---

Published as a conference paper at ICLR 2021
Thus the second row of K' is basically the same the first, only that the kernels swap with each other and are internally flipped. This is a special case of the outcome in Cohen Welling (2016a), which is also described clearly in /Weller et al. (2018b): in group convolutional kernels which are steerable with respect to finite groups, the kemels get copied and applied in all orientations de- manded by the group.
What we would still like to understand is if we can also reverse the direction: That is, assume that we start with group convolutional kernel K' of which know that (-z)(-)) Ki (v) and K22 (-y) = (1) for all E R If we then do a base change, we would like to know the resulting kernel consists of symmetric and antisymmetric entries. Namely, set
Kn
K12
K
K21
K22
-B-..'
Ki
Ki2
11
-1
K2
K2n
[K] + K + Kb2l [K] K
1
- K + K2z -
K22l
K K2n2
The reader can easily check that we can deduce that Kn and K22 are symmetric and that K12 and K21 are anti-symmetric. We have thus fully shown the equivalence of the kernel solutions in the setting of steerable CNNs compared to the setting of group convolutional CNNs for the specific group Zz.
E.4 (0(3)-STEERABL) KERNELS FOR COMPLEX REPRESENTATIONS.
In the first two sections, we have discussed SO(2) (eeeee)raaaaiann kernels (i.c., (equ))))cuvvrian neural networks) both over C and RR. The situation IR considerably more complicated and required new arguments. this section, we will discuss (O(3) (3)))qquivariant kernels (i.e., -quuivaanaa neural networks) for complex representations. In Section E.3 we will then look at the real case, which will essentially give the exact same results, thus differing somewhat from the considerations about SO(2). Different from the earlier sections, we will from now be less explicit and care more about the general properties of the different functions and coefficients we consider. SO(3)- equivariant networks with real representations have before been implemented in Weiler et al. 20188) and Thomas et al. (0188), among others.
E.4.1 THE IRREDUCIBLE REPRESENTATION OF SO(3) OVER THE COMPLEX NUMBERS
In this section, we state the complex irreducible representations of SO(3). We will not state the matrices explicitly since the matrix elements are considerably more complicated than in the earlier examples that we saw, For each E N20. there is one irreducible unitary representation
D1 SO(3) U(Vi,, where Vi2 C
The matrices Di(g) for g E SO(3) are called the Wigner D-matrices 20 lence, no other irreducible representations of SO(3) over C. A reference for all this the original work Wigner 1944).
There are, up equiva-
We note that the indices for the dimensions 221+1 are -1, -1+1, 1,/ general convention.
E.4.2 THE PETER-WERT THEOREM FOR (S2) A REPRESENTATION OF SO(3)
Here, we describe how (S?), considered as a unitary representation via SO(3) USL)) with [X())p]<x) = (gg ()), contains densely a direct sum of irreducible represen- tations. For doing so, we proceed by first describing spherical harmonics without formulas and stating their orthonormality properties, and then stating how they transform under rotation. This will then yield the result. Note that we do not need to describe explicit formulas for the spherical harmonics, which are again somewhat complicated since we more interested in their properties
Here, the letter "D" stands for "Darstellung" which is the German for "representation".
79

---

Published as a conference paper at ICLR 2021
in relation to Hilbert space theory and representation theory. A reference for all this is MacRobert 1947).
The spherical harmonics continuous functions Y," 8 + C for e N2o and n -(, Thus, they are elements of (5*)) They have the following properties:
1. (17 1r ów Om for all n,,,, n'.
2. The linear span of the spherical harmonics is dense in (S²).
3. They transform as follows under rotation: X(g)( = E Di (g)))"" where
DT (g) are the matrix elements of the Wigner D-matrices defined in Section 1.4.1
Properties 1 and together imply that the spherical harmonics form orthonormal basis of (g2))).
see Definition E.40 Let
Ví == spanc | n .......)..
Then already obtain (2 12 Vn1 Now, let em E C21+1 be the n th standard basis vector, for -l....,.. Then property 3 means that the linear map given basis vectors by
Vi + V1,, e" Y?"
is an isomorphism of unitary representations. More precisely, is clearly unitary transformation and linear isomorphism, and it is furthermore equivariant on basis vectors since
(Dilg)(e"))
Σ ()""(g)e" (O)e
DI (g)((e)(""))
(30)
E DF (g)
(xg)( (())((e))).
General equivariance then follows from equivariance on basis vectors. This concludes this section.
E.4.3 THE CLEBSCH-GORDAY DECOMPOSITION
Explicit formulas for the Clebsch-Gordan coefficients of SO(3) are given in Bohm & Löwe 1993] The most important fact is the following: There decomposition
[+
Vi
V
J
1-
of representations. Furthermore, the Sebbsch-Gordand coefficients [jM] In) are all real numbers, fact that will use in Section E.5
E.4.4 ENDOMORPHISMS OF VJ
As in the case of harmonic networks, this is again simple: considering representations over C, and so Schur's Lemma D.& tells us that Endgo(3) (Vy) is l-dimensional for each irrep We can therefore ignore the endomorphisms once again.
E.4.5 BRINGING EVERYTHING TOGETHER
Now, with all this prior work, let us determine the equivariant kernels K S2 Home Vy) for the irreducible representations D; SO(3) + U(Vi) and D, SO(3) + U(V)). For this, we use Eq. (23 Since each V, appears only once in the direct sum decomposition of (S2) according to Section E.4.2 and since V, can only appear once the direct sum decomposition a tensor do not need the indices and s. Furthermore, as
product V Vi according Section E.4.3
80

---

Published as a conference paper at ICLR 2021
mentioned in the last section, the endomorphisms trivial, which is why also do not need the index r. Overall, we see that we simply have basis kernels Kj S² - (Vom((Vi Vy) for all j with - --JJsjsi+ They are explicitly given by
(ilz CG)
(Glz) CG
for all x E S Remembering that (jm|z) 1 (x)) the individual matrix elements of K (a) are then given by
(=)lk) (JM)jm; In) YJ (x).
This ends the discussion.
E.5 (O(3)-STEERABLE KERNELS FOR REAL REPRESENTATIONS
In this section, we want argue why the results in the last section transfer over the real case as well. Most of the investigations in this section are probably well-known. However, we were able to find sources that explicitly explain the representation theory of SO(3) over the real numbers, and so we develop lots of it here from scratch. We thereby make use of the theory over C, some results about real spherical harmonics, and the general theory of real and quaternionic representations outlined in [Bröcker & Dieck (2003). We need to somewhat turn the order around in this section in order to develop the results. Therefore we first investigate the Peter- Peter-Weyl theorem, then look at the endomorphism spaces of the appearing irreducible representations and afterward, as a consequence, show that the representations appearing in the decomposition of L (S²) are already exhaustive.
E.5.1 THE PETER-WEE THEOREM FOR LR (S²) AS A REPRESENTATION OF SO(3)
The most important finding is the following, which is taken from Gallier & Quaintance (2000) One can do a base change for the spherical harmonics as follows to obtain real versions of them. Namely, let
(Yr" - (-1)-_)
if n < 0,
YYP
if n 0,
(31)
V2 +-
if 0.
One can then show that these functions real-valued continuous functions and therefore "Y," L3 (S²). Furthermore, they are an orthonormal basis of this space. We can then, as before, set Vi1 the span of the "Y," (S2) and obtain decomposition
(S?)
120 "Vu-
We need understand the transformation properties of these real-valued spherical harmonics under rotation. To understand this explicitly, we B₁ E ((2++))((1+1) as the (complex) base change matrix between the complex and real spherical harmonics. Its entries are given according to Eq. (31 such that the following relation holds for all .l....,::
"Y"
Σ T2'
B y
Since for given 1, both the complex and real spherical harmonics are linearly independent, the matrix B, is invertible. Let B-1 be its inverse. Then generally known from linear algebra that
30W saw that V, is direct summand of V, 6 Vi if and only if l - ol S J S I 1 By doing case distinctions, show that this the case if and only if sissl+ J.
81

---

Published as a conference paper at ICLR 2021
also obtain the inverse relation:
y (B "Y'
n'=-1
Using both these relations and the rotation properties of the complex spherical harmonics from Section E.4.2 we obtain the following rotation property for the real spherical harmonics:
(()(()P)
Σ B
My)((?"))
n
Bui
Duani (g) Y
n
Σ
B B
D (g) Σ
(Bi 1i'nn ryn
B
-2==1
n'==1
Σ
Σ
B 1nn'n D B "yp
--=-
MI==
n===1
1
Σ
(BT Di(g) Bi) n'n -Yp'
Now if we set "Di = B Di(g) Bj. then we obtain the transformation property
()) Σ
"D .Yy
(32)
which is analogous to the one in Section E.4.2
Lemma E.7. "Dilg)" n'n E for all l 2 0, n', n -l,..., and E SO(3).
Proof. Note that since "Y," is real-valued function, the rotation \(g((")") is real-valued as well. Thus, it is in the space L (S²). The real spherical harmonics are basis of this space, which means that the coefficients when expanding ("Y," this basis are necessarily real as well. These
coefficients are precisely given by the "Di(g)" n according to Eq. 32
L
Now, we have the choice to view "D, as either a real or a complex representation, but first we take the complex viewpoint and see it as a function "D; SO(3) - GL(CC++1). Notationwise, the following is important: the "," in "D; indicates that the elements in this matrix are real but does not tell us which space it acts. This will always be clarified by the context. We have the following:
Lemma E.S. DD: S3(3) + U(C2-11) is an irreducible unitary representation and isomorphic to Dr.
Proof. First of all. it is an actual linear representation since
"Digg'') = B Bi D( DIG') DD "Dig)) "Di(g")
where we used that D, is a linear representation. Now since Y," and ", are both orthonormal bases of (S*), the base change matrix B₁ needs to be unitary matrix. Consequently. "D(gg) B, B Di is product of unitary transformations itself unitary, which means that "D; is unitary representation. Furthermore, we obtain B "Di(g) = Di(g) B,, which means that B, gives isomorphism "D, z D; of unitary representations. From the fact that D, is irreducible, obtain that DD is irreducible as well.
Now we take the real viewpoint. Let "V 22
Lemma E.9. rD SO(3) O(VVi) is irreducible orthogonal representation.
82

---

Published as a conference paper at ICLR 2021
Proof. "gi(g) is unitary matrix for each g SO(3) by Lemma E.8 and since its matrix elements are real by Lemma E.7] it automatically is an orthogonal matrix. If it was reducible, then there would be real base change matrix that brings "D; in nontrivial block-diggonal shape. However, this base change would in particular he complex, meaning that we would conclude that the complex version of the representation "D; is reducible. But it is not, due Lemma E.8
Now, remember that ( - Diso VVn and that VV1 1 is generated from the real spherical har- monics. Also, remember that the real spherical harmonics transform in Eq. (32) Thus, with the same arguments as in Eq. (30) we obtain "Vi "V, which is from the preceding lemmas an irreducible orthogonal representation. Thus, we have found the Peter-Weyl decomposition of L (S*).
E.5.2 ENDOMORPHISMS OF ]VJ
In the next section, we will show that the "D, SO(3) O(VV)) -('V,) already given an exhaustive list of the irreducible representations SO(3) over the real numbers. this section, we first describe their endomorphism spaces since this will help in showing that there cannot be any other irreducible representations. Fortunately, the situation again simple:
Proposition E.10. (ndso((3).((V))
mimennmenio for each J 2
Proof. Let f TV, "Vy be an endomorphism. Since VV, 2R2 can view f as matrix in ((2++))((+++) That f is an endomorphism then means
f "D)(g) = "D>(g)- f
for all E SO(3). Now note that as real matrix, is in particular a complex matrix, i.e., f E ((2)+1)x22511) Also, remember that we can view "D., also as a complex irreducible representation "D, SO(3) U(C2J+1) by Lemma E.8 What this means is that / Endso03), c c(C2++1), which isomorphic to C by Schur's Lemma D.8 Thus, I is complex multiple of the identity. Since is a real matrix, it thus real multiple of the identity. The result follows.
E.5.3 GENERAL NOTES ON THE RELATION BETWEEN REAL AND COMPLEX REPRESENTATIONS
In the next section show that there can, up isomorphism, not be other irreducible represen- tations than the "D, SO(3) + O("V)) In order to do so, we first need to better understand the relationship between real and complex representations of compact groups. These investigations will carry over to the investigations for O(3) that we do in Section E.6 as well.
The following definition of classification of real irreducible representations of compact group G can be found in Bröcker & Dieck (2003). Theorem II.6.7. In this book, it is theorem, since the authors give an independent but equivalent definition of these notions.
Definition E. (Real, Complex, and Quaternionic Type Irreducible Representations). Let p: G O(V) be real irreducible representation of a compact group G. Then p is said to be of
1. real type if EndG,R (V) ) R,
2. complex type if EndG, (V) = C and
3. quaternionic type if (Vdg,n(V) = H, where H are the quaternions.
Here, these isomorphisms respect both addition and multiplication. The multiplication in the endo- morphism spaces thereby given by composition of functions.
Furthermore, Bröcker & Dieck 2003) shows in Theorem II.6.3 that there is other possibility for an irreducible real representation, i.c., they can be completely categorized by being of real, complex quaternionic type. Additionally, since R, C and H already differ in their IR R-dimension, it is enough to check whether the R-dimeession an endomorphism space is 1, or in order to do the classification.
83

---

Published as a conference paper at ICLR 2021
In order to compare real and complex representations need to define two functors between those
Definition E.12 (Restriction and Extension). Let "p: : G - G be a complex representation. Furthermore, let "p: G GL(VV) be real representation. Then we define their restriction and extension as follows:
1. Set r("V) as the Re-vector space that has the same underlying abelian group as VV and the scalar multiplication from R which is the restriction of the multiplication from C. The restriction r("p) G - GL(r(VV)) defined as the exact same map as Op, only that (g) : r(VV) r(vV) now viewed as an automorphism of real vector spaces.
2. We define the extension by e("V) = &R "V, where C is regarded as an R-vector space. This construction becomes C-vector space by scalar multiplication 2- @v) == (zz")@r. We can then define e(") p) G + GL(e("V)) by setting e(" (g) = idc (()())
Note that the extension operation doubles the IR-dimension, whereas for the restriction it stays equal. Therefore,
not hope that these operations are inverse each other. However, have the following, almost as nice statement:
Proposition E.13, For each real representation p + GL(V) there is a natural isomorphism r(e(V)) = V @ V of Re-representations.
Proof.
This is the first statement in Bröcker & Dieck 2003 Proposition I1.6.1.
The following definition is actually not the definition that Bröcker & Dieck (2003) formulate. How- ever, it is an equivalent characterization that follows from their Proposition 11.6.6 (vii), (vill) and (ix) and is more convenient for our needs:
Definition E.14 (Real Type Complex Representation). Let p G GL(V) be complex irre- ducible representation. Then p is called of real type there is an isomorphism real representations r(V) = U + U where
1. PU G GL(U) is an irreducible real representation and
2. r(p) G GL(r(V)) the restriction of p. as defined in Definition E.12 Proposition E.15. Assume G is compact group such that all complex irreducible representations are of real type. Then also all real irreducible representations are of real type.
Proof. This follows from Bröcker & Dieck 2003), Proposition I1.6.6 (ii) and (iii).
Proposition E.16. Let p:: GL(V) be an irreducible real representation of real type. Then its extension e(p) G - GL(e(V)) given in Definition E.12 is irreducible complex representa- tion (also of real type).
Proof. This is precisely Bröcker Dieck 2003). Proposition II.6.6().
E.5.4 THE IRREDUCIBLE REPRESENTATIONS SO(3) OVER THE REAL NUMBERS
The rough strategy is to use the fact that the "Di, viewed as complex irreducible representations, are an exhaustive list of all the complex irreps. Then, using the restriction and extension operators and between real and complex representations, we can show that in the specific case of SO(3), there can not be any other real irreducible representations than the "D,, viewed as real representations.
Lemma E.17. All complex irreducible representations of SO(3) are of real type.
We only detine these functors on objects and not on morphisms. The reason is that we will never explicitly use their definitions on morphisms. More details on this can be found in Bröcker & Dieck (2003). including other functors which are needed in the general theory. The reader should not worry if he or she does not know what functor is.
84

---

Published as a conference paper at ICLR 2021
Proof. From Section E.4.1 and Lemma E.8 we know that the "D, SO(3) U(C give up
to equivalence, all the complex irreducible representations of (3(3). According to Definition E.14 now need to understand that its restriction splits into the direct sum of twice the same irreducible real representation. We do this as follows:
We write r (( R + (IR)2++1 "Vi VVV, which is decomposition of C2/+1 when viewed as an R-vector space. Then, we can note that both
"D, : S0(3) - O(VV) and
"D, SO(3) O(iVV)
well-defined R-representations, which follows from the fact that the matrix elements all real. Furthermore, the first map is actually an irreducible real representation by Lemma E.9 The second one is isomorphic the first since
show that
i: VV 1' Vi, a-iia
an isomorphism of real SO(3) represettptsttnnn This gives precisely the splitting of r r(C2+1) as a representation that we were looking for.
Corollary E.18. All irreducible real representations of SO(3) are of real type.
Proof. This follows directly from Lemma E.17 and Proposition E.15
Proposition E.19, The "D₁ SO(3) + V())i) are, up to equivalence, all real irreducible repre- sentations of (3)..
Proof. Assume that p (O(3) GL(V) an irreducible real representation of SO(3). of
real type by Corollary E.18 By Proposition E. the extension e(p) G + GL(e(V)) is an irreducible complex representation. Since the "D₂ give us all complex irreducible representations up to equivalence by Section E.4.1 and Lemma E.8 there is an equivalence of complex SO(3)- representations e(V) = C21+1 for some Since functors respect isomorphisms (and equivalences are isomorphisms in the categories of G-representattonss) and the restriction operation is functor 32 and using Proposition 13 as well as the proof of Lemma E. we obtain:
V + V = r(e(V)) = r(( 'Ve 'Vi 4 VV.
Using the Krull Boull-Remak-Schiee Theorem B.39 we see that there is an isomorphism of SO(3)- representations V VV. This finishes the proof.
E.5.5 THE CLEBSCH-GORDAN DECOMPOSITION
We are almost there. The only thing left to understand is the Clebsch-Gorde decomposition. Re- member the following from Section E.4.3 decompositions
For the complex irreducible representations there are
V _
V
J=|1ー51
where each space, the representations D,, Dj and D, given by the Wigner D-matrices. Furthermore, the Clebsch-Gordan coefficients are all real. Now, we know that "Di is, as a complex representation, isomorphic by Lemma E.8 and such a representation then acts on 2 as well. Consequently, we also get the decomposition
221+1 21+1
2J41
J=ル|ー:
of the complex representations "D, and rD,, Obviously, the Clebsch-Gordan coefficients can be chosen to be exactly the same as before, and thus they are again real.
The reader does need know what functor is if he she believes these statements.
85

---

Published as a conference paper at ICLR 2021
Let the above isomorphism be called f. Now, view all involved vector spaces Revector spaces as well. Furthermore, we have subspaces "V, = IR2++1 'Vi = R and "V, = R which are also invariant under the representations " "D, and =D Consequently, restrict the isomorphism above to a map
just
H+
11 11'V, 8
J=1--i
which is well-defined since the Clebsch-Gordan coefficients are real. It needs to be injective, since a restriction of an isomorphism. For dimension reasons, the restriction then needs to be an isomorphism, and obviously, it has the exact same Clebsch-Gordee coefficients as the original map
35
E.5.6 BRINGING EVERYTHING TOGETHER
By what we've shown the last sections, we see that the situation is basically the same as in Section E.4.5 The only thing that changes is that therefore the complex conjugation disappears. What this overall means is the following: let "D, : SO(3) O('Vi) and "D,- SO(3) O("V,) be the representations determining the input and output fields. Then a basis for steerable kernels K : S² - HomR("V; "V,) is given by kernels K₃ S2 HomB ("Vi, VVy for all J| S
the real spherical harmonics, and
S 1 J. The matrix elements are given by
(z)(k)
(JMljm;bh) Inly (x)
(33)
E.6 ((3)-STEERABLE KERNELS FOR COMPLEX REPRESENTATIONS
In this section, we deal with ((3)-equivarit) equivariant kernels for complex representations and then, in the next section, will transport the results over to real representations. In the earlier examples, we saw that the Peter-Weyl decomposition of 2 (X) always contained each irreducible representation of the symmetry group exactly once. The example of 0(3) is the first in which this not the case: parity will play role in determining which irreducible representations make their way in the space of square-integrable functions and which do not. Overall, we hope that the example of 0(3) is sufficient justification for our of the multiplicities m, of irreducible representations that considered in all our theorems. -eq)iiqauva networks are to the best of our knowledge not described in any published work yet.
E. THE IRREDUCIBLE REPRESENTATION, OF O(3)
The most important observation is the following, after which we can deduce the irreducible repre- sentations (33) from those of (3):::
Lemma E.20. Let Za ((-1, +1),)) be the group with two elements. Then the map Le x SO(3) + 0(3), (s,g) +
an isomorphism of groups.
Proof. It is a group homomorphism since E {-1,+11 can be represented by multiple of the identity matrix, and as such it commutes with every matrix g. That an isomorphism follows since all matrices in O(3) either have determinant 1 or -1. The matrices with determinant form SO(3) and are the image of {+1] x SO(3). The matrices with determinant -1 are the image of {-1} SO(3).
Note the fact that for E SO(3), -g has determinant 11, which we used in the proof. This does only hold for g (O(d) with d being odd. Therefore, the above lemma is not true for d even. In the even case, we obtain semidirect product and the story complicates somewhat.
The reason for this is that the standard basis vectors in C* which are used for the Clebsch-Gordan coeffi cients exactly the standard basis vectors in R C* by definition of this embedding.
86

---

Published as a conference paper at ICLR 2021
Earlier, already considered tensor product representations of one and the same group. A related notion that tensor product representations two different groups
Definition E.21 (Tensor Product Representation). Let G and H be two compact groups. Let PG - GL(Va) and pn H + GL(VH) be representations of the two groups G and H. Then the lensor product reprexentation given by
PG pn G - GL(Ve @ Vn). [pG @ pH) hg (vg uH) po(l)(vc) @ (H)(((vH).
This is again linear representation.
Proposition E.22. Representatives of isomorphism classes of irreducible representations of x H are given precisely by all the pG pp, where pG and pn run through representatives of isomorphism classes of irreducible representations of and H, respectively.
Proof. This proven in chapter II, Proposition 4.14 and 4.15 of Bröcker Dieck (2003).
It is important to note that the proof of the above proposition uses the property of the complex numbers to be algebraically closed crucial steps, and therefore is unclear how exactly gener- alization representations over the real numbers looks like. Therefore, we will not use the above proposition in our later considerations for real representations of O(3).
However, in our current situation, we can apply it without problems. This proposition, together with Lemma E. suggests that we should understand the irreducible representations of Zz. We already saw this for real representations before and essentially obtain the same result:
Lemma E.23. The irreducible representations of Zz are up equivalence precisely the following two, which we state for simplicity only on the generator:
P+ Zz > GL(C), p-(-1) idc
p_ L2 + GL((C), (-(-1) = -ddc
Proof. This can be shown in exactly the same way as in Section E.3.1
Thus we are ready to state our result about the irreducible representations of 0(3):
Proposition E.24. The irreducible representations O(3) are up to equivalence given follows: for each 1 E IN 2o there are precisely two representations D4 0(3) - U(V++) and D (() U(V-) with V1+ = 2 Vi-, given as follows:
(sg) Di(g) for all E Z, g SO(3).
Di_ (sg) (g) for all s E Zz, E SO(3).
Proof. Remember from Section E.4.1 that the irreducible representations of SO(3) are given by the Wigner D-matrices D,. From Lemma E.23 know that the irreducible representations of Eg are given by pa and p.. From the isomorphism 0(3) Lo x SO(3) from Lemma E.20 and from Proposition E.2 we thus obtain that the irreducible representations of O(3) are precisely given by all p+ & D and p_ & D,. We now show that p- @ D, is equivalent to D We have
P- 8 & Dj 0(3) - GL(C & Vi), [pp= Di) (z @v) sz [D(g)] (o).
Now, consider the linear isomorphism f: C @ Vi V4+, 00 We only need to check that is equivariant and are then done:
s(le~ Di)(sg) (= v)) f(sz (Di)))(()))
[Di(g)] (v)
= [sDi())[<zv)
(D) @ v)).
The statement about D24 can be shown using the exact same map f-
St is direct generalization due the presence of two different group elements heing applied.
87

---

Published as a conference paper at ICLR 2021
E.6.2 THE PETER- WEYL THEOREM FOR (59) REPRESENTATION OF O(3)
The considerations in this section follow almost entirely from Section E.4.2 There we saw that, as representation over S0(3), we have decomposition
θ Va
with the spaces Vni being spanned by the spherical harmonics Y,", n -u,.....,4 We immediately see that in (5*), viewed as a representation over O(3), there is not enough space for all the irreducible representations, since they appear in pairs as shown in Proposition E. 55 Thus, we need figure which irreducible representations are present and which are not. The of this question is answered by the following proposition:
Lemma E.25 (Parity in spherical harmonics). The spherical harmonics obey the following parity rules:
(xrx)) = (x)
for all 2 -1,.-1,..... .1, E Z2 and x E 8
Proof. This is well-known property of the spherical harmonies.
Thus, together with Section E.4.2 get the following transformation behavior of spherical har- monics under the group O(3), where E Z, and g E SO(3):
(Vsg)( = ((g)))
Σ
"
Σ (s ()))"())
- (sg"
I even
Σ E Di"" ood.
Thus, we obtain the following decomposition of L2(())):
LA
VI++
Vn
120
120 I
Here, Vn+ and Vn are generated from the spherical harmonics of order and we have Vn4 Vi+ and Vn- 2 Vi_ representations according the transformation behavior we saw above.
E.6.3 THE CLEBSCH-GORDAN DECOMPOSITION
Remember from Section E.4.3 that we have a decomposition of reeeeeetttssssntaation
V, V
V
J=||ー
given by real Clebsch-Gordan coefficients. Now for 0(3), remember that vector spaces we have all j (and equally for 1 and J) equalities V, = V₁- Vj+. and so we guess that in the iso- morphism above, we just need to figure out the correct signs in order to be compatible with the
With this, mean the following: the irreducible representations of SO(3) already cover (.2 0(3) has irreucchlee representations than SO(3), it priori clear that they cannot all fit into (S*).
88

---

Published as a conference paper at ICLR 2021
corresponding representations. The idea is that "multiplying the signs the left" should lead to the "sign at the right", and this paradigm leads us to believe that there are the following isomorphisms:
L+
VJ+ 8 V4+
=
VJ+1
V3+ @ Vi
Vz-
J=K-=
J=
V V4+
Vi-,
V 2 Vi_ =
VJ_.
J=Hーン
We just show the lower-left isomorphism since the arguments are always the same. So, assume that Vi Ø= Vj is isomorphism and thus in particular intertwines the given representations. Now, take the exact same map VVj- Vi+ figure out that equivariant with respect the given representations, using the same property for the original isomorphism we started with:
ト+) J=([-jl V,- and only need
f [D (sg) & Dut (s0) = o [>(D)(() & D((g))
+
D)
J=
+
(s-
-= |
This shows the claim. From these considerations, it also follows that the Clebsch-Gordan coef- ficients do not in any way depend on the signs of the spaces V,, Vi, Vj. Thus, we write them generically as (JM (im; In).
E.6.4 ENDOMORPHISMS OF JJ
As always over C, Schur's Lemma D.8 shows that the endomorphism spaces are 1-dimensional, and thus we can ignore endomorphisms.
E.6.5 BRINGING EVERYTHING TOGETIIER
Now we can finally compute the basis for steerable kernels. The section on the Clebsch-Gordan decomposition suggests that we need to do case distinction for this, Namely, the possible kernlls depend on the signs of Vi and V,. The results basically follow analogously to the results Section E.4.5
STEERABLE KERNELS K S2 + Homo(Vi+, Vs+):
VJ+ only be in tensor product V, Vi+ if the sign of is positive. Spaces V ++ appear in the tensor product decomposition of (52) precisely for even j. according to Section E.6.2. Thus, a basis for steerable kernels is given by all K, with even j E {12 Jl...., / elements
J. It has matrix
(z) = Σ (JM(jm; In) YTm
exactly in Section E.4.5
STEERABLE KERNELS K 2 + Home ClomcVit. V)-):
Analogously, a basis for steerable kernels is given by all Ky, with odd j E {17 J|,..., J}.
STRERABLE KERNELS K: 22 + Home(Vi-, Vy+):
Again, basis for steerable kernels is given by all Kj with odd j {11 JJ...., J}.
89

---

Published as a conference paper at ICLR 2021
STEERABLE KERNELS K 8 Homa(Vi,, Vy-):
As in the first case, a basis for steerable kernels is given by all Kj with even j E {II-J....,I+))
Thus, we have determined all kernel bases for the group ((3) over the complex numbers. Compared to SO(3), we see that the kernel spaces get roughly halved. The reason for this is that with bigger symmetry group, the kernel needs to obey more rules, which means that the kernel constraint has fewer solutions.
E.7 0(3) (33)-TEERAABL KERNELS FOR REAL REPRESENTATIONS
Basically, argue exactly as in Section E.5.4 in order transport the results for complex rep- resentations over to the real world. We shortly sketch the procedure and outcome. As we know from Section E.3.1 P- Zg - ((R) and p+ Z₂ ((R) are the only irreduciile real representations of Z2. Thus, for each 2 we obtain two irreducible real representations "D : ((3) + O("Vi+ and "D,- 0(3) O(VV-). As before, they also act complex vector spaces and are such isomorphic to the complex irreducible representations of O(3). One can then show as in Lemma E.17 that all complex irreducible representations of the real version of this representation. Thus, by Corollary E.18] all real irreducible representa- tions are of real type, and this means that we can proceed exactly as in Proposition [E.19] in order to show that the "D1+ and "Di- are already all the irreducible real representations of O(3) up to equivalence.
of real type since they split into copies
For the Peter-Weyl Weyl decomposition of L (S²). we only need to note that the real spherical harmonics emerge with base change from the complex ones, as seen in Eq. (3T), and thus fulfill the same parity rules as the complex spherical harmonics. This gives us a decomposition
(S
('Vn+)
("Vn-).
120
120
For the Gebssch-Gordan coefficients, we again get decompositions
I+j
"V, & V
~
VV
J=||-ン
where the signs on the left must "multiply to" the signs on the right, as Section E.6.3 Finally, the endomorphism spaces must be 1-dimensional since the endomorphism spaces of the complex versions are 1-dimensional.
Overall, obtain the same kernels as in Section E.6.5 only that we need to the real spherical harmonics as our steerable filters and can get rid of the complex conjugation.
F MATHEMATICAL PRELIMINARIES
In this chapter, state mathematical preliminaries that we use throughout the earlier chapters. In this whole chapter, IK is one of the two fields IR or C.
F.1 TOPOLOGICAL SPACES, NORMED SPACES, AND METRIC SPACES
Since in this work, we want to develop the theory of representations over compact groups, and since this is topological property, need to formulate some topological concepts (Conway) 2014). Additionally, the vector spaces on which our compact groups act also carry a topology, mostly coming from their Hilbert space structure.
Definition F.1 (Topological Space, Open Sets, Closed Sets). A topological space (X, T) consists of a set X and set T of subsets of X, called the open sets, such that arbitrary unions and finite intersections of open sets open. In particular, X and the empty set are open. Closed sets the complements of open sets and fulfill dual axioms: arbitrary intersections and finite unions of closed sets are closed.
90

---

Published as a conference paper at ICLR 2021
Let in the following X and Y be topologiaal spaces.
Definition F.2 (Open Neighborhood), Let x E X. An open set U _ X is called open neighborhood of if U.
Definition F.3 (Hausdorff Space), X is called Hausdorff space if two distinct points can always be separated by open sets, i.e., for all 2, y E X there exist Uz, Uy open such that Uz. y Ug. and U, O UUy- 0 0.
In this work, all topological spaces are Hausdorff.
Definition F.4 (Subspace). Assume A X is subset. Then the set TA topology for and thus makes a topological space as well. is called a subspace of X.
(U A U T) is
Whenever we consider subset topological space, it is viewed as a topological space with this construction.
Definition E5 (Closure, Density). For A S X, its closure A is defined as the smallest closed subset of X that contains A. Equivalently, it is the intersection of all closed subsets of X containing A, which is closed by the axioms a topology. A is called dense in X Definition F.6 (Continuous Function, Homeomorphism). A function f X Y is called continu- ous if preimages of open sets are always open. Equivalently, for each point Øg E X and each open neighborhood V of f (zo) there is open neighborhood U of zo such that f(U) C V.
X.
homeomorphism is continuous bijective function with continuous inverse.
Note that compositions of continuous functions continuous well.
Definition F.7 (Open Cover, Compact Space), An open cover of X is a family of open sets hee that cover X, i.e., =Uses Ug. X is called compaet if all open covers have finite subcover, that is: For all open covers {Ut)er there exists finite subset J such that (UikeJ is still open cover of X.
Proposition F8. If X is compact and f X is continuous, then S(X) S Y compuct as well. In particular, if surjective, then Y is compact.
Proof. See Sutherland (1755), Proposition 13.15.
Proposition F.9. Let f - Y be a continuous bijection and assume that X is compact and that Y is Hausdorff. Then the inverse f-1 is continuous as well and thus I is homeomorphism.
Proof. See Sutherland 1975), Proposition 13.26.
Definition F.10 (Product Topology). The product topology on X Y is the coarsest (i.e., smallest in terms of inclusion) topology that makes both projections px X x Y - X and XxX Y continuous.
If Z is third topological space and we have two continuous functions fx :Z X and fy Z Y. then the function fx x fy Z + x Y, 2 + (fx()),frzz)) is continuous as well.
Definition F.11 (Quotient Map, Quotient Space). A continuous function f X Y is called quotient map if f is surjective and if U C Y is open if and only if (U) S X is open.
Let be any equivalence relation X and X/> be the quotient set formed by identifying equiva- lent elements. Let q: X + X/> be the canonical function sending each element to its equivalence class. We define U C X/> to be open if q -U)) C X is open. Then is quotient map and X/> is called quotient space.
Proposition F.12 (Universal property Quotient Maps). Let q + X/~ be a standard quotient map and f X + Y be any continuous function such that f(x) = f(x') whenever x'. Then there is unique continuous function f X X/~ - Y such that the following diagram commutes:
X
Y
X
91

---

Published as a conference paper at ICLR 2021
f is given equivalence classes by f((x)) = fx)).
Proof. See Conway 2014), Proposition 2.8.7.
can be shown that all quotient maps are equivalent to a construction of the form q X + X/> Namely, for a quotient map f X + Y, define by ~ x' if f(x) f(a'). Then the map f: X//> + Y. [x] f(x) is well-defined continuous map by the universal property of quotient maps Proposition F.12 One can show that this is a homeomorphism. Thus for a quotient map f: X Y we also call Y quotient space.
Our route for defining concrete topologies is in most cases through the existence inner products on Hilbert spaces, which will be defined in detail in Definition F.32 Namely, inner products define norms, which define metrics (Kaplansky. 2001), which in turn define topologies. For this, we need some definitions:
Definition F.13 (Norm). Let V be I<vector space, A norm on V is map || ||: V + R20 with the following properties for all À K and u, w E V:
1. |||||| = 0 if and only if v 0.
2. |Auil |^|
||=|||
3. Triangle inequality: |/v + will I ||a||
||w]|-
If (-1-)= V + K is an inner product on a Hilbert space, then it defines norm |||||: V + R20
by ||=||| = / (z/x).
Definition F.14 (Metric). Let Y be a set. A metric on Y is function d: Y x + R2o with the following properties for all z, y, 2 E Y:
1. d(z, if and only if y.
2. Symmetry: d(z, y) d(y, x.).
3. Triangle inequality: d(x, =) d(z,v) v) + d(y,=).
A norm || R>o defines a metric d : V x V + I by setting d(x, = || yl| In turn,
metric defines topology as follows: open balls are given by all sets of the form (z = {y V d(z, y) < E) for all x E V and > 0. Open sets are then defined as arbitrary unions of arbitrary open balls.
Additionally, we need notions about convergence in this work. Since we will deal with them mostly in the context metric spaces (with normed vector spaces and Hilbert spaces being special cases, as explained above), we focus on these notions for metric spaces.
Definition E.15 (Convergent Sequence). Let Y be a metric space. Then a sequence (gN)) in Y is said converge y if for all
there k; NN such that ys (y) for all > kr
With this in mind, one can give an equivalent definition of continuity that applies to metric spaces: Definition F.16 (Continuity in Metric Spaces). function Y - between metric spaces is continuous in y Y if for each sequence (yk) of points y/ Y converging to a point y Y, we also have that the sequence f()*) converges to f(y). This can be understood in terms of the function "commuting with limits":
lim f(yx) = f( lim yk). A R00
A000
Furthermore, Y + Z is called continuous if it continuous in all points y E Y.
Equivalently, the following holds: f Y + Z continuous in Y if and only of for all >> 0 there is such that (Bs (y)) C ()(())
Definition F.17 (Uniform Continuity). A function f Y + Z between metrie spaces is called uniformly continuous for each e there a 8 > 0 such that for all y E Y with dy (v. (y.8") < obtain (f)) <.
92

---

Published as a conference paper at ICLR 2021
The following is result several times in the main text:
Proposition F.18. Let f V - V be a linear function between normed vector spaces. Then the following equivalent:
1. f is uniformly continuous.
2. f is continuous.
3. f is continuous in O.
Proof. Trivially, implies which in turn implies 3. Now assume 3, i.e., is continuous in 0. Let
> 0. Then by continuity in O, there exists 5 > 0 such that for all V with ||a]| = 1
obtain [l]( fIf(v) f(O)]l Now let v, o' E be arbitrary with || o' < 6. Then
by the linearity of f we obtain:
(lf(w) - f(O') 1f(u -
which is exactly what we wanted to show.
Sometimes, sequences look like they converge since their elements get ever closer to each other. However, not all such sequences need to converge. Therefore, there is the following notion:
Definition E.19 (Cauchy Sequence). Let be metric space. sequence (y)) ). Y a Cauchy Sequence if for all 0 there is k2 IN such that for all k. k'' > lg we have d(ys ywe
For example, one can consider metric space R \ {0} together with the usual metric. Then sequence (L), 0, not in IR \ (0). Thus, the following notion is useful:
Cauchy sequence but does not converge since the limit (in R!), which would be
Definition F.20 (Complete Metric Space). A metric space Y is called complete if every Cauchy sequence converges.
Definition F.21 (Completion). Let Y be metric space. A completion of Y is metric space Y which contains Y as dense subspace and such that Y' complete.
Proposition F.22 (Universal Property of Completions). Assume that Y C Y' is pair of metric spaces, where Y a completion Y.Y. Then the following universal property holds:
Let Z be any complete metric space and f Y - Z be any uniformly continuous function. Then there is a unique continuous function f Y > Z that extends f, iee, such that f'ly = f. f furthermore is also uniformly continuous. This be expressed by the following commutative diagram, where i Y" is the canonical inclusion:
Y
yr
Proof. See, for example, Kaplansky (0001)
Definition F.23 (Boundedness). Let be metric space. subset A S V is called bounded there is constant C > 0 such that d(a, b) & C for all a, A.
Theorem F224 (Heine-Borel Theorem). A subset A Kd is compact if and only if it closed and bounded.
Proof. See Conway 2014), Theorem 1.4.8.
Corollary F.25 (Extreme Value Theorem), Let : X + compact topological space. Then f has maximum and a minimum.
be continuous, where X is any nonempty
Proof. By Proposition F.S. f(X) C R is compact. By Theorem F224 this means that f(X) is closed and bounded. Boundedness means that the supremum is finite and closedness means that the supremum must lie in f(X), and consequently it is a maximum. For the minimum, the same arguments apply.
93

---

Published as a conference paper at ICLR 2021
F.2 LIMITS OF NETS AND APPROXIMATED DIRAC DELTA FUNCTIONS
In this section, we discuss "limits of nets". where a net can be imagined as a sequence over an index set which may be "too big to be handled sequence over the natural numbers". They appear in the formulation of Theorem C.7, This material can, for example, be found in (Conway 2014).
Definition F.26 (Partially Ordered Set, Directed Set). Let be an index set and - a relation on it. 1= (1, S) is a partially ordered set if:
1. S is reflexive, i.e., i S for all i E T.
2. S is antisymmetriic, that is: i j and together imply i j.
3. S is transitive, that is: j j and together imply S
A partially ordered set / is called directed if for all i,j E / there exists & E I such that i S k and
j I
Example F.27. Clearly, the natural numbers together with the standard order relation form directed set.
important example for our purposes is the following: let be any topological space (for example, a homogeneous space X of compact group G) and E Z be any point. Furthermore, define U2 as the set of open neighborhoods of z, i.c., open sets [ Z such that æ e U. On this set, we define U V if 2 V, i... by reversed inclusion. Then (Ug, x) is directed set:
1. Reflexivity is clear since V 2 V for all V.
2. Antisymmetry is clear since U 2 V and V 2 U together clearly imply U = V.
3. Transitivity is clear since U 2 V and V W together clearly imply U W.
4. For directedness, let V Uz Define W = = Then W E Ug and clearly U W and V 2 W, which is what was to show.
Note that Ur is usually not totally ordered, i.e., there are usually U, V E Ug such that neither U 2 V nor V 2 U.
Definition F.28 (Net). Let Z be any topological space and / directed set. Then a net in Z is function x : + Z. We write net as (2,)kez, 1. in analogy to sequences.
Definition F.29 (Convergence of Nets). Let (a:)ser net in a topological space Z. Let x E Z. if the following holds: for all open
We say that (xi))ez converges to x, written limaer neighborhoods of x there is an io such that for 2 io we have x; E U.
Now we define the approximated Dirac delta for the special case that is homogeneous space of a compact group G. Remember that there is Haar measure on X.
Definition F.30 (Approximated Dirac Delta), For 0 + U C X open, we define the approximated Dirac delta by de X + K with
MUJ EU
(0)))
M(U) l0(x))
0, else.
We have dy E (X).
A priori, it is unclear that open sets have positive measure, which is needed for the well-definedness of this construction, since otherwise we divide by zero. Thus, we need the following lemma:
Lemma F31. Let 0 & U X be an open set. Then ((U)² > 0.
Proof. Consider the family of open sets (gU) That all these sets are necessarily open follows since the action G x X X is continuous, and thus by the definition of group action, each g E G induces a homeomorphism X + X, + gr. Now. since the action is transitive, (gU)) is an open cover of X, and since X is compact, see Definition E.7 it has open subcover (g:U), / with
94

---

Published as a conference paper at ICLR 2021
9: E G. Note that U((g,)) (()) for all i since the measure μ X is by definition left invariant under the action of G. Overall, we obtain
Σ u(g.U) Σ "
1 ==x(x)= = p
U 9:U
u(U)= = (U))
and thus udU) 2 0.
F.3 PRE-HILBERT SPACES AND HILBERT SPACES
Here, we state foundational concepts in the theory of Hilbert spaces (Debnath & Mikusinski (005).
Definition F.32 (pre-Hille Hilbert Space, Hilbert space), A pre-Hillert space V (V, (-1-)) consists of the following data:
1. A vector space V over K.
2. An inner product (-I-) V X - K, (z, + (zly).
has the following properties that hold for all x, x' U.y. E eV,
1. The inner product is conjugate linear in the first component: (x + x (aly) (xllv) and (Aælly) X (xly), where is the complex conjugate of A.
2. The inner product is linear in the second component: (æly + v²) (xly) (xrle) and (z/Ay) = x (zly).
3. The inner product is conjugate symmetrie: (y/ar) (xly)
4. The inner product is positive definite: (ar(z) unless x O.
If additionally, the following statement holds, then V is called a Hilbert Space:
5. V, together with the norm ||-1|= V + V induced from the inner product by |>||| = (x(a)) and consequently the metric defined by d(x, x y/|. complete metric space as in Definition F220
Remark F:33. Of course, all Hilbert Spaces are pre-Hilbert spaces, and so all Propositions about pre-Hilbert spaces in the following apply to Hilbert spaces just as well.
Note that the first property follows from the second and third. We also mention that usually, inner products on Hilbert spaces are assumed to be linear in the first and conjugate linear in the second component, contrast to how we view it. The reason for our choice is that our work is inspired by connections to physics where convention is more common. is basically the bra-ket convention. Furthermore, note that K = R. then conjugate linear maps are linear and thus the inner product will be linear in both components. Additionally, it will be symmetric instead of only conjugate symmetric.
Proposition F.34 (Cauchy-Schward) Inequality). For any two elements v, w in a pre-Hilbert space V, we have
(wlw)) | ||a|| ||
We have equality if and only v and w are linearly dependent.
Proof. See Debnath Mikusinski (005), Theorem 3.2.9.
Definition E.35 (Orthogonality). Two vectors v, in Hilbert space V are called orthogonal, written v 1 w. if( = 0.
Obviously, being orthogonal is symmetric relation.
Definition F.36 (Orthogonal Complement). Let V be a pre-Hilbert space and W C V a subset. E V is orthogonal to W if (v w) 0 for all W.
The orthogonal complement of W, denoted W the set of all vectors V that are orthogonal to W.
95

---

Published as a conference paper at ICLR 2021
Proposition F.37 (Closedness of Complements). Let W S V be subset of a pre-Hilbert space V. Then W is a topologically closed linear subspace of V.
Proof. See Debnath Mikusinski (2005) Theorem 3.6.2.
Proposition F.38 (Continuity of Scalar Product). For any pre-Hilbert space V. the scalar product V +
K is continuous.
Proof. See Debnath & Mikusinski (005). Theorem 3.3.12.
Definition F.39 (Orthonormal System). A family (Us)iez of clements in pre pre-Hil space is called orthonormal system if ||s||| = for all E and U; 1. My for all j-
Definition F.40 (Orthonormal Basis). An orthonormal system (ot)sz in Hilbert space V is called orthonormal basis if the linear span of all (urker is dense in V. If this is the case, then each er E V can be uniquely written as
EET
with only countably many ag E K being nonzero. The coefficients given by a; (w/lu).
We stress that while the index set can be uncountably infinite, the sequence expansions of each element in V only have countably many entries. It is obvious from the Peter-Weyl Theorem B.22 and this definition that the functions
Y 1 1 ()) E (1......d] di}
form an orthonormal basis of (X). Proposition F.11
Orthonormalization) For every linearly independent sequence (yx )N in a pre-Hillerr space V with N IN (o0) elements, one can find orthonormal sequence
(ex) & in V such that the following holds: for all n E IN, S N, the progressive linear span stays the same:
spank vn) spang (B
In particular, since every finite-dimensional Hilbert space has vector space basis, it necessarily also has orthonormal basis.
Proof. See Debnath & Mikusinski (2005), page 110.
Definition F.42 (Adjoint of an Operator). Let f + V" continuous linear function between
Hilbert spaces. Then there is unique continuous linear function f* V + V such that for all v E and o' V' one has:
(f())lu)) (v))
f is called the adjoint of
The existence of adjoints is, for example, discussed in Debnath & Mikusinski (2005), page 158. This book only considers the case of operators on Hilbert space to itself, but these considerations generalize to the setting with two different Hilbert spaces. One has the following:
Proposition F.4.. Let f Hilbert spaces. Then:
V and g V + V" be continuous linear functions between
1. (f*)* =
2. idl = idy.
3. (g f)" = f
Proof. All of these properties follow directly from the uniqueness of adjoints.
Proposition F.4. Let f V. + V be unitary transformation between Hilbert spaces, i.e., invertible linear function such that (f(v))((w) = (wln) for all v, w E V. Then the adjoint the inverse, iee, l = f-
96

---

Published as a conference paper at ICLR 2021
Proof. First of all, the inverse f-1 is again continuous due to the unitarity of f. Furthermore, due to the unitarity, we obtain
= = (fo])(
= (olff
for all u E V and v' E V'. Due to the uniqueness of adjoints, we obtain f f
The following proposition is sometimes used the main text:
Proposition E45. Let v, E V be two elements in pre-Hilbert space such that (ulu) = (wlu) for all u V. Then v = w.
Proof. We have
(0 w/u) = (wlu) - (w/lu) = for all u E V. In particular, when setting = we obtain
(u wi v w) 0
and thus v w i... v w.
Proposition E46 (Orthogonal Projection Operators). Let W S V be topologically closed sub- space of a Hilbert space. Then there is a continuous linear function P - W such that for all E V and W have
(P(v)lw) = (vlw).
Furthermore, if W is finitteddmmensional and Wi...., Wn and orthonormal basis, then P is given explicitly by
P(o) (wale) wy.
Proof. That W is topologically closed means that W, with the scalar product inherited from V, complete metric space. Thus, W is Hilbert space well. Therefore, the continuous linear emhedding W ~ V given by w + to has an adjoint i" V + W by Definition F.42 Set P i" For arbitrary E V and w W obtain:
(P(u)lw) (v)())
(uli(w))
(ulw).
For the second statement, note that for all E {I,...,, n we have, using that the are orthonormal:
(wi/w) w/ 203) = >n (walo) (welwry)
(vlwy)
(P(x)) (())))y))
By Proposition F.45 and since the wy generate W we obtain En (w)/ wi P(v) as claimed.
Proposition F.47. Let (V, (--)) be a fifiite-dimmesiinnll pre-Hilberl space. Then this space is already complete and thus Hilbert space.
In particular, all finte-ddimesoonal subspaces of Hilbert spaces are topologically closed.
Proof. The proof of the Gram Sehm-Shhmidt orthonormalization in Proposition F.41 does not make use of the completeness of the Hilbert space, and thus it holds for pre-Hilbert spaces as well. Consequently, V. being dimen--dmmnsional,, has an orthonormal basis. is thus isomorphic K together with the standard scalar product, which is well-known to be complete. Thus, V is Hilbert space.
Now, let W C V be Enite-dimensional subspace of Hilbert space V which may be infinite- dimensional. Then W is pre-Hilbert space and by what was just shown a Hilbert space. Con- sequently, all sequences in W which have a limit in need, by completeness, to have that limit already in W. This shows that W is topologically closed.
97


Reviews:
[{"review_id": "Z0qt_Wp_kMK", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe authors prove a theorem (thm 4.1) which describes a basis for the space of kernels in a G-steerable CNN for any compact group G.  Steerable CNNs are similar to CNNs but replace channels with G-reps and enforce an equivariance constraint on the kernels.  Though Cohen et al 2019 state the constraint, and Cohen et al 2019, Cohen & Welling 2016, and Weiler & Cesa 2019 and several other papers solve this constraint for different groups and representations, there has not been a general formulation which applies to all compact groups.  Here, solving the constraint means to construct a basis of the space of steerable kernels.  Any steerable kernel is then a linear combination of this basis and the network can then be trained by learning the coefficients.\n\nThe problem of finding a basis for the space of steerable kernels is non-trivial and critical for constructing G-steerable CNNs.  Up until now this has done group by group.  The theorem proved in this work unifies such previous efforts and provides a useful method for approaching further G.  This paper is a significant contribution to the field.\n\nThe appendix provides a complete and approachable background in the area as well as detailed and precise proofs.  Moreover, the effort by Cohen & Welling to frame equivariant deep learning in the proper context of representation theory is continued and extended here to good profit.  The appendix is quite verbose and the language is more casual than I am accustomed to in a mathematical text or research paper, but it serves the goal of being didactic and approachable.\n\nA practical consideration remains. Though theorem 4.1 reduces construction of a basis of steerable kernels to 1) finding Clebsch-Gordon decomposition of tensor products, 2) describing endomorphisms of irreps, and 3) describing harmonic functions, none of these problems is trivial (or even necessarily solved) for a general compact group G.  That said, Appendix E does a good job providing evidence that this can be done for many individual groups.  However, in that case, we are still back to solving the problem on a group by group basis.    \n\nSpecific Additional Points:\n1.My opinion is that the language of physics does not add to the paper.  While Clebsch-Gordan and harmonic functions first arose in physics, they can be described in terms of representation theory.  In this way, both steerable CNN and quantum mechanics are applications of rep. theory and so it is not necessary to use physics here to describe steerable CNN.\n2.Page 5, the notation $[j] = dim(V_j)$ seems unusual to me. It would be better to use something more standard.  In particular, in Defn 3.5, brackets are used as parenthesis, making this more confusing.\n3.Page 5, the fact that input and output representations $V_{in}$ and $V_{out}$ decompose into irreps does not immediately explain how to construct a steerable kernel basis for $V_{in} \\to V_{out}$ given ones between irreps $V_i \\to V_j$.  Though it is not complicated, I would include this.\n4.Page 5, I was confused by the inclusion of $End_{G,K}(V_j)$ at first since it does not appear when working over $\\mathbb{C}$ due to Shurr’s lemma.  It is explained in the paper and more so in the appendix that it is necessary over $\\mathbb{R}$, but it could be a bit clearer and earlier in the paper.  Namely, you could note that over $\\mathbb{C}$ $End_{G,K}(V_j) = \\mathbb{C}$ and give the possibilities over $\\mathbb{R}$.\n5.Page 5, Thm 3.4 and Page 43, Thm C.7.  Given that the purpose of this paper is partially to formulate Steerable CNN in precise terms, the fact that $\\delta_x$ is informally considered as in $L^2(X)$ is very imprecise.  Not only does this make the proof informal, it means the maps in the theorem are not even defined.  Can you replace $L^2(X)$ with an appropriate space of distributions in order to make the statement precise?  \n6.Page 6, “which is zero for almost all J” should be “which is zero for all but finitely many J”\n7.Appendix E, $U(1)$ is isomorphic to $SO(2)$, so it is strange to use both notations.  The difference between these subsections is whether the representations are real or complex.\n\n**Updates from Author Feedback**\nWhile I would have liked to see a draft with the changes, I feel reasonably sure the authors will improve Appendix C to make the statements mathematically precise. I am confident in the statements and proofs. While the presentation can be verbose and casual, I think it is justified to increase accessibility, so long as the proofs and statements are formal and precise. Based on Author responses I have increased my confidence.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "SxdbfEU11Vr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers Group Equivariant Convulation Neural Networks (GCNNs) which are convolutional neural networks that are equivariant wrt group symmetries of the underlying space. The equivariance requirement places constraints on the parameterization of the corresponding CNN. This work extends previous results for particular symmetries and outlines a method for obtaining these parameterizations for any compact group symmetry.\n\nTechnically, the paper establishes a Wigner-Eckert Theorem for G-steerable kernels, which in turn allows any admissible kernel to be expressed using a basis of kernels thereby establishing a natural parameterization. This procedure is carried out for U(1), SO(2), SO(3) among others.\n\nThe paper highlights important ideas from representation theory that can be used to obtain paramaterizations for symmetry-constrained learning models, and the mathematical methods could be of independent interest. The results obtained here are significant for any learning problem where there are inherent natural symmetries, as the authors point out this could be especially beneficial for data arising from physical processes.", "extra_scores": {}, "extra_text": {}}, {"review_id": "I5n5q2DzbR", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper under review is a very technical contribution to the study of group-equivariance of convolution kernels. The problem of group-equivariance is studied in the most general setup, thus encompassing the previous achievements of Cohen and Welling, Cohen, Geiger, and Weiler, Weiler and Cesa, etc.\n\nThe most general tools from classical harmonic analysis and Lie group representations are put to work in order to provide the most general framework for the analysis of equivariance. \n\nI learnt a lot about the mathematics of equivariance, a very interesting topic. However, I wonder wether the ICLR conference is the most appropriate venue for such thorough study. The application section is in particular way too sketchy to convince the novice that this impressive work will be useful to the machine learning community and an effort in this direction should be made to clarify the expected impact. \n\n \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "A4md1U3-RKb", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "For a physicist familiar with machine learning, the title of this work says it all.  It is a lengthy explanation of how to use well known techniques from physics in constructing convolutional neural networks with a group symmetry.  The paper also spells out what are likely to be the most used cases of U(1) and SU(2) and their quotients by discrete groups.\n\nI give the paper high marks on the value of the topic it addresses, but middling marks on presentation.  The results I spot checked were correct, but the paper reads like a compilation of material from the many math and physics textbooks on the topic, and lacks the coherence of a good textbook.  If I had a student who needed this material I would instead give him or her the original Cohen Welling paper and a representation theory textbook such as Hall 2015 or one of the several textbooks they cite.  In addition the explicit results of appendix E would be rather more useful if implemented in a ML package, and I hope that is on the author's to-do list.\n\nTo summarize, a serviceable reference work which will probably be made obsolete by the appearance of a proper textbook on group theory in ML before long.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers generally agree that the main result of the paper, which generalizes the classical Wigner-Eckart Theorem and provides a  basis for the space of G-steerable kernels for any compact group G, is a significant result. There are also several concerns
that need to be addressed. R4 notes that the use of the Dirac delta function (e.g. Theorem C.7) is informal and mathematically imprecise and needs to be fixed. R1 notes that it would be helpful to at least describe how this general formulation can be applied in machine learning.

Presentation and accessibility: the current version of the paper will be accessible to only  a small part of the machine learning audience, i.e. those already with advanced knowledge in mathematics and/or theoretical physics, in particular in representation theory. If the authors aim to make it more accessible, the writing would need to be substantially improved.
Decision: Accept (Poster)