Efficient Generalized Spherical CNNs
Authors: Oliver Cobb
Christopher G. R. Wallis
Augustine N. Mavor-Parker
Augustin Marignier
Matthew A. Price
Mayeul d'Avezac
Jason McEwen
Venue: ICLR 2021 Poster (2021)

Abstract: Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries.  We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other.  The only existing non-linear spherical CNN layer that is strictly equivariant has complexity $\mathcal{O}(C^2L^5)$, where $C$ is a measure of representational capacity and $L$ the spherical harmonic bandlimit.  Such a high computational cost often prohibits the use of strictly equivariant spherical CNNs.  We develop two new strictly equivariant layers with reduced complexity $\mathcal{O}(CL^4)$ and $\mathcal{O}(CL^3 \log L)$, making larger, more expressive models computationally feasible.  Moreover, we adopt efficient sampling theory to achieve further computational savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter efficiency on spherical benchmark problems.

Published as a conference paper at ICLR 2021
EFFICIENT GENERALIZED SPHERICAL CNNS
Oliver J. Cobb, Christopher G. R. Wallis, Augustine N. Mavor-Parker, Augustin Marignier, Matthew A. Price, Mayeul d' Avezac & Jason D. McEwen* Kagenova Limited, Guildford GUS 9LD, UK
ABSTRACT
Many problems across computer vision and the natural sciences require the anal- ysis of spherical data, for which representations may be learned efficiently by en- coding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN layer that is strictly equivariant has complexity O(C²T,²), where is measure of representational capacity and L the spherical harmonic bandlimit. Such high computational cost often prohibits the use of strictly equivariant spherical CNNs. We develop two new strictly equivariant layers with reduced complexity O(C)*) and O(CL' log L), making larger, more expressive models computationally fea- sible. Moreover, tional savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter ef- ficiency spherical benchmark problems.
adopt efficient sampling theory to achieve further computa-
INTRODUCTION
Many fields involve data that live inherently on spherical manifolds, e.g. 360 photo and video con- tent in virtual reality and computer vision, the cosmic microwave background radiation from the Big Bang in cosmology, topographic and gravitational maps in planetary sciences, and molecular shape orientations in molecular chemistry, to name just few. Convolutional neural networks (CNNs) have been tremendously effective for data defined on Euclidean domains, such as the 1D line, 2D plane, or nD volumes, thanks in part their translation invariance properties. However, these techniques are not effective for data defined on spherical manifolds, which have a very different geometric structure Euclidean spaces (see Appendix (A). To transfer the remarkable success of deep learning to data defined on spherical domains, deep learning techniques defined inherently on the sphere are required. Recently, number of spherical CNN constructions have been proposed.
Existing CNN constructions on the sphere fall broadly into three categories: fully real (i.c. pixel) space approaches (e.g. [Boomsma & Frellsen 2017 Jiang et al. 2019 Perraudin et al. 2019 Co- hen et al. 2019); combined real and harmonic space approaches (Cohen et al. [018]] Esteves et al.] 2018 2020); and fully harmonic space approaches (Kondor et al. 2018). Real space approaches can often be computed efficiently but they necessarily provide an approximate representation of spherical signals and the connection to the underlying continuous symmetries the sphere is lost.
Consequently, such approaches cannot fully capture rotational equivariance. Other constructions take a combined real and harmonic space approach (Cohen et al. [0018] Esteves et al.) 2018 2020), where sampling theorems (Driscoll & Healy 1994 Kostelec & Rockmore (008) are exploited to
connect with underlying continuous signal representations to capture the continuous symmetries of the sphere. However, in these approaches non-linear activation functions are computed pointwise in real space, which induces aliasing errors that break strict rotational equivariance. Fully harmonic space spherical CNNs have been constructed by Kondor et al. (2018). A continual connection with underlying continuous signal representations is captured by using harmonic signal representations throughout. Consequently, this the only approach exhibiting strict rotational equivariance. How- ever, strict equivariance comes at great computational cost, which can often prohibit usage.
*Coresponding author; jason, mcewen@kagenovaa.com

---

Published as a conference paper at ICLR 2021
In this article we present generalized framework for CNNs on the sphere (and rotation group), which encompasses and builds on the influential approaches of Cohen et al. (2018), Esteves et al. (2018) and Kondor et al. (2018) and allows them to be leveraged alongside each other. We adopt harmonic signal representation in order to retain the connection with underlying continuous repre- sentations and thus capture all symmetries and geometrie properties of the sphere. We construct new fully harmonic (non-linear) spherical layers that are strictly rotationally equivariant, are parameter- efficient, and dramatically reduce computational cost compared similar approaches. This is achieved by a channel-wise structure, constrained generalized convolutions, and an optimized de- gree mixing set determined by a minimum spanning tree. Furthermore, we adopt efficient sampling theorems on the sphere (McEwen & Wiaux 2011) and rotation group (MeEwen et al.) 2015a) to im- prove efficiency compared the sampling theorems used in existing approaches (Driscoll & Healy, 1994 Kostelec & Rockmore 2008). We demonstrate state-of-the-art performance on all spherical benchmark problems considered, both in terms of accuracy and parameter efficiency.
GENERALIZED SPHERICAL CNNS
We first overview the theoretical underpinnings of the spherical CNN frameworks introduced by Cohen et al. (2018), [Eseevs et al. (2018), and Kondor et al. (2018), which make a connection to underlying continuous signals through harmonic representations. For more in-depth treatments of
the underlying harmonic analysis we recommend Esteves (2020) Kennedy & Sadeghi (2013) and
Gallier & Quaintance| (2019). We then present a generalized spherical layer in which these and other existing frameworks are encompassed, allowing existing frameworks to be easily integrated
and leveraged alongside each other in hybrid networks.
Throughout the following we consider a network composed of S rotationally equivariant layers A ( where the i-th layer A maps an input activation f(i-1) E H (5-1) onto an out- put activation ((+ E H)) We focus on the case where the network input space H consists of spherical signals (but note that input signals on the rotation group may also be considered).
2.1 SIGNALS ON THE SPHERE AND ROTATION GROUP
Let (() denote the space of square-integrable functions over domain Ω. A signal f e L²( on the sphere (lt = 82) or rotation group (1) = SO(3) can be rotated by P E SO(3) by defining the action of rotation
signals by (wf(w) f(p ww) for Ω. An operator A : (2, (92)) where Ω2, S2 E (S², SO(3)), is then equivariant to rotations if (R()(()) = A(Rpf) for all f E (22) and E SO(3), i.e. rotating the function before application of the operator is equivalent to application of the operator first, followed by a rotation.
A spherical signal f E (S²) admits a harmonic representation (f (f f....) where fc C26+1 are the harmonic coefficients given by the inner product (f. Y where y are the spherical harmonic functions of degree and order [m] & &. Likewise a signal f e (S²(SO(3) on the rotation group ad- mits a harmonic representation (f² (!, where f E ((2++1)((2++) are the harmonic coefficients with (m, n)-th entry (f, D'..>> for integers |m/]/n/| & C, where D SO(3) - ((2++1)x(2E+1) is the unique 2 +1 1 dimensional irreducible group representation of SO(3) on C(2R+1), The rotation f - R of signal f e (S2) can be described in harmonic space by frr - (p)
A signal on the sphere or rotation group is said to be bandlimited at L if, respectively, (ʃ. </,Y²) 0 or (f.D = 0 for > L. Furthermore, a signal on the rotation group said to be azimuthally bandlimited at N if, additionally, (f, D D = 0 for |n|| > N. Bandlimited signals therefore ad- mit finite harmonic representations [f,, fL-1). In practice real-world signals can be accurately represented by suitably bandlimited signals; henceforth, we assume signals are bandlimited.
2.2 CONVOLUTION ON THE SPHERE AND ROTATION GROUP
A standard definition of convolution between two signals f. 1b L2((2) on either the sphere (s)
2 or rotation group (? = SO(3)) is given by
(f * (p) = (f. Rpp)) = [ d((w) (p-
(1)
2

---

Published as a conference paper at ICLR 2021
where du(w) denotes the Haar measure Ω and .* complex conjugation (e.g. Wandelt & Górski
2001 McEwen et al. 2007 2013: 2015b 2018 Cohen et al. 2018 Esteves et al. 2018). In partic-
ular, the convolution satisfies
(RR,f) (p)) = R,f.RR, (f. Ro = (Rplf (P))
(2)
and is therefore a rotationally equivariant linear operation, which we shall denote by C(w).
The convolution of bandlimited signals can be computed exactly and efficiently in harmonic space as
(
v5) f
{ O..,,
(3)
which for each degree l is vector product for signals the sphere and matrix product for signals on the rotation group (see Appendix for further details). Convolving in this manner results in signals on the rotation group (for inputs on both the sphere and rotation group). However, in the spherical case, if the filter is invariant to azimuthal rotations the resultant convolved signal may be interpreted signal on the sphere (see Appendix B
2.3 GENERALIZED SIGNAL REPRESENTATIONS
The harmonic representations and convolutions described above have proven useful for describing rotationally equivariant linear operators ((w), Cohen et al. (2018) and Esteves et al. (2018) define spherical CNNs that sequentially apply this operator, with intermediary representations taking the form of signals on S0(3) and S respectively. Alternatively, for intermediary representations we now consider the more general space of signals introduced by Kondor et al. (2018), to which the aforementioned notions of rotation and convolution naturally extend.
In describing the generalization we first note from Section 2.1 that all bandlimited signals on the sphere and rotation group can be represented as a set of variable length vectors of the form
f= (// 22++1 6 0,.,,
1; 1,, 1,..,,7T..
where 7f = 1 for signals on the sphere and rf min(2f + 1, 1) for signals on the rotation group. The generalization is to let FL be the space of all such sets variable length vectors, with Ty unrestricted. This more general space contains the spaces of bandlimited signals the sphere and rotation group as strict subspaces. For a generalized signal f E Fr we adopt the terminology of Kondor et al. 2018)) by referring to fl as the t-th fragment of degree and to Ty = (Ff..... 7/ 1) specilyıng the number of fragments for each degree, as the type of f. The action of rotations upon FL can be naturally extended from their action upon I,2(S") and (3)) For f FL we define the rotation operator f - Rpf by ft - (p) allowing us to extend the usual notion of equivariance to operators A FL FI.
2.4 GENERALIZED CONVOLUTIONS
The convolution described by Equation 1 provides a learnable linear operator ( that satisfies the desired property of equivariance. Nevertheless, given the generalized interpretation of signals g and (3))) as signals in Fr. the notion of convolution can also be generalized (Kondor et al. 2018
In order to linearly and equivariantly transform signal f F, of type Ty into new signal f 1/
Fr of any desired type ((f+))) we may specify filter v = { e Cr x l = 0, L 1), which in general is not an element of L. and define a transformation
* by
(f * 1))
Σ P SL
{= O,.,.,, 1; t= 1,..,, (f+0))
(5)
The degree-l fragments of the transformed signal (f * 3b) are simply linear combinations of the degree- fragments of f. with no mixing between degrees. Equation shows that this is precisely the form taken by convolution on the sphere and rotation group. In fact Kondor & Trivedi (2018) show that all equivariant línear operations take this general form; the standard notion of convolution is just special case. One benefit to the generalized notion is that the filter v is not forced to occupy the same domain the signal f. thus allowing control over the type T(fme) of the transformed
signal. We use c(#) to denote this generalized convolutional operator.
3

---

Published as a conference paper at ICLR 2021
2.5 NON-LINEAR ACTIVATION OPERATORS
For FL to be a useful representational space, it must be possible to not only linearly but also non- linearly transform its elements an equivariant manner. However, equivariance and non-linearity is not enough. Equivariant linear operators cannot mix information corresponding to different degrees. Therefore it is of crucial importance that degree mixing is achieved by the non-linear operator.
2.5.1 POINTWISE ACTIVATIONS
When the type Ty of f FL permits an interpretation as signal on S SO(3) we may perform an inverse harmonic transform to map the function onto a sample-base based representation (e.g. Driscoll & Healy] 1994, McEwen & Wiaux, 2011 Kostelec & Rockmore, 2008 McEwen et al. (015a). C may then be applied pointwise, T.e. separately to each sample, before performing harmonic transform to return to a representation in FL. We denote the corre- sponding non-linear operator as (f) = F(a(F-((()) where F represents the harmonic (i.e. Fourier) transform on 8 SO(3). The computational cost of the non-linear operator is dominated by the harmonic transforms. While costly, fast algorithms can be leveraged (see Appendix (A. While inverse and forward harmonic transforms on 5 SO(3) that are based on a sampling theory main- tain perfect equivariance for bandlimited signals, the pointwise application of a (most commonly ReL.U) only equivariant the continuous limit I, oo. For any finite bandlimit I., aliasing effects are introduced such that equivariance becomes approximate only, as shown by the following experiments.
A non-linear function a
We consider 100 random rotations p E SO(3), for each of 100 random signal-filter pairs (f, vt), and compute the mean equivariance error d(A(R))) (A()))) for operator A, where d(f,g) g) = Iff I -Illfl is the relative distance between signals. For convolutions the equivariance error is 4.4 x 10 for signals on S and 5.3 x 10-7 for signals on SO(3) (achieving floating point pre- cision). By comparison the equivariance error for a pointwise ReLU is 0.34 for signals on $ and 0.37 for signals on SO(3). Only approximate equivariance is achieved for the ReL.U since the non- linear operation spreads information to higher degrees that are captured at the original bandlimit, resulting in aliasing. To demonstrate this point we reduce aliasing error by oversampling the real- space signal. When oversampling by 2x or 8x for signals on S3) the equivariance error of the ReLU is reduced to 0.10 and 0.01, respectively. See Appendix D for further experimental details.
Despite the high cost of repeated harmonic transforms and imperfect equivariance, this is never- theless the approach adopted by Cohen et al. (2018). Esteves et al. (2018) and others, who find empirically that such models maintain a reasonable degree of equivariance.
2.5.2
ACTIVATIONS
In order to define a strictly equivariant non-linear operation that can be applied to a signal f e F, of any type Ty the decomposability of tensor products between group representations may be leveraged, as first considered by Thomas et al. (2018) in the context of neural networks.
Given two group representations e and Dd of SO(3) on C26++ and C2 respectively, the tensor-product group representation D @ 6 of SO(3) on C2 @ C26++1 is defined such that (D @ D (p) = DP (p) & (p) for all E SO(3). Decomposing D4 X D into a direct sum of irreducible group representations then constitutes finding a change of basis for C @ 2 such that (D' & D'a Xp) is block diagonal, where for each E there is a block equal to D DP(p). The necessary change of basis for ūf @ jf E C2 @ C22++1 is given by
Σ my
Σ
Chila G6 G62 mygm Myy mg
(6)
Eg
m22
Ea
where 82.E m2 E C denote Clebsch-Gordan coefficients whose symmetry properties are such that (ūf1 @ d is non-zero only for 1& - E2 & E &1 + f2. The use of Equation 6 arises naturally in quantum mechanics when coupling angular momenta.
This property is useful since if f E + and f e C2 are two fragments that are equivariant with respect to rotations of the network input, then a rotation of applied to the network input results
4

---

Published as a conference paper at ICLR 2021
in f @ fda transforming as
o ((D* @ (D = = @ (D) @ )
(8)
= =D'( @ f/..
(9)
where the final equality follows by block diagonality with respect to the chosen basis. Therefore, fragments f and fcz are equivariant with repsect to rotations of the network input, then so the fragment ( @ f(2) E C2++1 where we have written Equation 6 more compactly. We
now describe how Kondor et al. (2088) use this fact define equivariant non-linear transformations of elements in FI.
A signal f (f E C : O,.., - t 1,... 1...., Tf} E FL may be equivariantly and non-linearly transformed by an operator Ng Fr F defined as
(f) = ((C*).aa)) (T, f()) 8 .,,,, L- 1; (E,, (2) Pi 4 O,..,, tf';
O,... 12 0,,,,/'''
(10)
where for each degree E {0, ..., I, 1} the set
2 Pt. [( (2) E {0, L 13² - & & +2
(11)
defined in order to avoid the computation trivially equivariant all-zero fragments. We make the dependence on Py explicit since we redefine it in Section
Unlike the pointwise activations discussed in the previous section this operator is strictly equivariant, with a mean relative equivariance error of 5.0 x 10-7 (see Appendix D). Note that applying this operator to signals on the sphere or rotation group results in generalized signals that are no longer on the sphere rotation group. This is the rationale for the generalization FL: to unlock the ability to introduce non- nin-airyyy in a strictly equivariant manner. Note, however, that = (f) - = where T/ = (2)eee, Tf and therefore application of this non-
has type " (T)) -/ linear operator results in drastic expansion in representation size, which is problematic.
2.6 GENERALIZED SPHERICAL CNNS
Equipped with operators to both linearly and non-linearly transform elements of FL. with the latter also performing degree mixing, we may consider a network with representation spaces (0) = (5) = FL. We consider the s-th layer of the network to take the form of triple A =
(C,.NN L2) such that ((())( 1)) = L2(N(C, (f where L1, C2 FL - FL are linear operators and N FL F L is non-linear activation operator.
The approaches of Cohen et al. (2018) and Esteves et al. (2018) are encompassed in this framework as (( = (c(.), ,Na, where I denotes the identity operator and filters t may be defined to encode real-space properties such localization (see Appendix C The framework of Kondor al.
2018 is also encompassed as (* = NI,Ng, c(1)). Here the generalized convolution f comes last to counteract the representation--eppaaee exxpndiin effect of the pensortproduct activation and prevent it from compounding as signals pass through the network. Appendix E lends intuition regarding rela- tionships that may be captured by tensor-product activations followed by generalized convolutions.
For any intermediary representation f(() E FL may transition from equivariance with respect to the network input to invariance by discarding all but the scalar-valued fragments corresponding to l = 0 (equivalent to average pooling for signals on the sphere and rotation group). Finally, note that within this general framework we are free to consider hybrid approaches whereby layers proposed by Cohen et al. (2018); Esteves al. (2018); Kondor et al. (2018) and others, and those presented subsequently, can be leveraged alongside each other within a single model.
3 EFFICIENT GENERALIZED SPHERICAL CNNS
Existing approaches to spherical convolutional layers that are encompassed within the above frame- work are computationally demanding. They require the evaluation of costly harmonic transforms
5

---

Published as a conference paper at ICLR 2021
on the sphere and rotation group. Furthermore, the only strictly rotationally equivariant non-linear layer is that of Kondor et al. (2018), which has an even greater computational cost, scaling with the fifth power of bandlimit
thereby limiting spatial resolution and quadratically with the number of fragments per degree - thereby limiting representational capacity. This often prohibits the use of strictly equivariant spherical networks.
In this section we introduce a channel-wise structure, constrained generalized convolutions, and an optimized degree mixing set in order construct new strictly equivariant layers that exhibit much improved scaling properties and parameter efficiency. Furthermore, we adopt efficient sampling theory on the sphere and rotation group to achieve additional computational savings.
3.1 EFFICIENT GENERALIZED SPHERICAL LAYERS
For activation f Fr the value Fy + + represents resolution-indeeendenn proxy for its representational capacity. Kondor et al. (2018) consider the separate fragments contained within ʃ to subsume the traditional role of separate channels and therefore control the capacity of intermediary network representations through specification of Ty. This is problematic because, whereas activation functions usually act on each channel separately and therefore have a cost that scales linearly with representational capacity (usually controlled by the number of channels), for the activation function Ng not only does the cost scale quadratically with representational capacity 7t, but too does the size of Ngg()) (f). This feeds forward the quadratic dependence the cost of, and number of parameters required by, the proceeding generalized convolution.
More specifically, note that computation of g = N&((f) requires the computation of Σ/IO Ty frag- ments, where Tg- Eee,, of Thi T The size of F is O(LE) for each & and therefore the ex- panded representation has size 1/--1 TG of order L)) By exploiting the sparsity of Clebsch- Gordan coefficients (f.t mtg.m. 0 m2 + m2 # m) each fragment (C*h.f2.)) (fts @ fta) can be computed in (l min(A) (11, E2)). Hence, the total cost of computing all necessary fragments has complexity O(C2L5), where C' Fy captures representational capacity.
3.1.1 CHANNEL- WISE TENSOR-PRODUCE ACTIVATIONS
As is more standard for CNNs we maintain a separate channels axis, with network activations taking the form (J1...,,'' E F where Ji E Fr. all share the same type Ty- The non-linearity No may then be applied to each channel separately at cost that is reduced by K-times relative its applica- tion on a single channel with the same total number of fragments. This saving arises since for each i we need only compute K >i f2jerr Tf f fragments rather than (2)eg (KTT))
Figure 1 visualizes this reduction for the case K = 3. Note, however, that for practical applications K 100 is more typical. The K-times reduction in cost is therefore substantial and allows for intermediary activations with orders of magnitude more representational capacity.
By introducing this multi-channel approach and using C K rather than C = Fy to control representational capacity, reduce the complexity of Nog with respect to representational capacity from ((??) to OCC).
N
N
e
{-1
t-2
(=0, 1,2
(a) Prior approach to applying tensor prosuut based non-lincar operator
(b) Ours
Figure I: Comparison (to scale) of the expansion caused by the tensor-product activation applied to inputs of equal representational capacity but different structure. With depth representing the number of channels and width the number of fragments for each degree, it is clear that by grouping fragments into K separate channels the expansion (and therefore cost) can be K-times reduced. Visualization corresponds to inputs with ( (L, K)) equal to (3, 1) and (3, 3) for panel (a) and (b), respectively.

---

Published as a conference paper at ICLR 2021
3.1.2 CONSTRAINED GENERALIZED CONVOLUTION
Although much reduced, for a signal f F the channel-wise application of Ng still results in a drastically expanded representation g NoO to which representationn-ontttiing generalized convolution must be applied in order to project onto a new activation g' = c/ (g) E F of the desired type To' and number of channels Kout- However, under our multi-channel structure computational and parameter efficiency can be improved significantly by decomposing L into ((ts) three separate linear operators, C( C( and Lo
c(11) acts uniformly across channels, performing linear projection down onto the The first, G. desired type, and should be interpreted as a learned extension of Ne which undoes the drastic expansion. The second, c( then acts channel-wise, taking linear combinations of the (con- c(ws)
tracted number of) fragments within each channel. The third, GGs acts across channels, tak- ing linear combinations to learn new features. More concretely, the three filters are of the form 2/1 = =08f CT e :6= O..... L 13. t/2 (822 CT
O.... 1; 1, ..., Km) and 4/s = frig E CK Kon : l 0,.., L - 13, rather than a single filter of the form & = ( C Kaxxx : -===0..... O,..,, 13. leading to a large reduction in the number of pa- rameters as ry is invariably very large.
By applying the first step uniformly across channels we minimize the parametric dependence on the expanded representation and allow new features to be subsequently learned much more efficiently. Together the second and third steps can seen as analogous to depthwise separable convolutions often used in planar convolutional networks.
3.1.3 OPTIMIZED DEGREE MIXING SETS
We now consider approaches to reduce the (LL) complexity with respect to spatial resolution L. In the definition of Ng; each element of P., independently defines an equivariant fragment. Therefore restricted Ng in which only subset of TPÊ, is used for each degree & still defines a strictly equivariant operator, while reducing computational complexity. In order to make savings whilst remaining at resolution L it is necessary to consider subsets of IPt, that scale better than O(L²). The challenge is to find such subsets that do not hamper the ability of the resulting operator to inject non-linearity and mix information corresponding to different degrees f.
Whilst various subsetting approaches are possible, the following argument motivates an approach that have found be particularly effective. If (61, fa) IP4 then representational space is designated to capture the relationship between f and fa-degree information. However, resources have been designated already to capture the relationship between l and fy-degrees, well as between E2 and É3-degrees, then some notion of the relationship between L and f3-degrees has been captured already. Consequently, is unnecessary to designate further resources for this purpose.
More generally, consider the graph G = (Nz, PP))) with nodes N₂ = (.. L-13 and edges Pf, A restricted tensor-product activation can be constructed by using a subset of TPÉ, that corresponds to a subgraph of G²_- The subgraph of G captures some notion of the relationship between incoming & and fz-degree information if contains path between nodes é, and f2. Therefore we are interested in subgraphs for which there exists path between any two nodes if there exists such a path in the original graph, guaranteeing that any degree-mixxe relationship captured by the original graph is also captured by the subgraph.
The smallest subgraph satisfying this property minimum spanning tree (MST) of G¹,, The set of edges corresponding to any MST has at most L elements and we choose to consider its union with the set of loop-edges in Gf (of the form (f1, (1)). which proved particularly important for injecting non- non-linearity. We denote the resulting set as F and note that it satisfies |PA.1 < 2L. Therefore the tensor-product activation Nig corresponding to Equation I1 with Pf replaced by Ff has reduced spatial complexity (())). Given that many minimal spanning trees of the unweighted graph G² exist for each f, we select the ones that minimize the cost of the resulting activation Nop by assigning to each edge (f, f2) in G a weight equal to the cost of computing (C4.d24)) "(tt @ 22) and selecting the MST of the weighted graph.
7

---

Published as a conference paper at ICLR 2021
3
0
0
f
Full P set of size (TT.²)
(b) MST subset of size O(L)
(c) RMST subset of size (log T.)
Figure 2: Visualization of the mixing set PL (for L = 7 and f = 4) and the approaches to subsetting based on the minimum spanning tree (MST) and reduced minimum spanning tree (RMST) mixing polices, which reduces related computation costs from (I2) to, respectively, (L) or Olog L).
An example of PL and a MST subset PPg is shown Figure 2 where the dashed line in Figure 2c shows the general form of the MST. Using this as principled starting point we consider the further reduced MST (RMST) subset IP2 corresponding to centering the MST at the edge (f, E) and retaining only the edges that fall distance of 2' away the dotted line for some i N. We use Ng to denote the corresponding operator and note that it has further reduced spatial complexity of ( log I.).
We demonstrate in Section 4 that networks that make use of the MST tensor-product activation achieve state-of-the-arr performance. Replacing the MST with RMST activation results in a small but insignificant degradation in performance, which is offset by the reduced computational cost.
3.1.4 REDUCTION IN COMPUTATIONAL AND MEMORY FOUTPRINTS
The three modifications proposed in Sections 3.1.1 to 1.3 are motivated by improved scaling prop- erties. Importantly, they also translate to large reductions in the computational and memory cost of strictly equivariant layers in practice,
detailed in Appendix F Even at modest bandlimit of L = 64 and relatively small number of channels K = 4. for example, the modifications together lead to 51-times reduction in the number of flops required for computations and 16-times reduction in the amount of memory required to store representations, weights and gradients for training.
3.2 EFFICIENT SAMPLING THEORY
By adopting sampling theorems on the sphere we provide access to underlying continuous signal representations that fully capture the symmetries and geometric properties of the sphere, and allow standard convolutions be computed exactly and efficiently through their harmonic representations, as discussed in greater detail in Appendices/A [ and B We adopt the efficient sampling theorems on sphere and rotation group of McEwwn & Wiaux (201I) and McEwen et al. (2015a), respectively, which reduce the Nyquist rate by factor of two compared to those of Driscoll & Healy (1994)
and Kostelec Rockmore (2008), which have been adopted in other spherical CNN constructions (e.g. Cohen et al.) 2018 Kondor et al., 2018 Esteves al. 2018 2020). The sampling theorems adopted are equipped with fast algorithms to compute harmonic transforms, with complexity O(L for the sphere and (T.A)) for the rotation group. When imposing an azimuthal bandlimit N L,, the complexity of transforms on the rotation group can be reduced to O(NT,)) which we often exploit in our standard ) convolutional layers.
4 EXPERIMENTS
Using our efficient generalized spherical CNN framework (implemented in the fourpiAI code) we construct networks that apply to numerous spherical benchmark problems. We achieve state- of-the-art performance, demonstrating enhanced equivariance without compromising representa- tional capacity parameter efficiency. In all experiments of 2-3 standard convolutional layers (e.g. 8 or SO(3) convolutions proceeded by ReI.Us), followed by 2-3 of our efficient generalized layers. We adopt the efficient sampling theory described in Sec- tion 3.2 and encode localization of spatial filters as discussed in Appendix C Full experimental details may be found in Appendix G
a similar architecture, consisting
8

---

(2088), Published as a conference paper at ICLR 2021
Table Test accuracy for spherical MNIST digits clas-
Table 2: Test root mean squared (RMS)
sification problem
error for QM7 regression problem
NR/NR
R/R
NR/R
Params
RMS
Params
Planar CNN
99.32
90.74
11.36
58k
Montavon ct al. 2012
5.96
Cohen et al.
2018
95.59
94.62
93,40
58k
Cohen al. (218))
8.47
1.4M
Kondor ct
2018
96.40
96.60
96.00
286k
Kondor 2018
7.97
> 51.1M
Esteves ct
2020
99.37
99.37
99.08
58k
Ours (MST)
3.16
337k
Ours (MST)
99.35
99.38
99.34
58k
Ours (RMST)
3.46
335k
Ours (RMST)
99.29
99.17
99.18
57k
Table 3: SHREC'17 object retrieval competition metrics (perturbed micro-all)
P@N
R@N
@I@N
mAP
NDCG
Params
Kondor et al. (2018)
0.707
0.722
0.701
0.683
0.756
> ~1M
Cohen et al. 2018)
0.701
0.711
0.699
0.676
0.756
1.4M
Esteves
2018
0.717
0.737
0.685
500k
Ours
0.719
0,710
0.708
0.679
0.758
250k
4.1 ROTATED MNIST ON THE SPHERE
We consider the now standard benchmark problem of classifying MNIST digits projected onto the sphere. Three experimental modes NR/RR, R/R and NR/R are considered. indicating whether the training/test sets have been randomly rotated (R) or not (NR). Results are presented in Table [ which shows that closely match the prior state-of-the-ar performance obtained by EEttves et al. (2020) on the NR/NR and R/R modes, whilst outperforming all previous spherical CNNs on the NR/R mode, demonstrating the increased degree of equivariance achieved by our model.
Results are shown for models using both the MST-based and RMST-based mixing sets within the tensor-product activation. The results obtained when using the full sets IP, are very similar those obtained when using the MST-based sets (e.g. full sets achieved an accuracy of 99.39 for R/R).
4.2 ATOMIZATION ENERGY PREDICTION
We consider the problem of regressing the atomization energy of molecules given the molecule's Coulomb matrix and the positions of the atoms in space, using the QM7 dataset (Blum & Rey- mond 2009; Rupp et al. 2012). Results are presented in Table which shows that we dramatically outperform other approaches, whilst using significantly fewer parameters.
4.3 3D SHAPE RETRIEVAL
We consider the 3D shape retrieval problem on the SHREC'17 (aava et al., 2017) competition dataset, containing 51k 3D object meshes. We follow the pre-processing step of Cohen where several spherical projections of each mesh are computed, and use the official SHREC' 17 data splits. Results are presented Table 3 for the standard SHREC precision and recall metrics, which shows that we achieve state-of-the performance compared to other spherical CNN approaches, achieving the highest three of five performance metrics, whilst using significantly fewer parameters.
CONCLUSIONS
We have presented generalized framework for CNNs on the sphere that encompasses various ex- isting approaches. We developed new efficient layers to be used as primary building blocks in this framework by introducing channel-wise structure, constrained generalized convolutions, and opti- mized degree mixing sets determined by minimum spanning trees. These new efficient layers exhibit strict rotational equivariance, without compromising on representational capacity or parameter effi ciency. When combined with the flexibility of the generalized framework to leverage the strengths of alternative layers, powerful hybrid models can be constructed. On all spherical benchmark prob- lems considered we achieve state-of-the-ard performance, both in terms of accuracy and parameter efficiency. In future work we intend to improve the scalability of our generalized framework further still. In particular, scattering transforms (Malatt 2012) to the sphere, to further realize the potential deep learning on host of new applications where spherical data are prevalent.
plan to introduce additional highly scalable layers, for example by extending

---

Published as a conference paper at ICLR 2021
REFERENCES
Lorenz Blum and Jean-Louis Reymond. 970 million druglike small molecules for virtual screening in the chemical universe database GDB-13. Journal of the American Chemical Society, 131:8732, 2009.
Wouter Boomsma and Jes Frellsen. Spherical convolutions and their application in molecular mod- elling. In I. Guyon, U. V. Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 3433-3443. Cur- ran Associates, Inc., 2017.
Taco Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. In International Conference on Learning Representations, 2018. http:/s/////axtt.orr//bb////////
10130
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convo-
lutional networks and the icosahedral CNN. arXiv preprint arXiv:1902.04615, 2019. URL
04615
James Driscoll and Dennis Healy. Computing Fourier transforms and convolutions on the sphere. Advances in Applied Mathematics, 15:202-2555 250, 1994.
Carlos Esteves. Theoretical aspects of group equivariant neural networks. arXiv preprint 05154
arXiv:2004.05154 2020.
Carlos Esteves, Christine BlanchBBancettte, Ameesh Makadia, and Kostas Danilidis. Learning SO(3) equivariant representations with spherical CNNs. Proceedings of the European Con- ference Computer Vision (ECCV), pp- 52-68, 2018. URL | https://arxiv. wwrxxiv.org/aaa www. abg//bs//711. 06721
Carlos Esteves, Ameesh Makadia, and Kostas Danilidiss. Spin-weighted spherical CNNs. arXiv preprint arXiv:2006.1077 2020.
10731
Jean Gallier and Jocelyn Quaintance. Aspects of Harmonic Analysis and Representation Theory. 2019. URL /http::///we http:://wwwwwwwww..eeeem seas. upenn. iareen.eddu//ae wwtp: jean---jjaannccaamoniicc
Dennis Healy, Daniel Rockmore, Peter Kostelec, and Moore. FFTs for the 2-sphere - improve- ments and variations. Journal of Fourier Analysis and Applications, 9(4):341-385, 2003.
Chiyu Jiang, Jingwei Huang, Karthik Kashinath, Philip Marcus, Matthias Niessner, et al. Spherical CNNs on unstructured grids. arXiv preprint 1001.099000000399 2019. URL /ttpps://axxv..
org/abs/199 1901. 02039
Rodney A Kennedy and Parastoo Sadeghi. Hilbert space methods in signal processing. Cambridge University Press, 2013.
Diederik Kingma and Jimmy Lei Ba. Adam: A method for stochastic gradient descent. In ICLR: International Conference on Learning Representations, 2015. URL htttps://arxiv.org//aa 1412, 6980
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups In International Conference Machine Learning, pp- 2747-2755, 2018. URL htttt:///////iii http:://aaxxv..og/abb//////s otgg/bb///02.00333333 802.03690)
Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch-Gordand Gordan nets: a fully fourier space spherical convolutional neural network. In Advances in Neural Information Processing Systems, pp. 10117- 10126, 2018. URL http/::tts:
orgo aas/a//s/// /a0b/ 09231
Peter Kostelec and Daniel Rockmore. FFTs on the rotation group. Journal of Fourier Analysis and Applications, 14: 14:1 2008.
Stéphane Mallat. Group invariant scattering. Communications Pure and Applied Mathematics, 131-19913 2012. Utth/ttss://aan www.rviv.org/bb//11000 2286
Domenico Marinucci and Giovanni Peccati. Random Fields on the Sphere: Representation, Limit Theorem and TadCoonn Cosmological Applications. Cambridge University Press, 2011.
10

---

Published as a conference paper at ICLR 2021
Jason McEwen and Yves Wiaux. A novel sampling theorem the sphere. JEEE Transactions Signal Processing, 59(12)55876-588 5887, 2011. URL http::///axxiv.org/aata
6298
Jason McEwen, Michael P. Hobson, Daniel J. Mortlock, and Anthony N. Lasenby. Fast directional continuous spherical wavelet transform algorithms. IEEE Trans. Sig. Proc., 55(2):220-529, 2007.
www.:oggg/bb//.eeee URL http:////arriiiioog/////////stss
Jason McEwen, Pierre Vandergheynst, and Yves Wiaux. On the computation of directional scale- discretized wavelet transforms on the sphere. In Wavelets and Sparsity XV. SPIE international symposium optics and photonics, invited contribution, volume 8858, 2013. https:
//arxiv.ogg/abs//// 3tobs/3388.55 5706
Jason McEwen, Martin Büttner, Boris Leistedt, Hiranya Peiris, and Yves Wiaux. A novel sam- pling theorem the rotation group. IEEE Signal Processing Letters, 22(22)2245-22 2015,, URL htttps://arxiv.org/bbbssss
031011
Jason McEwen, Boris Leistedt, Martin Büttner, Hiranya Peiris, and Yves Wiaux. Directional spin wavelets on the sphere. IEEE Trans. Sig. Proc., submitted, 2015b. URL https:tts://aaxiioorr
abs/1509. 06749
Jason McEwen, Claudio Durastanti, and Yves Wiaux. Localisation of directional scale-discretised wavelets on the sphere, Applied Comput. Harm. Anal., 44(1):59-88, 2018. UtLhhttss///ariv
org/abs/1505 006756669990077
Grégoire Montavon, Katja Hansen, Siamac Fazli, Matthias Rupp, Franziska Biegler, Andreas Ziehe, Alexandre Tkatchenko, Anatole V. Lilienfeld, and Klaus-Robert Müller. Learning invariant repre- sentations of molecules for atomization energy prediction. F. Pereira, C. C. Burges, L. Bot- tou, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 25, pp- 440 448. Curran Associates, Inc., 2012.
Nathanaël Perraudin, Michaël Defferrard, Tomasz Kacprzak, and Raphael Sgier. Deepsphere: Ef- ficient spherical convolutional neural network with HEALPix sampling for cosmological appli- cations. Astronomy and Computing, 27:130-146, 2019. URL htttpss//aarrvv.org/aa/////////
12186
Matthias Rupp. Alexandre Tkatchenko, Klaus-Robert Müller, and Anatole von Lilienfeld. Fast and accurate modeling of molecular atomization energies with machine learning. Physical Review Letters, 108:058301, 2012. URL https://arxiv.org/ abs/aass//111 2618
Manolis Savva, Fisher Yu, Hao Su, Asako Kanezaki, Takahiko Furuya, Ryutarou Ohbuchi, Zhichao Zhou, Rui Yu, Song Bai, Xiang Bai, et al. Large-scale 3d shape retrieval from shapenet core55; Shree' 17 track. Proceedings of the Workshop on 3D Object Retrieval, pp. 39-50. Eurographics Association, 2017.
Max Tegmark. An Icosahedron- Based method for pixelizing the celestial sphere. Astrophys. J. Lett., 470:L81, October 1996.
Nathaniel Thomas, Tess Smidt, Steven Kearnes. Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translaiion-eqiivariant neural networks for 3d point clouds. arXiv preprint /rXiv:/802.0000
2018. URL htttss://aaxiii
08219
Stefano Trapani and Jorge Navaza. Calculation of spherical harmonics and Wigner d functions by FFT. Applications to fast rotational matching in molecular replacement and implementation into AMoRe. Acta Crystallographica Section A, 62(4):262-269) 2006.
Benjamin Wandelt and Krzysztof Górski. Fast convolution the sphere. Phys. Rev. D., 63(12): 123002, 2001. URL
11

---

Published as a conference paper at ICLR 2021
A REPRESENTATIONS OF SIGNALS ON THE SPHERE AND ROTATION GROUP
To provide further context for the discussion presented the introduction and to elucidate the prop- erties of different sampling theory on the sphere and rotation group, we concisely review represen- tations of signals on the sphere and rotation group.
A.1 DISCRETIZATION
It is well-known that a completely regular point distribution the sphere does in general exist (e.g- (e.g- [Tegmark] T 1996). Consequently, while a variety of spherical discretization schemes exists (e.g. icosahedron, HEALPix, graph, and other representations), it is not possible discretize (i.e. to sample or pixelize) the sphere in a manner that is invariant to rotations, i.e. a discrete sampling of rotations of the samples on the sphere will in general not map onto the same set of sample positions. This differs to the Euclidean setting and has important implications when constructing convolution operators on the sphere, which clearly are a critical component of CNNs.
Since convolution operators are in general built using a translation operator when on the sphere- - is thus not possible construct a convolution operator directly on discretized representation of the sphere that captures all of the symmetries of the underlying spher- ical manifold. While approximate discrete representations can be considered, and are nevertheless useful, such representations cannot capture all underlying spherical symmetries.
cerrattor-quiieennntte a rotation
A.2 SAMPLING THEORY
Alternative representations, however, can capture all underlying spherical symmetries. Sampling theories on the sphere (e.g. Driscoll & Healy 1994 McEwen & Wiaux 2011) provide mechanism to capture all information content of an underlying continuous function on the sphere from a finite of samples (and similarly the rotation group: Kostelec & Rockmore 2008 McEwwn et al. 2015a). A sampling theory on the sphere is equivalent to a cubature (i.e. quadrature) rule for the exact integration of bandlimited functions on the sphere. While optimal cubature on the sphere remains an open problem, the most efficient sampling theory on the sphere and rotation group is that developed by McEwen & Wiaux (201T) and McEwwn et al. 2015a),, respectively.
On compact manifold like the sphere (and rotation group), harmonic (i.e. Fourier) space is discrete. Hence, a finite set of harmonic coefficients captures all information content of an underlying contin- uous bandlimited signal. Since such representation provides access to the underlying continuous signal, all symmetries and geometric properties of the sphere are captured perfectly. Such repre- sentations have been employed extensively in the construction of wavelet transforms on the sphere, where the use of sampling theorems on the sphere and rotation group yield wavelet transforms of discretized continuous signals that are theoretically exact (c.g. McEwen et al.
2013 2015b 2018).
Harmonic signal representations have also been exploited in spherical CNNs to access all underlying spherical symmetries and develop equivariance network layers (Cohen et al.
2018 Kondor al.
2018 Esteves et al., 2018 2020).
A.3
EXACT AND EFFICIENT COMPUTATION
Signals on the sphere f LS(S?) may be decomposed into their harmonic representations
f(w) =
f
(,))
(12)
where their spherical harmonic coefficients are given by
f = -f.Y = [
Bu(w)f(w) y (w))
(13)
for w E S2 Similarly, signals on the rotation group g L²(SO(3)) may be decomposed into their harmonic representations as
26+1 8 8
g(p) = Σ
Σ
Σ
Iin De Dee (p)
(14)
m m=-
12

---

Published as a conference paper at ICLR 2021
where their harmonic (Wigner) coefficients are given by
Smn
{g,,
D mn/
dp(p)g(p) (p),
(15)
SSO(3)
for p E SO(3). Note that we adopt the convention where the conjugate of the Wigner D-function is used in Equation 14 since this leads to a convenient harmonic representation when considering convolutions (cf. McEwen et al. 2015a 2018).
As mentioned above, sampling theory pertains to strategies to capture all of the information content of band limited signals from a finite set of samples. Since the harmonic space of the sphere and rota- tion group is discrete, this is equivalent to exact quadrature rule for the computation of harmonic coefficients by Equation 13 and Equation 15 from sampled signals.
The canonical equiangular sampling theory on the sphere was that developed by Driscoll & Healy (1994), and subsequently extended to the rotation group by Kostelee & Rockmore (2008). More recently, novel sampling theorems on the sphere and rotation group were developed by McEwen & Wiaux (2011) and McEwen et al. (2013)), respectively, that reduce the Nyquist rate by a factor of two. Previous CNN constructions
the sphere (e.g. Cohen et al.; 2018 Kondor et al. 2018 Esteves et al., 2018: 2020) have adopted the more well-known sampling theories of Driscoll & Healy (1994) and Kostelec& Rockmore (2008). In contrast, we adopt the more efficient sampling theories of McEwen & Wiaux (2011) and McEwen et (2015a) to provide additional efficiency
savings, implemented in the open source ssht and so3 software packages (we also make use of a TensorFlow implementation of these algorithms in our private tensossht code). Note also that the sampling schemes associated with the theory Mc[Mn & Wiiux (201 (and other minor variants implemented in ssht align more closely with the one-to-two aspect ratio of common spherical data, such as 360 photos and videos.
All of the sampling theories discussed are equipped with fast algorithms to compute harmonic trans- forms, with complexity ( for transforms on the sphere (Driscoll & Healy 1994 McEwen & Wiaux 2011) and complexity O(L*) for transforms on the rotation group (Kostelec & Rockmore 2008 McEwen et al. (20 Note that algorithms that achieve slightly lower complexity have been
developed (Driscoll & Healy 1994 Healy et al. 2003 Kostelec & Rockmore 2008 but these
known to suffer stability issues (Healy et al.] 2003] Kostelec & Rockmore 2008). By imposing
an azimuthally bandlimit N, where typically N
the complexity of transforms on the rotation
group can be reduced to O(NL*) (McEwen et al. 2015a), which we exploit in our networks.
These fast algorithms compute harmonic transforms on the sphere and rotation group can be lever- aged to yield the exact and efficient computation of convolutions through their harmonic representa- tions (see Appendix B By computing convolutions in harmonic space, pixelization and quadrature errors are avoided and computational complexity is reduced to the cost of the respective harmonic transforms.
B CONVOLUTION ON THE SPHERE AND ROTATION GROUP
For completeness make explicit the standard (non-eeneralized convolution operations the sphere and rotation group that we adopt. The general form convolution for signals f E (22) either the sphere (Ω 82) rotation group (Ω SO(3)) ) is specified by Eqqattio [ with har- monic representation given by Equation 3 Here we provide specific expressions for the convolution for variety of cases, describe the normalization constants that arise and may be absorbed into learn- able filters, and derive the corresponding harmonic forms. In practice all convolutions are computed in harmonic space since the computation is then exact, avoiding pixelisation or quadrature errors, and efficient when fast algorithms to compute harmonic transforms are exploited (see Appendix A).
htttp:/www..pingg
http: www.wothree.e.mmm..wo /rg/
Available on request from https:: /ww. htttps://www.kggenovaaa.. www.kkaa..aa.oooooooookoo.... com/
13

---

Published as a conference paper at ICLR 2021
B.1 CONVOLUTION ON THE SPHERE
Given two spherical signals f, s L2(S?) their convolution, which in general is signal on the rotation group, may be decomposed as
( +b) Rov))
(16)
[((W) (p-tw)
(17)
Σ
f D (p)a IS dD(())
(1) (i)
(18)
8' l' mm'
Im D a (p) 6mm
(19)
Unite D'e (p),
(20)
Ennn
yielding harmonic coefficients
8x
f b
(21)
28 + 1
The constants 872/(26 1) may be absorbed into learnable parameters.
B.2 CONVOLUTION ON THE SPHERE WITH AXISYMMETRIC FILTERS
When convolving a spherical signal f (S())) with an axisymmetric spherical filter & E L (S())) that is invariant to azimuthal rotations, the resultant (f x)) may be interpreted signal the sphere. To see this note that an axisymmetric filter v has harmonic coefficients tbg v that non-zero only for substituting into Equation 20 we see that the convolution may be decomposed as
0. Denoting rotations by their zyz-Buler angles p (a, B, and
(ʃ * (a,,),,,,,1))
Σ (Ch (ono (a,
(22)
Σ 8m
fm 2o D (a, 8,0)
(23)
4m
ft 2
28 +1 y (B,
(24)
We may therefore interpret (f v) as signal on the sphere with spherical harmonic coefficients
4
(ʃ +
f 10
(25)
28 +
The constants / 44T/(2f 1) may be absorbed into learnable parameters.
14

---

Published as a conference paper at ICLR 2021
B.3 CONVOLUTION ON THE ROTATION GROUP
Given two signals S, v E L2(SO(3)) on the rotation group their convolution may then be decom- posed as
(f * (p) (S. Ro4b)
(26)
- (so(s)
(
(27)
-J du(p')
-
28 + 1
2
fe
Des
(0')
|--
28" +1
Σ
V De m'n (p-tp)
mn
[SO(3)
8T2
8
m'n'
(28)
28 +
28
Σ
du(p')
mn (p')
8T
mn
SmT
mn'n'
SSO(3)
mn
g
m'n'
(29)
+
Σ
20
t m''''
du(p')
mn (0') Σ Dams
8T
mn
mm
STT
m''''
SS((3)
(30)
28 +
20
8T²
Σ Σ
de c' m'n'
Σ D (p)
Ser Ommonn'
(31)
8T
mn
S
2f +
mn
m
8x
mom (p)
ts
Σ
m'n
(32)
mmmm
where for Equation
30
we make use of the relation (e.g. Marinucci & Peccati 2011 McEwen al.
2018)
D mm (p'1
Σ D&m (p) (c/).
(33)
This decomposition yields harmonic coefficients
(f )
> plam Lott
(34)
C FILTERS ON THE SPHERE AND ROTATION GROUP
When defining filters we look to encode desirable real-space properties, such as locality and regular- ity. However, practice considerable computation may saved by defining the filters in harmonic space and saving the cost of harmonic transforming ahead of harmonic space convolutions. We de- scribe here how filters motivated by their real space properties may defined directly in harmonic space.
C.I DIRAC DELTA FILTERS ON THE SPHERE
Spherical filters may be constructed as a weighted sum of Dirac delta functions on the sphere. This construction useful as the harmonic representation has an analytic form that may be computed efficiently. Furthermore, various real space properties can be encoded through sensible placement of the Dirac delta functions.
The spherical Dirac delta function dw centered at w/ = (0',6') E S² is defined as
(w) sin ó 0 cos 8')((( σ'),
(35)
where 0g is the familiar Dirac delta function on the reals centered O. The Dirac delta on the sphere may be represented in harmonic space by
(ow)) ) yd (w=))
Ne pe (cos 0e -ime!
(36)
15

---

Published as a conference paper at ICLR 2021
which follows form the sifting property of the Dirac delta, and where Y, denote the spherical harmonic functions, Pm (2) are associated Legendre functions and
28 + 1 (1 m)!
N
(37)
V
4x (1 + m)!
is a normalizing constant.
This representation may then be used define filter (S2) weighted sum of spher- ical Dirac delta functions, with weights Wij assigned to Dirac delta functions centered at points {(0,, Ø3) 1, 1,,,,,,,,, j L,,.. Noj. The associated harmonic space representation given by
Σ
Wij Not P (cos 0m)e
(38)
Σ No P (cos 0:) Σ Wije Wiie-iims, imds,
(39)
where fast Fourier transforms may be leveraged to compute the inner sum if the Dirac deltas are spaced evenly azimuthally (e.g. if Φ; Ng). Alternative arbitrary samplings can of course be considered if useful for a problem at hand.
When defining filters in this manner one should be careful not to over parametrize by assigning more weights than needed to define filter the harmonic bandlimit of the signal with which we wish to convolve. For example, if the filter is to be convolved with signal bandlimited at L then a maximum of 27, Dirac deltas should placed along each ring of constant 6. One may also choose to interpolate the weights from smaller number of learnable parameters acting anchor points, allowing higher resolution filters to be defined with fewer Iearnable parameters.
C.2 DIRAC DELTA FILTERS ON THE ROTATION GROUP
Similarly Dirac delta function 6pr on the rotation group SO(3) centered position p' = (',,8','') (0(3) is defined
(p) sin óz(a a')oz(cos - cos 3')0)7)))27 -7));
(40)
with harmonic form
Kmn Dinn e eimaa de (B")e iin'
(41)
where L are Wigner (small) d-matrices.
The filter (SO(3)) corresponding to weighted sum of Dirac deltas with weights Wijk signed to Dirac delta functions centered at points {(as, Bj, 7k) 1,..., Nai j = 1,... No: k 1,.., 3 has harmonic form
mm
Σ A Wijk
--may de (3i)e in
(42)
Σ (Bi)
Σ imay
Wizee
inn
(43)
where again fast Fourier transforms may be leveraged to compute the inner two sums assuming the Dirac deltas are spaced evenly in and 7. The outer sums of Equation and Equation 43 can also be computed by fast Fourier transforms by decomposing the Wigner d-matrices into their Fourier
representation (cf. (Trapani & Navaza 2006 McEwen & Wiaux 2011 One should again be careful not to over-parametrize.
D EQUIVARIANCE TESTS
N; in To test rotational equivariance of operators we consider N, 100 random signals {f 1.2 (St)) with harmonic coefficients sampled from the standard normal distribution and N₂ 100
16

---

Published as a conference paper at ICLR 2021
random rotations {pp} Nr sampled uniformly SO(3). In order measure the extent to which an operator A: (S11) (222)) is equivariant we evaluate the mean relative error
N ΣΣ
[IA(R fi) - RR (Afa)ll
Rp, (AI)) N₁ N
(44)
o1)
resulting from pre-rotation of the signal, followed by application of A, as opposed to post-rotation after application of AA where the operator norm || defined using the inner product <:, (P)2))
Table presents the mean relative equivariance errors computed. We consider the three standard convolutions described in Appendix B (with random filter dy for each signal ft, generated in
the same manner as fi), the pointwise ReLU activation described in Section [2.51] for signals on
the sphere (S2, 8*) and rotation group (S2) = 50(3)), and the composition of tensor-product activation with generalized convolution, described in Sections 2.5.2 and 2.4, respectively. We follow the tensor-pree cunsurpproduct activation with a generalized convolution in order to project down onto the sphere allow the same notion of errоr to be adopted as for the other operators. For consistency with the context in which we leverage these operators, all experiments are performed using single- precision arithmetic.
We see that all three standard notions of convolution and the composition of the tensor- prosurt activation and generalized convolution are all strictly equívariant floating point machine precision, with errors on the order of 10-7. The pointwise ReLUU operator is not strictly equivariant, with a mean relative error of 0.37 for signals on the rotation group and 0.34 for signals on the sphere. These errors reduce when the signals are oversampled before application of the ReLU, indicating that the error is due aliasing induced by the spreading of information to higher degrees not captured at the original bandlimit. For example, for the pointwise ReLU operator on the rotation group oversampling by factors of 2x, 4x and 8x results in a reduction in the mean relative equivariance error from 0.37 to 0.098, 0.032 and 0.0096, respectively.
Table 4: Layer equivariance tests
Layer
Mean Relative Error
S² to S conv.
4.4 x 10
Sa to SO(3) conv.
5.3 10-7
SO(3) SO(3) conv.
9.3 10-7
Tensor-product activation - Generalized conv.
5.0 10-7
S2 ReLU
3.4 x 10
8² ReLU (2x oversampling)
8.9 10-2
8
ReLU (4x oversampling)
2.9 10
S
ReLU (8x oversampling)
1.3 x 10-2
S0(3) ReLU
3.7 10
8O(3) ReLU (2x oversampling)
9.8 10
80(3) ReLU (4x oversampling)
3.2 10 10
S0(3) ReLU (8x oversampling)
9.6 x 1
E CONNECTION BETWEEN THE TENSOR PRODUCT ACTIVATION AND POINTWISE SQUARING
To provide some intuition on the manner in which the tensor-product based activation introduces non-linearity into representations we describe its relationship to pointwise squaring for signals on the sphere. Here we consider the operator N L²(3") (S2) satisfying (N))(z) for all x E S?, which differs subtly to No, with a(z) = x² (using notation from Section 2.5.1
(x
which corresponds to obtaining a sample-based representation at a finite bandlimit L and applying the squaring at the sample positions. For the special case oxx) = x² we can directly compute the harmonic representation corresponding to the equivariance-preseree
continuous limit /. - sc.
17

---

Published as a conference paper at ICLR 2021
1044
10
Memory (MB)
FLOPS
10th
10-
10a
Kondor et al. (2018)
10-1
Kondor et (2018)
Ours
30x
Reduction
Reductin
20x
50x
20
40
60
80
100
120
20
40
60
80
100
120
Bandlimit
Bandlimit
Computational cost
(b) Memory requirements'
Figure 3: Comparison of computation and memory footprints between the generalized spherical CNN layers of Kondor et al. (2018) and efficient generalized layers. The reduction in cost due to our efficient layers is given as multiplicative factors in the lower plot of each panel.
Given a spherical signal f E (S²) with generalized representation
( E C24+1 O,.., L - 1}, the generalized representation of the signal f L2()²) is given
-
Σ (Gh1 ( f : l 0, 1),
(45)
(11,2)EPP,,
where Gh.la. (126+)))x(2++))xxx are Gaunt coefficients defined as
Ghiaaja m
du(w)) (w) (w)Yt (w))
(46)
_ga
Gaunt coefficients are related to the Clebsch-Gordan coefficients by
Ghiaia mamms
w ii2za mimmmm3
Chiiejs
(47)
COUD Therefore, the continuous squaring operation corresponds to passing through tensor-product activation Ng. followed by generalized con- volution back down onto the sphere (single fragment per degree) with weight assigned to the (f1,62) Py, fragment in degree-l given by whll
where whijah = (-1)m², (2n 1) +1)
4n 4m +1)
This demonstrates that activations that are learnable within our framework can have very simple real- space interpretations. Even when confining outputs the sphere we found it be beneficial to allow the down-projection to be learnable rather than enforcing the weights given above for pointwise squaring. Learned activations will remain quadratic, however, given that output fragments linear combinations of products between input fragments.
F COMPARISON OF COMPUTATIONAL AND MEMORY FOOTPRINTS
We perform quantitative analysis of the computational cost and memory requirements of our pro- posed layers, and comparisons to prior approaches, to demonstrate how the complexity savings made through our proposals in Section 3 translate to tangible efficiency improvements.
We consider the simplest comparison, between a multi-channel generalized signal ff.....Kk FK where each channel has type Ty. (1,..., 1) (and therefore corresponds to signal on the sphere) and uni-channel signal E FL of type 7 (K..., K). The setting corre- sponding to signal ff captures the efficiency improvements of our proposed layers, while the setting corresponding to y represents the case without these improvements. Notice that the total number of fragments is the same in both and We compare the number of floating point operations (FLOPs) and amount of memory required to perform lensor-produce based activation followed by general- ized convolution projecting back down onto signal of the same type as the input. When applied to f. MST mixing sets of (Section 3.1.3) and constrained generalized convolutions (Section 3.1.2
18

---

Published as a conference paper at ICLR 2021
SO(3) Conv.
Coustrained Gen. Conv.
s Conv.
I
I
Tensor Products
Tensor Products
Tensor Products
ReLU
ReLU
Constrained Gen. Conv:
Constrained Conv.
Constrained Gen. Conv.
I
SO(3)
Efficient
Efficient
Efficient
S Layer
Layer
Layer
Gen. Layer
Gen. Layer
Figure 4: Visualization of the architecture used for the convolutional base in hybrid models. The input to the first convolutional layer is a signal on the sphere. The output from the final convolutional layer are scalar values corresponding to fragments of degree which are then mapped through some fully connected layers to give the model output.
used. When applied to g full mixing sets IPt and unconstrained generalized convolutions used, as in Kondor et al. (2018). Considered are the costs for single training instance (batch size of 1).
Figure shows the computational costs and memory requirements, in terms of floating point op- erations and megabytes respectively, for K = 4 and various spatial bandlimits L. We adopt the convention whereby complex addition and multiplication require and 6 floating point operations respectively. At low bandlimits we see the saving arising from the channel-wise structure. Note that the saving illustrated here relatively small since = for these experiments (so that the scaling is apparent), whereas in practice typically K ~ 100. The saving then increases linearly in response to increases the bandlimit the input, as expected given the O(L²) and O(L*) spa- tial complexities. We that even in this simple case, with relatively small number of channels (K = 4), both the computational and memory footprints are reduced by orders of magnitude. At bandlimit of 128 the computational cost is 101 1tmmtimes reduced and the memory requirement is 29-times reduced.
G ADDITIONAL INFORMATION ON EXPERIMENTS
G.1 ROTATED MNIST ON THE SPHERE
For our MNIST experiments we used hybrid model with the architecture shown in Figure The first block includes directional convolution the sphere that lifts the spherical input (r 1)
onto the rotation group (rfa) min(2f ff + 2,2, 1)). The second block includes a convolution on the rotation group, hence its input and output both live on the rotation group. We then apply restricted generalized convolution to map type rfae: /max/v22 1). where Tmax The same type is used for the following three channel-wise tensor-product activations and two restricted gen- eralized convolutions until the final restricted generalized convolution maps down to rotationally invariant representation (Tf00) = òzo). As is traditional in convolution networks we gradually de-
crease the resolution, with (o, L1, L2, La, La. Ls) (20, 10. 10, 6, 3, 1), and increase the number of channels, with (Ko, K1, K2, K3, K4, K5) tional layers with single dense layer of size 30, sandwiched between two dropout layers (keep probability 0.5), and then fully connect to the output of size 10.
(1, 20, 22, 24, 26, 28). We proceed these convolu-
We train the network for 50 epochs on batches of size 32, using the Adam optimizer (Kingma & Ba 2013) with a decaying learning rate starting at 0.001. For the restricted generalized convolutions follow the approach of Kondor al. (2018) by using L₃ regularization (regularization strength 10 5) and applying a restricted batch normalization across fragments, where the fragments are only scaled by their average and not translated (to preserve equivariance).
19

---

Published as a conference paper at ICLR 2021
G.2 ATOMIZATION ENERGY PREDICTION
When regressing the atomization energy of molecules there are two inputs to the model: the number of atoms of each element contained in the molecule; and spherical cross-sections of the potential energy around each atom. We adopt the high-level QM7-specific architecture of Cohen et al.] (2018) which contains spherical CNN as a sub-model. for which we substitute our own. This results in an overall model that is invariant to both rotations of the molecule around cach constituent atom and to permutations of the ordering of the atoms.
The first (non-spherical) input is mapped onto a scalar output using multi-layer perceptron (MLP) with three hidden layers of sizes 100, 100 and 10 (and ReLU activations). The second input, multiple spherical cross sections for each atom, are separately projected using a shared spherical CNN (of architecture described below) onto lower dimensional vectors of size 64. The mean vector is then taken across atoms (ensuring invariance w.r.t. permutations of the atoms) and mapped onto scalar output using an MLP with single hidden layer size 512 (with a ReLU activation). The predicted energy is then taken to be the of the two scalar outputs.
As a starting point we train the first MLP to regress the atomization energies alone (achieving RMS 20), before pairing it with the spherical model (and its connected MLP). We then train the joint model for 60 epochs, again with the Adam optimizer, decaying learning rate (starting at 2.5 x 10-1), regularizing the efficient generalized layers with L2 regularization (strength 2.5 10 10-6) and batch sizes of 32.
For the spherical component we again adopt the convolutional architecture shown in Figure 4 except with one fewer efficient generalized layer. We use bandlimits of (Lo, L1, L2, L3, L4)= 6, 3, 1), channels of (Ko, K1, K2, K3, K2)=(5, 16, 24, 32, 40) and Tmax = 6. One minor difference is that this time we include a skip connection between the & = 0 components of the fourth and fifth layer. We proceed the convolutional layers with two dense layers of size (256, 64) and use batch normalization between each layer.
G.3 3D SHAPE RETRIEVAL
To project the 3D meshes of the SHREC'17 data onto spherical representations (bandlimited at L = 128) adopt the preprocessing approach of Cohen et al. (2018) and augment the data with random rotations and translations.
We construct model with architecture that is again similar to that described in Appendix G. but with an additional axisymmetric convolutional layer prepended to the start the net- work and one fewer efficient generalized layers. We use bandlimits (Lo, L1, L2, La, L4, Ls) (128, 32, 16, 16, 6, 1), channels (Ko, K1, K2, K3, K4, Ks) = (6, 20, 30, 40, 60, 70) and Tmax = 6 for the efficient generalized layers. The convolutional layers are followed by dense layer of size 128 which is fully connected to the output (of size 55).
We again train with the Adam optimizer, a decaying learning rate (starting sizes of 8, this time until performance on the validation set showed no improvement for at least 4 epochs (36 epochs total). We perform batch normalization between convolutional layers and dropout preceding the dense layer. We regularize the efficient generalized layers with Le regu- larization (strength 10 10-5). When testing our model we average the output probabilities over 15 augmentations of the data.
10 10-4) and batch
20


Reviews:
[{"review_id": "xTWYk96CIM0", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summarize what the paper claims to contribute.**\nThe authors claim to introduce an efficient alternative to previous Spherical CNN models\n\n**Strengths:**\nThe authors consider the problem of spherical image processing using convolution\nThe authors present strong empirical results\n\n**Weaknesses:**\nBoth the mathematical presentation and discussion are difficult to follow\n\n**Clearly state your recommendation (accept or reject) and justification.**\nReject. It seems the authors have given considerable attention to the problem and produced compelling results; however, for me, the mathematical presentation and discussion are difficult to follow which I expect will make it difficult for readers to understand and build upon what has been done. My impression is that some of the difficulty could be resolved with more standard notational choices (e.g. nonlinearities are not often written \\mathcal{N}_\\otimes), and limiting the use of inline equations.\n\n**Ask questions you would like answered by the authors to help you clarify your understanding of the paper and provide the additional evidence you need to be confident in your assessment.**\n(first paragraph 2.1) Does the operator map spherical signals to signals on SO(3)?\nThe mathematical presentation is given with filters and functions in \\mathbb{C}, is reflected in the implementation?\nIt is unclear to me what the authors mean (specifically) by a hybrid approach\n\n**Provide additional feedback with the aim to improve the paper.**\nPerhaps state that \\mathcal{H} is the space of spherical signals and the superscript indicates the layer\nThe notation is a bit difficult to follow (and read since quite a bit is inline) and often is not explained, for example it could be helpful to say that L^2(S^2) are the square integrable functions on the sphere and show what that means.\nI think the paper would be easier to read if the language was consistent, for example, in the introduction the language of real and harmonic space is used and in section 2 it seems to change to real and Fourier space. I wonder if spatial and spectral are good words to use in place of these.\n(paragraph below eqn 3) remove (w.r.t)\nIn part because the authors attempt to describe [1,2] and [3] at the same time and because of the abundance of long inline equations, the mathematical presentation is difficult to follow\nMoreover, the mathematics are not trivial and not particularly well known, perhaps providing intuition along with the equations would improve readability\n\n**Possible typos:**\n(Conclusion) powerful hybrid model → powerful hybrid models\n(Introduction) Many fields involve → many fields use\n\n**Post rebuttal**\nWith consideration of the improved readability of the new submission and comments of other reviewers, I have modified both my initial rating and confidence.\n\n[1] Kondor, Risi, Zhen Lin, and Shubhendu Trivedi. \"Clebsch–gordan nets: a fully fourier space spherical convolutional neural network.\" Advances in Neural Information Processing Systems. 2018.\n[2] Taco Cohen,  Mario Geiger,  Jonas K ̈ohler,  and Max Welling.   Spherical CNNs.   InInternationalConference on Learning Representations, 2018\n[3] Carlos  Esteves,  Christine  Allen-Blanchette,  Ameesh  Makadia,  and  Kostas  Daniilidis.   LearningSO(3) equivariant representations with spherical CNNs.   InProceedings of the European Con-ference on Computer Vision (ECCV), pp. 52–68, 2018\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "hf9JjN8IWpz", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces a framework for computationally efficient and exactly rotation-equivariant spherical CNNs. The work most closely resembles the Fourier space method of Kondor et al., but improves on it in a number of ways: firstly, a channel-wise structure is introduced for the tensor product nonlinearities, which avoids the degree blowup of this operation while still allowing mixing between different harmonic degrees. Secondly, computational complexity of linear layers is reduced by factorizing it into three operators, two of which operate similar to depthwise-separable convolutions and one of which acts uniformly across channels. Thirdly, an optimized sparse degree mixing set is proposed, based on a minimum spanning tree. Finally, a more efficient sampling theorem is used that reduces the Nyquist rate by a factor of two compared to the ones used in previous works on spherical CNNs.\n\nThe paper is very well written and the authors clearly have a thorough understanding of the noncommutative harmonic analysis involved. This does not mean the paper will be easy to understand for all readers, but for those familiar with the relevant mathematics, either from textbooks or earlier works in the spherical CNN literature, the paper is very readable. The proposed improvements make a lot of sense to me, and their computational complexity improvements are clearly stated. The performance of a network architecture that includes the new layers is tested and shown to yield competitive or state of the art performance on several benchmark problems that have been used in many previous works.\n\nOverall I think this is a very nice paper, but I have a few minor concerns and points of improvement:\n\nThe degree mixing set (3.1.3) is a minimum spanning tree that minimizes a certain computational cost. This makes some sense, but it is not clear to me that this approach is optimal in any meaningful sense or necessary at all. I have personally experimented with sparse channel connectivity in planar CNNs, and found that it does not seem to matter much how exactly the channels are connected, with the main factor determining compute/accuracy being the number of connections. Full degree mixing does seem desirable, but this implies the need of a MST only if one wishes to use the same connectivity structure in multiple layers. An interesting baseline would be to do the degree mixing using a random pattern in each layer, with various sparsity levels. It may turn out that only the sparsity level but not the precise connectivity structure matters in practice. Such a finding would not diminish the paper's significance.\n\nIt should be clarified in the paper that full mixing happens only across several layers (as many as the maximum path length / tree width in the MST). The question then arises whether full mixing actually happens in the considered architecture, given that it is not very deep.\n\nIt would be interesting to see actual implementation details in some DL framework, as well as wallclock timings. Also, code would be much appreciated.\n\nThe appendix describes a method for enforcing spatial localization of the spectral filters, but it is not clear from the paper if/how this is actually used in the network architecture that is tested.\n\nIt would be nice to know why the initial convolution layers are necessary, instead of just using the generalized layers introduced in this paper in their full glory.\n\nI may have missed it, but could not figure out what L_G^(psi) refers to in 2.6.\n\nPost rebuttal update: I have read the author response and updated paper, as well as the other reviews, and have decided to maintain my rating.", "extra_scores": {}, "extra_text": {}}, {"review_id": "U0y8uGYjYRj", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a generalized spherical convolution operation that is strictly equivariant to rotation. The authors show that the spherical convolution operations introduced in prior works can be encompassed by the proposed approach. Because spherical convolutions introduce significant computational overhead, the authors also introduce an array of methods that reduce the computational cost while maintaining the model accuracy. Experiment results on multiple benchmark datasets show that the proposed approach outperforms the alternative approaches while having less number of parameters.\n\nThis paper studies an important problem. In particular, it addresses an important issue in spherical convolution operation, i.e. the computational cost of the operation. The proposed operation has the desirable property of strict rotational invariance, and it is general enough to replace existing spherical convolution operators and may be used as the basic component for CNN on spherical signals. The experiment results also verify the benefit of the proposed method.\n\nOn the other hand, there are several aspects on which the paper may be improved. First of all, there are some designs in the proposed method that are not carefully discussed or tested:\n1) While the authors use tensor-product to replace pointwise activation, it is unclear what's the relation between these two operations. Is tensor-product equivalent, more or less expressive than pointwise activation? Given that activation plays an important role in neural network, the authors should try to provide more information about the new activation function.\n2) The authors propose channel wise activation and degree mixing to reduce the computational cost. However, they also reduce the expressiveness of the model. Therefore, it is worthwhile to provide some study on how they impact the performance of the model. For example, what will the model performance be if these methods are not applied?\n\nSecond, the experiments are somehow limited. The authors only test the proposed convolution operations on a single model, and the model size is different from the baselines except for the MNIST experiment. It is unclear why the model sizes are not tied in the experiments. A more informative experiment will be comparing different methods over different model sizes. Also, while the main contribution of this work is to reduce the time complexity of the convolution operation, the experiments do not show the comparison in run time. The authors should also try to evaluate the model efficiency as well as memory overhead, as these are also important factors that limit the usage of spherical convolution operation.\n\nThe rebuttal provides valuable information that was missing in the original paper and improves the readability. Therefore, I recommend accepting the paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "EwuNng7dIAf", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors introduce channel-wise convolutions, and an optimized degree mixing set in order to construct equivariant layers that exhibit improved scaling properties and parameter efficiency on some prototypical spherical CNN tasks. \n\nSection 2 is unnecessarily math heavy with representations and terminologies introduced which are not relevant to the central claims in the paper and not reused in latter sections. The authors should pick out the essentials bits and place the rest of the technical bits to the supplement. The gained space should be used to expand and better explain section 3 which is extremely hard to understand. \n\nReading and re-reading Section 3 several times, I am still lost as to what figure 1 is trying to demonstrate. I do not understand what are the trade-offs involved with the constrained generalized convolution as opposed to generalized convolution. From the results it seems that it does not matter, which is counter intutitive. Group convolution has an adverse effect on performance on standard CNNs. I am not sure about the validity of the following statement: \"restricted N_b in which only a subset of P_L is used for each degree ` still defines a strictly equivariant operator\". Though it makes intutive sense, is there a proof of the same? \n\nThe results section leaves a lot to be desired. In table 3, we see that it is not state of the art on several metrics. The authors do not compare with the improved Esteves et. al. paper from 2020. This result should be added. What is the implication of the CL logL complexity. This should translate to reduced flop count, but there is no discussion on flop or timing of this approach anywhere in the paper. This makes me skeptical whether the proposed approach improves efficiency in practice. There are many typos, incomplete sentences, long hard to read sentences etc. I would recommend the authors to compare on all benchmarks provided in Esteves et. al. (2020). \n\nOverall, the paper is a very hard read which no clear message on the key contributions. It seems that the MST approach is driving the efficiency (without proof?), coupled with group convolutions which is well known in leterature and cannot be credited as a contribution. This coupled with the marginal improvement on select datasets and incomplete evaluation does not inspire acceptance. \n\nPost rebuttal comment: Having read the reviews from other reviewers who are subject matter experts, and the authors rebuttal which helped clarify most of my concerns, I am increasing my rating for the paper.  I recommend acceptance as two of the reviewers are convinced about the positive impact of the paper. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an efficient approach for computing equivariant spherical CNNs, significantly reducing the memory and computation costs. Experiments validate the effectiveness of the proposed approach.

Pros:
1. Speeding up equivariant spherical CNNs is a valuable topic in deep learning. 
2. The proposed approach is effective, in all parameter size, memory footprint and computation time.
3. The theory underpinning the speedup method is sound.

Cons:
1. The readability should be improved. Two of the reviewers complained that the paper is hard to read and only Reviewer #2 reflected that it is "easy" to read (but only under the condition that the readers are familiar with the relevant mathematics), and this situation is improved after rebuttal. Nonetheless, this should be further done.
2. The experiments are a bit limited. This may partially be due to limited benchmark datasets for spherical data, but for the existing datasets used for comparison, Esteves et al. (2020) is not compared on all of them. Esteves et al. (2020) is only reported on spherical MNIST, which has very close performance to the proposed one. This worries the AC, who is eager to see whether on QM7 and SHREC’17 the results would be similar. 

After rebuttal, three of the reviewers raised their scores. So the AC recommended acceptance.
Decision: Accept (Poster)