S CATTERING N ETWORKS ON THE S PHERE FOR S CALABLE AND R OTATIONALLY E QUIVARIANT S PHERICAL CNN S
A BSTRACT
Convolutional neural networks (CNNs) constructed natively on the sphere have been developed recently and shown to be highly effective for the analysis of spherical data.
While an efﬁcient framework has been formulated, spherical CNNs are nevertheless highly computationally demanding; typically they cannot scale beyond spherical signals of thousands of pixels.
We develop scattering networks constructed natively on the sphere that provide a powerful representational space for spherical data.
Spherical scattering networks are computationally scalable and exhibit rotational equivariance, while their representational space is invariant to isometries and provides efﬁcient and stable signal representations.
By integrating scattering networks as an additional type of layer in the generalized spherical CNN framework, we show how they can be leveraged to scale spherical CNNs to the high-resolution data typical of many practical applications, with spherical signals of many tens of megapixels and beyond.
I NTRODUCTION
The construction of convolutional neural networks (CNNs) on the sphere has received considerable attention recently, opening up their application to ﬁelds where standard (Euclidean) CNNs are not effective.
Such techniques are ﬁnding application in diverse ﬁelds; for example, for the analysis of 360 ◦ photos and videos in virtual reality and for the analysis of astrophysical observations on the celestial sphere in cosmology.
In both of these ﬁelds high-resolution spherical images are typically encountered, with image resolutions often reaching many tens of megapixels.
Current spherical CNN approaches that are deﬁned natively on the sphere and satisfy rotational equivariance, however, are limited to low resolutions and often cannot scale beyond thousands of pixels, severely restricting their applicability.
In this article we present a strategy to scale spherical CNNs to high resolution, further opening up their applicability to high-resolution spherical images encountered in virtual reality, cosmology, and many other ﬁelds beyond.
A number of spherical CNN constructions have been proposed recently, which can be broadly categorized into two approaches: (i) those that are deﬁned natively on the sphere and capture rotational equivariance (e.g. Cohen et al., 2018a; Esteves et al., 2018; 2020; Kondor et al., 2018;
Cobb et al., 2021); and (ii) those that are constructed on discretizations of the sphere and typically do not fully capture rotational equivariance (e.g. Boomsma & Frellsen, 2017; Jiang et al., 2018;
Perraudin et al., 2019; Cohen et al., 2019).
While constructions on discretized representations of the sphere can often be computed efﬁciently, and some capture rotational equivariance to a small set of rotations (Cohen et al., 2019), such approaches necessitate an approximate representation of spherical signals, losing the connection to the underlying continuous symmetries of the sphere and thus cannot fully capture rotational equivariance.
Approaches constructed natively on the sphere that do capture rotational equivariance, on the other hand, are highly computationally demanding.
Despite considerable improvements in computational efﬁciency proposed recently (Cobb et al., 2021), such approaches nevertheless remain computationally demanding and are not scalable to high resolution.
We seek a framework that captures the underlying continuous symmetries of the sphere and exhibits rotational equivariance, while also being computationally scalable to high resolution.
We draw inspiration from Cobb et al.
(2021), which presents a generalized spherical CNN framework and advocates hybrid networks where different types of spherical CNN layers are leveraged alongside each other.
We consider an additional layer to be integrated into this hybrid approach that must, critically, be scalable to high resolution and allow subsequent layers to operate effectively at low resolution, while also exhibiting rotational equivariance.
To provide an effective representational space it should also provide efﬁcient and stable signal representations, with control of the scale of signal invariances, and be sensitive to all signal content.
Scattering networks meet all requirements.
Scattering networks were ﬁrst proposed in the seminal work of Mallat (2012) in an effort to provide some further theoretical basis explaining the observed empirical effectiveness of CNNs (elaborated in Mallat 2016).
They are constructed from a cascade of linear transforms (typically convolutions), combined with pointwise non-linear activation functions, with carefully designed ﬁlters to ensure certain stability and invariance properties.
A scattering network is thus effectively a CNN but with designed, rather than learned, ﬁlters.
Typically wavelets that themselves satisfy certain stability and invariance properties are adopted for the ﬁlters and the absolute value function is adopted for the nonlinear activation function.
Scattering networks were initially constructed on Euclidean domains and have been applied in a variety of applications (e.g. Bruna & Mallat, 2011; 2013; And´en et al., 2019).
More recently scattering networks that support non-Euclidean data have been constructed through graph representations (Gama et al.
, 2019b;a; Gao et al., 2019; Zou & Lerman, 2020; Perlmutter et al., 2019) or on general Riemannian manifolds (Perlmutter et al., 2020).
While these approaches are ﬂexible, this ﬂexibility comes at a cost.
Just as there are considerable advantages in constructing
CNNs natively on the sphere (rather than, e.g., adopting graph CNNs), so too there are advantages in constructing scattering networks natively on the sphere.
For example, spherical scattering networks can be constructed to retain a connection to the underlying continuous symmetries of the sphere and to leverage fast algorithms tailored to the harmonic structure of the sphere.
In this article we construct scattering networks on the sphere, motivated by their use as an initial layer in hybrid spherical CNNs in order to yield rotationally equivariant spherical CNNs that scale to high resolution.
In Section 2 we review spherical signal representations and describe the invariance and stability properties required to provide an effective representational space.
In Section 3 we construct scattering networks on the sphere and discuss their invariance and stability properties.
In Section 4 we integrate scattering networks into the existing generalized spherical CNN framework.
Experiments studying the properties of spherical scattering networks and demonstrating their effectiveness in hybrid spherical CNNs are shown in Section 5.
Concluding remarks are made in Section 6.
S PHERICAL S IGNAL R EPRESENTATIONS
We seek a representational space that can be used as an initial layer in hybrid spherical CNNs to scale to high-resolution data.
Such a representational space must, critically, be scalable, allow subsequent
CNN layers to operate at low resolution, and be rotationally equivariant.
Moreover, to provide an effective representation for machine learning problems the space should provide efﬁcient and stable signal representations, with control over the scale of signal invariances, while being sensitive to all signal content.
In this section we concisely review common spherical signal representational spaces, including spatial, harmonic and wavelet signal representations, before discussing the desired invariance and stability properties of the representational space that we seek.
2. S PATIAL AND H ARMONIC R EPRESENTATIONS
Consider square integrable signals f, g ∈ L 2 ( S 2 ) on the sphere S 2 with inner product (cid:104) f, g (cid:105) .
The inner product induces the norm (cid:107) f (cid:107) 2 = (cid:112) (cid:104) f, f (cid:105) , with metric d L 2 ( S 2 ) ( f, g ) = (cid:107)
f − g
(cid:107) 2 .
We consider spherical signals bandlimited at L , with harmonic coefﬁcients ˆ f (cid:96)m = (cid:104) f, Y (cid:96)m (cid:105)
= 0 , ∀ (cid:96) ≥ L , where Y (cid:96)m are the spherical harmonic functions of natural degree (cid:96) and integer order | m | ≤ (cid:96) .
Sampling theories on the sphere (Driscoll & Healy, 1994; McEwen & Wiaux, 2011a) provide a mechanism to capture all information content of an underlying continuous function from a ﬁnite set of samples.
By providing access to the underlying continuous signal, such a representation perfectly captures all spherical symmetries and geometric properties.
We adopt the sampling theorem on the sphere of McEwen & Wiaux (2011a) since it provides the most efﬁcient sampled signal representation (reducing the Nyquist rate by a factor of two compared to Driscoll & Healy 1994).
2. W AVELET R EPRESENTATIONS
A number of wavelet frameworks on the sphere have been constructed (see Appendix A).
We focus on spherical scale-discretized wavelets since they have an underlying continuous representation, exhibit excellent localization and asymptotic correlation properties, are a Parseval frame, support directional wavelets, and exhibit fast algorithms for exact and efﬁcient computation (McEwen et al., 2007; Wiaux et al., 2008; Leistedt et al., 2013; McEwen et al., 2013; 2015c; 2018).
Furthermore, the wavelet transform is rotationally equivariant in theory (since it is based on spherical convolutions) and in practice (since its computation leverages underlying spherical sampling theory).
The (axisymmetric) spherical scale-discretized wavelet transform is deﬁned by the convolution of f ∈ L 2 ( S 2 ) with the wavelet ψ j ∈ L 2 ( S 2 ) , yielding wavelet coefﬁcients W j ( ω ) =
( f (cid:63) ψ j )( ω ) =
(cid:104) f, R ω ψ j (cid:105) , where ω
= ( θ, ϕ ) denotes a point on the sphere with colatitude θ ∈ [0 , π ] and longitude ϕ ∈ [0 , 2 π ) , and R ω
= R ( ϕ,θ, 0) denotes a three-dimensional (3D) rotation (we adopt the active zyz
Euler convention).
The wavelet scale j encodes the angular localization of ψ j , with increasing j corresponding to smaller scale (i.e. higher frequency) signal content.
The minimum and maximum wavelet scales considered are denoted J 0 and J , i.e. 0 ≤ J 0 ≤ j ≤ J .
The wavelet coefﬁcients capture high-frequency or bandpass information of the underlying signal; they do not capture lowfrequency signal content.
A scaling function φ ∈ L 2 ( S 2 ) is introduced to capture low-frequency signal content, with scaling coefﬁcients given by W ( ω ) =
( f (cid:63) φ )( ω ) =
(cid:104) f,
R ω φ (cid:105) .
For notational convenience we introduce the operator notation w j = Ψ
j f = f (cid:63) ψ j and w =
Φ f = f (cid:63) φ .
Further details regarding the spherical scale-discretized wavelet construction, synthesis and properties are contained in Appendix A. Note that the wavelet construction relies on a ﬁxed dilation parameter λ ∈ R + ∗ , λ > 1 , that controls the relative scaling of harmonic degrees one wishes each wavelet to probe.
A common choice is dyadic scaling with λ = 2 .
The minimum wavelet scale J 0 may be chosen freely or may be set by specifying a desired bandlimit L 0 for the scaling coefﬁcients, yielding
J 0 = (cid:100) log λ
L 0 (cid:101) .
For J 0 =
the wavelets probe the entire frequency content of the signal of interest except for its mean, which is encoded in the scaling coefﬁcients.
The wavelet transform of a signal bandlimited at L is therefore speciﬁed by the parameters ( L, λ, J 0 ) , or alternatively ( L, λ, L 0 ) .
2. I SOMETRIES AND D IFFEOMORPHISMS
While we have mentioned the signal representation we seek must satisfy invariance and stability properties, we are yet to elaborate on these properties in detail.
We deﬁne the transformations under which these properties are sought, before describing the properties themselves in the next subsection.
Consider transforms of spherical signals that are an isometry , preserving the distance between signals.
More precisely, consider the isometry ζ ∈ Isom( S 2 ) , where Isom( S 2 ) denotes the isometry group of the sphere.
The action of an isometric transformation V ζ : L 2 ( S 2 ) → L 2 ( S 2 ) is deﬁned as V ζ f ( ω ) =
f ( ζ − 1 ω ) .
The distance between signals f, g ∈ L 2 ( S 2 ) is then preserved under an isometry, i.e. d L 2 ( S 2 ) ( V ζ f, V ζ g ) =
d L 2 ( S 2 ) ( f, g ) .
One of the most typical isometries considered for spherical signals is a 3D rotation R ρ , often parameterized by Euler angles ρ =
( α, β, γ ) ∈ SO(3) .
Consider deformations of spherical signals, formally deﬁned by diffeomorphisms .
A diffeomorphism is a transformation that is a differentiable bijection (a one-to-one mapping that is invertible), with an inverse that is also differentiable.
Loosely speaking, a diffeomorphism can be considered as a “well-behaved” deformation.
Consider a diffeomorphism ζ ∈ Diﬀ( S 2 ) , where Diﬀ( S 2 ) denotes the diffeomorphism group of the sphere.
The action of a diffeomorphism on a spherical signal is deﬁned similarly to an isometric transformation.
It is useful to quantify the size of a diffeomorphism.
Let (cid:107) ζ (cid:107) ∞ = sup ω ∈ S 2 d S 2 ( ω, ζ ( ω )) represent the maximum displacement between points ω and ζ ( ω ) , where d
S 2 ( ω, ω (cid:48) ) is the geodesic distance on the sphere between ω and ω (cid:48) .
Often we are interested in small diffeomorphisms about an isometry such as a rotation, denoted ζ (cid:48)
=
ζ 1 ◦ ζ 2 for some isometry ζ 1 ∈ Isom( S 2 ) , e.g. ρ ∈ SO(3) , and some diffeomorphism ζ 2 ∈ Diﬀ( S 2 ) .
2. I NVARIANT AND S TABLE R EPRESENTATIONS
In representational learning, invariances with respect to isometries and stability with respect to diffeomorphisms play crucial roles.
As an illustrative example consider the classiﬁcation of handwritten digits in planar (spherical) images.
Translation (rotation) is a common isometry to which we often seek to encode various degrees of invariance as an inductive bias in machine learning models.
In some cases small translations (rotations) may be responsible for intra-class variations (e.g. if the digit is always located close to the center of the image), whereas in others global translations (rotations) may not alter class instances (e.g. if the digit may be located anywhere in the image).
Invariance to isometries up to some scale is therefore an important property of effective representation spaces.
Intra-class variations may also be due to small diffeomorphisms (e.g. deformations in the way a given digit is written), while inter-class variations may be due to larger diffeomorphisms (e.g. deformations mapping one digit to another).
Consequently, an effective representation space is one that is stable to diffeomorphisms, i.e. is insensitive to small diffeomorphisms and changes by an increasing amount as the size of the diffeomorphism increases.
It is well-known that scattering networks constructed from cascades of appropriate wavelet representations, combined with non-linear activation functions, satisfy these invariance and stability properties, thus providing an effective representation space (e.g.
Mallat, 2012; 2016).
S CATTERING N ETWORKS ON THE S PHERE
We present the construction of scattering networks on the sphere, which follows by a direct analogy with the Euclidean construction of Mallat (2012).
Firstly, we describe conceptually how scattering representations satisfy the invariance and stability properties that we seek.
Secondly, we deﬁne the spherical scattering propagator and transform, constructed using the spherical scale-discretized wavelet transform (e.g. McEwen et al., 2018).
Thirdly, we use the scattering transform to deﬁne scattering networks.
Finally, we formalize and prove that scattering networks are invariant to isometries up to a given scale and are stable to diffeomorphisms.
3. S CATTERING R EPRESENTATIONS
Scattering representations leverage stable wavelet representations to create powerful hierarchical representational spaces that satisfy isometric invariance up to a particular scale, are stable to diffeomorphisms and probe all signal content (both low and high frequencies).
The scaling coefﬁcients of the scale-discretized wavelet transform on the sphere are computed by an averaging (smoothing) operation.
Scaling coefﬁcients therefore satisfy isometric invariance associated with the scale of the scaling function (i.e. J 0 or equivalently L 0 ).
However, scaling coefﬁcients clearly do not probe all signal content as they are restricted to low frequencies.
Wavelet coefﬁcients of the scale-discretized wavelet transform on the sphere do not satisfy isometric invariance.
However, they are stable to diffeomorphisms by grouping frequencies into packets through the Meyer-like tiling of the spherical harmonic space on which they are constructed.
Furthermore, they probe signal content at a range of frequencies, including high frequency signal content.
A wavelet signal representation, including scaling and wavelet coefﬁcients, does not recover a representational space with the desired invariance and stability properties: scaling coefﬁcients satisfy some desired properties, whereas wavelet coefﬁcients satisfy others.
However, by combining cascades of scaling and wavelet coefﬁcient representations, with non-linear activation functions, it is possible to recover a representation that exhibits all of the desired properties.
Scattering representations do precisely this.
Furthermore, all signal content is then probed, with high frequency signal content non-linearly mixed into low frequencies (Mallat, 2012).
3. S CATTERING P ROPAGATOR AND T RANSFORM
Scattering networks are constructed from a scattering transform, which itself is constructed using a scattering propagator.
The spherical scattering propagator for scale j is deﬁned by
[Equation]
(1) where a non-linear activation function A : L 2 ( S 2 )
→ L 2 ( S 2 ) is applied to the wavelet coefﬁcients at scale j .
Following Mallat (2012), we adopt the absolute value (modulus) function,
Af = | f | , for the activation function since it is non-expansive and thus preserves the stability of wavelet representations.
Moreover, as it acts on spherical signals pointwise, it is rotationally equivariant.
Scattering propagators can then be constructed by applying a cascade of propagators:
[Equation]
for the path p
= ( j 1 , j 2 , . . . , j d ) with depth d . By adopting carefully designed ﬁlters, i.e. wavelets, combined with a non-expansive activation function, scattering propagators inherit stability properties f
U [ J 0 ] f •••
U [ j ] f •••
U [ J ] f
S [0] f
U [ J 0 ,J 0 ]
f ••• U [ J 0 ,j 0 ] f
U
[ J,J 0 ] f
S [ J 0 ] f
S [ J 0 ,J 0 ] f S [ J 0 ,j 0 ] f
S [ J,J 0 ] f
U [ j,J 0 ] f ••• U [ j,j 00 ] f ••• U [ j,J ] f
S [ j ] f
S [ j,J 0 ] f
S [ j,j 00 ] f
S [ j,J ] f ••• U [ J,J 0 ] f
U [ J,j 000 ] f ••• U [ J,J ] f
S [ J ] f
S [ J,J 0 ] f
S [ J,j 000 ] f
S [ J,J ] f
[Equation]
for path p .
We adopt the convention that S [ p = 0] f = Φ f .
By projecting onto the wavelet scaling function we inherit the isometric invariance of the scaling coefﬁcients, with the scale of invariance controlled by the scale of the scaling function (i.e. by the J 0 or L 0 parameter).
Consequently, the scattering transform yields a representational space that satisﬁes both the desired invariance and stability properties (shown formally in Section 3.4).
Furthermore, since scattering representations are constructed from wavelet transforms and pointwise non-linear activation functions, both of which are rotationally equivariant, the resulting scattered representation is also rotationally equivariant.
For J 0 =
the scaling function is constant and we recover a representation that is completely invariant to isometries (in many settings we do not desire full invariance to rotations and so we set J 0 > 0 in the numerical experiments that follow).
Formally, in the case where J
=
and the network is fully rotationally invariant, equivariance still holds, it is just that the spherical scattering coefﬁcients for each path are reduced to a single constant value over the sphere.
Note also that the computation of the spherical scattering transform is scalable since it is based on spherical scale-discretized wavelet transforms that are themselves computationally scalable (McEwen et al., 2007; 2013; 2015c).
3. S CATTERING N ETWORKS
A scattering network is then simply a collection of scattering transforms for a number of paths:
S P f = { S [ p ] f : p ∈ P } , where the general path set P denotes the inﬁnite set of all possible paths P = { p = ( j 1 , j 2 , . . . , j d ) : J 0 ≤ j i ≤ J, 1 ≤ i ≤ d, d ∈ N 0 } .
Since a scattering network is a collection of scattering transforms that are rotationally equivariant, so too are scattering networks.
In practice one typically considers a path set deﬁned by paths up to some maximum depth D , i.e. P D = { p = [Equation]
[Equation] [Equation] [Equation] i ≤ J, 1 ≤ i
[Equation] d, 0 ≤ d ≤ D } . A diagram of a scattering network is illustrated in Fig.
A scattering network is effectively a spherical CNN with carefully designed ﬁlters (i.e. wavelets) and typically with an activation function given by the absolute value function.
Since the intended use of spherical scattering networks here is as an initial layer in generalized spherical CNNs, where we seek to mix information from high to low frequencies so that subsequent CNN layers can operate at low resolution, we are often interested in descending paths where the wavelet scale j reduces along the path.
The descending path set, up to maximum depth D , is given by P descending D
[Equation]
Since scattering networks are simply a collection of scattering transforms, the network inherits the properties of scattering transforms discussed previously and thus satisﬁes all requirements of the representational space that we seek.
3. I SOMETRIC I NVARIANCE AND S TABILITY TO D IFFEOMORPHISMS
While we have explained conceptually how spherical scattering networks exhibit isometric invariance and are stable to diffeomorphisms, here we formalize these results.
Theorem 1 ( Isometric Invariance ) .
Let ζ ∈ Isom( S 2 ) , then there exists a constant C such that for all f ∈ L 2 ( S 2 ) ,
[Equation]
Theorem 1 is analogous to Theorem 3.2 of Perlmutter et al.
(2020) that considers scattering networks on general Riemannian manifolds.
The manifold wavelet construction considered in Perlmutter et al.
(2020) is analogous to the spherical scale-discretized wavelet construction (e.g. McEwen et al., 2018) since both are based on a Meyer-like tiling of the harmonic frequency line.
However, in each approach different wavelet generating functions are considered, with an exponential kernel adopted in Perlmutter et al.
(2020) and an inﬁnitely differential Schwartz kernel adopted for spherical scale-discretized wavelets.
Consequently, the proof of Theorem 1 follows directly from the proof presented in Perlmutter et al.
but accounting for different wavelet generating functions and noting that (cid:80) (cid:96) ˆ φ (cid:96)
≤ L 1 / 2 λ J 0 .
Theorem 1 shows that the scattering network representation is invariant to isometries up to the scale controlled by J 0 , which is directly related to the bandlimit Lof the spherical wavelet scaling function.
Theorem 2 ( Stability to Diffeomorphisms ) .
Let
ζ ∈ Diﬀ
( S 2 ) .
If ζ = ζ 1 ◦ ζ 2 for some isometry ζ 1 ∈ Isom( S 2 ) and diffeomorphism
ζ 2 ∈ Diﬀ( S 2 ) , then there exists a constant C such that for all f ∈ L 2 ( S 2 ) ,
[Equation]
Theorem 2 is analogous to Theorem 4.1 of Perlmutter et al.
and the proof again follows similarly.
Theorem 2 shows that the scattering network representation is stable to small diffeomorphisms ζ 2 about an isometry.
Moreover the change in signal representation depends on the size of the diffeomorphism (cid:107) ζ 2 (cid:107) ∞ .
Consequently, for a classiﬁcation problem, for example, the scattering network representation is likely to be relatively insensitive to intra-class variations due to small diffeomorphisms but sensitive to inter-class variations due to large diffeomorphisms.
L EVERAGING S CATTERING N ETWORKS FOR S CALABLE S PHERICAL CNN S
We describe how spherical scattering networks may be integrated into the generalized spherical CNN framework of Cobb et al.
(2021) to scale to high-resolution data in a rotationally equivariant manner.
4. G ENERALIZED S PHERICAL CNN S
A variety of spherical CNN constructions have been presented recently in the inﬂuential works of Cohen et al.
(2018a), Esteves et al.
(2018) and Kondor et al.
These were uniﬁed in a generalized framework by Cobb et al.
(2021), who advocate hybrid networks where varying types of spherical layers can be leveraged alongside each other, and where alternative techniques to substantially improve efﬁciency were also developed.
Following Cobb et al.
(2021), consider the generalized spherical CNN framework where the s -th layer of the network takes the form of a triple A ( s ) =
( L 1 , N , L 2 ) , comprised of linear, non-linear, and linear operators, respectively.
The output of layer s is then given by f ( s ) =
A ( s ) ( f ( s − 1) ) =
( L 1 ◦ N ◦ L 2 )( f ( s − 1) ) , where f ( s − 1) is the signal inputted to the layer.
The linear and non-linear operators can take a variety of forms, such as those introduced in Cohen et al.
(2018a)
; Esteves et al.
; Kondor et al.
Cobb et al.
The linear layers are typically convolutions, either on the sphere, rotation group, or generalized convolutions (Kondor et al., 2018;
Cobb et al., 2021), while the non-linear layers are pointwise or tensor-product activations based on Clebsch-Gordan decompositions (Kondor et al., 2018; Cobb et al., 2021).
While generalized spherical CNNs have been shown to be highly effective and techniques to compute them efﬁciently have been developed (Cobb et al., 2021), they remain highly computationally demanding and cannot scale to high-resolution data.
4. S CALABLE G ENERALIZED S PHERICAL CNN S
Spherical scattering networks (Section 3) can be leveraged to scale spherical CNNs to high resolution, whilst maintaining rotational equivariance.
Adopting the generalized spherical CNN framework of
Cobb et al.
(2021), we advocate the use of scattering networks as an additional type of layer. [Figure] [Figure] [Table]
[Figure] [Table] [Figure] Spherical scattering networks are computationally scalable, rotationally equivariant, exhibit isometric invariance up to some scale, provide efﬁcient and stable representations, and are sensitive to all signal content, including both low and high frequencies.
Consequently, scattering networks provide a highly effective representational space for machine learning problems.
Each set of scattering coefﬁcients, in the parlance of CNNs, simply provides an additional channel
Computation of the spherical scattering network scales as O ( L 3 ) , for bandlimit L , due to the adoption of fast spherical wavelet transforms (McEwen et al., 2007; Leistedt et al., 2013;
McEwen et al., 2013; 2015c), which themselves leverage fast harmonic transforms (McEwen & Wiaux, 2011b; McEwen et al., 2015b).
In contrast, even the efﬁcient spherical CNNs of Cobb et al.
(2021) typically scale as
O ( L 4 ) .
More critically, however, scattering networks mix signal content from high to low frequencies.
Thus, they are an ideal ﬁrst layer in a hybrid spherical CNN so that subsequent layers may then operate at low resolution, while still capturing high-frequency signal content.
Since scattering networks use designed ﬁlters, rather than learned ﬁlters, they do not need to be trained.
Consequently, a scattering network layer acting as the ﬁrst layer of a spherical CNN can be considered as a data preprocessing stage before training.
The outputs of the scattering network therefore need only be computed once, rather than repeatedly during training.
Furthermore, the size of the output data of a scattering network is modest since all sets of output scattering coefﬁcients (i.e. output channels) are limited to the resolution of the scaling function L 0 and so need only be stored at low resolution.
These features provide considerable computational and memory savings, allowing scattering networks to scale spherical CNNs to high resolutions.
For example, the spherical scale-discretized wavelet transform on which the scattering network is based has so far been applied up to resolution L =
(Rogers et al., 2016a;b) and higher resolutions are feasible.
While evaluation of a scattering network at very high resolution may take some time, this is not a concern since the computation is trivially parallelizable and need only be computed once outside of training.
E XPERIMENTS
We perform numerical experiments to verify the theoretical properties of spherical scattering networks.
Using spherical scattering networks as an initial layer in the generalized spherical CNN framework, we show how scattering networks can be leveraged to scale spherical CNNs to high resolutions.
Spherical scattering networks are implemented in the fourpiAI 1 software package.
We use the s2let 2 code to perform spherical wavelet transforms.
5. P OWER OF S CATTERED S IGNALS
The absolute value function adopted as the activation function within scattering networks acts to spread signal content, predominantly from high to low frequencies.
To demonstrate this we generate 100 simulations of random spherical signals, with spherical harmonic coefﬁcients drawn from a standard Gaussian distribution.
In Fig.
2 we plot the average power spectra of the wavelet coefﬁcients of these signals before and after taking the absolute value.
Notice in Fig.
2(a) that the harmonic support of the wavelet coefﬁcients matches the support of the underlying wavelets, as expected.
Notice in Fig.
2(b) that after taking the absolute value the power of the signal is spread in harmonic space, predominantly mixed from high to low frequencies, and that variability is reduced, as expected.
5. E QUIVARIANCE T ESTS
To test the equivariance of the scattering transform we consider 100 random test signals and compute the mean relative equivariance error between rotating signals before and after the scattering transform (see Appendix B.1 for further details).
Results are given in Table 1, averaged over all scattering coefﬁcients with the same path depth d .
Notice that equivariance errors are small, as expected.
In fact, equivariance errors are considerably smaller than the standard spherical ReLU layer commonly used in spherical CNNs, which has an error on the order of ∼ 35 % (Cobb et al., 2021).
Notice also that the signal energy for each channel is small for a path length of three and greater, again as expected.
5. D IFFEOMORPHISM T ESTS
[Figure]
To test the stability of spherical scattering networks to diffeomorphisms we consider the following experiment.
Taking the prototypical example of classiﬁcation of hand-written digits, we compute differences in scattering network representations for both inter- and intra-class instances using MNIST digits projected onto the sphere (Cohen et al., 2018b).
See Appendix B.2 for further details.
Results are shown in Fig.
3 for all combinations of digits, normalized relative to intra-class variations of the digit one (which, incidentally, exhibits the smallest intra-class variation, as might be expected).
As discussed in Section 2.4, small diffeomorphisms are likely responsible for intra-class variations and large diffeomorphisms for inter-class variations.
Furthermore, by Theorem 2, a larger diffeomorphism leads to a larger difference in scattering representation.
For MNIST digits, we therefore expect the average difference between scattering representations to increase as the size of diffeomorphisms within and between digits increases.
This is indeed what we observe.
For each digit, intra-class differences are smaller than inter-class differences, i.e. the diagonal elements of Fig.
3 are smaller than the other elements on the same row/column.
While it is difﬁcult to quantify the size of a diffeomorphism mapping one digit to another, the results generally match ones intuition about similarities between digits, e.g. a three is more similar to a seven and nine, than to a zero or four.
In general, the ﬁndings of this experiment support the theoretical result that spherical scattering networks are stable to diffeomorphisms.
5. S PHERICAL MNIST AT V ARYING R ESOLUTION
We consider a variant of the standard benchmark of classifying MNIST digits projected onto the sphere (Cohen et al., 2018b), varying the resolution of spherical images and considering digits projected at smaller angular sizes (see Fig. 4).
The intention is to evaluate the effectiveness of scattering networks to represent high-resolution signal content in low-resolution scattering coefﬁcients.
To this end, we consider two classiﬁcation models (see Appendix B.3).
The ﬁrst is the hybrid model proposed in Cobb et al.
Since this model is too computationally demanding to run at high resolution, it considers harmonic content of the spherical MNIST images up to L = 32 only.
The second model prepends a scattering network to the ﬁrst model with output scattering coefﬁcients at resolution L 0 = 32 .
We stress that this experiment is constructed speciﬁcally to test the effectiveness of the scattering network to spread signal content to low frequencies for subsequent low-frequency spherical CNN layers.
The model without a scattering network will not perform well since it loses high-frequency signal content.
As resolution is increased, the digit size is reduced so that signal content is concentrated at higher frequencies.
The problem thus becomes increasingly more difﬁcult for the models considered and we are thus able to observe how effective the scattering network is at mixing information into low frequencies.
We consider two scenarios: digits that are centered and not rotated during training and testing (NR/NR); and digits that are not rotated during training but rotated to arbitrary positions on the sphere when testing (NR/R).
Effective classiﬁcation for the latter scenario requires invariance of the model.
The results of these experiments are shown in Table 2.
Since the problem is constructed to be more difﬁcult as resolution increases, we observe accuracy decreasing as resolution increases.
In all scenarios classiﬁcation accuracy is signiﬁcantly improved when incorporating a scattering
[Figure]
[Table]
network since the scattering network provides an effective representational space that mixes highfrequency signal content to low frequencies, where it may be captured in the following spherical
CNN layers.
The scattering network evaluation takes approximately ﬁve seconds at L = 256 (based on an unoptimized, single-threaded implementation).
We do not observe any signiﬁcant differences in speed of convergence between settings with and without the scattering network.
5. G AUSSIANITY OF THE C OSMIC M ICROWAVE B ACKGROUND
[Figure]
In the standard inﬂationary cosmological model the temperature anisotropies of the cosmic microwave background (CMB), the relic radiation of the Big Bang, are predicted to be a realization of a Gaussian random ﬁeld on the sphere.
However, many alternative cosmological models predict weak deviations from Gaussianity.
The statistical properties of the CMB are thus a powerful probe for distinguishing cosmological models and their study in observational data is of topical interest (e.g. Planck Collaboration
VII, 2020; Planck Collaboration XXIII, 2014).
We generate Gaussian and non-Gaussian simulations of the CMB (Rocha et al. 2005) at resolution L = 1024 (see Fig. 5).
We train similar models to those considered for the MNIST experiment above to distinguish between CMB simulations (see Appendix B.4).
Since the non-Gaussianity considered is weak this is a challenging classiﬁcation problem.
The ﬁrst model, without a scattering network, achieves a classiﬁcation accuracy of 53.1%, demonstrating the challenging nature of the problem.
The second model, including a scattering network, achieves a classiﬁcation accuracy of 95.3%.
Since the scattering network provides a highly effective representational space, which captures high-frequency signal content in the low-resolution scattering coefﬁcients, the model is able to achieve a considerably greater accuracy.
The scattering network evaluation takes a little under two minutes at L = 1024 (based on an unoptimized, single-threaded implementation); recall that this need only be computed once, outside of training.
C ONCLUSIONS
We have developed scattering networks on the sphere that yield a powerful and scalable representational space for spherical signals, providing an efﬁcient signal representation that is rotationally equivariant, invariant to isometries up to a particular scale, stable to diffeomorphisms, and sensitive to all signal content, including high and low frequencies.
When incorporated as an additional initial layer in the generalized spherical CNN framework, scattering networks allow spherical CNNs to be scaled to high-resolution data, while preserving rotational equivariance.
In future work spherical scattering networks can be straightforwardly extended to spin signals and directional wavelets (by adopting spin scale-discretized spherical wavelets; McEwen et al. 2015c).
While Euclidean scattering networks have been shown to be highly effective and have been used in a number of studies, they nevertheless have not experienced pervasive use since Euclidean CNNs are already computational scalable.
In the spherical setting, on the other hand, existing spherical CNNs are not computationally scalable.
We therefore anticipate scattering networks, integrated as a layer in the generalized spherical CNN framework, to be of critical use to scale spherical CNNs to the high resolution data typical of many applications.
A W AVELETS ON THE S PHERE A. R EVIEW OF A PPROACHES
Numerous wavelet frameworks on the sphere have been constructed.
Early constructions were based either on fully continuous frameworks (e.g. Antoine & Vandergheynst, 1999; 1998; McEwen et al., 2006) or on fully discrete frameworks (Schr¨oder & Sweldens, 1995; Barreiro et al., 2000; McEwen &
Scaife, 2008; McEwen et al., 2011)
based on the lifting scheme (Sweldens, 1997).
While the former continuous frameworks lacked the ability to perfectly synthesize a signal from its wavelet coefﬁcients in practice, the latter discrete frameworks lacked stable representations.
More recently, a number of exact discrete wavelet frameworks on the sphere have been developed with underlying continuous representations facilitated through the adoption of sampling theorems (e.g. McEwen & Wiaux, 2011a; McEwen et al., 2015a), including needlets (Narcowich et al., 2006;
Baldi et al., 2009; Marinucci et al., 2008), directional scale-discretized wavelets (Wiaux et al., 2008;
Leistedt et al., 2013; McEwen et al., 2015c; 2018) and the isotropic undecimated and pyramidal wavelet transforms (Starck et al., 2006).
These approaches have been extended to spin functions on the sphere (Geller et al., 2008; Geller & Marinucci, 2010; 2011;
Geller et al., 2009; McEwen et al., 2015c; 2014; Starck et al., 2009), extended to functions deﬁned on the three-dimensional ball (Durastanti et al., 2014; Leistedt & McEwen, 2012; McEwen & Leistedt, 2013; Lanusse et al., 2012), and in some cases exhibit algorithms for efﬁcient and exact computation (McEwen et al., 2007; 2013; 2015c).
Scale-discretized wavelet on the sphere (e
g. McEwen et al., 2018) have found widespread use in a variety of ﬁelds, including cosmology (e.g. Price et al., 2020;
Rogers et al., 2016b;a; Leistedt et al., 2017; 2015) and geophysics (e.g. Marignier et al., 2021), and are well-suited for the construction of spherical scattering networks, as we elaborate in Appendix A.6.
We therefore provide a concise review of the spherical scale-discretized wavelet framework and its properties.
A. W AVELET A NALYSIS
While the scale-discretized wavelet transform on the sphere is reviewed very concisely in Section 2.2, we provide further details here.
Since we provide a more complete review of the forward wavelet transform here, this subsection overlaps with the material of Section 2.2.
However, proceeding subsections provide details not discussed in the main body of the article.
The scale-discretized wavelet transform of a function f ∈ L 2 ( S 2 ) on the sphere S 2 is deﬁned by the convolution of f with the wavelet ψ j ∈ L 2 ( S 2 ) .
The wavelet coefﬁcients W j ∈ L 2 ( S 2 ) thus read
[Equation]
[Equation]
where d µ ( ω ) = sin θ d θ d ϕ denotes the rotationally invariant Haar measure on the sphere and ∗ denotes complex conjugation.
The wavelet scale j encodes the angular localization of ψ j , with increasing j corresponding to smaller scale (i.e. higher frequency) signal content.
The minimum and maximum wavelet scales considered are denoted J 0 and J respectively, with 0 ≤ J 0 ≤ j ≤ J .
In general directional wavelets are supported by the scale-discretized wavelet framework (i.e. wavelets that are not azimuthally symmetric), resulting in wavelet coefﬁcients that live on the rotation group
SO(3) ; nevertheless, in this article we restrict our attention to axisymmetric wavelets (i.e. wavelets that are azimuthally symmetric).
The wavelet coefﬁcients capture high-frequency or bandpass information of the underlying signal; they do not capture low-frequency signal content.
A scaling function is introduced to capture low-frequency signal content.
Scaling coefﬁcients ψ ∈ L 2 ( S 2 ) are given by convolution of f with the scaling function φ ∈ L 2 ( S 2 ) :
[Equation]
A. W AVELET S YNTHESIS
The signal f can be synthesized perfectly from its wavelet and scaling coefﬁcients by
[Equation]
provided that the following admissibility condition holds:
[Equation]
where the spherical harmonic coefﬁcients of the wavelet and scaling function are given by, respectively, (
[Equation]
ˆ φ (cid:96)m
[Equation]
A. W AVELET C ONSTRUCTION
Scale-discretized wavelets are constructed to satisfy the admissibility property of Equation 9 in order to ensure perfect signal synthesis and to be inﬁnitely differentiable in harmonic space, resulting in excellent spatial localization properties.
Consider the inﬁnitely differentiable Schwartz function with compact support t ∈ [ λ − 1 , 1] , for dilation parameter λ ∈ R + ∗ , λ > 1 :
[Equation] [Equation]
where
[Equation]
Deﬁne the smoothly decreasing function k λ by
[Equation]
which is unity for t < λ − 1 , zero for t > 1 , and is smoothly decreasing from unity to zero for t ∈
[ λ − 1 , 1] .
Deﬁne the wavelet kernel generating function by
[Equation]
13) which has compact support t ∈
[ λ − 1 , λ ] and reaches a peak of unity at t = 1 .
The scale-discretized wavelet kernel for scale j is then deﬁned by
[Equation]
14) which has compact support on (cid:96) ∈ (cid:2)
λ j − 1 , λ j +1 (cid:3) and reaches a peak of unity at λ j .
Scaling functions are required to probe the low-frequency content of the signal of interest not probed by the wavelets and are thus deﬁned by
[Equation]
The scaling function has compact support on (cid:96) ∈ [0 , λ J 0 ] and is unity up to λ J 0 − 1 , decaying smoothly to zero on [ λ J 0 − 1 , λ J 0 ] .
An illustration of the harmonic support of the scaling function and wavelets is shown in Fig.
While the dilation parameter λ can be selected arbitrarily depending on the harmonic scales one wishes each wavelet to probe, provided λ > 1 , a common choice is dyadic scaling with λ = 2 .
The maximum wavelet scale J is set to ensure the wavelets reach the bandlimit L of the signal of interest, yielding J = (cid:100) log λ L (cid:101) , where (cid:100)·(cid:101) is the ceiling function.
The minimum wavelet scale J 0 may be freely chosen, provided≤ J 0 < J .
Alternatively, the minimum scale considered may be set by specifying a desired bandlimit L 0 for the scaling coefﬁcients, yielding J
=
(cid:100) log λ L 0 (cid:101) .
For J 0 =
the wavelets probe the entire frequency content of the signal of interest except its mean, which is encoded in the scaling coefﬁcients.
The wavelet transform of a signal bandlimited at L is therefore speciﬁed by the parameters ( L, λ, J 0 ) , or alternatively ( L, λ, L 0 ) .
[Figure]
L =
[Figure]
J φ ψ Jψ j ψ J . . . A.
[Figure]
E XACT AND E FFICIENT C OMPUTATION
The scale-discretized wavelet transform of signals on the sphere can be computed exactly and efﬁciently by appealing to sampling theorems on the sphere (e.g. McEwen & Wiaux, 2011b) (and rotation group for directional wavelets; McEwen et al. 2015b) and corresponding fast algorithms (McEwen et al., 2007; 2013; 2015c).
By exploiting sampling theorems access to the underlying continuous signals is afforded and wavelet transforms can be computed in a manner that is theoretically exact, up to machine precision.
We adopt the s2let 3 code implementing this scale-discretized wavelet transform on the sphere.
For the axisymmetric wavelet transforms considered herein, the complexity of the wavelet transform scales as O ( L 3 ) .
Further computational savings can be achieved by considering a multi-resolution setting where wavelet coefﬁcients for scale j are computed at the minimum resolution required to capture all information content, i.e. at L = λ j +
(Leistedt et al., 2013).
We adopt the multi-resolution setting in our experiments since we found this does not markedly impact model equivariance or accuracy, while considerably reducing the computational time of wavelet transforms.
A. P ROPERTIES
As discussed, the computation of the scale-discretized wavelet transform is scalable.
The wavelet transform is rotationally equivariant in theory, since it is based on spherical convolutions that themselves are clearly rotationally equivariance (shown in e.g. Cohen et al. 2018a), and in practice, since its computation leverages underlying sampling theory on the sphere.
Moreover, scale-discretized spherical wavelets have excellent localization and stability properties, and constitute a Parseval frame (McEwen et al., 2018; 2015c).
Finally, they are clearly sensitive to both low and high frequency signal content, with the scaling coefﬁcients capturing low frequency content and wavelet coefﬁcients capturing frequencies associated with the scale of the corresponding wavelet.
Scale-discretized wavelets on the sphere thus provide an ideal wavelet framework on which to build spherical scattering networks.
B A DDITIONAL I NFORMATION ON E XPERIMENTS B. E QUIVARIANCE T ESTS
To test the equivariance of the scattering transform we perform similar experiments to those of Cobb et al.
(2021, Appendix D), considering N f = 100 random spherical signals { f i } N f i
=1 with harmonic coefﬁcients sampled from the standard normal distribution and N ρ = 100 random rotations { ρ j } N ρ j = sampled uniformly on SO(3) .
To measure the extent to which an operator A : L 2 ( S 2 ) → L 2 ( S 2 ) is equivariant we evaluate the mean relative error [Equation]
[Equation]
resulting from pre-rotation of the signal, followed by application of A , as opposed to post-rotation after application of A , where the operator norm (cid:107) · (cid:107) is deﬁned using the inner product (cid:104)· , ·(cid:105)
L 2 ( S 2 ) .
B. D IFFEOMORPHISM T ESTS
To test the stability of spherical scattering networks to diffeomorphisms we compute differences in scattering network representations for spherical MNIST digits, averaged over 10,00 randomly selected digits.
We deﬁne the difference between scattering network representations for spherical signals f ( k ) i , for class k and instance i , by
[Equation]
where the norm (cid:107) · (cid:107) in this case denotes the inner product (cid:104)· , ·(cid:105) L 2 ( S 2 ) applied to each scattering coefﬁcient channel, then summed over all channels.
For diffeomorphism tests we consider a spherical scattering network with parameters ( L, λ, L 0 ) =
(128 , 2 , 32) , descending paths only, and depth
D = 2 (matching the settings of Section 5.4).
B. S PHERICAL MNIST AT V ARYING R ESOLUTION
The ﬁrst model considered for our MNIST classiﬁcation experiments is the hybrid model proposed in Cobb et al.
The ﬁrst layer includes a directional convolution on the sphere that lifts the spherical input onto the rotation group.
The second layer includes a convolution on the rotation group, followed by three layers comprising constrained generalized convolutions and tensor-product activations.
The ﬁnal restricted generalized convolution maps down to a rotationally invariant representation.
As is traditional in convolution networks we gradually decrease the resolution, with ( L 0 , L 1 , L 2 , L 3 , L 4 , L 5 ) =
(20 , 10 , 10 , 6 , 3 , 1) , and increase the number of channels, with ( K 0 , K 1 , K 2 , K 3 , K 4 , K 5 ) =
We proceed these convolutional layers with a single dense layer of size 30 , sandwiched between two dropout layers (keep probability 0.5), and then fully connect to the output of size 10.
This model considers harmonic content of the spherical MNIST images up to L = 32 only (since it is too computationally demanding to run at high resolution).
The resulting model has 59,396 learnable parameters.
The second model considered prepends a spherical scattering network to the ﬁrst model.
We consider a scattering network with λ = 2 , descending paths only, and depth D = 2 .
The scattering network takes the spherical image at its original resolution and outputs scattering coefﬁcients (channels) at resolution L 0 = 32 .
The non-scattering part of the model thus runs at the same resolution for both settings ( L = 32 ).
The resulting model has 60,396 learnable parameters for L = 64 .
We train the network for 20 epochs on batches of size 32 , using the Adam optimizer (Kingma & Ba, 2015) with a decaying learning rate starting at 0 .
For the restricted generalized convolutions we follow the approach of Kondor et al.
by using L 1 regularization (regularization strength 10 − 5 ) and applying a restricted batch normalization across fragments, where the fragments are only scaled by their average and not translated (to preserve equivariance).
We adopt the same core hybrid model, with the same parameters, for both scenarios with and without the scattering network.
The only different is the number of input channels since for the case without scattering we have one greyscale spherical image of a digit, whereas for the scattering case we have a channel for each set of scattering coefﬁcients (i.e. each blue node in Fig. 1).
This difference in number of input channels results in a small difference in overall number of learnable parameters for the two models, since a different convolutional ﬁlter is applied to each of the input channels.
To ensure this does not have a signiﬁcant impact on results, we also consider a hybrid model without a scattering network with K 1 = 21 (instead of K 1 = 20 ) channels in the ﬁrst layer, which results in a model with 60,507 learnable parameters (more learnable parameters than the model with a scattering network for L = 64 ).
We ﬁnd no signiﬁcant difference between the models with K 1 = 20 and
K 1 = 21 , suggesting the superiority of the approach with a scattering network in Table 2 is due to the effectiveness of the scattering network in mixing high-frequency signal content to low frequencies, rather than the very small increase in number of parameters of the models that include a scattering network.
B. G AUSSIANITY OF THE C OSMIC M ICROWAVE B ACKGROUND
To simulate CMB observations we adopt the Lambda Cold Dark Matter (LCDM)
CMB power spectrum that best ﬁts CMB observations made by the ESA Planck satellite (Planck Collaboration I, 2020).
CMB simulations are generated following the approach of Rocha et al.
(2005), where the nonGaussian distribution is derived from the wavefunctions of the harmonic oscillator.
We set α 3 = 0 .to
recover Gaussian simulations and α 3 = 0 .
2 to recover non-Gaussian simulations(following Rocha et al. 2005).
We consider simulations at resolution L = 1024 , training on 1024 simulations and evaluating on a test set of 128 simulations.
We consider similar models and training conﬁguration as the MNIST experiment presented above but with scattering paths such that j 2 =
j 1 − 1 (to avoid generating a large number of output channels and to promote mixing from high to low frequencies) and train for 200 epochs.