Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs

1 INTRODUCTION . The construction of convolutional neural networks ( CNNs ) on the sphere has received considerable attention recently , opening up their application to fields where standard ( Euclidean ) CNNs are not effective . Such techniques are finding application in diverse fields ; for example , for the analysis of 360◦ photos and videos in virtual reality and for the analysis of astrophysical observations on the celestial sphere in cosmology . In both of these fields high-resolution spherical images are typically encountered , with image resolutions often reaching many tens of megapixels . Current spherical CNN approaches that are defined natively on the sphere and satisfy rotational equivariance , however , are limited to low resolutions and often can not scale beyond thousands of pixels , severely restricting their applicability . In this article we present a strategy to scale spherical CNNs to high resolution , further opening up their applicability to high-resolution spherical images encountered in virtual reality , cosmology , and many other fields beyond . A number of spherical CNN constructions have been proposed recently , which can be broadly categorized into two approaches : ( i ) those that are defined natively on the sphere and capture rotational equivariance ( e.g . Cohen et al. , 2018a ; Esteves et al. , 2018 ; 2020 ; Kondor et al. , 2018 ; Cobb et al. , 2021 ) ; and ( ii ) those that are constructed on discretizations of the sphere and typically do not fully capture rotational equivariance ( e.g . Boomsma & Frellsen , 2017 ; Jiang et al. , 2018 ; Perraudin et al. , 2019 ; Cohen et al. , 2019 ) . While constructions on discretized representations of the sphere can often be computed efficiently , and some capture rotational equivariance to a small set of rotations ( Cohen et al. , 2019 ) , such approaches necessitate an approximate representation of spherical signals , losing the connection to the underlying continuous symmetries of the sphere and thus can not fully capture rotational equivariance . Approaches constructed natively on the sphere that do capture rotational equivariance , on the other hand , are highly computationally demanding . Despite considerable improvements in computational efficiency proposed recently ( Cobb et al. , 2021 ) , such approaches nevertheless remain computationally demanding and are not scalable to high resolution . We seek a framework that captures the underlying continuous symmetries of the sphere and exhibits rotational equivariance , while also being computationally scalable to high resolution . We draw ∗Corresponding author : jason.mcewen @ kagenova.com inspiration from Cobb et al . ( 2021 ) , which presents a generalized spherical CNN framework and advocates hybrid networks where different types of spherical CNN layers are leveraged alongside each other . We consider an additional layer to be integrated into this hybrid approach that must , critically , be scalable to high resolution and allow subsequent layers to operate effectively at low resolution , while also exhibiting rotational equivariance . To provide an effective representational space it should also provide efficient and stable signal representations , with control of the scale of signal invariances , and be sensitive to all signal content . Scattering networks meet all requirements . Scattering networks were first proposed in the seminal work of Mallat ( 2012 ) in an effort to provide some further theoretical basis explaining the observed empirical effectiveness of CNNs ( elaborated in Mallat 2016 ) . They are constructed from a cascade of linear transforms ( typically convolutions ) , combined with pointwise non-linear activation functions , with carefully designed filters to ensure certain stability and invariance properties . A scattering network is thus effectively a CNN but with designed , rather than learned , filters . Typically wavelets that themselves satisfy certain stability and invariance properties are adopted for the filters and the absolute value function is adopted for the nonlinear activation function . Scattering networks were initially constructed on Euclidean domains and have been applied in a variety of applications ( e.g . Bruna & Mallat , 2011 ; 2013 ; Andén et al. , 2019 ) . More recently scattering networks that support non-Euclidean data have been constructed through graph representations ( Gama et al. , 2019b ; a ; Gao et al. , 2019 ; Zou & Lerman , 2020 ; Perlmutter et al. , 2019 ) or on general Riemannian manifolds ( Perlmutter et al. , 2020 ) . While these approaches are flexible , this flexibility comes at a cost . Just as there are considerable advantages in constructing CNNs natively on the sphere ( rather than , e.g. , adopting graph CNNs ) , so too there are advantages in constructing scattering networks natively on the sphere . For example , spherical scattering networks can be constructed to retain a connection to the underlying continuous symmetries of the sphere and to leverage fast algorithms tailored to the harmonic structure of the sphere . In this article we construct scattering networks on the sphere , motivated by their use as an initial layer in hybrid spherical CNNs in order to yield rotationally equivariant spherical CNNs that scale to high resolution . In Section 2 we review spherical signal representations and describe the invariance and stability properties required to provide an effective representational space . In Section 3 we construct scattering networks on the sphere and discuss their invariance and stability properties . In Section 4 we integrate scattering networks into the existing generalized spherical CNN framework . Experiments studying the properties of spherical scattering networks and demonstrating their effectiveness in hybrid spherical CNNs are shown in Section 5 . Concluding remarks are made in Section 6 . 2 SPHERICAL SIGNAL REPRESENTATIONS . We seek a representational space that can be used as an initial layer in hybrid spherical CNNs to scale to high-resolution data . Such a representational space must , critically , be scalable , allow subsequent CNN layers to operate at low resolution , and be rotationally equivariant . Moreover , to provide an effective representation for machine learning problems the space should provide efficient and stable signal representations , with control over the scale of signal invariances , while being sensitive to all signal content . In this section we concisely review common spherical signal representational spaces , including spatial , harmonic and wavelet signal representations , before discussing the desired invariance and stability properties of the representational space that we seek . 2.1 SPATIAL AND HARMONIC REPRESENTATIONS . Consider square integrable signals f , g ∈ L2 ( S2 ) on the sphere S2 with inner product 〈f , g〉 . The inner product induces the norm ‖f‖2 = √ 〈f , f〉 , with metric dL2 ( S2 ) ( f , g ) = ‖f −g‖2 . We consider spherical signals bandlimited at L , with harmonic coefficients f̂ ` m = 〈f , Y ` m〉 = 0 , ∀ ` ≥ L , where Y ` m are the spherical harmonic functions of natural degree ` and integer order |m| ≤ ` . Sampling theories on the sphere ( Driscoll & Healy , 1994 ; McEwen & Wiaux , 2011a ) provide a mechanism to capture all information content of an underlying continuous function from a finite set of samples . By providing access to the underlying continuous signal , such a representation perfectly captures all spherical symmetries and geometric properties . We adopt the sampling theorem on the sphere of McEwen & Wiaux ( 2011a ) since it provides the most efficient sampled signal representation ( reducing the Nyquist rate by a factor of two compared to Driscoll & Healy 1994 ) . 2.2 WAVELET REPRESENTATIONS . A number of wavelet frameworks on the sphere have been constructed ( see Appendix A ) . We focus on spherical scale-discretized wavelets since they have an underlying continuous representation , exhibit excellent localization and asymptotic correlation properties , are a Parseval frame , support directional wavelets , and exhibit fast algorithms for exact and efficient computation ( McEwen et al. , 2007 ; Wiaux et al. , 2008 ; Leistedt et al. , 2013 ; McEwen et al. , 2013 ; 2015c ; 2018 ) . Furthermore , the wavelet transform is rotationally equivariant in theory ( since it is based on spherical convolutions ) and in practice ( since its computation leverages underlying spherical sampling theory ) . The ( axisymmetric ) spherical scale-discretized wavelet transform is defined by the convolution of f ∈ L2 ( S2 ) with the wavelet ψj ∈ L2 ( S2 ) , yielding wavelet coefficients Wj ( ω ) = ( f ? ψj ) ( ω ) = 〈f , Rωψj〉 , where ω = ( θ , ϕ ) denotes a point on the sphere with colatitude θ ∈ [ 0 , π ] and longitude ϕ ∈ [ 0 , 2π ) , and Rω = R ( ϕ , θ,0 ) denotes a three-dimensional ( 3D ) rotation ( we adopt the active zyz Euler convention ) . The wavelet scale j encodes the angular localization of ψj , with increasing j corresponding to smaller scale ( i.e . higher frequency ) signal content . The minimum and maximum wavelet scales considered are denoted J0 and J , i.e . 0 ≤ J0 ≤ j ≤ J . The wavelet coefficients capture high-frequency or bandpass information of the underlying signal ; they do not capture lowfrequency signal content . A scaling function φ ∈ L2 ( S2 ) is introduced to capture low-frequency signal content , with scaling coefficients given by W ( ω ) = ( f ? φ ) ( ω ) = 〈f , Rωφ〉 . For notational convenience we introduce the operator notation wj = Ψjf = f ? ψj and w = Φf = f ? φ . Further details regarding the spherical scale-discretized wavelet construction , synthesis and properties are contained in Appendix A . Note that the wavelet construction relies on a fixed dilation parameter λ ∈ R+∗ , λ > 1 , that controls the relative scaling of harmonic degrees one wishes each wavelet to probe . A common choice is dyadic scaling with λ = 2 . The minimum wavelet scale J0 may be chosen freely or may be set by specifying a desired bandlimit L0 for the scaling coefficients , yielding J0 = dlogλ L0e . For J0 = 0 the wavelets probe the entire frequency content of the signal of interest except for its mean , which is encoded in the scaling coefficients . The wavelet transform of a signal bandlimited at L is therefore specified by the parameters ( L , λ , J0 ) , or alternatively ( L , λ , L0 ) . 2.3 ISOMETRIES AND DIFFEOMORPHISMS . While we have mentioned the signal representation we seek must satisfy invariance and stability properties , we are yet to elaborate on these properties in detail . We define the transformations under which these properties are sought , before describing the properties themselves in the next subsection . Consider transforms of spherical signals that are an isometry , preserving the distance between signals . More precisely , consider the isometry ζ ∈ Isom ( S2 ) , where Isom ( S2 ) denotes the isometry group of the sphere . The action of an isometric transformation Vζ : L2 ( S2 ) → L2 ( S2 ) is defined as Vζf ( ω ) = f ( ζ−1ω ) . The distance between signals f , g ∈ L2 ( S2 ) is then preserved under an isometry , i.e . dL2 ( S2 ) ( Vζf , Vζg ) = dL2 ( S2 ) ( f , g ) . One of the most typical isometries considered for spherical signals is a 3D rotation Rρ , often parameterized by Euler angles ρ = ( α , β , γ ) ∈ SO ( 3 ) . Consider deformations of spherical signals , formally defined by diffeomorphisms . A diffeomorphism is a transformation that is a differentiable bijection ( a one-to-one mapping that is invertible ) , with an inverse that is also differentiable . Loosely speaking , a diffeomorphism can be considered as a “ well-behaved ” deformation . Consider a diffeomorphism ζ ∈ Diff ( S2 ) , where Diff ( S2 ) denotes the diffeomorphism group of the sphere . The action of a diffeomorphism on a spherical signal is defined similarly to an isometric transformation . It is useful to quantify the size of a diffeomorphism . Let ‖ζ‖∞ = supω∈S2 dS2 ( ω , ζ ( ω ) ) represent the maximum displacement between points ω and ζ ( ω ) , where dS2 ( ω , ω′ ) is the geodesic distance on the sphere between ω and ω′ . Often we are interested in small diffeomorphisms about an isometry such as a rotation , denoted ζ ′ = ζ1 ◦ ζ2 for some isometry ζ1 ∈ Isom ( S2 ) , e.g . ρ ∈ SO ( 3 ) , and some diffeomorphism ζ2 ∈ Diff ( S2 ) .

Summary: The authors propose to process signals on the sphere via an equivariant CNN where, as a pre-processing step, a scattering network is used. As these scattering networks can be computed efficiently at high resolutions, this allows for spherical CNNs to be used on higher-resolution signals. As experiments, the authors compare a spherical CNN with and without the scattering network pre-computation step on spherical MNIST and a cosmic background radiation dataset and find that the pre-computation step aids performance.