D EEP S PHERE : A GRAPH - BASED SPHERICAL CNN
A BSTRACT
Designing a convolution for a spherical neural network requires a delicate trade off between efﬁciency and rotation equivariance.
DeepSphere, a method based on a graph representation of the sampled sphere, strikes a controllable balance between these two desiderata
This contribution is twofold.
First, we study both theoretically and empirically how equivariance is affected by the underlying graph with respect to the number of vertices and neighbors.
Second, we evaluate DeepSphere on relevant problems.
Experiments show state-of-the-art performance and demonstrates the efﬁciency and ﬂexibility of this formulation.
Perhaps surprisingly, comparison with previous work suggests that anisotropic ﬁlters might be an unnecessary price to pay.
Our code is available at https: //github.com/deepsphere .
I NTRODUCTION
Spherical data is found in many applications (ﬁgure 1).
Planetary data (such as meteorological or geological measurements) and brain activity are example of intrinsically spherical data.
The observation of the universe, LIDAR scans, and the digitalization of 3D objects are examples of projections due to observation.
Labels or variables are often to be inferred from them.
Examples are the inference of cosmological parameters from the distribution of mass in the universe (Perraudin et al., 2019), the segmentation of omnidirectional images (Khasanova & Frossard, 2017), and the segmentation of cyclones from Earth observation (Mudigonda et al., 2017). MEGevokedpotential, 0 . 1 s (a) CMB temperature map (Planck 2015)
[Figure]
As neural networks (NNs) have proved to be great tools for inference, variants have been developed to handle spherical data.
Exploiting the locally Euclidean property of the sphere, early attempts used standard 2D convolutions on a grid sampling of the sphere (Boomsma & Frellsen, 2017; Su & Grauman, 2017; Coors et al., 2018).
While simple and efﬁcient, those convolutions are not equivariant to rotations.
On the other side of this tradeoff, Cohen et al.
(2018) and Esteves et al.
(2018) proposed to perform proper spherical convolutions through the spherical harmonic transform.
While equivariant to rotations, those convolutions are expensive (section 2).
As a lack of equivariance can penalize performance (section 4.2) and expensive convolutions prohibit their application to some real-world problems, methods standing between these two extremes are desired.
Cohen et al.
(2019) proposed to reduce costs by limiting the size of the representation of the symmetry group by projecting the data from the sphere to the icosahedron.
The distortions introduced by this projection might however hinder performance (section 4.3).
Another approach is to represent the sampled sphere as a graph connecting pixels according to the distance between them (Bruna et al., 2013; Khasanova & Frossard, 2017; Perraudin et al., 2019).
While Laplacian-based graph convolutions are more efﬁcient than spherical convolutions, they are not exactly equivariant (Defferrard et al., 2019).
In this work, we argue that graph-based spherical CNNs strike an interesting balance, with a controllable tradeoff between cost and equivariance (which is linked to performance).
Experiments on multiple problems of practical interest show the competitiveness and ﬂexibility of this approach.
M ETHOD
DeepSphere leverages graph convolutions to achieve the following properties: (i) computational efﬁciency, (ii) sampling ﬂexibility, and (iii) rotation equivariance (section 3).
The main idea is to model the sampled sphere as a graph of connected pixels: the length of the shortest path between two pixels is an approximation of the geodesic distance between them.
We use the graph CNN formulation introduced in (Defferrard et al., 2016) and a pooling strategy that exploits hierarchical samplings of the sphere.
Sampling.
A sampling scheme V = { x i ∈ S 2 } ni =1 is deﬁned to be the discrete subset of the sphere containing the n points where the values of the signals that we want to analyse are known.
For a given continuous signal f , we represent such values in a vector f ∈ R n .
As there is no analogue of uniform sampling on the sphere, many samplings have been proposed with different tradeoffs.
In this work, depending on the considered application, we will use the equiangular (Driscoll &
Healy, 1994), HEALPix (Gorski et al., 2005), and icosahedral (Baumgardner & Frederickson, 1985) samplings.
Graph.
From V , we construct a weighted undirected graph G =
( V , w ) , where the elements of
V are the vertices and the weight w ij = w ji is a similarity measure between vertices x i and x j .
The combinatorial graph Laplacian L ∈ R n × n is deﬁned as L = D − A , where A = ( w ij ) is the weighted adjacency matrix, D = ( d ii ) is the diagonal degree matrix, and d ii = (cid:80) j w ij is the weighted degree of vertex x i .
Given a sampling V , usually ﬁxed by the application or the available measurements, the freedom in constructing G is in setting w .
Section 3 shows how to set w to minimize the equivariance error.
Convolution.
On Euclidean domains, convolutions are efﬁciently implemented by sliding a window in the signal domain.
On the sphere however, there is no straightforward way to implement a convolution in the signal domain due to non-uniform samplings.
Convolutions are most often performed in the spectral domain through a spherical harmonic transform (SHT).
That is the approach taken by Cohen et al.
(2018) and Esteves et al.
(2018), which has a computational cost of O ( n 3 / 2 ) on isolatitude samplings (such as the HEALPix and equiangular samplings) and O ( n 2 ) in general.
On the other hand, following Defferrard et al.
(2016), graph convolutions can be deﬁned as
[Equation]
where P is the polynomial order (which corresponds to the ﬁlter’s size) and α i are the coefﬁcients to be optimized during training.
Those convolutions are used by Khasanova & Frossard (2017) and
Perraudin et al.
and cost O ( n ) operations through a recursive application of L .
Pooling.
Down- and up-sampling is natural for hierarchical samplings, 5 where each subdivision divides a pixel in (an equal number of) child sub-pixels.
To pool (down-sample), the data supported on the sub-pixels is summarized by a permutation invariant function such as the maximum or the average.
To unpool (up-sample), the data supported on a pixel is copied to all its sub-pixels.
Architecture.
All our NNs are fully convolutional, and employ a global average pooling (GAP) for rotation invariant tasks.
Graph convolutional layers are always followed by batch normalization and ReLU activation, except in the last layer.
Note that batch normalization and activation act on the elements of f independently, and hence don’t depend on the domain of f .
G RAPH CONVOLUTION AND EQUIVARIANCE
While the graph framework offers great ﬂexibility, its ability to faithfully represent the underlying sphere — for graph convolutions to be rotation equivariant — highly depends on the sampling locations and the graph construction.
3. P ROBLEM FORMULATION
A continuous function f : C ( S 2 )
⊃ F V → R is sampled as T V ( f ) =
f by the sampling operator
T V : C ( S 2 )
⊃ F V → R n deﬁned as f : f i = f ( x i ) .
We require F V to be a suitable subspace of continuous functions such that T V is invertible, i.e., the function f ∈ F V can be unambiguously reconstructed from its sampled values f .
The existence of such a subspace depends on the sampling
V and its characterization is a common problem in signal processing (Driscoll & Healy, 1994).
For most samplings, it is not known if F V exists and hence if T V is invertible.
A special case is the equiangular sampling where a sampling theorem holds, and thus a closed-form of T − 1 V is known.
For samplings where no such sampling formula is available, we leverage the discrete SHT to reconstruct f from f = T V f , thus approximating T − 1 V .
For all theoretical considerations, we assume that F V exists and f ∈ F V .
By deﬁnition, the (spherical) graph convolution is rotation equivariant if and only if it commutes with the rotation operator deﬁned as R ( g ) , g ∈ SO (3) : R ( g ) f
( x ) =
f (cid:0) g − 1 x (cid:1) .
In the context of this work, graph convolution is performed by recursive applications of the graph Laplacian (1).
Hence, if R ( g ) commutes with L , then, by recursion, it will also commute with the convolution h ( L ) .
As a result, h ( L ) is rotation equivariant if and only if
[Equation]
where R V ( g ) =
T V R ( g ) T − 1 V .
For an empirical evaluation of equivariance, we deﬁne the normalized equivariance error for a signal f and a rotation g as
[Equation]
More generally for a class of signals f ∈ C ⊂ F V , the mean equivariance error deﬁned as
[Equation]
represents the overall equivariance error.
The expected value is obtained by averaging over a ﬁnite number of random functions and random rotations.
As long as the graph is sparsiﬁed such that the number of edges, i.e., the number of non-zeros in A , is proportional to the number of vertices n .
This can always be done as most weights are very small.
[Figure] [Figure]
[Figure]
3. F INDING THE OPTIMAL WEIGHTING SCHEME
Considering the equiangular sampling and graphs where each vertex is connected to 4 neighbors (north, south, east, west)
[Figure]
Khasanova & Frossard (2017) designed a weighting scheme to minimize (3) for longitudinal and latitudinal rotations 6 .
Their solution gives weights inversely proportional to
Euclidean distances:
[Equation]
While the resulting convolution is not equivariant to the whole of SO (3) (ﬁgure 2), it is enough for omnidirectional imaging because, as gravity consistently orients the sphere, objects only rotate longitudinally or latitudinally.
To achieve equivariance to all rotations, we take inspiration from Belkin & Niyogi (2008).
They prove that for a random uniform sampling , the graph Laplacian L built from weights
[Equation]
converges to the Laplace-Beltrami operator ∆ S 2 as the number of samples grows to inﬁnity.
This result is a good starting point as ∆ S 2 commutes with rotation, i.e., ∆ S 2 R ( g ) =
R ( g )∆ S 2 .
While the weighting scheme is full (i.e., every vertex is connected to every other vertex), most weights are small due to the exponential.
We hence make an approximation to limit the cost of the convolution (1) by only considering the k nearest neighbors ( k -NN) of each vertex.
Given k , the optimal kernel width t is found by searching for the minimizer of (3).
Figure 3 shows the optimal kernel widths found for various resolutions of the HEALPix sampling.
As predicted by the theory, t n ∝ n β , β ∈ R .
Importantly however, the optimal t also depends on the number of neighbors k .
Considering the HEALPix sampling, Perraudin et al.
(2019) connected each vertex to their 8 adjacent vertices in the tiling of the sphere, computed the weights with (5), and heuristically set t to half the average squared Euclidean distance between connected vertices.
This heuristic however overestimates t (ﬁgure 3) and leads to an increased equivariance error (ﬁgure 2).
3. A NALYSIS OF THE PROPOSED WEIGHTING SCHEME
We analyze the proposed weighting scheme both theoretically and empirically.
Theoretical convergence.
We extend the work of (Belkin & Niyogi, 2008) to a sufﬁciently regular, deterministic sampling.
Following their setting, we work with the extended graph Laplacian operator as the linear operator L tn :
L 2 ( S 2 ) →
L 2 ( S 2 ) such that
[Equation]
This operator extends the graph Laplacian with the weighting scheme (5) to each point of the sphere (i.e., L tn f = T V L tn f ).
As the radius of the kernel t will be adapted to the number of samples, we scale the operator as ˆ
L tn :=
| S 2 | (4 πt 2 ) − 1 L tn .
Given a sampling V , we deﬁne σ i to be the patch of the surface of the sphere corresponding to x i , A i its corresponding area, and d i the largest distance between the center x i and any point on the surface σ i .
Deﬁne d ( n ) := max i =1 ,...,n d i and A ( n ) :
= max i
=1 ,...,n A i .
Theorem 3.1.
For a sampling V of the sphere that is equi-area and such that d ( n ) ≤ Cn α , α ∈ (0 , 1 / 2] , for all f : S 2 → R Lipschitz with respect to the Euclidean distance in R 3 , for all y ∈ S 2 , there exists a sequence t n = n β , β ∈ R such that
[Equation]
This is a major step towards equivariance, as the Laplace-Beltrami operator commutes with rotation.
Based on this property, we show the equivariance of the scaled extended graph Laplacian.
Theorem 3.2.
Under the hypothesis of theorem 3.1, the scaled graph Laplacian commutes with any rotation, in the limit of inﬁnite sampling, i.e.,
[Equation]
From this theorem, it follows that the discrete graph Laplacian will be equivariant in the limit of n → ∞ as by construction L tn f = T V L tn f and as the scaling does not affect the equivariance property of L tn .
Importantly, the proof of Theorem 3.1 (in Appendix A) inspires our construction of the graph Laplacian.
In particular, it tells us that t should scale as n β , which has been empirically veriﬁed (ﬁgure 3).
Nevertheless, it is important to keep in mind the limits of Theorem 3.1 and 3.2.
Both theorems present asymptotic results, but in practice we will always work with ﬁnite samplings.
Furthermore, since this method is based on the capability of the eigenvectors of the graph Laplacian to approximate the spherical harmonics, a stronger type of convergence of the graph Laplacian would be preferable, i.e., spectral convergence (that is proved for a full graph in the case of random sampling for a class of Lipschitz functions in (Belkin & Niyogi, 2007)).
Finally, while we do not have a formal proof for it, we strongly believe that the HEALPix sampling does satisfy the hypothesis d ( n ) ≤ Cn α , α ∈ (0 , 1 / 2] , with α very close or equal to 1 / 2 .
The empirical results discussed in the next paragraph also points in this direction.
This is further discussed in Appendix A.
Empirical convergence.
Figure 2 shows the equivariance error (3) for different parameter sets of DeepSphere for the HEALPix sampling as well as for the graph construction of Khasanova &
Frossard (2017) for the equiangular sampling.
The error is estimated as a function of the sampling resolution and signal frequency.
The resolution is controlled by the number of pixels n = 12 N 2 side for HEALPix and n = 4 b 2 for the equiangular sampling.
The frequency is controlled by setting the set C to functions f made of spherical harmonics of a single degree (cid:96) .
To allow for an almost perfect implementation (up to numerical errors) of the operator R V , the degree (cid:96) was chosen in the range (0 , 3 N side − 1) for HEALPix and (0 , b ) for the equiangular sampling (Gorski et al., 1999).
Using these parameters, the measured error is mostly due to imperfections in the empirical approximation of the Laplace-Beltrami operator and not to the sampling.
[Table]
al.
( b = 128 )
[Table]
al.
(2018) (simpliﬁed, 9 b = 64 )
[Table]
Esteves [Table] al.
[Table] =
[Table]
DeepSphere (equiangular, b = 64 )
[Table]
DeepSphere (HEALPix, N side = 32 )
[Table]
Figure 2 shows that the weighting scheme (4) from (Khasanova & Frossard, 2017) does indeed not lead to a convolution that is equivariant to all rotations g ∈ SO (3) .
For k = 8 neighbors, selecting the optimal kernel width t improves on (Perraudin et al., 2019) at no cost, highlighting the importance of this parameter.
Increasing the resolution decreases the equivariance error in the high frequencies, an effect most probably due to the sampling.
Most importantly, the equivariance error decreases when connecting more neighbors.
Hence, the number of neighbors k gives us a precise control of the tradeoff between cost and equivariance.
E XPERIMENTS 4. 3D OBJECTS RECOGNITION
The recognition of 3D shapes is a rotation invariant task: rotating an object doesn’t change its nature.
While 3D shapes are usually represented as meshes or point clouds, representing them as spherical maps (ﬁgure 4) naturally allows a rotation invariant treatment.
The SHREC’17 shape retrieval contest (Savva et al., 2017) contains 51,300 randomly oriented 3D models from ShapeNet (Chang et al., 2015), to be classiﬁed in 55 categories (tables, lamps, airplanes, etc.).
As in (Cohen et al., 2018), objects are represented by 6 spherical maps.
At each pixel, a ray is traced towards the center of the sphere.
The distance from the sphere to the object forms a depth map.
The cos and sin of the surface angle forms two normal maps.
The same is done for the object’s convex hull.
The maps are sampled by an equiangular sampling with bandwidth b =( n = 4 b 2 = 16 , 384 pixels) or an HEALPix sampling with N side = 32 ( n =
N 2 side = 12 ,pixels).
The equiangular graph is built with (4) and k = 4 neighbors (following Khasanova & Frossard, 2017).
The HEALPix graph is built with (5), k = 8 , and a kernel width t set to the average of the distances (following Perraudin et al., 2019).
The NN is made of 5 graph convolutional layers, each followed by a max pooling layer which down-samples by 4 .
A GAP and a fully connected layer with softmax follow.
The polynomials are all of order P = 3 and the number of channels per layer is 16 , 32 , 64 , 128 , 256 , respectively.
Following Esteves et al.
(2018), the cross-entropy plus a triplet loss is optimized with Adam for 30 epochs on the dataset augmented by 3 random translations.
The learning rate is 5 · 10 − 2 and the batch size is 32.
Results are shown in table 1.
As the network is trained for shape classiﬁcation rather than retrieval, we report the classiﬁcation F1 alongside the mAP used in the retrieval contest.
10 DeepSphere achieves the same performance as Cohen et al.
(2018) and Esteves et al.
(2018) at a much lower cost, suggesting that anisotropic ﬁlters are an unnecessary price to pay.
As the information in those spherical maps resides in the low frequencies (ﬁgure 5), reducing the equivariance error didn’t translate into improved performance.
For the same reason, using the more uniform HEALPix sampling or lowering the resolution down to N side = 8 ( n = 768 pixels) didn’t impact performance either.
accuracy time
Perraudin et al.
(2019), 2D CNN baseline 54.
104 ms
Perraudin et al.
(2019), CNN variant, k =62.
185 ms
Perraudin et al.
FCN variant, k =83.
185 ms k = 8 neighbors, t from section 3.
185 ms k = 20 neighbors, t from section 3.
250 ms k = 40 neighbors, t from section 3.
363 ms
[Figure]
4. C OSMOLOGICAL MODEL CLASSIFICATION
Given observations, cosmologists estimate the posterior probability of cosmological parameters, such as the matter density Ω m and the normalization of the matter power spectrum σ 8 .
Those parameters are typically estimated by likelihood-free inference, which requires a function to predict the parameters from simulations.
As that is complicated to setup, prediction methods are typically benchmarked on the classiﬁcation of spherical maps instead (Schmelzle et al., 2017).
We used the same task, data, and setup as Perraudin et al.
(2019): the classiﬁcation of 720 partial convergence maps made of n ≈ 10 6 pixels ( 1 / 12 ≈ 8% of a sphere at N side = 1024 ) from two Λ CDM cosmological models, ( Ω m = 0 . 31 , σ 8 = 0 . 82) and ( Ω m = 0 .
26 , σ 8 = 0 . 91) , at a relative noise level of 3 . 5 (i.e., the signal is hidden in noise of 3 .
5 times higher standard deviation).
Convergence maps represent the distribution of over- and under-densities of mass in the universe (see Bartelmann, 2010, for a review of gravitational lensing).
Graphs are built with (5), k = 8 , 20 , 40 neighbors, and the corresponding optimal kernel widths t given in section 3.2.
Following Perraudin et al.
, the NN is made of 5 graph convolutional layers, each followed by a max pooling layer which down-samples by 4 .
A GAP and a fully connected layer with softmax follow.
The polynomials are all of order P = 4 and the number of channels per layer is 16 , 32 , 64 , 64 , 64 , respectively.
The cross-entropy loss is optimized with Adam forepochs.
The learning rate is 2 · 10 − 4 · 0 .
999 step and the batch size is 8.
Unlike on SHREC’17, results (table 2) show that a lower equivariance error on the convolutions translates to higher performance.
That is probably due to the high frequency content of those maps (ﬁgure 5).
There is a clear cost-accuracy tradeoff, controlled by the number of neighbors k (ﬁgure 7).
This experiment moreover demonstrates DeepSphere’s ﬂexibility (using partial spherical maps) and scalability (competing spherical CNNs were tested on maps of at most 10 , 000 pixels).
4. C LIMATE EVENT SEGMENTATION
We evaluate our method on a task proposed by (Mudigonda et al., 2017): the segmentation of extreme climate events, Tropical Cyclones (TC) and Atmospheric Rivers (AR), in global climate simulations (ﬁgure 1c).
The data was produced by a 20-year run of the Community Atmospheric Model v5 (CAM5) and consists of 16 channels such as temperature, wind, humidity, and pressure at multiple altitudes.
We used the pre-processed dataset from (Jiang et al., 2019).
11 There is 1,072, spherical maps, down-sampled to a level-5 icosahedral sampling ( n = 10 · 4 l + 2 = 10 , 242 pixels).
The labels are heavily unbalanced with 0.1% TC, 2.2% AR, and 97.7% background (BG) pixels.
The graph is built with (5), k = 6 neighbors, and a kernel width t set to the average of the distances.
Following Jiang et al.
(2019), the NN is an encoder-decoder with skip connections.
Details in section C.3.
The polynomials are all of order P = 3 .
The cross-entropy loss (weighted or nonweighted) is optimized with Adam for 30 epochs.
The learning rate is 1 · 10 − 3 and the batch size is 64.
Results are shown in table 3 (details in tables 6, 7 and 8).
The mean and standard deviation are computed over 5 runs.
Note that while Jiang et al.
and Cohen et al.
(2019) use a weighted cross-entropy loss, that is a suboptimal proxy for the mAP metric.
DeepSphere achieves state-of-
[Table]
[Table] [Table]
the-art performance, suggesting again that anisotropic ﬁlters are unnecessary.
Note that results from
Mudigonda et al.
(2017) cannot be directly compared as they don’t use the same input channels.
Compared to Cohen et al.
(2019)’s conclusion
[Table]
it is surprising that S2R does worse than DeepSphere (which is limited to S2S).
Potential explanations are (i) that their icosahedral projection introduces harmful distortions [Table] or (ii) that a larger architecture can compensate for the lack of generality.
We indeed observed that more feature maps and depth led to higher performance (section C.3).
4. U NEVEN SAMPLING
To demonstrate the ﬂexibility of modeling the sampled sphere by a graph, we collected historical measurements from n ≈ 10 , 000 weather stations scattered across the Earth.
The spherical data is heavily non-uniformly sampled, with a much higher density of weather stations over North America than the Paciﬁc (ﬁgure 1d).
For illustration, we devised two artiﬁcial tasks.
A dense regression: predict the temperature on a given day knowing the temperature on the previous 5 days.
A global regression: predict the day (represented as one period of a sine over the year) from temperature or precipitations.
Predicting from temperature is much easier as it has a clear yearly pattern.
The graph is built with (5), k = 5 neighbors, and a kernel width t set to the average of the distances.
The equivariance property of the resulting graph has not been tested, and we don’t expect it to be good due to the heavily non-uniform sampling.
The NN is made of 3 graph convolutional layers.
The polynomials are all of order P = 0 or 4 and the number of channels per layer is 50 , 100 , 100 , respectively.
For the global regression, a GAP and a fully connected layer follow.
For the dense regression, a graph convolutional layer follows instead.
The MSE loss is optimized with RMSprop for 250 epochs.
The learning rate is 1 · 10 − 3 and the batch size is 64.
Results are shown in table 4.
While using a polynomial order P = 0 is like modeling each time series independently with an MLP, orders P > 0 integrate neighborhood information.
Results show that using the structure induced by the spherical geometry always yields better performance.
C ONCLUSION
This work showed that DeepSphere strikes an interesting, and we think currently optimal, balance between desiderata for a spherical CNN.
A single parameter, the number of neighbors k
a pixel is connected to in the graph, controls the tradeoff between cost and equivariance (which is linked to performance).
As computational cost and memory consumption scales linearly with the number of pixels, DeepSphere scales to spherical maps made of millions of pixels, a required resolution to faithfully represent cosmological and climate data.
Also relevant in scientiﬁc applications is the ﬂexibility offered by a graph representation (for partial coverage, missing data, and non-uniform samplings).
Finally, the implementation of the graph convolution is straightforward, and the ubiquity of graph neural networks — pushing for their ﬁrst-class support in DL frameworks — will make implementations even easier and more efﬁcient.
A potential drawback of graph Laplacian-based approaches is the isotropy of graph ﬁlters, reducing in principle the expressive power of the NN.
Experiments from Cohen et al.
(2019) and Boscaini et al.
(2016) indeed suggest that more general convolutions achieve better performance.
Our experiments on 3D shapes (section 4.1) and climate (section 4.3) however show that DeepSphere’s isotropic ﬁlters do not hinder performance.
Possible explanations for this discrepancy are that NNs somehow compensate for the lack of anisotropic ﬁlters, or that some tasks can be solved with isotropic ﬁlters.
The distortions induced by the icosahedral projection in (Cohen et al., 2019) or the leakage of curvature information in (Boscaini et al., 2016) might also alter performance.
Developing graph convolutions on irregular samplings that respect the geometry of the sphere is another research direction of importance.
Practitioners currently interpolate their measurements (coming from arbitrarily positioned weather stations, satellites or telescopes) to regular samplings.
This practice either results in a waste of resolution or computational and storage resources.
Our ultimate goal is for practitioners to be able to work directly on their measurements, however distributed.
A CKNOWLEDGMENTS
We thank Pierre Vandergheynst for advices, and Taco Cohen for his inputs on the intriguing results of our comparison with Cohen et al. We thank the anonymous reviewers for their constructive feedback. The following software packages were used for computation and plotting: PyGSP (Defferrard et al.) , healpy (Zonca et al., 2019), matplotlib (Hunter, 2007), SciPy (Virtanen et al., 2020), NumPy (Walt et al., 2011), TensorFlow (Abadi et al., 2015).
S UPPLEMENTARY M ATERIAL A P ROOF OF THEOREM 3.
Preliminaries.
The proof of theorem 3.1 is inspired from the work of Belkin & Niyogi (2008).
As a result, we start by restating some of their results.
Given a sampling V = { x i ∈ M} ni =1 of a closed, compact and inﬁnitely differentiable manifold M , a smooth ( ∈ C ∞ ( M ) ) function f : M → R , and deﬁned the vector f of samples of f as follows: T V f = f ∈ R n , f i = f
( x i ) .
The proof is constructed by leveraging 3 different operators:
• The extended graph Laplacian operator, already presented in (6), is a linear operator L tn :
L 2 ( M )
→ L 2 ( M ) deﬁned as
[Equation]
Note that we have the following relation L tn f = T V L tn f .
• The functional approximation to the Laplace-Beltrami operator is a linear operator L t : L
[Equation]
→
[Equation]
deﬁned as
[Equation]
where µ is the uniform probability measure on the manifold M , and vol ( M ) is the volume of M .
• The Laplace-Beltrami operator ∆ M is deﬁned as the divergence of the gradient
[Equation]
of a differentiable function f : M → R .
The gradient ∇ f : M → T p M is a vector ﬁeld deﬁned on the manifold pointing towards the direction of steepest ascent of f , where T p M is the afﬁne space of all vectors tangent to M at p .
Leveraging these three operators, Belkin & Niyogi (2008; 2007) have build proofs of both pointwise and spectral convergence of the extended graph Laplacian towards the Laplace-Beltrami operator in the general setting of any compact, closed and inﬁnitely differentiable manifold M , where the sampling V is drawn randomly on the manifold.
For this reason, their results are all to be interpreted in a probabilistic sense.
Their proofs consist in establishing that (6) converges in probability towards (8) as n → ∞ and (8) converges towards (9) as t → 0 .
In particular, this second step is given by the following:
Proposition 1 (Belkin & Niyogi (2008), Proposition 4.4) .
Let M be a k -dimensional compact smooth manifold embedded in some Euclidean space R N , and ﬁx y ∈ M .
Let f ∈ C ∞ ( M ) .
Then
[Equation]
Building the proof.
As the sphere is a compact smooth manifold embedded in R 3 , we can reuse proposition 1.
Thus, our strategy to prove Theorem 3.1 is to (i) show that
[Equation]
for a particular class of deterministic samplings, and (ii) apply Proposition 1.
We start by proving that for smooth functions, for any ﬁxed t , the extended graph Laplacian L tn converges towards its continuous counterpart L t as the sampling increases in size.
Proposition 2.
For an equal area sampling { x i ∈ S 2 } ni =1 : A i =
A j ∀ i, j of the sphere it is true that for all f : S 2 → R Lipschitz with respect to the Euclidean distance (cid:107)·(cid:107) with Lipschitz constant
C f (cid:12)(cid:12)(cid:12)(cid:12) (cid:12)
[Equation]
(cid:12) ≤ C f d ( n ) .
Furthermore, for all y ∈ S 2 the Heat Kernel Graph Laplacian operator L tn converges pointwise to the functional approximation of the Laplace Beltrami operator L t
[Equation]
Proof.
Assuming f : S 2 → R is Lipschitz with Lipschitz constant C f , we have
[Equation]
where σ i ⊂ S 2 is the subset of the sphere corresponding to the patch around x i .
Remember that the sampling is equal area.
Hence, using the triangular inequality and summing all the contributions of the n patches, we obtain
[Equation]
A direct application of this result leads to the following pointwise convergences
[Equation]
Deﬁnitions 6 and 8 end the proof.
The last proposition show that for a ﬁxed t , L tn f ( x ) → 1 / 4 π 2 L t f ( x ) .
To utilize Proposition 1 and complete the proof, we need to ﬁnd a sequence of t n for which this holds as t n → 0 .
Furthermore this should hold with a faster decay than
[Equation]
Proposition 3.
Given a sampling regular enough, i.e., for which we assume A i =
A j ∀ i, j and d ( n ) ≤ Cn α , α ∈ (0 , 1 / 2] , a Lipschitz function f and a point y ∈ S 2 there exists a sequence t n = n β , β < 0 such that
[Equation]
Proof.
To ease the notation, we deﬁne
[Equation]
We start with the following inequality
[Equation] [Equation]
where C φ ty is the Lipschitz constant of x → φ t ( x, y ) and the last inequality follows from Proposition 2.
Using the assumption d ( n ) ≤ C √ n we ﬁnd
[Equation]
now
[Equation]
where C K ty is the Lipschitz constant of the function x → K t ( x ; y ) .
We note that this constant does not depend on y :
[Equation]
Hence we have
[Equation]
Inculding this result in (14) and rescaling by 1 / 4 πt 2 , we obtain
[Equation]
In order for C 4 π (cid:104) (cid:107) f (cid:107) ∞ √ 2 e
[Equation]
0 , we need
[Equation]
It happens if (cid:26)
[Equation]
=
[Equation]
β
[Equation]
( −
[Equation]
Indeed, we have n a lphat 5 / 2 = n 5 / 2 β + α n →∞ −−−−→ ∞ since 52 β + α >
⇐⇒ β > − 2 αand n α t 2 = n 2 β + α n →∞ −−−−→ ∞ since 2 β + α >
⇐⇒ β > − α 2 .
As a result, for t = n β with β ∈ ( − 15 , 0) we have
[Equation]
which concludes the proof.
Theorem 3.1, is then an immediate consequence of Proposition 3 and 1.
Proof of Theorem 3.1.
Thanks to Proposition 3 and Proposition 1 we conclude that ∀ y ∈ S
[Equation]
In (Belkin & Niyogi, 2008), the sampling is drawn from a uniform random distribution on the sphere, and their proof heavily relies on the uniformity properties of the distribution from which the sampling is drawn.
In our case the sampling is deterministic, and this is indeed a problem that we need to overcome by imposing the regularity conditions above.
[Table]
To conclude, we see that the result obtained is of similar form than the result obtained in (Belkin &
Niyogi, 2008).
Given the kernel density t ( n ) = n β , Belkin & Niyogi (2008) proved convergence in the random case for β ∈ ( − 14 , 0) and we proved convergence in the deterministic case for β ∈ ( − 2 α 5 , 0) , where α ∈ (0 , 1 / 2] (for the spherical manifold).
B P ROOF OF T HEOREM 3.
Proof.
Fix x ∈ S 2 .
Since any rotation R ( g ) is an isometry, and the Laplacian ∆ commutes with all isometries of a Riemanniann manifold, and deﬁning R ( g )
f
=: f (cid:48) for ease of notation, we can write that
[Equation]
Since g − 1 ( x ) ∈ S 2 and f (cid:48) still satisﬁes hypothesis
[Equation]
we can apply theorem
[Equation]
1 to say that
[Equation]
to conclude that
[Equation]
C E XPERIMENTAL DETAILS C. 3D OBJECTS RECOGNITION
Table 5 shows the results obtained from the SHREC’17 competition’s ofﬁcial evaluation script.
[Equation]
C. C OSMOLOGICAL MODEL CLASSIFICATION
[Equation] [Equation]
[Table] [Table]
C. C LIMATE EVENT SEGMENTATION
Table 6, 7, and 8 show the accuracy, mAP, and efﬁciency of all the NNs we ran.
The experiment with the model from Jiang et al.
(2019) was rerun in order to obtain the AP metrics, but with a batch size of 64 instead of 256 due to GPU memory limit.
Several experiments were run with different architectures for DeepSphere (DS).
Jiang architecture use a similar one as Jiang et al.
(2019), with only the convolutional operators replaced.
DeepSphere only is the original architecture giving the best results, deeper and with four times more feature maps than Jiang architecture.
And the wider architecture is the same as the previous one with two times the number of feature maps.
Regarding the weighted loss [Table] the weights are chosen with scikit-learn function compute class weight on the training set.
[Table]
DeepSphere with Jiang architecture
encoder:
[Equation]
decoder: (18)
[Equation]
Concat is the operation that concatenate the results of the corresponding encoder layer.
Original DeepSphere architecture with encoder decoder
encoder: (19)
[Equation]
+ Pool + [ GC 512 ] Ldecoder: (20)
Unpool + [ GC 512 + BN + ReLU ] L 1 + concat + [ GC 512 + BN + ReLU ] L+ Unpool + [ GC 256 + BN + ReLU ] L 2 + concat + [ GC 256 + BN + ReLU ]
L 2 + Unpool + [ GC 128 + BN + ReLU ] L+ concat + [ GC 128 + BN + ReLU ]
L 3 + Unpool + [ GC 64 + BN + ReLU ]
L 4 + concat + [ GC 64 + BN + ReLU ] L+ Unpool + [ GC 32 + BN + ReLU ]
L 5 + [ GC 3 ] L
C. U NEVEN SAMPLING
Architecture for dense regression:
[Equation]
Architecture for global regression
[Equation]