DeepSphere: a graph-based spherical CNN
Authors: Michaël Defferrard
Martino Milani
Frédérick Gusset
Nathanaël Perraudin
Venue:  (2020)
Keywords: spherical cnns
graph neural networks
geometric deep learning

Abstract: Designing a convolution for a spherical neural network requires a delicate tradeoff between efficiency and rotation equivariance. DeepSphere, a method based on a graph representation of the discretized sphere, strikes a controllable balance between these two desiderata. This contribution is twofold. First, we study both theoretically and empirically how equivariance is affected by the underlying graph with respect to the number of pixels and neighbors. Second, we evaluate DeepSphere on relevant problems. Experiments show state-of-the-art performance and demonstrates the efficiency and flexibility of this formulation. Perhaps surprisingly, comparison with previous work suggests that anisotropic filters might be an unnecessary price to pay. Our code is available at https://github.com/deepsphere.

Published as a conference paper at ICLR 2020
DEEPSPHERE: A GRAPH-BASED SPHERICAL CNN
Michaël Defferrard, Martino Milani & Frédérick Gusset
École Polytechnique Fédérale de Lausanne (EPFL), Switzerland (michael. defferaad, martino mi lani, Trederick.gusset ch
Nathanaël Perraudin
Swiss Data Science Center (SDSC), Switzerland nathanael perraudin sascc wtc.eestss.............. ch
ABSTRACT
Designing a convolution for a spherical neural network requires a delicate trade- off between efficiency and rotation equivariance. DeepSphere, method based on a graph representation of the sampled sphere, strikes a controllable balance between these two desiderata. This contribution is twofold. First, we study both theoretically and empirically how equivariance is affected by the underly- ing graph with respect to the number of vertices and neighbors. Second, we evaluate DeepSphere on relevant problems. Experiments show state-of-the-art performance and demonstrates the efficiency and flexibility of this formulation. Perhaps surprisingly, comparison with previous work suggests that anisotropic filters might be an unnecessary price to pay. Our code is available at hhtpps: //github.comm com.hbppeeo//deeppeeeeee
INTRODUCTION
Spherical data is found in many applications (figure D. Planetary data (such meteorological or geological measurements) and brain activity are example of intrinsically spherical data. The observation of the universe, LIDAR scans, and the digitalization of 3D objects are examples of projections due to observation. Labels or variables are often to be inferred from them. Examples are the inference of cosmological parameters from the distribution of mass in the universe (Perraudin) et al. 2019). the segmentation of omnidirectional images (Khasanova & Frossard 2017), and the segmentation of cyclones from Earth observation (Mudigonda et al. (017)
Mal modes paiedal R.1.
CRB Sorgr 22
LRUHEES 1MAN JEALER
of GHCN sttter
2017
(a)
(b)
(c)
(d)
(e)
Figure 1: Examples of spherical data: (a) brain activity recorded through magnetoencephalooraa
phy (MEG) ()) the cosmic microwave background (CMB) temperature from Planck Collaboration
(b)
(2016), (c) hourly precipitation from a climate simulation (Jiang et al., 2019). (d) daily maximum temperature from the Global Historical Climatology Network (GHCN) F rigid full-sphere sam- pling not ideal: brain activity only measured on the scalp, the Milky Way's galactic plane masks observations, climate scientists desire variable resolution, and the position of weather sta- tions is arbitrary and changes over time. (c) Graphs can faithfully and efficiently represent sampled spherical data by placing vertices where it matters.

---

Published as a conference paper at ICLR 2020
As neural networks (NNs) have proved to be great tools for inference, variants have been developed to handle spherical data. Exploiting the locally Euclidean property of the sphere, early attempts used standard 2D convolutions
grid sampling of the sphere (Boomsma & Frellsen 2017| Su & Grau- man 2017 Coors et al. 2018). While simple and efficient, those convolutions are not equivariant to rotations. On the other side of this tradeoff, [Cohen al. (2018) Esteves al. (2018) proposed to
perform proper spherical convolutions through the spherical harmonic transform. While equivariant to rotations, those convolutions are expensive (section 2
As a lack of equivariance can penalize performance (section 4.2) and expensive convolutions pro- hibit their application some real-world problems, methods standing between these two extremes are desired. (Cohen et al. (2019) proposed to reduce costs by limiting the size of the representation of the symmetry group by projecting the data from the sphere to the icosahedron. The distortions introduced by this projection might however hinder performance (section
Another approach to represent the sampled sphere as a graph connecting pixels according to the distance between them (Bruna al.) [2013] Khasanova & Frossard 2017]] Perraudin et al.) 2019). While Laplacian-based graph convolutions are more efficient than spherical convolutions, they are not exactly equivariant (Defterrard et al, 2019). In this work, we argue that graph-based spheri- cal CNNs strike an interesting balance, with a controllable tradeoff between cost and equivariance (which is linked performance). Experiments multiple problems of practical interest show the competitiveness and flexibility of this approach.
METHOD
DeepSphere leverages graph convolutions to achieve the following properties: (i) computational efficiency, (ii) sampling flexibility, and (iii) rotation equivariance (section B3. The main idea is to model the sampled sphere as a graph of connected pixels: the length of the shortest path between two pixels is an approximation of the geodesic distance between them. We use the graph CNN formulation introduced (Defferrard al. 2016) and a pooling strategy that exploits hierarchical samplings of the sphere.
Sampling. A sampling scheme = {ar; S2) is defined be the discrete subset of the sphere containing the n points where the values of the signals that we want to analyse are known. For a given continuous signal f. we represent such values in vector E R" As there no analogue of uniform sampling on the sphere, many samplings have been proposed with different tradeoffs. In this work, depending on the considered application, we will use the equiangular (Driscoll & ( HEALPix (Gorski et al. 2005), and icosahedral (Baumgardner & Frederickson 1985) samplings.
Graph. From V, we construct a weighted undirected graph G = (V, w). where the elements of are the vertices and the weight Wij Wye is a similarity measure between vertices z; and 2z. The combinatorial graph Laplacian L E R**= is defined as L D A, where A = (wij) is the weighted adjacency matrix, D (d,)) is the diagonal degree matrix, and d, = E, Wi the weighted degree of vertex x,. Given a sampling V. usually fixed by the application or the available measurements, the freedom in constructing G is in setting w. Section 3 shows how to set w to minimize the equivariance error.
Convolution. On Euclidean domains, convolutions are efficiently implemented by sliding a win- dow in the signal domain. On the sphere however, there is straightforward way to implement convolution in the signal domain due to non-uniform samplings. Convolutions are most often per- formed in the spectral domain through a spherical harmonic transform (SHT). That is the approach taken by Cohen et al. 2018 and Esteves et al. (2018), which has computational cost of (n3/2) on isolatitude samplings (such as the HEALPix and equiangular samplings) and O(n²) in general.
attps: http:///mtttiiiaaaaaaarrrrrar/as
2

---

Published as a conference paper at ICLR 2020
On the other hand, following Defferrard et al. (2016), graph convolutions be defined as
h {
Σ ==0
LgL' f:
where P is the polynomial order (which corresponds to the filter's size) and a; are the coefficients to be optimized during training Those convolutions are used by Khasanova & Frossard (017)) and Perraudin et al. 2019) and cost (n) operations through a recursive application of L
Pooling. Down- and up-sampling is natural for hierarchical samplings 2 where each subdivision divides a pixel in (an equal number of) child sub-pixels. To pool (down-sample), the data supported on the sub-pixels is summarized by a permutation invariant function such the maximum or the average. To unpool (up-sample), the data supported on a pixel is copied to all its sub-pixels.
Architecture. All our NNs are fully convolutional, and employ global average pooling (GAP) for rotation invariant tasks. Graph convolutional layers are always followed by batch normalization and ReLU activation, except in the last layer. Note that batch normalization and activation act the elements of f independently, and hence don't depend on the domain of f.
GRAPH CONVOLUTION AND EQUIVARIANCE
While the graph framework offers great flexibility, its ability faithfully represent the underlying sphere - for graph convolutions to be rotation equivariant - highly depends on the sampling locations and the graph construction.
3.1 PROBLEM FORMULATION
A continuous function f C(S²) C Fv - is sampled as Ty(J) f by the sampling operator Ty C(S?) C Fy + R" defined as f f f(z)). We require Fy be a suitable subspace of continuous functions such that Tv is invertible, i.e., the function f E Fy can be unambiguously reconstructed from its sampled values f. The existence such a subspace depends on the sampling V and its characterization is a common problem in signal processing (Driscoll & Healy 1994). For most samplings,
f
not known if Fy exists and hence Ty is invertible. special case the equiangular sampling where a sampling theorem holds, and thus closed-for of Z is known. For samplings where no such sampling formula is available, we leverage the discrete SHT to reconstruct from Tv.f, thus approximating T For all theoretical considerations, assume that Fy exists and f E Fy.
By definition, the (spherical) graph convolution is rotation equivariant if and only if it commutes with the rotation operator defined as R(g),g E SO(3): R(g) = (g- In the context of this work, graph convolution is performed by recursive applications of the graph Laplacian (T). Hence, if R(g) commutes with L, then, by recursion, it will also commute with the convolution h(L). As a result, ((L) is rotation equivariant if and only if
Lf = LRv(g)f,
Vf e Fy and Vgg E SO(3),
where Rv(g) = Tvg) For an empirical evaluation of equivariance, we define the normalized equivariance error for a signal f and a rotation g as
(f(f.g) = -(11 (
LRv()))II
(2)
|f
More generally for class of signals f E TCC Fv, the mean equivariance error defined FLCC = REEC. (z(f.g)
(3)
represents the overall equivariance error. The expected value is obtained by averaging over finite number random functions and random rotations.
'In practice, training with Chebyshev polynomials (instead of monomials) is slightly more stable. We believe it to be due to their orthogonality and uniformity.
+ "As long as the graph is sparsified such that number of edges, i.c., the number non-zeros in A, proportional to the number of vertices n. This can always be done as most weights are very small.
The equiangular, HEAL.Ps and icosahedral samplings are of this kind.
3

---

Published as a conference paper at ICLR 2020
10
32
kemel width
groph,
6:30 praph, 6-
109
RX 128
1X
10%
6.5 smoph, R- Permoalo
praph.
ton
meem oqpivarianes ernar Ei.c
12N
idds pixels
Figure 3: Kernel widths.
10
10a
Klasanova Frossard, b=
Figure 4: 3D object represented as spherical depth map.
Perraudin et al., Am
NNNN graph, 8= neighbors graph, noighbors
power spectrum
N-NN graph, k 40 neighbors
coomologe
clmase 216 aerabled
10a
102
spherical harmonic degree
.0
10
Figure 2: Mean equivariance error There is a clear tradeoff between equivariance and computational cost, gov- erned by the number of vertices n and edges kn.
spheri cal harmonic degree
Figure 5: Power spectral densities.
3.2 FINDING THE OPTIMAL WEIGHTING SCHEME
Considering the equiangular sampling and graphs where each vertex is connected to 4 neighbors (north, south, east, west), Khasanova & Frossard (2017) designed a weighting scheme to minimize 3 for longitudinal and latitudinal rotations Their solution gives weights inversely proportional to
Euclidean distances:
Wij
While the resulting convolution is equivariant the whole of SO(3) (figure 2). is enough for omnidirectional imaging because, as gravity consistently orients the sphere, objects only rotate longitudinally or latitudinally.
To achieve equivariance to all rotations, we take inspiration from Belkin & Niyogi (2008). They prove that for random uniform sampling, the graph Laplacian L built from weights
Wij = e + 10
(5)
converges to the Laplace- Beltram operator Aga as the number of samples grows to infinity. This result good starting point as Aga commutes with rotation, i.c., Aga Ag=)(g) R(g) While the weighting scheme is full (i.e., every vertex is connected to every other vertex), most weights are small due to the exponential. We hence make an approximation to limit the cost of the convolution (T) by only considering the i nearest neighbors (k-NN) of each vertex. Given k the optimal kernel width is found by searching for the minimizer of 3 Figure 3 shows the optimal kernel widths found for various resolutions of the HEALPix sampling. As predicted by the theory. In n R. Importantly however, the optimal t also depends on the number of neighbors k.
2019 connected each vertex to their 8 adjacent vertices in the tiling of the sphere, computed the weights with (5), and heuristically set t to half the average squared Euclidean distance between connected vertices. This heuristic however over- estimates t (figure and leads to an increased equivariance error (figure 2)
Considering the HEALPix Campling,Perrrauue
'Fquivariance longitudinal rotation is essentially given by the equiangular sampling.
4

---

Published as a conference paper at ICLR 2020
3.3 ANALYSIS OF THE PROPOSED WEIGHTING SCHEME
We analyze the proposed weighting scheme both theoretically and empirically.
Theoretical convergence. We extend the work of (Belkin & Niyogi 2008) to a sufficiently regular, deterministic sampling. Following their set- ting. we work with the extended graph Laplacian operator as the linear op- erator L : (S²) - (8²) such that
- 1=:-112 (f(y)
(,)(() 71
(6)
This operator extends the graph Laplacian with the weighting scheme { to each point of the sphere (i.e., L = TyL, As the radius of the kernel / will be adapted to the number of samples, we scale the operator as it (s*(( Li Given a sampling V. we define a; to be the patch of the surface of the sphere corresponding to Tú, A; its corresponding area, and di the largest distance between the center r, and any point on the surface o, Define d( == max;= di and A == max,=11..... I..... Aj.
Figure 6: Patch.
Theorem 3.1. For a sampling V of the sphere that is equi-area and such that d( S n a E (o, 1/2], for all f : + IR Lipschitz with respect to the Euclidean distance in , for all E S?, there exists sequence in = n², 8 E IR such that
C
lim 1 f(y) Asaf(ly) n
This is major step towards equivariance, as the Laplace-Beltrann operator commutes with rotation.
Based on this property, we show the equivariance of the scaled extended graph Laplacian.
Theorem 3.2. Under the hypothesis of theorem 3.1 the scaled graph Laplacian commutes with any rotation, the limit of infinite sampling, i.e.,
Vy E g2
R(o) f(U) - f. R(g)f(v)
11000 0.
From this theorem, it follows that the discrete graph Laplacian will be equivariant in the limit of oo by construction L TyLy and as the scaling does not affect the equivariance property of L
Importantly, the proof of Theorem 3.1 (in Appendix[A) A inspires our construction of the graph Lapla- cian. In particular, tells us that t should scale as n², which has been empirically verified (figure 3 Nevertheless, it is important to keep in mind the limits of Theorem 3.1 and 3.2 Both theorems present asymptotic results, but in practice we will always work with finite samplings. Furthermore, since this method is based the capability of the eigenvectors of the graph Laplacian to approx- imate the spherical harmonics, a stronger type of convergence of the graph Laplacian would be preferable, i.e., spectral convergence (that is proved for a full graph in the case of random sam- pling for a class of Lipschitz functions in (Belkin & Niyogi 2007)). Finally, while we do not have formal proof for it, we strongly believe that the HEALPix sampling does satisfy the hypothesis d(n) < no E (0, 1/2], with very close equal to 1/2. The empirical results discussed in the next paragraph also points in this direction. This is further discussed in Appendix A
C.
Empirical convergence. Figure 2 shows the equivariance error 3 for different parameter sets of DeepSphere for the HEALPix sampling as well as for the graph construction of Khasanova & Frossard (017)) for the equiangular sampling. The error estimated as a function the sampling resolution and signal frequency. The resolution is controlled by the number of pixels n 12N2 side for HEALPix and n = 4b2 for the equiangular sampling. The frequency is controlled by setting the set C' to functions f made of spherical harmonics of a single degree &. To allow for an almost perfect implementation (up to numerical errors) of the operator Ry, the degree was chosen in the range (0, 3...N - 1) for HEALPix and (0, b) for the equiangular sampling (Gorski et al. 1999). Using these parameters, the measured error is mostly due to imperfections in the empirical approximation of the Laplace- Beltrami operator and not to the sampling.
5

---

Published as a conference paper at ICLR 2020
performance
size
speed
F1
mAP
params
inference
training
Cohen et al.
(2018) (b = 128)
67.6
1400 k
38.0 ms
50h
Cohen et al.
(2018) (simplified 64)
78.9
66.5
400 k
12.0
32h
Esteves et al.
(2018) (b 64)
79.4
68.5
500 k
9.8 ms
3h
DeepSppere (equangular, b = 64)
79.4
66.5
190k
0.9 ms
50m m
DeepSphere (HEALPix, Noide 32)
80.7
68.6
190
0.9 ms
50
Table 1: Results on SHREC' 17 (3D shapes). DeepSphere achieves similar performance at a much lower cost, suggesting that anisotropic filters are an unnecessary price pay.
Figure shows that the weighting scheme from Khassaova & Frossard 2017 does indeed not lead convolution that is equivariant all rotations E SO(3) For k 8 neighbors, selecting the optimal kernel width improves on (Perraudin et al. 2019 at no cost, highlighting the importance of this parameter. Increasing the resolution decreases the equivariance error in the high frequencies, an effect most probably due to the sampling. Most importantly, the equivariance error decreases when connecting more neighbors. Hence, the number of neighbors gives us precise control of the tradeoff between cost and equivariance.
4 EXPERIMENTS
4.1 3D OBJECTS RECOGNITION
The recognition of 3D shapes is a rotation invariant task: While 3D shapes are usually represented as meshes or point clouds, representing them as spherical maps (figure naturally allows a rotation invariant treatment.
rotating an object doesn't 't change its nature.
The SHREC' 17 shape retrieval contest (Savva et al., 2017) contains 51,300 randomly oriented 3D models from ShapeNet (Chang et al. 2015). to be classified in 55 categories (tables, lamps, air- planes, etc.). As in (Cohen et al. 2018). objects are represented by 6 spherical maps. At each pixel, ray is traced towards the center of the sphere. The distance from the sphere to the object forms depth map. The cos and sin of the surface angle forms two normal maps. The same is done for the object's convex hull [8 The maps are sampled by an equiangular sampling with bandwidth b 64 (n = 402 = 16, 384 pixels) or an HEALPix sampling with Ngide = 32 (n 12N2 side 12, 288 pixels).
The equiangular graph is built with 4 and k = 4 neighbors (following Khasanova & Frossard 2017). The HEALPix graph is built with (5) A 8, and kernel width set to the average of the distances (following Perraudin et al. 2019p. The NN is made of 5 graph convolutional layers, each followed by a max pooling layer which down-samples by with softmax follow. The polynomials are all of order P = 3 and the number of channels per layer 16, 32, 64, 128, 256, respectively. Following Esteves et al. (2018), the cross-entropy plus a triplet loss is optimized with Adam for 30 epochs on the dataset augmented by 3 random translations. The learning rate 5 10-2 and the batch size 32.
A GAP and a fully connected layer
Results are shown in table [ As the network is trained for shape classification rather than trieval, we report the classification alongside the mAP used the retrieval contest I0 DeepSphere achieves the same performance Cohen et al. (2018) and Esteves al. 2018)) much lower cost, suggesting that anisotropic filters are an unnecessary price to pay. As the information in those spher- ical maps resides in the low frequencies (figure 5). reducing the equivariance error didn't 't translate into improved performance. For the same reason, using the more uniform HEALPix sampling or lowering the resolution down to Nside (n 768 pixels) didn't impact performance either.
We however verified that the convolution is equivariant to longitudinal and latitudinal rotations, as intended.
Albeit didn't 't. observe much improvement by using the convex hull.
7 As implemented in https: //qi thub. com/ jonas koeh er /22nn
We omit the F1 for Cohen al. 2018 didn't I get the mAP reported in the paper when running it.

---

Published as a conference paper at ICLR 2020
accuracy
time
92
accuracy [%]
Perraudin et al.
(2019), 2D CNN baseline
54.2
104 ms
20
Perraudin al.
(0T9), CNN variant,
62.1
185 ms
90-
Perraudin et al.
(2019), FCN variant, h 8
83.8
185 ms
neighbors, from section 3.2
87.1
185
88
k 20 neighbors, / from section 3.2
91.3
250 ms
k 40 neighbors, & from section
92.5
363 ms
200 250 300
inference time [ms]
Table 2: Results on the classification of partial convergence maps, Lower equivariance error translates to higher performance.
Figure 7: Tradeoff be- tween cost and accuracy.
4.2 COSMOLOGICAL MODEL CLASSIFICATION
Given observations, cosmologists estimate the posterior probability of cosmological parameters, such as the matter density Ωm and the normalization of the matter power spectrum dg. Those pa- rameters are typically estimated by likelihood- free inference, which requires function to predict the parameters from simulations. As that is complicated to setup, prediction methods are typically benchmarked on the classification of spherical maps instead (Schmelzle et al. 2017). We used the same task, data, and setup as Perraudin et al. (2019): : the classification of 720 partial convergence maps made of z 10 pixels (1/12 = 8% of sphere at Naide = 1024) from two ACDM cosmo- logical models, (92m 0.31, dg 0.82) and (92m= 0.26, ds = 0.91), at relative noise level of 3.5 (i.e., the signal is hidden in noise of 3.5 times higher standard deviation). Convergence maps represent the distribution of over- and under-densities of mass in the universe (see Bartelmann 20T0 for a review of gravitational lensing).
Graphs are built with 5). k 8, 20, 40 neighbors, and the corresponding optimal kernel widths t given in section 3.2 Following Perraudin et al. (2019), the NN is made of 5 graph convolutional lay- ers, each followed by max pooling layer which down-samples by 4. A GAP and a fully connected layer with softmax follow. The polynomials are all of order P = 4 and the number of channels per layer is 16, 32, 64, 64, 64, respectively. The cross-entropy loss is optimized with Adam for 80 epochs. The learning rate is 2 10-1 0.999stcp and the batch size is 8.
Unlike on SHREC' results (table 2 show that lower equivariance error on the convolutions translates to higher performance. That is probably due to the high frequency content of those maps (figure5). There a clear cost-accuracy tradeoff, controlled by the number of neighbors & (figure This experiment moreover demonstrates DeepSphere's 's flexibility (using partial spherical maps) and scalability (competing spherical CNNs were tested on maps of at most 10, 000 pixels).
4.3 CLIMATE EVENT SEGMENTATION
We evaluate our method on a task proposed by (Mudigonda et al.) 2017): :) the segmentation of ex- treme climate events, Tropical Cyclones (TC) and Atmospheric Rivers (AR), global climate sim- ulations (figure [Tc.. The data was produced by 20-year run of the Community Atmospheric Model v5 (CAM5) and consists of 16 channels such as temperature, wind. humidity, and pressure at mul- tiple altitudes. We used the pre-processed dataset from (Jiang et al. 2019 There is 1,072,805 spherical maps, down-sampled to a level-5 icosahedral sampling (n 10- +2 10, 242 pixels). The labels heavily unbalanced with 0.1% TC, 2.2% AR, and 97.7% background (BG) pixels.
The graph is built with (5).
neighbors, and kernel width set to the average of the distances. Following Jiang et al. (2019) the NN is encoder-decoder with skip connections. Details in section C The polynomials are all of order = 3. The cross-entropy loss (weighted or non- weighted) is optimized with Adam for 30 epochs. The learning rate is 1 10-3 and the batch size is 64.
Results are shown in table 3 (details in tables 6, 7 and 8 The mean and standard deviation computed over runs. Note that while Jiang et al. 2019) and Cohen et al. (2019) use a weighted cross-entropy loss. that suboptimal proxy for the mAp metric. DeepSphere achieves state-of-
Available http: / /isl and.d wer.beeeeeeee. www.eeey.eeeeeeeeeoeeeeeeo.ee.ee www..ydduuggcnn////e

---

Published as a conference paper at ICLR 2020
accuracy
mAP
Jiang et al. (2019)
(rerun)
94.95
38,41
Cohen al. (20T9) (S2R)
97.5
68.6
Cohen et al. (2019) (R2R)
97.7
75.9
DeepSphere (weighted loss)
97.8 + 0.3
77.15 + 1.94
DeepSphere (non-weighted loss)
87.8 + 0.5
89.16 + 1.37
Table 3: Results on climate event segmentation: mean accuracy (over TC, AR, BG) and mean average precision (over TC and AR). DeepSphere achieves state-of-the-ard performance.
temp. (from past temp.)
day (from temperature)
day (from precipitations)
order P
MSE
MAE
R2
MSE
MAE
R2
MSE
MAE
R2
0
10.88
2.42
0.896
0.10
0.10
0.882
0.58
0.42
-0.980
4
8.20
2.11
0.919
0.05
0.05
0.969
0.50
0.18
0.597
Table 4: Prediction results data from weather stations. Structure always improves performance.
the-art performance, suggesting again that anisotropic filters unnecessary. Note that results from Mudigonda et al. (2017) cannot be directly compared as they don't It use the same input channels.
Compared to Cohen et al. (2019) conclusion, it surprising that S2R does worse than DeepSphere (which is limited to S2S). Potential explanations are (i) that their icosahedral projection introduces harmful distortions, or (ii) that a larger architecture can compensate for the lack of generality. We
indeed observed that more feature maps and depth led to higher performance (section C.3
4.4 UNEVEN SAMPLING
To demonstrate the flexibility of modeling the sampled sphere by graph, collected historical measurements from n z= 10, 000 weather stations scattered across the Earth The The spherical data is heavily non-uniformly sampled, with much higher density of weather stations than the Pacific (figure ldp. For illustration, we devised two artificial tasks. A dense regression: predict the temperature on given day knowing the temperature on the previous 5 days. A global regression: predict the day (represented as one period of sine over the year) from temperature or precipitations. Predicting from temperature is much easier has a clear yearly pattern.
North America
The graph is built with (5) k neighbors, and kernel width set to the average of the distances. The equivariance property of the resulting graph has not been tested, and we don't 't expect it to be good due to the heavily non-uniform sampling. The NN is made of 3 graph convolutional layers. The polynomials are all of order P respectively. For the global regression, GAP and fully connected layer follow. For the dense regression, a graph convolutional layer follows instead. The MSE loss is optimized with R.MSprop for 250 epochs. The learning rate is 1 10-3 and the batch size is 64.
and the number of channels per layer is 50, 100, 100,
like modeling each time series independently with MLP, orders P > O integrate neighborhood information. Results show that using the structure induced by the spherical geometry always yields better performance.
Results are shown table 4 While using a polynomial order
5 CONCLUSION
This work showed that DeepSphere strikes interesting, and we think currently optimal, balance between desiderata for spherical CNN. A single parameter, the number of neighbors & a pixel is connected to in the graph, controls the tradeoff between cost and equivariance (which is linked to performance). As computational cost and memory consumption scales linearly with the number of pixels, DeepSphere scales to spherical maps made of millions of pixels, required resolution
noaa won.a.gov/ghccooo
8

---

Published as a conference paper at ICLR 2020
to faithfully represent cosmological and climate data. Also relevant in scientific applications is the flexibility offered by a graph representation (for partial coverage, missing data, and mon-unifome samplings). Finally, the implementation of the graph convolution is straightforward, and the ubiquity of graph neural networks - pushing for their first-class support in DL frameworks - will make implementations
easier and more efficient.
potential drawback of graph Laplacian-bar based approaches is the isotropy of graph filters, reducing in principle the expressive power of the NN. Experiments from Cohen al. (2019) and [oscainii et al. (2016) indeed suggest that more general convolutions achieve better performance. Our ex- periments on 3D shapes (section 4.1 and climate (section 4.3) however show that DeepSphere's isotropic filters do not hinder performance. Possible explanations for this discrepancy are that NNs somehow compensate for the lack of anisotropic filters, or that some tasks can be solved with isotropic filters. The distortions induced by the icosahedral projection in (Cohen et al., (019)) or the leakage of curvature information (Boscaini et al. (016) might also alter performance.
Developing graph convolutions on irregular samplings that respect the geometry of the sphere is an- other research direction of importance. Practitioners currently interpolate their measurements (com- ing from arbitrarily positioned weather stations, satellites or telescopes) to regular samplings. This practice either results in a waste of resolution or computational and storage resources. Our ultimate goal is for practitioners to be able to work directly their measurements, however distributed.
ACKNOWLEDGMENTS
We thank Pierre Vandergheynst for advices, and Taco Cohen for his inputs on the intriguing results of our comparison with Cohen et al. (2019), We thank the anonymous reviewers for their construc- tive feedback. The following software packages were used for computation and plotting: PyGSP (Defferrard et al.), healpy (Zonca et al., 2019), matplotlib (Hunter) 2007). SciPy (Virtanen et al. 2020), NumPy (Walt et al. 2011). TensorFlow (Abadi et al., 2015).
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing on heterogeneous systems, 2015. URL https: http:s://wwww.o. http://www.ttttet www.owffwoffl...oooooooooo org/ Software available from tensorflow.org.
M. Bartelmann. Gravitational lensing. Classical and Quantum Gravity, 2010.
John Baumgardner and Paul O Frederickson. Icosahedral discretization of the two-sphere. SIAM Journal on Numerical Analysis, 1985.
Mikhail Belkin and Partha Niyogi. Convergence of laplacian eigenmaps. In Advances in Neural Information Processing Systems, 2007.
Mikhail Belkin and Partha Niyogi. Towards a theoretical foundation for laplacian based manifold methods. Journal of Computer and System Sciences, 2008
Wouter Boomsma and Jes Frellsen. Spherical convolutions and their application in molecular mod- elling. In Advances in Neural Information Processing Systems, 2017.
Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. Learning shape cor- respondence with anisotropic convolutional neural networks. Advances in Neural Information Processing Systems, 2016.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv:1312.6203, 2013. URL htttss://aaatt org/ abs/ 131 2. 6203

---

Published as a conference paper at ICLR 2020
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rien 3d model repository. arXiv:1512.03012 2015.
Taco S Cohen, Mario Geiger, Jonas Koehler, and Max Welling. Spherical cnns. In International Conference on Learning Representations (ICLR), 2018. URL httph:t//sxxii...t// ort/
1801. 10130
Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant con-
volutional networks and the icosahedral cnn. In International Conference on Machine Learning (ICML). 2019. URL
abs/ wttp////....... 1902 04615,
Benjamin Coors, Alexandru Paul Condurache, and Andreas Geiger. Spherenet: Learning spherical representations for detection and classification in omnidirectional images. In European Confer- ence on Computer Vision, 2018.
Michaäl Defferrard, Lionel Martin, Rodrigo Pena, and Nathanaël Perraudin. Pygsp: processing python. httppp/://g...ab...oo//e//iii//p
Graph signal
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, 2016. httts:/t:
org/ abs 09375
Michaël Defferrard, Nathanaël Perraudin, Tomasz Kacprzak, and Raphael Sgier. Deepsphere: to- wards an equivariant graph-based spherical cnn. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019. URLhttpss : / http//xxivv.ooo http://wwwwwwoooo////////ooo 904 05146
J. R. Driscoll and D. M. Healy. Computing fourier transforms and convolutions on the 2-sphere. 1tt.::00//.amm1...1111.......
Adv. Appl. Math., 1994. URL
aama. 1994.
Carlos Esteves, Christine Allen-Blanchesn Blanchette, Ameesh Makadia, and Kostas Danilidis. Learning so(3) equivariant representations with spherical cnns. Proceedings of the European Conference on Computer Vision (ECCV), 2018. URL https: // //tpivivvvv////////////aa///awaa org/ abs 1t///11111111111111
Krzysztof M Gorski, Benjamin D Wandelt, Frode K Hansen, Eric Hivon, and Anthony J Banday. The healpix primer. arXiv preprint astro-ph/9905255, 1999.
Krzysztof Gorski, Eric Hivon, AJ Banday, Benjamin D Wandelt, Frode K Hansen, Mstvos Rei- necke, and Mathhi Bartelmann. Healpix: framework for high-resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 2005.
J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3): 90 90-95, 2007. doi: 10.1 109/MCSE.2007.55.
Chiyu "Max" Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, and Matthias Niess- ner. Spherical cnns on unstructured grids. In International Conference on Learning Representa- tions (ICLR), 2019.
attp//ass//11 02039
Renata Khasanova and Pascal Frossard. Graph-based classification of omnidirectional images. In Proceedings of the IEEE International Conference on Computer Vision, 2017. URL https:
//axxiv.org/////o http://wwrr/aab//777..33333 08301
Mayur Mudigonda, Sookyung Kim, Ankur Mahesh, Samira Kahou, Karthik Kashinath, Dean Williams, Vincen Michalski, Travis O'Brien, and Mr Prabhat. Segmenting and tracking extreme climate events using neural networks. In Deep Learning for Physical Sciences (DLPS) Workshop, io/
held with NIPS Conference, 2017. URL httpl/tta..//lppppssaaaeeeeeeeee 2017 20. 20.2df. pdf]
files/ wttps//iis___dd_
Nathanaël Perraudin, Michaël Defferrard, Tomasz Kacprzak, and Raphael Sgier. Deepsphere: Effi- cient spherical convolutional neural network with healpix sampling for cosmological applications. Astronomy and Computing, 2019. URL
12186
Planck Collaboration. Planck 2015 results. i. overview of products and scientific results. Astronomy & Astrophysics, 2016.
10

---

Published as a conference paper at ICLR 2020
Manolis Savva, Fisher Yu, Hao Su, Asako Kanezaki, Takahiko Furuya, Ryutarou Ohbuchi, Zhichao Zhou, Rui Yu, Song Bai, Xiang Bai, et al. Large-scale 3d shape retrieval from shapenet core55: Shrec' 17 track. In Eurographics Workshop 3D Object Retrieval, 2017.
Schmelzle, A. Lucchi, T. Kacprzak, A. Amara, R. Sgier, A. Réfrégier, and T. Hofmann. Cosmo- logical model discrimination with deep learning. arxiv:1707.05167, 2017.
Yu-Chuan Su and Kristen Grauman. Learning spherical convolution for fast features from 360 imagery. In Advances in Neural Information Processing Systems, 2017.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. Nel- son, Erie Jones, Robert Kern, Eric Larson, CJ Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake Vand erPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1. 0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 2020. doi: https://doi.org 1r. 1001///113222000999
Stéfan van der Walt, Chris Colbert, and Gael Varoquaux. The numpy array: a structure for efficient numerical computation. Compuling in Science & Engineering, 13(2):22-30, 2011.
Andrea Zonca, Leo Singer, Daniel Lenz, Martin Reinecke, Cyrille Rosset, Eric Hivon, and Krzysztof Gorski. healpy: equal area pixelization and spherical harmonics transforms for data on the sphere in python. Journal of Open Source Software, 1(35) March 2019. doi: 10.21 105/joss. 01 298. URL /tc=:s////olloorr//02221/1//////

---

Published as a conference paper at ICLR 2020
SUPPLEMENTARY MATERIAL
A PROOF OF THEOREM 3.1
Preliminaries. The proof of theorem 3.1 is inspired from the work of Belkin & Niyogi (008)). As a result, we start by restating some of their results. Given sampling V= {z; E MM compact and infinitely differentiable manifold M, smooth (E Coe (M)) function f := of a closed, M - R, and defined the vector f of samples of f as follows: Tyf = =f- E R f: = f(x;). The proof is constructed by leveraging 3 different operators:
The extended graph Laplacian operator, already presented in 6 is a linear operator L (2( + (2M defined as
(v) 12
(f(D) - f(x)).
Note that we have the following relation f = T
The functional approximation to the Laplace-Beltrame operator is a linear operator T,² (.² I.2 (MM defined
f'f())= - S/1 c Ia-v13 (F() f(x)) du(x),
where j is the uniform probability measure on the manifold M, and vol is the volume of M.
The Laplace-
Laplace-Beltrami Beltrami operator AM is defined as the divergence of the gradient
A = -iviivV
of a differentiable function S MM + IR. The gradient V M + TnMM is a vector field
defined on the manifold pointing towards the direction of steepest ascent of where T, M is the affine space of all vectors tangent to M at
Leveraging these three operators, Belkin & Niyogi (2008 2007) have build proofs of both pointwise and spectral convergence of the extended graph Laplacian towards the Laplace-Belle Beltrami operator in the general setting of any compact, closed and infinitely differentiable manifold M,, where the sampling V is drawn randomly on the manifold. For this reason, their results are all to be interpreted probabilistic sense. Their proofs consist in establishing that (G) converges in probability towards (8) as + co and ( converges towards (9) as t O. In particular, this second step is given by the
following:
Proposition 1 Belkin & Niyogi (2008), Proposition 4,4). Let M be k-dimensional compact smooth manifold embedded in some Euclidean space RN,, and fix y E M. Let f (MM. Then
1
1 L' fly) t-, vol(M) Amf(s). t (4T1)
(10)
Building the proof. As the sphere is compact smooth manifold embedded in TR3,, we can reuse proposition]
[ Thus, our strategy to prove Theorem 3.1 is to (i) show that
lim fgy L'(y)
(11)
R-000
for particular class of deterministic samplings, and (ii) apply Proposition !
We start by proving that for smooth functions, for any fixed t, the extended graph Laplacian 4 converges towards its continuous counterpart It* as the sampling increases in size.
Proposition For an equal area sampling {ar E 82) A, A,Vi,j j of the sphere it is true that for all f 82 R. Lipschitz with respect the Euclidean distance |1-Il with Lipschitz constant C₁
| f(x)du(>)- - n
f(x)) s &C
12

---

Published as a conference paper at ICLR 2020
Furthermore, for all S the Heat Kernel Graph Laplacian operator L,, converges pointwise to the functional approximation of the Laplace Beltrami operator L'
[ 110000 L' L'f(y).
Proof. Assuming / - RR is Lipschitz with Lipschitz constant Cj, we have
| [())d)())
- n ((x)) S Cyd(n)
n
where C S the subset of the sphere corresponding to the patch around æy. Remember that the sampling is equal area. Hence, using the triangular inequality and summing all the contributions of the n patches, we obtain
Ue - f(x)
Σ
/
- f(x)dp(x)- ((.) & nn = Cyd(n)
4T
A direct application of this result leads to the following pointwise convergences
xl1
/ 1=- du(x)
ff Lipschitz,
Vy eye: 82,
1i f(x;) + +f fe 1ا
Vf Lipschitz,
Vy Es,
f(x)dp(x)
Definitions 6 and
g
end the proof.
The last proposition show that for fixed I, Lzf(x) + f(x). To utilize Proposition I and complete the proof, we need to find a sequence of tn for which this holds as tn - O. Furthermore this should hold with a faster decay than Amaz.
Proposition 3. Given a sampling regular enough, iee, for which we assume A 10,1. Lipschitz function and point y S there exists
A, Vi,j and d(m) I noy sequence t, n², 3 such that
C
1 4T
ff Lipschitz, Vx E ?
((x) L s(x))
1000 0.
Proof. To ease the notation, we define
x11
Kt K' v)
(12)
o' (x;,))
(f(2) - f(x))
(13)
We start with the following inequality
[ - It' f1l max
Tn)(v) Lt I'v(
yES2
n o' (xxz)) -f d' v)d(())
max vE8
(xx -f d'(x;))du(x))
max yES2
< (m) max Cag.
(14)
yESa
where Cas is the Lipschitz constant of x + Φx y) and the last inequality follows from Proposition
2 Using the assumption d(n) <
find
vn
| - L' L'
max
yE8a
13

---

Published as a conference paper at ICLR 2020
We now find the explicit dependence between and Cos
Cay ;
= 10 10:00(::)ll
(:1))l
10 (K)())) llse
fl0
Cx311//1l=+|00///1b
= Ck₃ -xx11f11<+++
where OK₃ is the Lipschitz constant of the function x + (); We note that this constant does not depend on y/:
= (2et)- xc 3
47
00
2t
00
2t
2=
Hence we have
max Cot vn
C
(2et) 11f1lo +Cg)
WE82
vn
Cllflla
C
+ na C₁.
no (2et))11
Inculding this result in (14 and rescaling by 1/4Tt?, we obtain
1
(L/ L'f)
(1 - l'
4
4v72
C'
na/5/2 + nay2 1
1
Cf
4m
Vee
[n + 00
In order for
4x
C [lfll
n5
+ not²
n-+00
0, we need
(nay² -
((nn)= n²
BE (- 20 0)
happens if Indeed, we have
t(n) = n² BE (-a- 20 .).
(t(n) = n B (- 2' 0)
a
5l1phat5/2 5/2 a a oo since 3/3 +
B 20
and n = ta -000 oo since 23 + a > + 83-2
(tm) n+00,
As result, for t 8 with 3 E (- have
Liv
L'v
fn
1stz
1xt2
which concludes the proof.
Theorem 3.1 is then an immediate consequence of Proposition 3 and
Proof of Theorem 3.1 Thanks to Proposition 3 and Proposition ! we conclude that Vy S
lim
L' f(y)
lim L f(v)
Aga f(v)
4T13
n-000 4-13.
32
In (Bllin & Niyogi 2008), the sampling is drawn from a uniform random distribution on the sphere, and their proof heavily relies on the uniformity properties of the distribution from which the sampling is drawn. In our case the sampling is deterministic, and this is indeed problem that we need overcome by imposing the regularity conditions above.
14

---

Published as a conference paper at ICLR 2020
micro (label average)
macro (instance average)
P@N
R@N
FI@N
MAP
P@N
R@N
FI@N
mAP
Cohen et ml
2018 1b
128)
0.701
0.711
0.699
0.676
2015
(simplified.
0.701
0.665
0.385
Ealeves di
2018
(b 64)
0.717
0.737
0.685
0.450
0.550
0.444
DeeeSpphere esquangular
64)
0.665
DeepSphere (HEALPsx Naide
0.725
0.717
0.715
0.686
0.475
0.508
0.468
0.428
Table 5: Official metries from the SHREC' 17 object retrieval competition.
To conclude, we see that the result obtained is of similar form than the result obtained in (Belkin &
Niyogi 2008). Given the kernel density t(n)) = n Belkin & Niyogi (2008) proved convergence
in the random case for 3 E (-4 4- 0) and we proved convergence in the deterministic case for B E (0, 1/2] (for the spherical manifold).
20: 5 0), where
B PROOF OF THEOREM 3.2
Proof. Fix a E S Since any rotation R(g) is an isometry, and the Laplacian A commutes with all isometries of Riemanniann manifold, and defining R(g)f f' for ease of notation, write that
R(g) 1/ f(x) - i. R(g)(())
< R(g) [(x) R(g) (1x) + ((1) - 1 RG)
f + (x) -
$ ( - fo (x)) + f 1: fx))
Since (x) E S2 and f' still satisfies hypothesis, we can apply theorem 3.1 to say that
(i:: l 1) (x))
A f() - 1. f(x)
n=000
to conclude that
Vr E S
f( fs R(g)f(z))) 22020 0
C EXPERIMENTAL DETAILS
C. 3D OBJECTS RECOGNITION
Table 5 shows the results obtained from the SHREC'17 competition's official evaluation script.
[GC16 + BN + ReLU'nnesdes + Pool [GOs2 BN ReLU ReLUUnnsddeL InsideGG Pool
(15)
+ |GCs1 + BN + ReLU nsides + Pool + [GC12s + BN + ReLU nsided
( ReLUUnssdee Inside2 Pool GAP + FCN + softmax
+ Pool + |GC256 + BN + ReLU
C.2 COSMOLOGICAL MODEL CLASSIFICATION
[GC16 + BN + ReLU mssde1024 + Pool + [GCs2 + BN + ReLU mside512
Pool |GCos + BN Inidd220 Pool
(16)
ReLUnssidea2)) + [GC64 + BN + ReLU mside128 Pool + [GCs4 + BN + ReLU moide64
Pool (GC2)asidess) GAP + softmax
15

---

Published as a conference paper at ICLR 2020
TC
AR
BG
mean
Mudigonda et al.
(2017)
74
65
78.67
Jiang
(2019)
(paper)
94
93
97
94.67
Jiang et al.
(0119)
(rerun)
93.9
95.7
95.2
94.95
Cohen al.
(2019
(S2R)
97.8
97.3
97.3
97.5
Cohen et al.
2019
(R2R)
97.9
97.8
97.4
97.7
DS (Jiang architecture, weighted loss)
97.1
97.6
96.5
97.1
(weighted loss)
97.4 1.1
97.7 + 0.7
98.2 + 0.5
97.8 0.3
DS (wider architecture, weighted loss)
91.5
93.4
99.0
94.6
DS (Jiang architecture, non-weighted loss)
33.6
93.6
99.3
75.5
DS (non-weighted loss)
69.2 + 3.7
94.5 + 2.9
99.7 + 0.1
87.8 + 0.5
DS (wider architecture, non-weighted loss)
73.4
92.7
99.8
88.7
Table 6: Results on climate event segmentation: accuracy. Tropical cyclones (TC) and atmospheric rivers (AR) are the two positive classes, against the background (BG). MMddgonda al. (2017) is not directly comparable as they don't use the same input feature maps. Note that a non-weighted cross-entropy loss not optimal for the accuracy metric.
TC
AR
mean
Jiang et al.
(2019)
(rerun)
11.08
65.21
38.41
Cohen et al.
(2019
(S2R)
68.6
Cohen et al.
2019
(R2R)
75.9
DS (Jiang architecture, non-weighted loss)
46.2
93.9
70.0
DS (non-weighted loss)
80.86 + 2.42
97.45 + 0.38
89.16 + 1.37
DS (wider architecture, non-weighted loss)
84.71
98.05
91.38
DS (Jiang architecture, weighted loss)
49.7
89.2
69.5
DS (weighted loss)
58.88 + 3.17
95.41
+ 1.51
77.15 + 1.94
DS (wider architecture, weighted loss)
52.80
94.78
73.79
Table 7: Results climate event segmentation: average precision. Tropical cyclones (TC) and atmospheric rivers (AR) are the two positive classes. Note that a weighted cross-entropy loss is not optimal for the average precision metric.
C.3 CLIMATE EVENT SEGMENTATION
Table and show the accuracy, mAP, and efficiency of the NNs we ran.
The experiment with the model from Jiang et al. (2019) was rerun in order to obtain the AP metrics, but with batch size of 64 instead 256 due to GPU memory limit.
Several experiments were with different architectures for DeepSphere (DS). Jiang architecture use a similar one as Jiang et al. (2019), with only the convolutional operators replaced. DeepSphere only is the original architecture giving the best results, deeper and with four times more feature maps than Jiang architecture. And the wider architecture is the same as the previous one with two times the number of feature maps.
Regarding the weighted loss, the weights are chosen with scikit -lean function compute al class wed whight the training set.
16

---

Published as a conference paper at ICLR 2020
size
speed
params
inference
training
Jiang et al. (2019)
330
10 ms
10
DeepSphere (Jiang architecture)
590k
5 ms
3h
DeepSphere
13
33 ms
13h
DpepSphere (wider architecture)
52M
50 ms
20
Table 8: Results on climate event segmentation: size and speed.
DeepSphere with Jiang architecture encoder:
[GC& + BN + ReLU + Pool + |GO + BN + ReLUULA + Pool
(17)
+ [GO'2 + BN + ReLUIra + Pool + [GCs4 + BN + ReLU + Pool
+ [GC128 + BN + LL + Pool + [GC128 + BN + ReLU
decoder:
Unpool + [GC128 + BN + ReLU L1 + concat + [GC12s + BN + ReLU
+ Unpool |GCs4 + BN ReLU L2 concat
(18)
+ [GC&4 BN + ReLU + Unpool [GC32 + BN ReLUlta
+ concat |GOs2 + BN ReLU + Unpool
+ [GC16 BN + ReLU L4 + concat + [GC16 + BN + ReLU L4 + Unpool
+ [GCs + BN + ReLU + concat + [GC' + BN + ReLU - [GCa]
Concat is the operation that concatenate the results of the corresponding encoder layer.
Original DeepSphere architecture with encoder decoder encoder:
[G + BN + ReLU + [GOs1 + BN + ReLU
Pool + [GC128 BN + ReLU" + Pool
(19)
|GC258 + BN + ReLU La + Pool + [GCs12 + BN + ReLU
Pool + [GOs12 BN + ReLU Pool [GCs12l40
decoder:
Unpool + [GCs12 + BN + ReLU L1 + concat [GCs12 + BN + ReLU
+ Unpool + |GC236 + BN + ReLU L7 + concat
(20)
+ [GC256 + BN + ReLU r.2 + Unpool + [GC128 + BN + ReLU
+ concat + |GC12s + BN + ReLU + Unpool
+ [GCs1 + BN + ReLU + concat [GCss + BN ReLU
Unpool + (GCs2 BN + ReLU [GC3)t8
17

---

Published as a conference paper at ICLR 2020
C.4 UNEVEN SAMPLING
Architecture for dense regression:
[GC50 + BN + ReLUU [GC100 BN + ReL.U + [GC100 + BN + ReLUJ + [GC|] (21)
Architecture for global regression:
[GCsn BN + ReLUJ [GC100 BN ReLU
(22)
+ |GC + BN + RELUJ + GAP + FCN
18


Reviews:
[{"review_id": "r1xdde8nKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, CNNs specialized for spherical data are studied. The proposed architecture is a combination of existing frameworks based on the discretization of a sphere as a graph. As a main result, the paper shows a convergence result, which is related to the rotation equivalence on a sphere. The experiments show the proposed model achieves a good tradeoff between the prediction performance and the computational cost. \n\nAlthough the theoretical result is not strong enough, the empirical results show the proposed approach is promising. Therefore I vote for acceptance. \n\nThe paper is overall clearly written. It is nice that the authors try to mitigate from overclaiming of the analysis. \n\nAs a non-expert of spherical CNN, I don't understand clearly the gap between the result Theorem 3.1 and showing the rotation equivalence. It would be nice to add some counterexample (i.e., in what situation the proposed approach does not have rotational equivalence while Theorem 3.1 holds).", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "Bkx5kg6RKH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe paper presents DeepSphere, a method for learning over spherical data via a graphical representation and graph-convolutions. The primary goal is to develop a method that encodes equivariance to rotations, cheaply. The graph is formed by sampling the surface of the sphere and connecting neighbors according to a distance-based similarity measure. The equivariance of the representation is demonstrated empirically and theoretical background on its convergence properties are shown. DeepSphere is then demonstrated on several problems as well as shown how it applies to non-uniform data.\n\nThe paper is interesting and clear. The projection of structured data to graphical representations is both efficient in utilizing existing algorithmic techniques for graph convolutions and useful for approaching the spherical structure of the data. The theoretical analysis and discussion of sampling is interesting, though should be more clearly stated throughout and potentially visualized in figures.\n\nThe experiments performed are thorough and interesting. The approach both outperforms baselines in inference time and accuracy. However, one wonders the performance on the well-researched tasks such as the performance on 3D imagery, e.g., Su & Grauman, 2017; Coors et al., 2018. \n\nThe unevenly sampled data is a nice extension showing the generality of the approach. How does the approach work for data connected within a radius rather than a k-nearest approach?\n\nMinor:\n- A figure detailing the parameters and setup for theorem 3.1 and figure 2 would be useful.\n- The statement on the dispersion of the sampling sequence states “the smallest ball in \\R^3 containing \\sigma_i”, but I believe it should be “containing only \\sigma_i”.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BkgixEgAKr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies the problem of designing a convolution for a spherical neural network. The authors use the existing graph CNN formulation and a pooling strategy that exploits hierarchical pixelations of the sphere to learn from the discretized sphere. The main idea is to model the discretized sphere as a graph of connected pixels: the length of the shortest path between two pixels is an approximation of the geodesic distance between them. To show the computational efficiency, sampling flexibility and rotation equivariance, extensive experiments are conducted, including 3D object recognition, cosmological mode classification, climate event segmentation and uneven sampling.\nPros: \n1. The application and combination of different techniques in this paper are smart.\n2. The experiments show that the proposed method outperforms other baseline methods.\n3. The paper is well organized and written. \nCons:\n1. It is a good application of known techniques, but the novelty is limited.\n2. It is suggested to add more baselines in the experiments.\n\n[1] Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information ProcessingSystems, 2016", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel methodology for applying convolutional networks to spherical data through a graph-based discretization.   The reviewers all found the methodology sensible and the experiments convincing.  A common concern of the reviewers was the amount of novelty in the approach, as in it involves the combination of established methods, but ultimately they found that the empirical performance compared to baselines outweighed this.
Decision: Accept (Spotlight)