ChebLieNet: Invariant spectral graph NNs turned equivariant by Riemannian geometry on Lie groups
Authors: Hugo Aguettaz
Erik J Bekkers
Michaël Defferrard
Venue: NeurIPS 2021 Submitted (2021)
Keywords: Lie Group
Riemannian Geometry
Spectral Graph NN
Machine Learning

Abstract: We introduce ChebLieNet, a group-equivariant method on (anisotropic) manifolds. Surfing on the success of graph- and group-based neural networks, we take advantage of the recent developments in the geometric deep learning field to derive a new approach to exploit any anisotropies in data. Via discrete approximations of Lie groups, we develop a graph neural network made of anisotropic convolutional layers (Chebyshev convolutions), spatial pooling and unpooling layers, and global pooling layers. Group equivariance is achieved via equivariant and invariant operators on graphs with anisotropic left-invariant Riemannian distance-based affinities encoded on the edges. Thanks to its simple form, the Riemannian metric can model any anisotropies, both in the spatial and orientation domains. This control on anisotropies of the Riemannian metrics allows to balance equivariance (anisotropic metric) against invariance (isotropic metric) of the graph convolution layers. Hence we open the doors to a better understanding of anisotropic properties. We empirically prove the existence of (data-dependent) sweet spots for anisotropic parameters on CIFAR10. This crucial result is evidence of the necessity of using tunable anisotropic kernels. We also evaluate the scalability of this approach on STL10 (image data) and ClimateNet (spherical data), showing its remarkable adaptability to diverse tasks and the benefit of defining anisotropic space.

ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups
Anonymous Author(s)
Affiliation
Address
email
Abstract
We introduce ChebLieNet, group-equivariant method (anisotropic) manifolds. Surfing on the success of graph- and group- based neural networks, we take advan- tage of the recent developments in the geometric deep learning field to derive a new approach to exploit any anisotropies data. Via discrete approximations Lie groups, develop graph neural network made of anisotropic convolutional layers (Chebyshev convolutions), spatial pooling and unpooling layers, and global pooling layers. Group equivariance is achieved via equivariant and invariant opera- tors on graphs with anisotropic left-invariant Riemannian iistance-based affinities encoded on the edges. Thanks to its simple form, the Riemannian metric can model any anisotropies, both in the spatial and orientation domains. This control on anisotropies the Riemannian metries allows balance equivariance (anisotropic metric) against invariance (isotropic metric) of the graph convolution layers. Hence we open the doors to better understanding of anisotropic properties. Furthermore, we empirically prove the existence of ddata-dependent) sweet spots for anisotropic parameters on CIFAR1O. This crucial result is evidence of the benefice we could get by exploiting anisotropic properties in data. We also evaluate the scalability of this approach on STL.10 (image data) and ClimateNet (spherical data), showing its remarkable adaptability to diverse tasks.
2
4
6
8
10
11
12
13
14
15
17
Introduction
20
Deep learning is class of machine learning algorithms inspired by the human brain's network of neurons [Goodfellow et al., (016). These algorithms use a hierarchical structure of neural layers to extract higher-level features from the raw input progressively. In the past few years, the growing computational power modern GPU-based computers and the availability large training datasets in the field of machine learning have made it possible to successfully train neural networks with many layers and degrees of freedom. Consequently, deep learning has revolutionized many machine learning tasks in recent years, ranging from image and video processing to speech recognition and natural language understanding.
21
23
24
25
27
28 Many neuroscientific research results served as focal points in the development of deep learning algorithms. When Hubel and Wiesel| [1962] studied the visual cortex in the brain, they made three important discoveries. First, they observed one-to-one correspondence between spatial locations the retina and neurons in the brain that fired as response to line-like visual stimuli. Second, the activity of the neurons changed depending on the orientation of the line, uncovering a neat
29
30
31
33 organization based on local orientations. Last, the neurons sometimes fired only when the line was moving in particular direction. Later, [Bosking al. [1997 showed that neurons that aligned fire together, indicating the presence of a type of long-range interactions. All these results motivated the development of mathematical framework for modeling visual perception based on aub-Riemannian
35
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

37
geometry on the space of positions and orientations, which is typically modeled with the Lie group
SE(2) [Petitot 2003 Citti and Sarti, 2006, Duits et al., 2014]. Apart from the neurophysiological
38
39
inspiration, group equivariance has also been proven to be an excellent inductive bias (Cohen and
40
Welling 2016] not only in computer vision (as the translation equivariance property of CNNs as
41
shown) but also in physics (Finzi et al. 2020 and molecular data analysis (Fuchs et al. [202]] Jumper ct al., 2020]. In this work, we propose to build group equivariant graph neural networks via the same principle that underlie the nub-Riemannian, neurogeometrical modeling of the visual cortex.
42
43
44
Our work connects the observations by Hubel and Wiesel 1962 and Bosking et al. [1997] on two levels. First, the organization of visual data based on their location and orientation [Hubel and Wiesel [962] is modeled by Lie group convolutions [Bekkers] 2019]. in which feature maps encode response for every position and every orientation. Second, long-range interactions between aligned neurons Bosking et al. 1997] are modeled by building graphs with affinity matrices based on (approximate) sub-Riemannian distances on the Lie groups, inspired by sub-Riemannia image analysis methods
45
46
47
48
49
50
such as [Franken and Duits 2009 Bekkers et al., 2015 Favali et al. 2016 Mashtakov et al., 2017
51
Boscain et al. 2018
Duits al., 2018 Baspinar et al. 2021].
52
2020 showed how to construct powerful graph NNs that are faithful the manifolds on which they are defined. Nevertheless, the layers themselves are based on rotationally invariant (Laplacian) convolutions. In order to exploit directional cues in the data, group convolutions are
Defferrard
58
54
55
desirable [Cohen et al., [018 Kondor and Trivedi 2018 [ohen and Welling, 2016 Bekkers [2019]. However, since Laplacian operators are intrinsically isotropic, there is no point applying them the lifted feature maps on the group unless we construct anisotropic metrics on the groups. Therefore,
56
57
58 we adopt the Lie group viewpoint by [Sanguinetti et al.] [2015) to define anisotropic Riemannian metries based on left-invariant vector fields on the group. Once an anisotropic Riemannian graph is constructed, any spectral method can directly be applied to this graph. The resulting graph neural networks will then, by construction, be equívariant and capable of utilizing directional cues in data.
59
60
61
62
Before going further into the details, we summarize our main contributions:
63
We introduce ChebLieNet, an equivariant graph Laplacian-based neural network based on Lie groups equipped with an anisotropic Riemannian metric.
64
The Riemannian geometry is automatically derived from a standard base space (e.g. R²2 or the sphere), which makes our approach flexible and effective in building group equivariant graph neural networks for a variety of data structures (e.g. 2D and spherical data).
65
66
67
68
We demonstrate the equivariance property of ChebLieNet, both in theory and in practice. This property guarantees that the neural network's 's predictions are robust against given transformations, which is not necessarily the case with methods based on data augmentation.
69
70
71
We show that the use of directional information via anisotropic Riemannian spaces could benefit many tasks.
72
73
We show the flexibility of the method by considering two different problems; we validate on classification problems with 2D image data and segmentation problem on spherical data via the construction of a sub-Riemannial geometry on SE()) and SO(3) respectively.
74
75
2
Related works
78
77
2.1
Group equivariant convolutional neural networks
78
Deep convolutional neural networks [LeCun et al. 1995] have proven to be compelling models for pattern recognition tasks on images, vidco, and audio data. Although a robust theory of neural network design is currently lacking, a large amount of empirical evidence supports the notion that both convolutional weight sharing, depth, and width are essential for good predictive performance. Such properties are enabled through the equivariance property of convolutions (convolving a shifted image is the same translating its result).
80
81
82
89
84 Lene and Vedaldi (2055) showed that the AlexNet CNN Krizhevsky et al. 2012 trained on ImageNet learns representations equivariant to flips, scalings, and rotations spontaneously. This supports the
85
86 idea that equivariance is an excellent inductive bias for deep convolutional networks. In the last few years, joint effort has been made to build group equivariant networks. By the introduction of group

---

88
convolutions in deep learning, Cohen and Welling 2016] generalize the translation equivariance property to larger groups of symmetries, including rotations and reflections. [Kndor and Trivedi [20T8) gave a rigorous, theoretical treatment of convolution and equivariance in neural networks concerning any compact group's 8 action. One of the main contributions of that work was to show that, given some natural constraints, the convolutional structure is not just sufficient but also necessary condition for equivariance to a compact group's action. In a similar spirit, in [Bekkers] 2019] is shown that any bounded linear operator is equivariant to Lie groups if and only if it is a group convolution. In our work, we propose to build group equivariant neural networks via left-invariant Laplace operators on Lie groups, which indeed can be seen as group convolutions with kernels that are the fundamental solutions of the Laplace operator. The Chebyshev-yye seppp neural network (Defferrard et al. 2016 that we will refer to ChebLieNet.
89
90
91
92
93
94
95
97
is a Lie group equivariant
99
2.2
Graph neural networks
100
Using the term geometric deep learning, Bronstein et al. [2117] 2021] give an overview of deep learning methods in the -unEuulidean domain, including graphs and manifolds. They present differ- ent examples of geometric deep learning problems and available solutions, fundamental difficulties, applications, and future research directions in this nascent field.
102
103
104
One of the main challenges when working with graph data it to deal with the inter-dependencies between points. Indeed, the derivations of most standard machine learning models firmly base on an independence assumption. For this reason, transferring existing methods on a graph appears doomed to failure, and it seems necessary to build models acting directly on graphs Due to its success on Euclidean data, the development of a convolution- operator on graphs has been largely studied. Because the notion of space is not naturally defined
105
106
108
graph, we lack straightforward generalization of the convolutional operator from grid data to graphs (Scarselli et al. 2008 Bruna al. 2013 Henaff al.) 2015 Defferrard et al.) 2016 Kipf and Welling 2016 Masci et al. 2013
110
112
Boscaii et al.] |016)] Monti et al.] 2017].
Spectral approaches have a solid mathematical foundation in graph signal processing. Rather than using the traditional spatial definition of the convolution, it proposes to see this operation from a spectral perspective. Based on the convolution theorem, it defines the convolution operator from the graph spectral domain via the eigendecomposition of the graph Laplacian (see App.A.3) A.3)
114
115
116
Definition 2.1 (Spectral graph convolution) Lei = (V,8, W) be graph with Laplacian å and let I and y be two functions defined on V. We define the G-convolution #g of J and g as:
117
118
f 9 2 260 2 3 fT
with eigenvectors T obtained through the unique eigendecompositionn À TAΦT
119
120
While this definition alleviates the difficulty of deriving a convolution operator in the spatial domain, other difficulties arise. First of all, because the Laplacian of a graph is an intrinsic operator, it is domain-dependent, and the framework cannot be easily transferred from a graph another as expressed in different "language". Nevertheless, this is not a problem for us since we are focusing on fixed manifold graphs. Next, there no guarantee that filters represented in the spectral domain are spatially localized. [Henaff al. 2015 successfully bypassed this problem by defining smooth spectral filter coefficients, arguing that if spectral filters are smooth, they are spatially localized. Last but not least, the Laplacian's eigendecomposition makes the method expensive in terms of memory and time. Indeed, the forward and inverse graph Fourier transforms (via & and ④) incur expensive multiplications as no FFT-like algorithm exists on general graphs. Defferrard al. (2016) alleviated the cost of explicitly computing the graph Laplacian using spatially- localize filters with Chebyshev polynomials.
121
122
is too. It implies that a model built on this
123
124
125
126
127
128
129
130
131
132
Definition 2.2 (Chebyshev convolutional layer) Let G = EV,E, W) be a graph with rescaled
Laplaciar 0 A x E V xd, be an input features' vector and θ E Rdixdd learnable filters. The
133
Because Chebyshev polynomials are defined in the range |-1, 1), it is necessary rescalc the graph Laplacian with À 21_1x Á where Amax is the largest eigenvalue of Á.
3

---

output features' vector E xlv is computed as:
134
R-1 with 20 ===
y =
21
a
and 2, = -== - 22-2 vj (2)
1=0
135
Kipf and Welling| 2016) simplified this formulation bit by considering the construction of single- parametric filters that are linear with relation to A. They further approximate Xmax = 2 they
136
137
expect that neural network parameters will adapt to this change in scale during training.
3
Method
138
139
Our method can be seen as extension of the original ChebNet (Defferrard et al. 2016 Perraudin
140
et al. 2019]. Instead of directly working on a homogeneous base space, we first extend it to a higher
141
dimensional space (Lie group). The goal of this extension is to convert the previously invariant
spectral convolutional layers into equivariant layers 2
142
143
3.1
Anisotropic manifold graph
In order to define the anisotropic manifold graphs we have to consider two types of manifolds. The base manifold M and a Lie group G that acts transitively on M. The latter implies that MM is a 145
homogeneous space of G, which means that any two points m,, m2 .M can be mapped to each other via the action of a group element g E G via m2 = 9 mi- E.g., the plane M = R2 is a homogeneous space of the special Euclidean motion group G = S (2) any two points can be mapped to each other through a rotation and a translation. Such groups G, which have M as a homogeneous space, can always be split in two parts via the semi-direct product G = M x H, with H a sub-group of G' that leaves some reference point my E M invariant, i.e., VAEH: mg = h mo- g.g. rotations leave the zero vector in M R² invariant, and thus H SO()) in the S2) case. Conversely, any homogenous space can be modeled with a group quotient M = G/H.
147
149
150
151
152
153
We define an anisotropic manifold graph to be a discretization of Lie group G of which M homogeneous space. It consists of a finite set of vertices corresponding to a random sampling of group elements, and a finite set of similarity- -based edges that are constructed via left-invariant Riemannian metric on G. In our work we consider two anisotropic manifold graphs: one associated with the base manifold M IR2 which extend with additional orientation/rotation dimension
155
157
158
H = S0()) to come to the Lie group G = S (2) = R2 x S0(2), and the other associated with the sphere M 5 which we similarly "lift" to the Lie group G = SO(3) by adding an additional rotation dimension. Considering the similarity between the two cases (the sphere locally looks like
159
160
161
162
R22) we will refer M the "spatial" part, and H/ the "orientation" part of the group.
163
Uniform sampling of the vertices. The first step to construct an anisotropic manifold graph is to sample elements the group uniformly as uniformly as possible if the manifold does not permit uniform grid. We split the grid construction in two parts, a grid on A which is sampled with [..] points and grid H that is sampled with |V] points, leading a total of |V = [V.[V. vertices.
164
185
166
187
Left-in invarinnt anisotropic Riemannian distance. Once vertices have been uniformly sampled on the group manifold, a similarity measure between vertices is computed. This measure is based on a Riemannian distance between points in G. The only thing one needs in our algorithm is the implementation of the logarithmic map the Lie group (see e.g. [Bekkers] 2019]). and diagonal Riemannian metric tensor (see e.g. (Sanguinetti et al.) 2013) and |Mashtakov et al. 2017) for the SE()) and SO(3) case respectively). In the following we provide the essential idea and intuition
169
170
171
172
173
behind the construction of the similarity measure and provide a more extensive treatment in App. B
In Riemannian geometry on Lie groups it is common to express tangent vectors of curves in basis of left-invariant vector fields as it allows to measure their lengths with a single Riemannian metric
175
tensor that is shared over the entire group. This works as follows. Consider curve 7 [0, 1] - G with its tangent vectors ((t) Σ u' (t))) expressed a basis/moving frame of reference
177
2 Because spectral graph NNs are able to capture the geometry of the space, which in this work we equip with anisotropic metrics, any spectral method could be made equivariant using our method.

---

(All Hi in which A, are left-invariant vector fields. The length of these tangent vectors is then measured by Riemannian metric tensor that we denote with (f)(t)
178
a(())"Ru(( with R a symmetric positive definite matrix defined relative to the basis (Ailot)) and with
180
m(t) (t)( un (t),,...)) .". The A, are left-invariant vector fields and the notation Ailg means the vector in the vector field A; at location g. The vector fields are constructed by choosing a vector A; in the tangent space at origin (the Lie algebra) which then defines complete vector field on G' via the push-forward of lefl-multipicaation In less technical terms this means that if we pick direction vector at the origin, and we move it to another point in, e.g. G = SE(2), via roto-translation, this vector will move and rotate along. By defining everything in terms of these left-invariant vector fields, every tangent space (G) at each g E G can be identified with the tangent space at the origin, and a single Riemannian metric tensor R can be shared over the entire space. Moreover, the induced Riemannian distance d(g. h) between any two points g. h E G then by construction left-invariant,
181
182
183
184
185
186
187
188
189
190
i.e., Vg,A. d(g h,g g i) = d(g, i).
Expressing tangent vectors in such left- ieft-invariant vector fields allows to reason in terms of the generators of the group. Consider the G = (2) case. As a basis we pick the 3 generators of the group: a forward motion represented by a vector A pointing in the forward direction within the plane, a side-ways motion represented by a perpendicular planar vector A2, and rotation/change of orientation represented by a vector Aa that points vertically in along the Hi-dimension. We then work with diagonal Riemannian metric tensors R diag(1, 2 which penalize each type of motion (represented by the vector components) differently. When + 0 one arrives the Rib-Riemannian geometry which forms the basis for the mathematical modeling of visual perception. It quantifies a notion of alignment through the aub-Riemnnnian distance; the length of a distance- minimiiiin geodesic that connects two local orientations that lie in the extend of each other will be much smaller
192
193
194
195
196
197
198
199
200
201
that that of geodesic connecting two local orientations parallel each other. An analogy can be found with the example of car in parking lot where it can move forward/backwant backward (A,) ) and change orientation (Aa) [Reeds and Shepp. 1990]. will be easier to move it to the more aligned spot directly ahead then it will to the spot next to the car, as sideways motion (A2) is impossible.
202
203
205
Parameters and & will respectively be referred to as spatial and orientation anisotropy parameters. With 1 the metric is isotropic and there will be no distinction between different orientations. When e < 1, & determines the flexibilty/curvature of the geodesics as it balances spatial motion against angular motion. In a sense it defines how easily one connects local orientations that are not optimally aligned. In Figure this behavior is visualized by running a diffusion process on the anisotropic manifold graph. In the anisotropic case (c < 1) diffusion is faster along the forward direction within a 8-plane. From a graph NN perspective this suggests that information propagated
208
207
208
209
210
211
more quickly between vertices that are aligned, nevertheless, Chow's theorem (see e.g. (Montgomery 2006)) guarantees that any point pair in the (sub-) Riemaniian manfiold can interact with one another. 213
214
The exact computation of the (sub-) Ruub))RRemnnnan distances is challenging and can generally not be done in closed form, but can be done numerically via method such as [Bekkers et al., 2015 Sanguinetti] et al. 2015) Mashtakov et al., 2017). In order to keep our graph construction algorithm efficient though, we will approximate the Riemannian distances via an efficient analytic formula based on
215
216
217
218
those in [Bekkers et al., 2018)) that only involves the Lie group's 'ss logarithmic map log G - (G)
219
and the Riemannian metric tensor R. We then approximate the distance between points g. h E G by
d(g, h) = d(e,g 1 hh) = || log[g h)]lR
(3)
220
Similarity measure. Encoding a similarity measure in the edges of graph requires defining weighting scheme. It is common to use a Gaussian kernel and set the weights via
221
w(Vt, 0y)) = exp (- dl (v.,0s)
if e( 0z) E otherwise
4t.
(4)
0
222
The choice for kernel bandwidth t is essentially arbitrary, but good heuristics exist. Perraudin et al. [2019) set it to half the average squared distance between connected vertices. [Defferrard et al. [0220] however, showed that this heuristic has the tendency to overestimate it and preterred to choose it as
223
224
225
the minimizer of the mean equivariance error. Following this overestimation observation, we fix the kernel bandwidth as 20% of the average squared Riemannian distance between connected vertices.
As such, the weights diversely cover values in the whole range [0, 1]. The most similar vertices are connected with close-to-one weighted edges whereas the lowest connections are close to zero
227
5

---

Quality of the approximation. In theory, we would like our approximation to be as precise as possible. In practice, high-resolution approximation leads computational issues time and memory. Hence, tuning of the graph parameters becomes trade-off between theoretical consisteney and practical feasibility. First of all, the graph resolution (or the number of vertices we sample) is directly related to the quality of the approximation. While the spatial resolution [)]a is usually determined by the data (up to up- and swm-sasmllngg), the orientation resolution [V/ a design choice. An important remark is to notice that large orientation resolution does not necessarily help two different orientations are not distinguishable because of a poor spatial resolution [Weiler et al. 236
230
232
234
2018 Bekkers 2019]. Secondly, the connectivity of the graph is also crucial parameter. A fully
238
connected graph is theoretically the best approximation. Nevertheless, for computational reasons, we K-NN graphs to sparsify the graph Laplacians.
(a) Base space M with isotropic Riemannian metric
(b) Lic group extension G with isotropic Riemannian metric
(c) Lie group extension G with an anisotropic Riemannian metric
Figure I: Isotropic diffusion applied an impulse signal on Riemannian manifolds M = IR and = SS).)..
Theoretical group equivariance of the graph Laplacian. Due to the success of machine learning algorithms based on graph Laplacian, the theoretical convergence of the graph Laplacian to its continuous analogue has been largely studied (Hein et al. [005 Singer [006]. Belkin and Niyogi 2006) noticed that many graph-based algorithms, central role is played by the graph Laplacian's eigenvectors. Thus, they focused on proving convergence in eigenmaps as it is sufficient in this case. They proved that if the graph's 's vertices are sampled uniformly from an unknown submanifold M Rd then the eigenvectors of suitably constructed graph Laplacian converges to the eigenfunctions of the Laplace-Bellee Beltrami operator on M. Consequently, as the latter operator is left-invariant, as we show in theorem A.1 the graph Laplacian is asymptotically group equivariant.
241
243
245
247
249
Empirical group equivariance of the graph Laplacian. We empirically confirm the group equiv- ariance property of the graph Laplacian applied to our anisotropic manifold graphs. By checking PT ÁP = A where P is permutation matrix, we can verify that the graph Laplacian is invariant vertices corresponding
250
251
252
group transformation (e.g. a rotation of the graph). Moreover, we can also compare the eigenmaps of graph Laplacian and its continuous counterpart if well-known. For further discussion about this, see App.
under given permutation
254
3.2 ChebLieNet
Chebyshev convolutional layer. As introduced in Defferrard et al. (016)) a Chebyshev convolu- tional layer is a spectral layer based on continuous kernel parametrization with graph Laplacians. This parameterization makes such layers highly suitable for our method, as they intrinsically cap- ture the Riemannian geometry of the graphs on Moreover, the Chebyshev convolutions on the anisotropic manifold graphs are equivariant by construction because the graph Laplacians are equivariant operators (see Figure 2
257
259
261
Note that in implementation, K -NN graphs does mean that each vertex has K neighbors but most K neighbors. Indeed, if the graph domain has boundaries, using exactly neighbors for each vertex could lead to asymmetries that may introduce biases and harm the permutation invariances in the graph.
'The asymptotic corresponds [V] and Gaussian weight kernel with kemel bandwidth t 0.

---

Figure 2: Rotation equiv- ariance of randomly ini- tialized SE Chebyshev convolutional layer. From left to right shows differ- ent rotations of input (top row) and the activa- tions for different slices of E [0, π in the graph (bot- tom 6 rows). A rotation of an imput image followed by Chebyshev convolution is equivalent to first convolu- tion followed by planar tation in each 0 slice and roll in the 0-axis.
7
Spatial pooling and unpooling layers. Graph pooling is central component in myriad of graph neural network architectures. Producing coarsened graphs from a finer graph have two main advantages: first, it reduces the computational cost, and second, it could improve performance by
263
285
reducing the overfitting effect and adding a multiscale perspective. As an inheritance from traditional CNNs, most approaches formulate graph pooling
288
cluster assignment problem, extending local patches' idea in regular grids to graphs [Dhillon et al. 2007 Ying et al.) 2018 Khasahmadi et al.
287
268
Mesquita al. 2020 We propose similar operations on the base space (spatial domain) and involving ving two steps (see Figure B First, each sample is assigned to cluster that will correspond to
2020
270
the output sample; this is the down- (resp. up-) sampling phase. With a well designed method, this change of data-resolution can be made equivariant any group transformation! Then, each cluster is reduced (resp. expanded) according a given scheme (e.g. maximum, average or random); this the reduction (resp. expansion) phase. When the reduction and expansion steps are permutation-invariant operations, such layers are automatically invariant under any transformation the group.
272
274
(a) R2RddPPool
S2MaxPool
(c) RnRanodnpool
(d) S2AvgUnpool
Figure 3: Spatial pooling and unpooling layers on the 2D grid and the sphere.
275
Global pooling (projection) layer and point-wise operations. When the neural network does not need to be equivariant but invariant (e.g. classification task), it is common to rely on global pooling layer (or simply projection layer). This layer reduces the d-dimensional signal on the graph's vertices to d-dimensional vector of features derived from information on the whole graph. As commutation-ivariinn operation, such a layer does not break the equivariance property the neural network. Finally, point-wisg wise operations do not affect the equivariance of a neural network.
276
277
278
279
Altough down- and up samplinlsngs are naturally defined on the Euclidean grid, this task is more complicated on the sphere. However. using an icosahedron decomposition of the sphere, we make it more natural as down- and up-sampling consists of decreasing increasing the subdivision level.

---

4
Experiments
281
In this section, we show the benefits of working on the anisotropic manifold graphs compared the base manifold graphs. We believe that further improvements could be achieved through tuning and hyper-parameter optimization of the models Yu and Zhu 2020), using high-capacity networks, or via more advanced training process, but this is not the goal of our work. We here intent to illustrate the 285
282
284
286
adaptability our approach to different tasks such as classification and segmentation 2D images or spherical data. In the first couple of experiments, we motive the use of anisotropic spaces. By varying the anisotropies, we show the existence of sweet spots, both for the spatial anisotropy parameter e and the orientation anisotropy parameter& In the second couple of experiments,
286
show that even we add a new orientation dimension, our method remains scalable using a proper implementation.
290
291
url We perform all the experiments on single GeForce GTX 1080 Ti gpu and track them with the Weights & Biases library [Biewald] 2020]. The details of the experiments are given in the App. D
Our implementation is fully PyTorch
Paszke et al. 2019] and available at
292
293
294
4.1
Why using tunable anisotropic kernels?
295
As introduced in Section 3.1 the anisotropies are tunable via the parameters and & of the Riemannian
296
metric. respectively responsible for the spatial and orientation anisotropies. As the & parameter should
depend on the spatial and orientation resolutions, use the following parameterisation: ç² [V
[y_
Setting
yields 40/60 ratio of neighbors within versus outside the orientation plane. We
ran different experiments with a Wide Residual architecture [Zagoruyko and Komodakis) (0016) on
299
300
CIFARIO Krrihhvsky al. 2009] varying the spatial and orientation anisotropic parameters.
Test-accuracy [%)
Test- Test-accural accurcy [%)
85
isotropic
85
& invariant
80
80
cquivariant
2
4
8
16
32
0.2
0.4
0.6
0.8
1
2
O
(a) Test-accuracy against orientation anisotropics
(b) Test-accuracy against spatial anisotropies
Figure 4: Empirical proof of existence of sweet spots for data-dependent anisotropic parameters.
301
Orientation anisotropy. The orientation anisotropy & controls how strongly orientation layers are connected. At the limit & oc, orientation layers decoupled. It is like test-time augmentation with rotations:
running CNN working with one anisotropic Laplacian (e.g., only vertically aligned filters) and testing the network for different input rotations before averaging the output. The other extreme & + ( keeps all layers equally close to each other, and features are essentially identified with just a spatial coordinate. This would then correspond to a WideResNet with isotropic Chebyshev convolutions. For reasonable values of {, interactions between orientation layers take place. Figure 4ah is evidence of the existence of sweet spot for this parameter in the range of reasonable values. At 308
303
305
308
307
309
the moment, we expect with no certainty that this parameter could be set a priori of the data, only considering the data resolution. As rule of thumb, we set E such that each vertex has approximately 40% of its neighbors in the same orientation layer and 60% on others.
310
311
312
Spatial anisotropy. The spatial anisotropy regulates the anisotropy of the space on the spatial domain. For 1, the Riemannian metric is spatially isotropic; all directions are treated equally and the resulting model would effectively be WideResNet with isotropic Chebyshev convolutions. At the limit e - 0, the main direction has minimal cost, and the resulting space is highly spatially anisotropic. figure 4a we observe that using anisotropic spaces instead of isotropic ones is relevant,
313
314
315
316
as we almost get an 8% test-accuracy improvement. Unlike the orientation anisotropic parameter, our opinion, this parameter is task/data-depeeddett different datasets could benefit in different
318
degrees from the utilization of directional information through different spatial anisotropy settings.
8

---

320
4.2
How scalable is the method?
321
Scalability is often an important limitation of graph- and group- neural networks. By adding an orientation dimension, we do not run from this rule as we necessarily increase the number of vertices of the anisotropic manifold graphs. To permit experiments on larger images, it becomes crucial to pre-compute anisotropic manifold graphs and their Laplacians. Dedicated librairies like PyKeops [Charlier et al. 2020] enable this without memory issues. Nevertheless, the graph operations (convolutions, pooling or unpooling) still scale with the size of the graph. Fortunately, Py PyTorch
322
323
324
325
326
327
provides sparse operations that increase efficiency in terms of time and memory compared to dense
operations in cases of sufficiently sparse graph Laplacians (typically a sparsity S(Ā) 98.5%).
328
329
We evaluate our models on an image classification task on STL10 (Coates et al. 2011 and an image segmentation task on ClimateNet (Kashinath et al. [202]] We show the adaptability of our method by using a Wide Residual architecture [Zagoruyko and Komodakis 2016) on STL10 and a U-Net-like network [Ronneberger et al. 2015] on Nei We also demonstrate the potential of our approach and the benefits of using anisotropic spaces. Indeed, while on ClimateNet the use of anisotropies is neither beneficial nor detrimental, the difference in performance on STL10 significant.
330
331
332
333
334
Table 1: Mean of test performance and training duration on ClimateNet and STL10. Errorbars are 1 standard deviation computed over 5 trials.
ClimateNet
STL10
Test F1
Duration
Test accuracy
Duration
85.62 + 0.09% (invariant)
22 ~ 7
68.98 0.56%
hh ~ 16h
0.1
(equivariant)
85.25 + 0.19%
74.02 + 1.10%
335
5
Conclusion
336
With our method, geometric graph NNs are made equivariant Lie groups Via the groups SE() and E3), we can construct roto-translation equivariant networks for 2D image data and 3D volumetric data. Based on the group SO(3), our method can deal with metcorological or cosmological data while preserving rotation equivariance. We believe that our flexible approach is ideal for further explorations on the relevance of group equivariance in tasks not considered in this work.
Scope.
337
338
339
340
341
The main weakness of our method is its relatively high memory requirement. Al- though all experiments ran single gpu, by adding an orientation axis, significantly enlarge the feature maps. As a result, anisotropic graph manifolds are memory-heavier than isotropic ones and prone slowdown during the forward- and backward-pass. Nevertheless, with the emergence of ge- ometric deep learning, we expect improvement in the hardware and implementation of graph-oriented operations. Another challenge is the increased number of hyper-pun apper-parameters for which we only have derived rules of thumb. The graph connectivity and resolutions require a tradcoff between efficiency and quality of the manifold approximation. The anisotropic parameters require an analysis of the dataset and some intuition about the amount of anisotropy to set. With systematic aaper-paramete optimization, we can find an optimal combination, but requires more computational resources.
Limitations.
342
343
344
345
348
347
348
349
350
351
Potential and future research. Thanks to its easy-to-tune anisotropic properties, our model can be used to better understand anisotropic properties in data. In particular, one could explore the effect of using anisotropic spaces instead of isotropic ones on many tasks and conclude when such anisotropic information is relevant. In this vein, it could also be interesting to derive anisotropic pooling and unpooling layers based on anisotropic spaces instead of isotropic ones as it is usually done. More generally, our method simple enough be extended shapes/surfaces with a Riemannian manifold structure [Cohen et al. [ In this work, we focused on 2D images and spherical data on, but the method is readily extendable to higher dimensional Lie groups such as the SE group to obtain
352
353
354
355
356
357
358
359
3D roto- iro-ttranslation equivariant ChebLieNets. Moreover, our method for constructing anisotropic geometries could directly improve other successful Euclidean distance-based graph NNs such as
360
Satorras et al. 2021 by making them fully equivariant. Last but not least, despite graph-
algorithms being computationally sub-optimal compared to CNNs, their flexibility a real asset. We see high potential in the exploration of graph sparsification to reduce computational complexity.
362

---

References
384
365
Emre Baspinar, Luca Calatroni, Valentina Franceschi, and Dario Prandi. A cortical-inspired sub- riemannian model for poggendorll-type visual illusions. Journal of Imaging, 7(3):41, 2021.
John R Baumgardner and Paul O Frederickson. Icosahedral discretization of the two-sphere. SIAM Journal on Numerical Analysis, 22(6):1 107-1115, 1985.
368
389
Erik Bekkers. Retinal image analysis using sub-riemannian geometry in se (2). 2017.
370
Erik J Bekkers, B-spline cnns on lie groups. In International Conference on Learning Representations, 2019.
371
372
Erik Bekkers, Remco Duits, Alexey P Mashtakov, and Gonzalo R Sanguinetti. A PDE Approach to Data-Drive Sub- Ribmmmminnnnn Geodesics in SE(2). SIAM Journal on Imaging Sciences, 8(4): 2740-2770, 2015. doi: 10.1 10.1137/15M101846) URL https://p///.. org/ 10. 1137/15M1018460
373
374
375
Erik J Bekkers, Da Chen, and Jorg M Portegies. Nilpotent approximations of sub-riemannian distances for fast perceptual grouping of blood vessels in 2d and 3d. Journal of mathematical imaging and vision, 60(6):882-899, 2018.
377
378
Mikhail Belkin and Partha Niyogi. Convergence of laplacian eigenmaps. Advances in neural information processing systems, 19:129-136, 2006.
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL com/
381
Software available from wandb.com.
382
Ugo V Boscain, Roman Chertovskih, Jean-Paul Gauthier, Dario Prandi, and Alexey Remizov. Highly corrupted image inpainting through hypoelliptic diffusion. Journal of Mathematical Imaging and Vision, 1031)11231 2018.
384
385
Davide Boscaini, Jonathan Masei, Emanuele Rodoià, and Michael Bronstein. Learning shape correspondence with anisotropic convolutional neural networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 3197-3205, 2016.
386
387
388
William H Bosking, Ying Zhang, Brett Schofield, and David Fitzpatrick. Orientation selectivity and the urrangement of horizontal connections in tree shrew striate cortex. Journal of neuroscience, 17 12):212 1997.
389
390
391
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
392
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint 2XXx:21044.3478 2021.
395
396
Joan Bruna, Wojeiech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks graphs. arXiv preprint arXiv:1312.6200 2013.
Benjamin Charlier, Jean Feydy, Joan Alexis Glaunès, François- Dravid Collin, and Ghislain Du- rif. Kernel operations on the gpu, with autodiff, without memory overflows. arXiv preprint arXiv:2004. / /27, 2020.
389
400
Fan RK Chung and Fan Chung Graham. Spectral graph theory. Number 92, American Mathematical Soc., 1997.
402
403
Giovanna Citti and Alessandro Sarti. A cortical based model of perceptual completion in the ato--translation space. Journal of Mathematical Imaging and Vision, 24(3):307-326, 2006.
404
405
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215-223. JMLR Workshop and Conference Proceedings. 2011.
406
407
408
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference machine learning, pages 2990-2999, 2016.
10

---

410
Taco Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns on homogeneous spaces. arXiv preprint arXiv:1811.0211,, 2018.
411
412
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolu- tional networks and the icosahedral cnn. arXiv preprint arXiv:1902.0460 2019.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks graphs with fast localized spectral filtering. In Advances in neural information processing systems, pages 3844-3852, 2016.
415
416
417
Michaël Defferrard,
Martino Milani, Frédérick Gusset, and Nathanaël Perraudin. Deepsphere: graph-based spherical cnn. arXiv preprint 2rXiv:2012.15000, 2020.
418
Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. Weighted graph cuts without eigenvectors multilevel approach. IEEE transactions on pattern analysis and machine intelligence, 29(11), (1): 1947-1957, 2007.
420
James R Driscoll and Dennis M Healy. Computing fourier transforms and convolutions the 2 2-sphere. Advances in applied mathematics, 15(2):202-250, 1994.
423
424
R. Duits, S. P. L. Meesters, J.-M. Mirebeau, and J. M. Portegies. Optimal Paths for Variants of the 2d and 3d Reeds-Shepp Car with Applications in Image Analysis. Journal of Mathematical Imaging and Vision, February 2018. ISSN 1573-7683 doi: (0.1007/110831-018-777777 --99--2. URL https / org/ 10. 1007 018- 0795-
425
428
427
428
Remco Duits, Ugo Boscain, Francesco Rossi, and Yuri Sachkov. Association fields via cuspless nub-Riemannian geodesics in SE (2). Journal of mathematical imuging and vision, 49(2):384-417, 2014.
430
431
Marta Favali, Samaneh Abbasi-Sureshjant Bart ter Haar Romeny, and Alessandro Sarti. Analysis of vessel connectivities in retinal images by cortically inspired spectral clustering. Journal of Mathematical Imaging and Vision, 56(1):158-11)) 172- 2016.
433
434
Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International Conference on Machine Learning, pages 3165-3176. PMLR, 2020.
435
436
437
Erik Franken and Remco Duits. Crossing- preservin coherence-enhancing diffusion on invertible orientation scores. International Journal of Computer Vision, 85(3):253, 2009.
438
439
Fabian B Fuchs, Edward Wagstaff, Justas Dauparas, and Ingmar Posner. Iterative se (3)-transformers. arXiv preprint arX1v:2102.13419 2021.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.
442
443
Krzysztof M Gorski, Eric Hivon, Anthony J Banday, Benjamin D Wandelt, Frode K Hansen, Mstvos Reinecke, and Matthia Bartelmann. Healpix: A framework for high-resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759, 2005.
444
445
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015.
447
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition,
450
pages 770-778, 2016.
452
Matthias Hein, Jean-Yves Audibert, and Ulrike Von Luxburg. From graphs manifolds-wedt and strong pointwise consistency of graph laplacians. In International Conference on Computational Learning Theory, pages 470-485. Springer, 2005.
453
454
455
Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint
2015.

---

457
David H Hubel and Torsten N Wiesel. Receptive fields, binocular interaction and functional architec- ture in the cat's visual cortex. The Journal of physiology, 160(1):106-155 1962.
458
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint 1502::3302 2015.
480
461
John Jumper, R Evans, A Pritzel, Green, M Figurnov, K Tunyasuvunakool, O Ronneberger, R Bates, A Zidek, A Bridgland, et al. High accuracy protein structure prediction using deep learning. Fourteenth Critical Assessment of Techniques for Protein Structure Prediction (Abstract Book), 22:
462
463
24, 2020.
465
Karthik Kashinath, Mayur Mudigonda, Sol Kim, Lukas Kapp-Schwoeree Andre Graubner, Ege Karaismailoglu, Leo Kleist, Thorsten Kurth, Annette Greiner, Ankur Mahesh, al. Climatenet: an expert-labeled open dataset and deep learning architecture for enabling high-precision analyses of extreme weather. Geoscientific Model Development, 14(1):107-124, 2021.
466
467
468
Nicolas Keriven, Alberto Bietti, and Samuel Vaiter. Convergence and stability of graph convolutional networks on large random graphs arXiv preprint arXiv:2006.01868, 2020.
470
Amir Hosein Khasahmadi, Kaveh Hassani, Parsa Moradi, Leo Lee, and Quaid Morris. Memory-based graph networks. arXiv preprint arXiv:2002.09578, 2020.
472
473
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69888 69800 2014.
475
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.000 02907, 2016.
478
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning, pages 2747-2755. PMLR, 2018.
478
480
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
481
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep con- volutional neural networks. Advances in neural information processing systems, 25:1097- 1105, 2012.
483
484
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL http://yann lecun. www/exdd/ bexdb/mnist.
485
486
Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, (10) 1995.
488
Karel Lene and Andrea Vedaldi. Understanding image representations by measuring their equivariance and equivalence. Proceedings the IEEE conference on computer vision and pattern recognition, pages 991-999, 2015.
489
490
491
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic con- volutional neural networks riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops, pages 37-45, 2015.
483
494
Alexey Mashtakov, Remco Duits, Yu Sachkov, Erik Bekkers, and Ivan Beschastnyi. Tracking of lines in spherical images via sub-riemannian geodesics in SO(3). Journal of mathematical imaging and vision 58(2):239-264 2017.
496
Diego Mesquita, Amauri H Souza, and Samuel Kaski. Rethinking pooling in graph neural networks. arXiv preprint arXiv:2010.1 /418, 2020.
488
499
Richard Montgomery. A tour of aub-Riemnnnian geometries, their geodesics and applications. Number 91, American Mathematical Soc., 2006.
12

---

501
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model enns. In Proceedings of the IEEF conference on computer vision and pattern recognition, pages 51 15-5142,
502
504
2017.
505
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-perttee performaroeee deep learning library. In Advances neural information processing systems,
506
507
508
pages 8026-8037, 2019.
509
Nathanaël Perraudin, Michaël Defferaard Tomasz Kacprzak, and Raphael Sgier. Deepsphere: Effi- cient spherical convolutional neural network with healpix sampling for cosmological applications. Astronomy and Computing, 27:130-146, 2019.
510
511
512
Jean Petitot. The neurogeometry of pinwheels as a uub-Riemannian contact structure. Journal of Physiology-Paris, 9702-3):65-309, 2003.
513
Jorg Portegies, Gonzalo Sanguinetti, Stephan Meesters, and Remco Duits. New approximation of a scale space kernel on se (3) and applications in neuroimaging. In International Conference on Scale Space and Variational Methods in Computer Vision, pages 40-52. Springer, 2015.
515
518
James Reeds and Lawrence Shepp. Optimal paths for car that goes both forwards and backwards. Pacific journal of mathematics, 145(2)3367-333, 1990.
518
519
Olinde Rodrigues. Des lois géométriques qui régissent les déplacements d' I'un système solide dans I'espace: et de la variation des cordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire. 1840.
521
522
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.
524
525
Gonzalo Sanguinetti, Erik Bekkers, Remco Duits, Michiel HJ Janssen, Alexey Mashtakov, and Jean-Marie Mirebeau. Sub-riemannian fast marching in (2). In Iberoamerican Congress Pattern Recognition, pages 366-374. Springer, 2015.
527
528
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. arXiv preprint arXiv:2102.09844, 2021.
530
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. JEEE transactions on neural networks, 20(1):61-8)) 2008
531
532
David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending nigh-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine, 30(3):83 98, 2013.
533
534
535
Amit Singer. From graph to manifold laplacian: The convergence rate. Applied and Computational Harmonic Analysis, 21(1): 2006.
536
537
Maurice Weiler, Fred A Hamprecht, and Martin Storath. Learning Steerable Filters for Rotation
538
Equivariant CNNs. In International Conference on Computer Vision and Pattern Recognition, 2018.
539
540
Douglas Brent West et al. Introduction to graph theory, volume 2. Prentice hall Upper Saddle River, NJ, 1996.
Rex Ying. Jiaxuan You, Christopher Morris, Xiang Ren, William L Hamilton, and Jure Leskovec. Hier- archical graph representation learning with differentiable pooling, arXiv preprint arXiv:1806.08804, 2018.
543
Tong Yu and Hong Zhu. Hyper-parameter optimization: A review of algorithms and applications. Xrxiv preprint arXiv:2003.0566 2020.
546
547
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint 1605.000000.07 146, 2016.
13

---

Checklist
549
550
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
552
contributions and scope? [Yes] See Section T
553
(b) Did you descrihe the limitations of your work? [Yes] See Section S
554
(c) Did you discuss any potential negative societal impacts of your work? [Yes] The environmental impact is a direct consequence of the time and memory issues we discussed in Section 5
555
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
558
559
2. If you are including theoretical results...
580
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section
561
(b) Did you include complete proofs of all theoretical results? [Yes] We refer the reader to original publication with proofs.
563
3. If you ran experiments...
564
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See Section S
586
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section 4
587
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] See Section 4
569
570
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
573
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4
(b) Did you mention the license of the assets? [Yes] We always refered to the original papers of the datasets we used.
575
576
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes] See Section for the URL. We also send zip file containing the whole implementation.
578
(d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [N/A]
579
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
581
582
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
584
585
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
586
587
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "fzo3ngmhHz", "reviewer": "Reviewer_MH5E", "summary": "In the paper, a new graph neural network called ChebLieNet was proposed. The proposed network was designed to achieve equivariance properties with respect to actions of Lie groups. The proposed ChebLieNet was analyzed on vanilla datasets.\n\nIn summary, the proposed ChebLieNet was well motivated and structured. However, there are various major and minor problems with the paper including missing notation, definitions, incomplete and insufficient mathematical results and experimental analyses, especially in comparison with the state-of-the-art.\n\nAfter Rebuttal: Thank you for your response to my comments. I checked the comments of other reviewers and the response to their comments as well. The authors answered most of the questions in the rebuttal. However, the paper should be improved for a clear acceptance. Therefore, I slightly increase my grade, and recommend authors address the theoretical details and improve the experimental analyses, esp. in comparison with state-of-the-art related methods..", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Comments and suggestions on major and minor issues with the paper:\n\n1.\tThere are various unclear statements in the paper. For instance, even the first sentence in the abstract “We introduce ChebLieNet, a group-equivariant method on (anisotropic) manifolds” is not clear. More precisely, is ChebLieNet a method, algorithm or a new neural network? What do you mean by “group-equivariant method”? The title should be also updated with a more clear description.\n\n2.\tThere are also various undefined terms and notation (I also checked supp. mat.). For instance, definitions of invariance/equivariant isotropic/anisotropic metric/manifold/layer/network/method are missing and these terms are used interchangeably without considering terminological and mathematical precision. Therefore, a reader needs to make educated guess on various terms, and the paper is not easily readable and most of the parts should be rewritten. Please also provide definition of the notation as it is used (e.g. for eqn. (1)).\n\n3.\tMost of the claims proposed in the paper should be revised since they are not verified mathematically or experimentally. Note that, the mathematical statements given in the paper construct the formalism of the proposed method/network but not introduces mathematical/theoretical results supporting the strong claims. For instance, the following claims should be explored and analyzed more precisely:\n\na.\tWith our method, geometric graph NNs are made equivariant to Lie groups: Needs mathematical and experimental verification. This property is not proved and detailed experimental analyses were not given.\n\nb.\tWe demonstrate the equivariance property of ChebLieNet, both in theory and in practice. This property guarantees that the neural network’s predictions are robust against given transformations, which is not necessarily the case with methods based on data augmentation: Needs mathematical and experimental verification: Needs mathematical and experimental verification. This property is not proved and detailed experimental analyses were not given. You need to provide results and analyses at least for different transformations and data augmentation methods in comparison with state-of-the-art methods.\n\nc.\tWe show that the use of directional information via anisotropic Riemannian spaces could benefit many tasks: Needs mathematical and experimental verification. This property is not proved and detailed experimental analyses were not given. You need to provide results and analyses at least for 4-5 different tasks.\n\nd.\tWe show the flexibility of the method by considering two different problems; we validate on classification problems with 2D image data and a segmentation problem on spherical data via the construction of a sub-Riemannian geometry on SE(2) and SO(3) respectively. Needs more detailed experimental verification. If you would like to address segmentation problem, please also consider the problem on image data.\n\n4.\tPlease explain more clearly how “the sub-Riemannian geometry forms the basis for the mathematical modeling of visual perception.”\n\n5.\tThe experimental analyses should be improved also by comparison of the proposed method with state-of-the-art invariant and equivariant methods and network models on different datasets including benchmark graph datasets.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Partially. Please check the above comments.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "d36q-vy0oOP", "reviewer": "Reviewer_JM2z", "summary": "The authors introduce ChebLieNet as a group-equivariant method on manifolds that extends the original ChebNet. The extension is based on enforcing group equivariance via encoding anisotropic left-invariance Riemannian distance on the edges.", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Quality**: I checked most of the mathematical details from the paper. Except for a small typo on the left-invariance definition on line 190, all other details seem to be correct.\n\n**Clarity**: The paper is in general very clear. The introduction and related work sections are written nicely, and the authors did a good job introducing the various components of their proposed network.\n\n**Originality and Significance**: My research area is essentially disjoint from the area of the paper, and I do not have the expertise to comment on the originality or significance of the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors addressed the limitations and potential negative societal impact in Section 5 of the paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "b0YNVhcme4", "reviewer": "Reviewer_hWjk", "summary": "Authors introduce graphs equivariant convolutional neural networks that are group-equivariant. They build on Defferrard et al. 2017, which introduced scalable convolutional neural networks on graphs, leveraging Chebyshev polynomials to bypass the need to compute the Fourier basis. In particular, they lift the (homogeneous) base space on which the data is defined to a Lie group (which acts on the base space) where they parametrize an anisotropic Riemannian metric. The edges weight are then computed using the geodesic distance induced that the metric, reflecting anisotropy.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n## Strengths\nThis work is relevant to the community as it contributes to building models that encode inductive biases in the form of group symmetry. The authors seem well versed in Riemannian geometry, group theory and spectral theory.\nEmpirically, they show that incorporating both 'spatial' and 'orientation' anisotropy for CIFAR10 helps on generalization performance.\n\n## Weaknesses\nThe main weakness of this work perhaps is clarity. I expand on this below.\nAdditionally, empirical evidence backing up the motivation of this work is somewhat limited. For instance, on Figure 4, the best test accuracy is still quite low, WideResNet reported an an accuracy of 96.11%. (I may be missing something). Similarly for STL10. I would argue to include a simple synthetic dataset with anisotropy baked in, and showing some of the learned filters so as to help the reader understand the method better.\n\n## Clarity\nGenerally the paper needs some improvement with regards to clarity. The method is grounded in a mathematical framework, which some readers may only be partially familiar with. As a consequence, particular care is required when introducing core ideas. For instance, it is somewhat stated that an anisotropic metric leads to the equivariance of the graph convolution layers whereas an isotropic metric leads to invariance. As this seems key I would suggest to expand and clarify this. \nAs section 3 is where the method is formally introduced, it needs to convey efficiently the ideas that the method builds on. I believe that the paragraph on local frames and co could be moved to the appendix as it is not adding much. The paragraphs that follow are key as they introduce the parametrization of the Riemannian metric with the spatial and orientation anisotropies. I believe that a bit more work on these is necessary to give better intuition to the reader. Perhaps an illustration would help? Figure 1 is helping but still not filling the gap.\n\n## Correctness\nThis work claims that the proposed ChebLieNet is group-equivariant as graph Laplacians are equivariant operators. This is true as the Laplacian is intrisic, hence invariant under isometries. As this is a core claim of the paper, it would nonetheless  be useful for the reader to have a bit more explanations and intuition.\nThe convolution is in practice only approximately equivariant due to the Laplacian being asymptotically equivariant. How come this requires the kernel bandwidth to go to infinity? Wouldn't all the weights go to zero then?\n\n## Reproducibility\nThe code source has been provided in the supplementary material and after a quick inspection, it seems to allow to reproduce their results.\n\n## Additional feedback\n- As the base space is lifted, how is the $H$ dimension (grid) initialized for the data?\n- 197: \"which forms the basis for the mathematical modeling of visual perception.\" This sentence would need more context, or could be simply removed.\n- 218: Can we really not analytically compute the distance the geodesic distance in $SE(3)$? Is it because of the particular choice of Riemannian metric R? Does the Lie group logarithm map depends on $\\epsilon$ and $\\xi$? The approximation from Equation 3 linearize the group around $h$ (or symmetrically $g$) which sounds reasonable as the filter is localized. This may be highlighted.\n- Figure 1: Perhaps adding a 4th subfigure `Base space M with an anisotropic Riemannian metric' would help?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are discussed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "VH1BbrIAT3g", "reviewer": "Reviewer_9mff", "summary": "This paper proposes a learning method for graphs using equivariance for Lie groups on manifolds.\nSpecifically, it is an equivariant graph Laplacian-based neural network based on Lie groups equipped with an anisotropic Riemannian metric.\nThey have also provided some experiments for this approach.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "There are a few things that I think are not good.\nThe first one is what kind of functions this model is a universal approximator for.\nFor example, in Maron's model, the universality of invariant continuous functions is known for the induced action of node permutation on adjacency matrices. Such a theoretical analysis is needed for this model.\nI also have a question about discrete approximations of the Lie group: as the dimension of the Lie group increases, this kind of approximation becomes very computationally expensive, as seen in Lie conv. Is it always possible to approximate the Lie group to the level of practical accuracy with this method?\nMy last is about the experiment.\nIn Table 1, there are only results for the proposed method and no comparison method.\nIn this table, it is unclear whether the accuracy of the proposed method is higher than that of the previous methods or not.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors present a novel type of layer for neural networks on manifolds, which can take into account the inherent anisotropy of the manifold. There was a consensus among reviewers that the paper lacked clarity and the presentation could be improved. The one reviewer who recommended acceptance only wrote a very cursory review and admitted that the paper was far outside their expertise, and so I have not taken their review into account. Therefore, I recommend the paper be rejected.
Decision: Reject