ChebLieNet: Invariant spectral graph NNs turned equivariant by Riemannian geometry on Lie groups

We introduce ChebLieNet , a group-equivariant method on ( anisotropic ) manifolds.1 Surfing on the success of graph- and group-based neural networks , we take advan-2 tage of the recent developments in the geometric deep learning field to derive a3 new approach to exploit any anisotropies in data . Via discrete approximations of4 Lie groups , we develop a graph neural network made of anisotropic convolutional5 layers ( Chebyshev convolutions ) , spatial pooling and unpooling layers , and global6 pooling layers . Group equivariance is achieved via equivariant and invariant opera-7 tors on graphs with anisotropic left-invariant Riemannian distance-based affinities8 encoded on the edges . Thanks to its simple form , the Riemannian metric can model9 any anisotropies , both in the spatial and orientation domains . This control on10 anisotropies of the Riemannian metrics allows to balance equivariance ( anisotropic11 metric ) against invariance ( isotropic metric ) of the graph convolution layers . Hence12 we open the doors to a better understanding of anisotropic properties . Furthermore,13 we empirically prove the existence of ( data-dependent ) sweet spots for anisotropic14 parameters on CIFAR10 . This crucial result is evidence of the benefice we could15 get by exploiting anisotropic properties in data . We also evaluate the scalability of16 this approach on STL10 ( image data ) and ClimateNet ( spherical data ) , showing its17 remarkable adaptability to diverse tasks.18 1 Introduction19 Deep learning is a class of machine learning algorithms inspired by the human brain ’ s network of20 neurons [ Goodfellow et al. , 2016 ] . These algorithms use a hierarchical structure of neural layers to21 extract higher-level features from the raw input progressively . In the past few years , the growing22 computational power of modern GPU-based computers and the availability of large training datasets23 in the field of machine learning have made it possible to successfully train neural networks with24 many layers and degrees of freedom . Consequently , deep learning has revolutionized many machine25 learning tasks in recent years , ranging from image and video processing to speech recognition and26 natural language understanding.27 Many neuroscientific research results served as focal points in the development of deep learning28 algorithms . When Hubel and Wiesel [ 1962 ] studied the visual cortex in the brain , they made three29 important discoveries . First , they observed a one-to-one correspondence between spatial locations30 in the retina and neurons in the brain that fired as a response to line-like visual stimuli . Second,31 the activity of the neurons changed depending on the orientation of the line , uncovering a neat32 organization based on local orientations . Last , the neurons sometimes fired only when the line was33 moving in a particular direction . Later , Bosking et al . [ 1997 ] showed that neurons that are aligned fire34 together , indicating the presence of a type of long-range interactions . All these results motivated the35 development of a mathematical framework for modeling visual perception based on sub-Riemannian36 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . geometry on the space of positions and orientations , which is typically modeled with the Lie group37 SE ( 2 ) [ Petitot , 2003 , Citti and Sarti , 2006 , Duits et al. , 2014 ] . Apart from the neurophysiological38 inspiration , group equivariance has also been proven to be an excellent inductive bias [ Cohen and39 Welling , 2016 ] not only in computer vision ( as the translation equivariance property of CNNs as40 shown ) but also in physics [ Finzi et al. , 2020 ] and molecular data analysis [ Fuchs et al. , 2021 , Jumper41 et al. , 2020 ] . In this work , we propose to build group equivariant graph neural networks via the same42 principle that underlie the sub-Riemannian , neurogeometrical modeling of the visual cortex.43 Our work connects the observations by Hubel and Wiesel [ 1962 ] and Bosking et al . [ 1997 ] on two44 levels . First , the organization of visual data based on their location and orientation [ Hubel and Wiesel,45 1962 ] is modeled by Lie group convolutions [ Bekkers , 2019 ] , in which feature maps encode response46 for every position and every orientation . Second , long-range interactions between aligned neurons47 [ Bosking et al. , 1997 ] are modeled by building graphs with affinity matrices based on ( approximate ) 48 sub-Riemannian distances on the Lie groups , inspired by sub-Riemannian image analysis methods49 such as [ Franken and Duits , 2009 , Bekkers et al. , 2015 , Favali et al. , 2016 , Mashtakov et al. , 2017,50 Boscain et al. , 2018 , Duits et al. , 2018 , Baspinar et al. , 2021 ] .51 Defferrard et al . [ 2020 ] showed how to construct powerful graph NNs that are faithful to the manifolds52 on which they are defined . Nevertheless , the layers themselves are based on rotationally invariant53 ( Laplacian ) convolutions . In order to exploit directional cues in the data , group convolutions are54 desirable [ Cohen et al. , 2018 , Kondor and Trivedi , 2018 , Cohen and Welling , 2016 , Bekkers , 2019 ] .55 However , since Laplacian operators are intrinsically isotropic , there is no point applying them to the56 lifted feature maps on the group unless we construct anisotropic metrics on the groups . Therefore,57 we adopt the Lie group viewpoint by Sanguinetti et al . [ 2015 ] to define anisotropic Riemannian58 metrics based on left-invariant vector fields on the group . Once an anisotropic Riemannian graph is59 constructed , any spectral method can directly be applied to this graph . The resulting graph neural60 networks will then , by construction , be equivariant and capable of utilizing directional cues in data.61 Before going further into the details , we summarize our main contributions:62 • We introduce ChebLieNet , an equivariant graph Laplacian-based neural network based on63 Lie groups equipped with an anisotropic Riemannian metric.64 • The Riemannian geometry is automatically derived from a standard base space ( e.g . R2 or65 the sphere ) , which makes our approach flexible and effective in building group equivariant66 graph neural networks for a variety of data structures ( e.g . 2D and spherical data ) .67 • We demonstrate the equivariance property of ChebLieNet , both in theory and in practice.68 This property guarantees that the neural network ’ s predictions are robust against given69 transformations , which is not necessarily the case with methods based on data augmentation.70 • We show that the use of directional information via anisotropic Riemannian spaces could71 benefit many tasks.72 • We show the flexibility of the method by considering two different problems ; we validate on73 classification problems with 2D image data and a segmentation problem on spherical data74 via the construction of a sub-Riemannian geometry on SE ( 2 ) and SO ( 3 ) respectively.75 2 Related works76 2.1 Group equivariant convolutional neural networks77 Deep convolutional neural networks [ LeCun et al. , 1995 ] have proven to be compelling models78 for pattern recognition tasks on images , video , and audio data . Although a robust theory of neural79 network design is currently lacking , a large amount of empirical evidence supports the notion that80 both convolutional weight sharing , depth , and width are essential for good predictive performance.81 Such properties are enabled through the equivariance property of convolutions ( convolving a shifted82 image is the same as translating its result ) .83 Lenc and Vedaldi [ 2015 ] showed that the AlexNet CNN Krizhevsky et al . [ 2012 ] trained on ImageNet84 learns representations equivariant to flips , scalings , and rotations spontaneously . This supports the85 idea that equivariance is an excellent inductive bias for deep convolutional networks . In the last few86 years , a joint effort has been made to build group equivariant networks . By the introduction of group87 convolutions in deep learning , Cohen and Welling [ 2016 ] generalize the translation equivariance88 property to larger groups of symmetries , including rotations and reflections . Kondor and Trivedi89 [ 2018 ] gave a rigorous , theoretical treatment of convolution and equivariance in neural networks90 concerning any compact group ’ s action . One of the main contributions of that work was to show that,91 given some natural constraints , the convolutional structure is not just a sufficient but also a necessary92 condition for equivariance to a compact group ’ s action . In a similar spirit , in [ Bekkers , 2019 ] it93 is shown that any bounded linear operator is equivariant to Lie groups if and only if it is a group94 convolution . In our work , we propose to build group equivariant neural networks via left-invariant95 Laplace operators on Lie groups , which indeed can be seen as group convolutions with kernels96 that are the fundamental solutions of the Laplace operator . The result is a Lie group equivariant97 Chebyshev-type neural network [ Defferrard et al. , 2016 ] that we will refer to as ChebLieNet.98 2.2 Graph neural networks99 Using the term geometric deep learning , Bronstein et al . [ 2017 , 2021 ] give an overview of deep100 learning methods in the non-Euclidean domain , including graphs and manifolds . They present differ-101 ent examples of geometric deep learning problems and available solutions , fundamental difficulties,102 applications , and future research directions in this nascent field.103 One of the main challenges when working with graph data it to deal with the inter-dependencies104 between points . Indeed , the derivations of most standard machine learning models firmly base on105 an independence assumption . For this reason , transferring existing methods on a graph appears106 doomed to failure , and it seems necessary to build models acting directly on graphs . Due to its107 success on Euclidean data , the development of a convolution-like operator on graphs has been largely108 studied . Because the notion of space is not naturally defined on a graph , we lack a straightforward109 generalization of the convolutional operator from grid data to graphs [ Scarselli et al. , 2008 , Bruna110 et al. , 2013 , Henaff et al. , 2015 , Defferrard et al. , 2016 , Kipf and Welling , 2016 , Masci et al. , 2015,111 Boscaini et al. , 2016 , Monti et al. , 2017 ] .112 Spectral approaches have a solid mathematical foundation in graph signal processing . Rather than113 using the traditional spatial definition of the convolution , it proposes to see this operation from a114 spectral perspective . Based on the convolution theorem , it defines the convolution operator from the115 graph spectral domain via the eigendecomposition of the graph Laplacian ( see App . A.3 ) .116 Definition 2.1 ( Spectral graph convolution ) Let G = ( V , E , W ) be a graph with Laplacian ∆̂ and117 let f and g be two functions defined on V . We define the G-convolution ∗G of f and g as:118 f ∗G g = Φ ( ĝ f̂ ) = Φ ( Φ > g Φ > f ) , ( 1 ) with eigenvectors Φ obtained through the unique eigendecomposition ∆̂ = ΦΛΦT .119 While this definition alleviates the difficulty of deriving a convolution operator in the spatial domain,120 other difficulties arise . First of all , because the Laplacian of a graph is an intrinsic operator , it121 is domain-dependent , and the spectral-convolution is too . It implies that a model built on this122 framework can not be easily transferred from a graph to another as expressed in a different `` language '' .123 Nevertheless , this is not a problem for us since we are focusing on fixed manifold graphs . Next , there124 is no guarantee that filters represented in the spectral domain are spatially localized . Henaff et al.125 [ 2015 ] successfully bypassed this problem by defining smooth spectral filter coefficients , arguing126 that if spectral filters are smooth , they are spatially localized . Last but not least , the Laplacian ’ s127 eigendecomposition makes the method expensive in terms of memory and time . Indeed , the forward128 and inverse graph Fourier transforms ( via ΦT and Φ ) incur expensive multiplications as no FFT-like129 algorithm exists on general graphs . Defferrard et al . [ 2016 ] alleviated the cost of explicitly computing130 the graph Laplacian using spatially-localized filters with Chebyshev polynomials.131 Definition 2.2 ( Chebyshev convolutional layer ) Let G = ( V , E , W ) be a graph with rescaled132 Laplacian1 ∆̃ , x ∈ R|V|×di be an input features ’ vector and Θj ∈ Rdi×do learnable filters . The133 1Because Chebyshev polynomials are defined in the range [ −1 , 1 ] , it is necessary to rescale the graph Laplacian with ∆̃ = 2λ−1max∆̂− I where λmax is the largest eigenvalue of ∆̂ . output features ’ vector y ∈ R|V|×do is computed as:134 y = R−1∑ j=0 zjΘj with z0 = x , z1 = ∆̃x and zj = 2∆̃zj−1 − zj−2 . ∀j ≥ 2 . ( 2 ) Kipf and Welling [ 2016 ] simplified this formulation a bit by considering the construction of single-135 parametric filters that are linear with relation to ∆̃ . They further approximate λmax ' 2 as they136 expect that neural network parameters will adapt to this change in scale during training.137 3 Method138 Our method can be seen as an extension of the original ChebNet [ Defferrard et al. , 2016 , Perraudin139 et al. , 2019 ] . Instead of directly working on a homogeneous base space , we first extend it to a higher140 dimensional space ( Lie group ) . The goal of this extension is to convert the previously invariant141 spectral convolutional layers into equivariant layers.2142 3.1 Anisotropic manifold graph143 In order to define the anisotropic manifold graphs we have to consider two types of manifolds . The144 base manifoldM and a Lie group G that acts transitively onM . The latter implies thatM is a145 homogeneous space of G , which means that any two points m1 , m2 ∈ M can be mapped to each146 other via the action of a group element g ∈ G via m2 = g · m1 . E.g. , the plane M = R2 is a147 homogeneous space of the special Euclidean motion group G = SE ( 2 ) as any two points can be148 mapped to each other through a rotation and a translation . Such groups G , which have M as a149 homogeneous space , can always be split in two parts via the semi-direct product G =MoH , with150 H a sub-group of G that leaves some reference point m0 ∈M invariant , i.e. , ∀h∈H : m0 = h ·m0.151 E.g. , rotations leave the zero vector inM = R2 invariant , and thus H = SO ( 2 ) in the SE ( 2 ) case.152 Conversely , any homogenous space can be modeled with a group quotientM = G/H .153 We define an anisotropic manifold graph to be a discretization of a Lie group G of whichM is a154 homogeneous space . It consists of a finite set of vertices corresponding to a random sampling of155 group elements , and a finite set of similarity-based edges that are constructed via a left-invariant156 Riemannian metric on G. In our work we consider two anisotropic manifold graphs : one associated157 with the base manifoldM = R2 which we extend with an additional orientation/rotation dimension158 H = SO ( 2 ) to come to the Lie group G = SE ( 2 ) = R2 o SO ( 2 ) , and the other associated with159 the sphereM = S2 which we similarly `` lift '' to the Lie group G = SO ( 3 ) by adding an additional160 rotation dimension . Considering the similarity between the two cases ( the sphere locally looks like161 R2 ) we will refer toM as the `` spatial '' part , and H as the `` orientation '' part of the group.162 Uniform sampling of the vertices . The first step to construct an anisotropic manifold graph is to163 sample elements on the group uniformly or as uniformly as possible if the manifold does not permit a164 uniform grid . We split the grid construction in two parts , a grid onM which is sampled with |Vs|165 points and a grid on H that is sampled with |Vo| points , leading to a total of |V| = |Vs||Vo| vertices.166 Left-invariant anisotropic Riemannian distance . Once vertices have been uniformly sampled167 on the group manifold , a similarity measure between vertices is computed . This measure is based168 on a Riemannian distance between points in G. The only thing one needs in our algorithm is the169 implementation of the logarithmic map on the Lie group ( see e.g . [ Bekkers , 2019 ] ) , and a diagonal170 Riemannian metric tensor ( see e.g . [ Sanguinetti et al. , 2015 ] and [ Mashtakov et al. , 2017 ] for the171 SE ( 2 ) and SO ( 3 ) case respectively ) . In the following we provide the essential idea and intuition172 behind the construction of the similarity measure and provide a more extensive treatment in App . B.173 In Riemannian geometry on Lie groups it is common to express tangent vectors of curves in a basis174 of left-invariant vector fields as it allows to measure their lengths with a single Riemannian metric175 tensor that is shared over the entire group . This works as follows . Consider curve γ : [ 0 , 1 ] → G176 with its tangent vectors γ̇ ( t ) = ∑d i=1 u i ( t ) Ai|γ ( t ) expressed in a basis/moving frame of reference177 2Because spectral graph NNs are able to capture the geometry of the space , which in this work we equip with anisotropic metrics , any spectral method could be made equivariant using our method . { Ai|γ ( t ) } di=1 , in which Ai are left-invariant vector fields . The length of these tangent vectors178 is then measured by a Riemannian metric tensor that we denote with ‖γ̇ ( t ) ‖2R : = u ( t ) TRu ( t ) ,179 with R a symmetric positive definite matrix defined relative to the basis { Ai|γ ( t ) } di=1 , and with180 u ( t ) = ( u0 ( t ) , u1 ( t ) , . . . ) T . The Ai are left-invariant vector fields and the notation Ai|g means the181 vector in the vector field Ai at location g. The vector fields are constructed by choosing a vector Ai182 in the tangent space at origin ( the Lie algebra ) which then defines a complete vector field on G via183 the push-forward of left-multiplication . In less technical terms this means that if we pick a direction184 vector at the origin , and we move it to another point in , e.g . G = SE ( 2 ) , via a roto-translation , this185 vector will move and rotate along . By defining everything in terms of these left-invariant vector fields,186 every tangent space Tg ( G ) at each g ∈ G can be identified with the tangent space at the origin , and187 a single Riemannian metric tensor R can be shared over the entire space . Moreover , the induced188 Riemannian distance d ( g , h ) between any two points g , h ∈ G is then by construction left-invariant,189 i.e. , ∀g , h , i∈G : d ( g · h , g · i ) = d ( g , i ) .190 Expressing tangent vectors in such left-invariant vector fields allows us to reason in terms of the191 generators of the group . Consider the G = SE ( 2 ) case . As a basis we pick the 3 generators of the192 group : a forward motion represented by a vector A1 pointing in the forward direction within the193 plane , a side-ways motion represented by a perpendicular planar vector A2 , and a rotation/change of194 orientation represented by a vector A3 that points vertically in along the H-dimension . We then work195 with diagonal Riemannian metric tensors R = diag ( 1 , −2 , ξ2 ) , which penalize each type of motion196 ( represented by the vector components ) differently . When → 0 one arrives at the sub-Riemannian197 geometry which forms the basis for the mathematical modeling of visual perception . It quantifies198 a notion of alignment through the sub-Riemannian distance ; the length of a distance-minimizing199 geodesic that connects two local orientations that lie in the extend of each other will be much smaller200 that that of a geodesic connecting two local orientations parallel to each other . An analogy can be201 found with the example of a car in a parking lot where it can move forward/backward ( A1 ) and202 change orientation ( A3 ) [ Reeds and Shepp , 1990 ] . It will be easier to move it to the more aligned203 spot directly ahead then it will to the spot next to the car , as sideways motion ( A2 ) is impossible.204 Parameters and ξ will respectively be referred to as spatial and orientation anisotropy parameters.205 With = 1 the metric is isotropic and there will be no distinction between different orientations.206 When < 1 , ξ determines the flexibilty/curvature of the geodesics as it balances spatial motion207 against angular motion . In a sense it defines how easily one connects local orientations that are208 not optimally aligned . In Figure 1 this behavior is visualized by running a diffusion process on the209 anisotropic manifold graph . In the anisotropic case ( < 1 ) diffusion is faster along the forward210 direction within a θ-plane . From a graph NN perspective this suggests that information is propagated211 more quickly between vertices that are aligned , nevertheless , Chow ’ s theorem ( see e.g . [ Montgomery,212 2006 ] ) guarantees that any point pair in the ( sub- ) Riemannian manfiold can interact with one another.213 The exact computation of the ( sub- ) Riemannian distances is challenging and can generally not be done214 in closed form , but can be done numerically via method such as [ Bekkers et al. , 2015 , Sanguinetti215 et al. , 2015 , Mashtakov et al. , 2017 ] . In order to keep our graph construction algorithm efficient216 though , we will approximate the Riemannian distances via an efficient analytic formula based on217 those in [ Bekkers et al. , 2018 ] that only involves the Lie group ’ s logarithmic map log : G→ Te ( G ) 218 and the Riemannian metric tensor R. We then approximate the distance between points g , h ∈ G by219 d ( g , h ) = d ( e , g−1 · h ) ' || log ( g−1 · h ) ||R . ( 3 ) Similarity measure . Encoding a similarity measure in the edges of a graph requires defining a220 weighting scheme . It is common to use a Gaussian kernel and set the weights via221 w ( vi , vj ) = { exp ( −d 2 ( vi , vj ) 4t ) if e ( vi , vj ) ∈ E 0 otherwise . ( 4 ) The choice for kernel bandwidth t is essentially arbitrary , but good heuristics exist . Perraudin et al.222 [ 2019 ] set it to half the average squared distance between connected vertices . Defferrard et al . [ 2020 ] ,223 however , showed that this heuristic has the tendency to overestimate it and preferred to choose it as224 the minimizer of the mean equivariance error . Following this overestimation observation , we fix the225 kernel bandwidth as 20 % of the average squared Riemannian distance between connected vertices.226 As such , the weights diversely cover values in the whole range [ 0 , 1 ] . The most similar vertices are227 connected with close-to-one weighted edges whereas the lowest connections are close to zero.228 Quality of the approximation . In theory , we would like our approximation to be as precise as229 possible . In practice , a high-resolution approximation leads to computational issues in time and230 memory . Hence , tuning of the graph parameters becomes a trade-off between theoretical consistency231 and practical feasibility . First of all , the graph resolution ( or the number of vertices we sample ) 232 is directly related to the quality of the approximation . While the spatial resolution |V|s is usually233 determined by the data ( up to up- and down-samplings ) , the orientation resolution |V|o is a design234 choice . An important remark is to notice that a large orientation resolution does not necessarily help235 if two different orientations are not distinguishable because of a poor spatial resolution [ Weiler et al.,236 2018 , Bekkers , 2019 ] . Secondly , the connectivity of the graph is also a crucial parameter . A fully237 connected graph is theoretically the best approximation . Nevertheless , for computational reasons , we238 use K-NN graphs3 to sparsify the graph Laplacians.239 Theoretical group equivariance of the graph Laplacian . Due to the success of machine learning240 algorithms based on graph Laplacian , the theoretical convergence of the graph Laplacian to its241 continuous analogue has been largely studied [ Hein et al. , 2005 , Singer , 2006 ] . Belkin and Niyogi242 [ 2006 ] noticed that in many graph-based algorithms , a central role is played by the graph Laplacian ’ s243 eigenvectors . Thus , they focused on proving convergence in eigenmaps as it is sufficient in this case.244 They proved that if the graph ’ s vertices are sampled uniformly from an unknown submanifoldM∈245 Rd , then the eigenvectors of a suitably constructed graph Laplacian converges to the eigenfunctions246 of the Laplace-Beltrami operator onM . Consequently , as the latter operator is left-invariant , as we247 show in theorem A.1 , the graph Laplacian is asymptotically 4 group equivariant.248 Empirical group equivariance of the graph Laplacian . We empirically confirm the group equiv-249 ariance property of the graph Laplacian applied to our anisotropic manifold graphs . By checking250 P > ∆̃P = ∆̃ where P is a permutation matrix , we can verify that the graph Laplacian is invariant251 under a given permutation of vertices corresponding to a group transformation ( e.g . a rotation of252 the graph ) . Moreover , we can also compare the eigenmaps of a graph Laplacian and its continuous253 counterpart if it is well-known . For a further discussion about this , see App . C.254 3.2 ChebLieNet255 Chebyshev convolutional layer . As introduced in Defferrard et al . [ 2016 ] , a Chebyshev convolu-256 tional layer is a spectral layer based on a continuous kernel parametrization with graph Laplacians.257 This parameterization makes such layers highly suitable for our method , as they intrinsically cap-258 ture the Riemannian geometry of the graphs on G. Moreover , the Chebyshev convolutions on259 the anisotropic manifold graphs are equivariant by construction because the graph Laplacians are260 equivariant operators ( see Figure 2 ) .261 3Note that in our implementation , a K-NN graphs does not mean that each vertex has K neighbors but at most K neighbors . Indeed , if the graph domain has boundaries , using exactly K neighbors for each vertex could lead to asymmetries that may introduce biases and harm the permutation invariances in the graph . 4The asymptotic case corresponds to |V| → ∞ and a Gaussian weight kernel with kernel bandwidth t→ 0 . Spatial pooling and unpooling layers . Graph pooling is a central component in a myriad of262 graph neural network architectures . Producing coarsened graphs from a finer graph have two main263 advantages : first , it reduces the computational cost , and second , it could improve performance by264 reducing the overfitting effect and adding a multiscale perspective . As an inheritance from traditional265 CNNs , most approaches formulate graph pooling as a cluster assignment problem , extending local266 patches ’ idea in regular grids to graphs [ Dhillon et al. , 2007 , Ying et al. , 2018 , Khasahmadi et al.,267 2020 , Mesquita et al. , 2020 ] . We propose similar operations on the base space ( spatial domain ) and268 involving two steps ( see Figure 3 ) . First , each sample is assigned to a cluster that will correspond to269 the output sample ; this is the down- ( resp . up- ) sampling phase . With a well designed method , this270 change of data-resolution can be made equivariant to any group transformation.5 Then , each cluster is271 reduced ( resp . expanded ) according to a given scheme ( e.g . maximum , average or random ) ; this is the272 reduction ( resp . expansion ) phase . When the reduction and expansion steps are permutation-invariant273 operations , such layers are automatically invariant under any transformation in the group.274 Global pooling ( projection ) layer and point-wise operations . When the neural network does275 not need to be equivariant but invariant ( e.g . classification task ) , it is common to rely on a global276 pooling layer ( or simply projection layer ) . This layer reduces the d-dimensional signal on the graph ’ s277 vertices to a d-dimensional vector of features derived from information on the whole graph . As a278 permutation-invariant operation , such a layer does not break the equivariance property of the neural279 network . Finally , point-wise operations do not affect the equivariance of a neural network.280 5Altough down- and up-samplings are naturally defined on the Euclidean grid , this task is more complicated on the sphere . However , using an icosahedron decomposition of the sphere , we make it more natural as downand up-sampling consists of decreasing or increasing the subdivision level . 4 Experiments281 In this section , we show the benefits of working on the anisotropic manifold graphs compared to the282 base manifold graphs . We believe that further improvements could be achieved through tuning and283 hyper-parameter optimization of the models [ Yu and Zhu , 2020 ] , using high-capacity networks , or via284 a more advanced training process , but this is not the goal of our work . We here intent to illustrate the285 adaptability of our approach to different tasks such as classification and segmentation in 2D images or286 spherical data . In the first couple of experiments , we motive the use of anisotropic spaces . By varying287 the anisotropies , we show the existence of sweet spots , both for the spatial anisotropy parameter 288 and the orientation anisotropy parameter ξ . In the second couple of experiments , we show that even289 if we add a new orientation dimension , our method remains scalable using a proper implementation.290 Our implementation is fully PyTorch [ Paszke et al. , 2019 ] and available at https : //anonymous.url.291 We perform all the experiments on a single GeForce GTX 1080 Ti gpu and track them with the292 Weights & Biases library [ Biewald , 2020 ] . The details of the experiments are given in the App . D.293 4.1 Why using tunable anisotropic kernels ? 294 As introduced in Section 3.1 , the anisotropies are tunable via the parameters and ξ of the Riemannian295 metric , respectively responsible for the spatial and orientation anisotropies . As the ξ parameter should296 depend on the spatial and orientation resolutions , we use the following parameterisation : ξ2 = α |Vo||Vs| .297 Setting α = 1 yields a 40/60 ratio of neighbors within versus outside the orientation plane . We298 ran different experiments with a Wide Residual architecture [ Zagoruyko and Komodakis , 2016 ] on299 CIFAR10 [ Krizhevsky et al. , 2009 ] , varying the spatial and orientation anisotropic parameters.300 Orientation anisotropy . The orientation anisotropy ξ controls how strongly orientation layers are301 connected . At the limit ξ →∞ , orientation layers are decoupled . It is like test-time augmentation302 with rotations : running a CNN working with one anisotropic Laplacian ( e.g. , only vertically aligned303 filters ) and testing the network for different input rotations before averaging the output . The other304 extreme ξ → 0 keeps all layers equally close to each other , and features are essentially identified with305 just a spatial coordinate . This would then correspond to a WideResNet with isotropic Chebyshev306 convolutions . For reasonable values of ξ , interactions between orientation layers take place . Figure307 4a is evidence of the existence of a sweet spot for this parameter in the range of reasonable values . At308 the moment , we expect with no certainty that this parameter could be set a priori of the data , only309 considering the data resolution . As a rule of thumb , we set ξ such that each vertex has approximately310 40 % of its neighbors in the same orientation layer and 60 % on others.311 Spatial anisotropy . The spatial anisotropy regulates the anisotropy of the space on the spatial312 domain . For = 1 , the Riemannian metric is spatially isotropic ; all directions are treated equally313 and the resulting model would effectively be a WideResNet with isotropic Chebyshev convolutions.314 At the limit → 0 , the main direction has a minimal cost , and the resulting space is highly spatially315 anisotropic . In figure 4a we observe that using anisotropic spaces instead of isotropic ones is relevant,316 as we almost get an 8 % test-accuracy improvement . Unlike the orientation anisotropic parameter,317 in our opinion , this parameter is task/data-dependent ; different datasets could benefit in different318 degrees from the utilization of directional information through different spatial anisotropy settings.319 4.2 How scalable is the method ? 320 Scalability is often an important limitation of graph- and group-based neural networks . By adding321 an orientation dimension , we do not run from this rule as we necessarily increase the number of322 vertices of the anisotropic manifold graphs . To permit experiments on larger images , it becomes323 crucial to pre-compute anisotropic manifold graphs and their Laplacians . Dedicated librairies like324 PyKeops [ Charlier et al. , 2020 ] enable this without memory issues . Nevertheless , the graph operations325 ( convolutions , pooling or unpooling ) still scale with the size of the graph . Fortunately , PyTorch326 provides sparse operations that increase efficiency in terms of time and memory compared to dense327 operations in cases of sufficiently sparse graph Laplacians ( typically a sparsity S ( ∆̃ ) ≥ 98.5 % ) .328 We evaluate our models on an image classification task on STL10 [ Coates et al. , 2011 ] and an image329 segmentation task on ClimateNet [ Kashinath et al. , 2021 ] . We show the adaptability of our method by330 using a Wide Residual architecture [ Zagoruyko and Komodakis , 2016 ] on STL10 and a U-Net-like331 network [ Ronneberger et al. , 2015 ] on ClimateNet . We also demonstrate the potential of our approach332 and the benefits of using anisotropic spaces . Indeed , while on ClimateNet the use of anisotropies is333 neither beneficial nor detrimental , the difference in performance on STL10 is significant.334 5 Conclusion335 Scope . With our method , geometric graph NNs are made equivariant to Lie groups . Via the groups336 SE ( 2 ) and SE ( 3 ) , we can construct roto-translation equivariant networks for 2D image data and 3D337 volumetric data . Based on the group SO ( 3 ) , our method can deal with meteorological or cosmological338 data while preserving rotation equivariance . We believe that our flexible approach is ideal for further339 explorations on the relevance of group equivariance in tasks not considered in this work.340 Limitations . The main weakness of our method is its relatively high memory requirement . Al-341 though all experiments ran on a single gpu , by adding an orientation axis , we significantly enlarge the342 feature maps . As a result , anisotropic graph manifolds are memory-heavier than isotropic ones and343 prone to a slowdown during the forward- and backward-pass . Nevertheless , with the emergence of ge-344 ometric deep learning , we expect improvement in the hardware and implementation of graph-oriented345 operations . Another challenge is the increased number of hyper-parameters for which we only have346 derived rules of thumb . The graph connectivity and resolutions require a tradeoff between efficiency347 and quality of the manifold approximation . The anisotropic parameters require an analysis of the348 dataset and some intuition about the amount of anisotropy to set . With systematic hyper-parameter349 optimization , we can find an optimal combination , but requires more computational resources.350 Potential and future research . Thanks to its easy-to-tune anisotropic properties , our model can be351 used to better understand anisotropic properties in data . In particular , one could explore the effect of352 using anisotropic spaces instead of isotropic ones on many tasks and conclude when such anisotropic353 information is relevant . In this vein , it could also be interesting to derive anisotropic pooling and354 unpooling layers based on anisotropic spaces instead of isotropic ones as it is usually done . More355 generally , our method is simple enough to be extended to shapes/surfaces with a Riemannian manifold356 structure [ Cohen et al. , 2019 ] . In this work , we focused on 2D images and spherical data on , but the357 method is readily extendable to higher dimensional Lie groups such as the SE ( 3 ) group to obtain358 3D roto-translation equivariant ChebLieNets . Moreover , our method for constructing anisotropic359 geometries could directly improve other successful Euclidean distance-based graph NNs such as360 [ Satorras et al. , 2021 ] by making them fully equivariant . Last but not least , despite graph-based361 algorithms being computationally sub-optimal compared to CNNs , their flexibility is a real asset . We362 see high potential in the exploration of graph sparsification to reduce computational complexity.363 References364 Emre Baspinar , Luca Calatroni , Valentina Franceschi , and Dario Prandi . A cortical-inspired sub-365 riemannian model for poggendorff-type visual illusions . Journal of Imaging , 7 ( 3 ) :41 , 2021.366 John R Baumgardner and Paul O Frederickson . Icosahedral discretization of the two-sphere . SIAM367 Journal on Numerical Analysis , 22 ( 6 ) :1107–1115 , 1985.368 Erik J Bekkers . Retinal image analysis using sub-riemannian geometry in se ( 2 ) . 2017.369 Erik J Bekkers . B-spline cnns on lie groups . In International Conference on Learning Representations,370 2019.371 Erik J Bekkers , Remco Duits , Alexey P Mashtakov , and Gonzalo R Sanguinetti . A PDE Approach372 to Data-Driven Sub-Riemannian Geodesics in SE ( 2 ) . SIAM Journal on Imaging Sciences , 8 ( 4 ) :373 2740–2770 , 2015. doi : 10.1137/15M1018460 . URL https : //doi.org/10.1137/15M1018460.374 Erik J Bekkers , Da Chen , and Jorg M Portegies . Nilpotent approximations of sub-riemannian375 distances for fast perceptual grouping of blood vessels in 2d and 3d . Journal of mathematical376 imaging and vision , 60 ( 6 ) :882–899 , 2018.377 Mikhail Belkin and Partha Niyogi . Convergence of laplacian eigenmaps . Advances in neural378 information processing systems , 19:129–136 , 2006.379 Lukas Biewald . Experiment tracking with weights and biases , 2020 . URL https : //www.wandb.380 com/ . Software available from wandb.com.381 Ugo V Boscain , Roman Chertovskih , Jean-Paul Gauthier , Dario Prandi , and Alexey Remizov . Highly382 corrupted image inpainting through hypoelliptic diffusion . Journal of Mathematical Imaging and383 Vision , 60 ( 8 ) :1231–1245 , 2018.384 Davide Boscaini , Jonathan Masci , Emanuele Rodoià , and Michael Bronstein . Learning shape385 correspondence with anisotropic convolutional neural networks . In Proceedings of the 30th386 International Conference on Neural Information Processing Systems , pages 3197–3205 , 2016.387 William H Bosking , Ying Zhang , Brett Schofield , and David Fitzpatrick . Orientation selectivity and388 the arrangement of horizontal connections in tree shrew striate cortex . Journal of neuroscience , 17389 ( 6 ) :2112–2127 , 1997.390 Michael M Bronstein , Joan Bruna , Yann LeCun , Arthur Szlam , and Pierre Vandergheynst . Geometric391 deep learning : going beyond euclidean data . IEEE Signal Processing Magazine , 34 ( 4 ) :18–42,392 2017.393 Michael M Bronstein , Joan Bruna , Taco Cohen , and Petar Veličković . Geometric deep learning:394 Grids , groups , graphs , geodesics , and gauges . arXiv preprint arXiv:2104.13478 , 2021.395 Joan Bruna , Wojciech Zaremba , Arthur Szlam , and Yann LeCun . Spectral networks and locally396 connected networks on graphs . arXiv preprint arXiv:1312.6203 , 2013.397 Benjamin Charlier , Jean Feydy , Joan Alexis Glaunès , François-David Collin , and Ghislain Du-398 rif . Kernel operations on the gpu , with autodiff , without memory overflows . arXiv preprint399 arXiv:2004.11127 , 2020.400 Fan RK Chung and Fan Chung Graham . Spectral graph theory . Number 92 . American Mathematical401 Soc. , 1997.402 Giovanna Citti and Alessandro Sarti . A cortical based model of perceptual completion in the403 roto-translation space . Journal of Mathematical Imaging and Vision , 24 ( 3 ) :307–326 , 2006.404 Adam Coates , Andrew Ng , and Honglak Lee . An analysis of single-layer networks in unsupervised405 feature learning . In Proceedings of the fourteenth international conference on artificial intelligence406 and statistics , pages 215–223 . JMLR Workshop and Conference Proceedings , 2011.407 Taco Cohen and Max Welling . Group equivariant convolutional networks . In International conference408 on machine learning , pages 2990–2999 , 2016.409 Taco Cohen , Mario Geiger , and Maurice Weiler . A general theory of equivariant cnns on homogeneous410 spaces . arXiv preprint arXiv:1811.02017 , 2018.411 Taco S Cohen , Maurice Weiler , Berkay Kicanaoglu , and Max Welling . Gauge equivariant convolu-412 tional networks and the icosahedral cnn . arXiv preprint arXiv:1902.04615 , 2019.413 Michaël Defferrard , Xavier Bresson , and Pierre Vandergheynst . Convolutional neural networks on414 graphs with fast localized spectral filtering . In Advances in neural information processing systems,415 pages 3844–3852 , 2016.416 Michaël Defferrard , Martino Milani , Frédérick Gusset , and Nathanaël Perraudin . Deepsphere : a417 graph-based spherical cnn . arXiv preprint arXiv:2012.15000 , 2020.418 Inderjit S Dhillon , Yuqiang Guan , and Brian Kulis . Weighted graph cuts without eigenvectors a419 multilevel approach . IEEE transactions on pattern analysis and machine intelligence , 29 ( 11 ) :420 1944–1957 , 2007.421 James R Driscoll and Dennis M Healy . Computing fourier transforms and convolutions on the422 2-sphere . Advances in applied mathematics , 15 ( 2 ) :202–250 , 1994.423 R. Duits , S. P. L. Meesters , J.-M. Mirebeau , and J. M. Portegies . Optimal Paths for Variants of424 the 2d and 3d Reeds–Shepp Car with Applications in Image Analysis . Journal of Mathematical425 Imaging and Vision , February 2018 . ISSN 1573-7683. doi : 10.1007/s10851-018-0795-z . URL426 https : //doi.org/10.1007/s10851-018-0795-z.427 Remco Duits , Ugo Boscain , Francesco Rossi , and Yuri Sachkov . Association fields via cuspless428 sub-Riemannian geodesics in SE ( 2 ) . Journal of mathematical imaging and vision , 49 ( 2 ) :384–417,429 2014.430 Marta Favali , Samaneh Abbasi-Sureshjani , Bart ter Haar Romeny , and Alessandro Sarti . Analysis431 of vessel connectivities in retinal images by cortically inspired spectral clustering . Journal of432 Mathematical Imaging and Vision , 56 ( 1 ) :158–172 , 2016.433 Marc Finzi , Samuel Stanton , Pavel Izmailov , and Andrew Gordon Wilson . Generalizing convolutional434 neural networks for equivariance to lie groups on arbitrary continuous data . In International435 Conference on Machine Learning , pages 3165–3176 . PMLR , 2020.436 Erik Franken and Remco Duits . Crossing-preserving coherence-enhancing diffusion on invertible437 orientation scores . International Journal of Computer Vision , 85 ( 3 ) :253 , 2009.438 Fabian B Fuchs , Edward Wagstaff , Justas Dauparas , and Ingmar Posner . Iterative se ( 3 ) -transformers.439 arXiv preprint arXiv:2102.13419 , 2021.440 Ian Goodfellow , Yoshua Bengio , Aaron Courville , and Yoshua Bengio . Deep learning , volume 1.441 MIT press Cambridge , 2016.442 Krzysztof M Gorski , Eric Hivon , Anthony J Banday , Benjamin D Wandelt , Frode K Hansen , Mstvos443 Reinecke , and Matthia Bartelmann . Healpix : A framework for high-resolution discretization and444 fast analysis of data distributed on the sphere . The Astrophysical Journal , 622 ( 2 ) :759 , 2005.445 Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . Delving deep into rectifiers : Surpassing446 human-level performance on imagenet classification . In Proceedings of the IEEE international447 conference on computer vision , pages 1026–1034 , 2015.448 Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . Deep residual learning for image449 recognition . In Proceedings of the IEEE conference on computer vision and pattern recognition,450 pages 770–778 , 2016.451 Matthias Hein , Jean-Yves Audibert , and Ulrike Von Luxburg . From graphs to manifolds–weak and452 strong pointwise consistency of graph laplacians . In International Conference on Computational453 Learning Theory , pages 470–485 . Springer , 2005.454 Mikael Henaff , Joan Bruna , and Yann LeCun . Deep convolutional networks on graph-structured data.455 arXiv preprint arXiv:1506.05163 , 2015.456 David H Hubel and Torsten N Wiesel . Receptive fields , binocular interaction and functional architec-457 ture in the cat ’ s visual cortex . The Journal of physiology , 160 ( 1 ) :106–154 , 1962.458 Sergey Ioffe and Christian Szegedy . Batch normalization : Accelerating deep network training by459 reducing internal covariate shift . arXiv preprint arXiv:1502.03167 , 2015.460 John Jumper , R Evans , A Pritzel , T Green , M Figurnov , K Tunyasuvunakool , O Ronneberger , R Bates,461 A Zidek , A Bridgland , et al . High accuracy protein structure prediction using deep learning.462 Fourteenth Critical Assessment of Techniques for Protein Structure Prediction ( Abstract Book ) , 22:463 24 , 2020.464 Karthik Kashinath , Mayur Mudigonda , Sol Kim , Lukas Kapp-Schwoerer , Andre Graubner , Ege465 Karaismailoglu , Leo von Kleist , Thorsten Kurth , Annette Greiner , Ankur Mahesh , et al . Climatenet:466 an expert-labeled open dataset and deep learning architecture for enabling high-precision analyses467 of extreme weather . Geoscientific Model Development , 14 ( 1 ) :107–124 , 2021.468 Nicolas Keriven , Alberto Bietti , and Samuel Vaiter . Convergence and stability of graph convolutional469 networks on large random graphs . arXiv preprint arXiv:2006.01868 , 2020.470 Amir Hosein Khasahmadi , Kaveh Hassani , Parsa Moradi , Leo Lee , and Quaid Morris . Memory-based471 graph networks . arXiv preprint arXiv:2002.09518 , 2020.472 Diederik P Kingma and Jimmy Ba . Adam : A method for stochastic optimization . arXiv preprint473 arXiv:1412.6980 , 2014.474 Thomas N Kipf and Max Welling . Semi-supervised classification with graph convolutional networks.475 arXiv preprint arXiv:1609.02907 , 2016.476 Risi Kondor and Shubhendu Trivedi . On the generalization of equivariance and convolution in neural477 networks to the action of compact groups . In International Conference on Machine Learning,478 pages 2747–2755 . PMLR , 2018.479 Alex Krizhevsky , Geoffrey Hinton , et al . Learning multiple layers of features from tiny images . 2009.480 Alex Krizhevsky , Ilya Sutskever , and Geoffrey E Hinton . Imagenet classification with deep con-481 volutional neural networks . Advances in neural information processing systems , 25:1097–1105,482 2012.483 Yann LeCun and Corinna Cortes . MNIST handwritten digit database . 2010 . URL http : //yann.484 lecun.com/exdb/mnist/.485 Yann LeCun , Yoshua Bengio , et al . Convolutional networks for images , speech , and time series . The486 handbook of brain theory and neural networks , 3361 ( 10 ) :1995 , 1995.487 Karel Lenc and Andrea Vedaldi . Understanding image representations by measuring their equivariance488 and equivalence . In Proceedings of the IEEE conference on computer vision and pattern recognition,489 pages 991–999 , 2015.490 Jonathan Masci , Davide Boscaini , Michael Bronstein , and Pierre Vandergheynst . Geodesic con-491 volutional neural networks on riemannian manifolds . In Proceedings of the IEEE international492 conference on computer vision workshops , pages 37–45 , 2015.493 Alexey Mashtakov , Remco Duits , Yu Sachkov , Erik J Bekkers , and Ivan Beschastnyi . Tracking of494 lines in spherical images via sub-riemannian geodesics in SO ( 3 ) . Journal of mathematical imaging495 and vision , 58 ( 2 ) :239–264 , 2017.496 Diego Mesquita , Amauri H Souza , and Samuel Kaski . Rethinking pooling in graph neural networks.497 arXiv preprint arXiv:2010.11418 , 2020.498 Richard Montgomery . A tour of sub-Riemannian geometries , their geodesics and applications.499 Number 91 . American Mathematical Soc. , 2006.500 Federico Monti , Davide Boscaini , Jonathan Masci , Emanuele Rodola , Jan Svoboda , and Michael M501 Bronstein . Geometric deep learning on graphs and manifolds using mixture model cnns . In502 Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5115–5124,503 2017.504 Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor505 Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , et al . Pytorch : An imperative style,506 high-performance deep learning library . In Advances in neural information processing systems,507 pages 8026–8037 , 2019.508 Nathanaël Perraudin , Michaël Defferrard , Tomasz Kacprzak , and Raphael Sgier . Deepsphere : Effi-509 cient spherical convolutional neural network with healpix sampling for cosmological applications.510 Astronomy and Computing , 27:130–146 , 2019.511 Jean Petitot . The neurogeometry of pinwheels as a sub-Riemannian contact structure . Journal of512 Physiology-Paris , 97 ( 2-3 ) :265–309 , 2003.513 Jorg Portegies , Gonzalo Sanguinetti , Stephan Meesters , and Remco Duits . New approximation of514 a scale space kernel on se ( 3 ) and applications in neuroimaging . In International Conference on515 Scale Space and Variational Methods in Computer Vision , pages 40–52 . Springer , 2015.516 James Reeds and Lawrence Shepp . Optimal paths for a car that goes both forwards and backwards.517 Pacific journal of mathematics , 145 ( 2 ) :367–393 , 1990.518 Olinde Rodrigues . Des lois géométriques qui régissent les déplacements d ’ un système solide519 dans l ’ espace : et de la variation des cordonnées provenant de ces déplacements considérés520 indépendamment des causes qui peuvent les produire . 1840.521 Olaf Ronneberger , Philipp Fischer , and Thomas Brox . U-net : Convolutional networks for biomedical522 image segmentation . In International Conference on Medical image computing and computer-523 assisted intervention , pages 234–241 . Springer , 2015.524 Gonzalo Sanguinetti , Erik Bekkers , Remco Duits , Michiel HJ Janssen , Alexey Mashtakov , and525 Jean-Marie Mirebeau . Sub-riemannian fast marching in se ( 2 ) . In Iberoamerican Congress on526 Pattern Recognition , pages 366–374 . Springer , 2015.527 Victor Garcia Satorras , Emiel Hoogeboom , and Max Welling . E ( n ) equivariant graph neural networks.528 arXiv preprint arXiv:2102.09844 , 2021.529 Franco Scarselli , Marco Gori , Ah Chung Tsoi , Markus Hagenbuchner , and Gabriele Monfardini . The530 graph neural network model . IEEE transactions on neural networks , 20 ( 1 ) :61–80 , 2008.531 David I Shuman , Sunil K Narang , Pascal Frossard , Antonio Ortega , and Pierre Vandergheynst.532 The emerging field of signal processing on graphs : Extending high-dimensional data analysis to533 networks and other irregular domains . IEEE signal processing magazine , 30 ( 3 ) :83–98 , 2013.534 Amit Singer . From graph to manifold laplacian : The convergence rate . Applied and Computational535 Harmonic Analysis , 21 ( 1 ) :128–134 , 2006.536 Maurice Weiler , Fred A Hamprecht , and Martin Storath . Learning Steerable Filters for Rotation537 Equivariant CNNs . In International Conference on Computer Vision and Pattern Recognition,538 2018.539 Douglas Brent West et al . Introduction to graph theory , volume 2 . Prentice hall Upper Saddle River,540 NJ , 1996.541 Rex Ying , Jiaxuan You , Christopher Morris , Xiang Ren , William L Hamilton , and Jure Leskovec . Hier-542 archical graph representation learning with differentiable pooling . arXiv preprint arXiv:1806.08804,543 2018.544 Tong Yu and Hong Zhu . Hyper-parameter optimization : A review of algorithms and applications.545 arXiv preprint arXiv:2003.05689 , 2020.546 Sergey Zagoruyko and Nikos Komodakis . Wide residual networks . arXiv preprint arXiv:1605.07146,547 2016.548 Checklist549 1 . For all authors ... 550 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s551 contributions and scope ? [ Yes ] See Section 1552 ( b ) Did you describe the limitations of your work ? [ Yes ] See Section 5553 ( c ) Did you discuss any potential negative societal impacts of your work ? [ Yes ] The554 environmental impact is a direct consequence of the time and memory issues we555 discussed in Section 5.556 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to557 them ? [ Yes ] 558 2 . If you are including theoretical results ... 559 ( a ) Did you state the full set of assumptions of all theoretical results ? [ Yes ] See Section 3560 ( b ) Did you include complete proofs of all theoretical results ? [ Yes ] We refer the reader to561 original publication with proofs.562 3 . If you ran experiments ... 563 ( a ) Did you include the code , data , and instructions needed to reproduce the main experi-564 mental results ( either in the supplemental material or as a URL ) ? [ Yes ] See Section 3565 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they566 were chosen ) ? [ Yes ] See Section 4567 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running experi-568 ments multiple times ) ? [ Yes ] See Section 4569 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type570 of GPUs , internal cluster , or cloud provider ) ? [ Yes ] See Section 4571 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 572 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] See Section 4573 ( b ) Did you mention the license of the assets ? [ Yes ] We always refered to the original574 papers of the datasets we used.575 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] 576 See Section 4 for the URL . We also send a zip file containing the whole implementation.577 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re578 using/curating ? [ N/A ] 579 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable580 information or offensive content ? [ N/A ] 581 5 . If you used crowdsourcing or conducted research with human subjects ... 582 ( a ) Did you include the full text of instructions given to participants and screenshots , if583 applicable ? [ N/A ] 584 ( b ) Did you describe any potential participant risks , with links to Institutional Review585 Board ( IRB ) approvals , if applicable ? [ N/A ] 586 ( c ) Did you include the estimated hourly wage paid to participants and the total amount587 spent on participant compensation ? [ N/A ] 588

Summary: In the paper, a new graph neural network called ChebLieNet was proposed. The proposed network was designed to achieve equivariance properties with respect to actions of Lie groups. The proposed ChebLieNet was analyzed on vanilla datasets.  In summary, the proposed ChebLieNet was well motivated and structured. However, there are various major and minor problems with the paper including missing notation, definitions, incomplete and insufficient mathematical results and experimental analyses, especially in comparison with the state-of-the-art.  After Rebuttal: Thank you for your response to my comments. I checked the comments of other reviewers and the response to their comments as well. The authors answered most of the questions in the rebuttal. However, the paper should be improved for a clear acceptance. Therefore, I slightly increase my grade, and recommend authors address the theoretical details and improve the experimental analyses, esp. in comparison with state-of-the-art related methods..