Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
Authors: Pim De Haan
Maurice Weiler
Taco Cohen
Max Welling
Venue: ICLR 2021 Spotlight (2021)
Keywords: symmetry
equivariance
mesh
geometric
convolution

Abstract: A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs).  Such GCNs utilize isotropic kernels and are therefore insensitive to the relative orientation of vertices and thus to the geometry of the mesh as a whole. We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels. Since the resulting features carry orientation information, we introduce a geometric message passing scheme defined by parallel transporting features over mesh edges. Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.

Published as a conference paper at ICLR 2021
GAUGE EQUIVARIANT MESH CNNS ANISOTROPIC CONVOLUTIONS ON GEOMETRIC GRAPHS
Pim de Haan
Maurice Weiler
Taco Cohen
Qualcomm Al Research
QUVA Lab
Qualcomm Al Research
University of Amsterdam
University of Amsterdam
Max Welling
Qualcomm Al Research University of Amsterdam
ABSTRACT
A common approach define convolutions on meshes is to interpret them as graph and apply graph convolutional networks (GCNs). Such GCNs utilize isotropic kernels and are therefore insensitive to the relative orientation of vertices and thus to the geometry of the mesh whole. We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels. Since the resulting features carry orientation information, introduce geometric message passing scheme defined by parallel transporting features over mesh edges. Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.
INTRODUCTION
Convolutional neural networks (CNNs) have been established the default method for many machine learning tasks like speech recognition or planar and volumetric image classification and segmentation. Most CNNs are restricted to flat or spherical geometries, where convolutions are easily defined and optimized implementations generated interest to generalize convolutions to more general spaces like graphs or Riemannian manifolds, creating field now known as geometric deep learning (Bronstein al.. 2017).
available. The empirical success of CNNs such spaces has
case of specific interest is convolution on meshes, the discrete analog of 2-dimensional embedded Riemannian manifolds. Mesh CNNs can be applied to tasks such detecting shapes, registering different poses the same shape and shape segmentation. If we forget the positions of vertices, and which vertices form faces, mesh M can be represented by graph This allows for the application of graph convolutional networks (GCNs) to processing signals on meshes.
"Equal Contribution
QQallomm AI Research is initiative of Qualcomm Technologies, Inc.
P
41
Figure 1: Two local neighbourhoods around vertices p and their representations in the tangent plancs T,M. The distinct geometry the neighbourhoods is reflected in the different angles 6pu, of incident edges from neighbours qi Graph convolutional networks apply isotropic kernels can therefore not distinguish both neighbourhoods. Gauge Equivariant Mesh CNNs apply anisotropic kernels and are therefore sensitive to orientations. The arbitrariness of reference orientations, determined by choice of neighbour go. is accounted for by the gauge equivariance of the model.

---

Published as a conference paper at ICLR 2021
However, when representing mesh by a graph, we lose important geometrical information. In particular, in a graph there is no notion of angle between or ordering of two of a node's 's incident edges (see figure DT. Hence, GCNs output at node p is designed to be independent of relative angles and invariant to any permutation of its neighbours qi E WN(p). (p). A graph convolution on a mesh graph therefore corresponds applying an isotropic convolution kernel. Isotropic filters are insensitive to the orientation of input patterns, so their features are strictly less expressive than those of orientation aware anisotropic filters.
To address this limitation of graph networks we propose Gauge Equivariant Mesh CNNs (GEM-CNN), which minimally modify GCNs such that they are able to use anisotropic filters while sharing weights across different positions and respecting the local geometry. One obstacle in sharing anisotropic kernels, which are functions of the angle pg of neighbour y with respect to vertex p, over multiple vertices of mesh is that there is no unique way of selecting reference neighbour qo. which has the direction pgo O. The reference neighbour, and hence the orientation of the neighbours, needs to be chosen arbitrarily. In order to guarantee the equivalence of the features resulting from different choices of orientations, we adapt Gauge Equivariant CNNs (Cohen et al., 2019b) to general meshes. The kernels of our model are thus designed to be equivariant under gauge transformations, that is, to guarantee that the responses for different kernel orientations are related by a prespecified transformation law. Such features are identified as geometric objects like scalars, vectors, tensors, etc., depending on the specific choice of transformation law. In order to compare such geometric features at neighbouring vertices, they need be parallel transported along the connecting edge.
In our implementation we first specify the transformation laws of the feature spaces and compute a space of gauge equivariant kernels. Then we pick arbitrary reference orientations at each node, relative to which we compute neighbour orientations and compute the corresponding edge transporters. Given these quantities, we define the forward pass as a message passing step via edge transporters followed by a contraction with the equivariant kernels evaluated at the neighbour orientations. Algorithmically, Gauge Equivariant Mesh CNNs are therefore just GCNs with anisotropic, gauge equivariant kernels and message passing via parallel transporters. Conventional GCNs are covered in this framework for the specific choice of isotoopic kernels and trivial edge transporters, given by identity maps.
In Sec. 2 we will give outline of our method, deferring details to Secs. B In Sec. 3.2 we describe how to compute general geometric quantities, not specific to our method, used for the computation of the convolution. In experiments in Sec. 6.1 we find that the enhanced expressiveness of Gauge Equivariant Mesh CNNs enables them to outperform conventional GCNs and other prior work in shape correspondence task.
2 CONVOLUTIONS ON GRAPHS WITH GEOMETRY
We consider the problem of processing signals on discrete 2-dimensional manifolds, or meshes M. Such meshes are deseribed by set of vertices in R&3 together with a set F of tuples, each consisting of the vertices at the corners of face. For a mesh to describe proper manifold, each edge needs to be connected to two faces, and the neighbourhood of each vertex needs to be homeomorphic to disk. Mesh M induces a graph G by forgetting the coordinates of the vertices while preserving the edges.
conventional graph convolution between kernel and signal f, evaluated at a vertex p, can be defined by
(K f)p K + E gEN, K Ku
where N₂ is the set of neighbours of p in G, and Kger E CC and Kneigh E RCaxCoo are two linear maps which model a self interaction and the neighbour contribution, respectively. Importantly, graph convolution does not distinguish different neighbours, because each feature vector fg is multiplied by the same matrix Knsigh and then summed. For this reason we say the kernel is isotropic.
Consider the example in figure [[ where on the left and right, the neighbourhood of one vertex p. containing neighbours g E NNp. is visualized. An isotropic kernel would propagate the signal from the neighbours to p in exactly the same way in both neighbourhoods, even though the neighbourhoods geometrically distinct. For this reason, our method uses direction sensitive (anisotropic) kemels instead of isotropic kernels. Anisotropic kernels are inherently more expressive than isotropic ones which is why they used universally in conventional planar CNNs.
2

---

Published as a conference paper at ICLR 2021
Algorithm 1 Gauge Equivariant Mesh CNN layer
Input: mesh M, input/output feature types Pin- Poot. reference neighbours (of E Np) Compute basis kernels Katt- Hasigh (0)
D Sec. 3
Initialise weights waer and w Wheigh- For each neighbour pair, p E M. q N.::
D> App.
compute neighbor angles Bgg relative to reference neighbor compute parallel transporters 9e-p
Forward (input features (Sp)peлr, weights w mhngh):
fo + Ei " Karfs ",igee,,nnen K neigh (Pgg)PPm88 Pin(30 +p)f
We propose the Gauge Equivariant Mesh Convolution, a minimal modification of graph convolution that allows for anisotropic kernels (() whose value depends on an orientation 0 [0, 2x)) To define the orientations 8oy of neighbouring vertices E NN of p. we first map them to the tangent plane TpM at p, visualized in figure 1 We then pick arbitrary reference neighbour of to determine reference orientation 8 pug := 0, marked orange in figure I This induces a basis the tangent plane, which, when expressed in polar coordinates, defines the angles 0pg of the other neighbours.
As we will motivate in the next section, features in Gauge Equivariant CNN are coefficients of geometric quantities. For example, a tangent vector at vertex p can be described either geometrically by 3 dimensional vector orthogonal to the normal p by two coefficients in the basis the tangent plane. In order to perform convolution, geometric features at different vertices need to be linearly combined, for which it is required first. "parallel transport" the features to the same vertex. This is done by applying matrix p(9q=t) E IR CCa to the coefficients of the feature at in order obtain the coefficients of the feature vector transported to p, which can be used for the convolution at The transporter depends on the geometric type (group representation) of the feature, denoted by p and described in more detail below. Details of how the tangent space is defined, how to compute the map to the tangent space, angles 8yg, and the parallel transporter are given in Appendix A
In combination, this leads to the GEM-CNN convolution
(K f) = -Kettfp f + EyEN, (0x)gn(0p
(2)
which differs from the conventional graph convolution, defined in Eq. only by the use of an anisotropic kernel and the parallel transport message passing.
We require the outcome of the convolution to be equivalent for any choice of reference orientation. This is not the case for any anisotropic kernel but only for those which are equivariant under changes of reference orientations (gauge consformations). Equivariance imposes linear constraint on the kernels. We therefore solve for complete sets of "basis-kerelss" Kiar and Kneigh satisfying this constraint and linearly combine them with parameters w'etr and w Whigh such that K Σ, w K
and Kneigh = E. Whniigh Ki Khiighh Details on the computation basis kernels are given in section 3 The full algorithm for initialisation and forward pass, which is of time and space complexity linear in the number of vertices, for a GEM-CNN layer are listed in algorithm Gradients can be computed by automatic differentiation.
The GEM-CNN is gauge equivariant, but furthermore satisfies two important properties. Firstly, depends only on the intrinsic shape of the 2D mesh not on the embedding of the mesh in R3 Secondly, whenever map from the mesh to itself exists that preserves distances and orientation, the convolution is equivariant to moving the signal along such transformations. These properties are proven in Appendix D and empirically shown in Appendix F2
IIn principle, the kernel could be made dependent the radial distance of neighboring nodes, by Kneish (r, 8) Kaegas where F(r) is unconstrained and Kneaa (9) as presented this paper. As this dependency did not improve the performance in our empirical evaluation, we omit it. Mathematically, this corresponds choice of local reference frame or gauge.
3

---

Published as a conference paper at ICLR 2021
CONY gzuge
CURO gauge
12 3hhnd
pick gauge
map
8 o Coweeaaaeoe afard
pack gauge 9F
01 wwwpaaaaaaan a uifud
9.A
82 1-1 Moggei www.gnagggnnn......
QL
geomenic COIN
goometiic CURY
q.A
QA
qA
4C
pick
back mush
pick 00
sncill
50
W
CODY gange
CORV
UIA
97A
Euage IJ
(a) Convolution from scalar scalar features.
(b) Convolution from scalar vector features.
Visualization of the Gauge Equivariant Mesh Convolution two configurations, scalar to scalar and scalar to vector. The convolution operates in a gauge, so that vectors are expressed in coefficients in basis and neighbours have polar coordinates, but can also be seen as geometric convolution, a gauge- gouge-independen map from an input signal on the mesh to output signal the mesh. The convolution is equivariant if this geometric convolution does not depend on the intermediate chosen gauge, if the diagram commutes.
Figure 2:
3 GAUGE EQUIVARIANCE & GEOMETRIC FEATURES
On a general mesh, the choice of the reference neighbour, gauge, which defines the orientation of the kernel, can only be made arbitrarily. However, this choice should not arbitrarily affect the outcome of the convolution, as this would impede the generalization between different locations and different meshes. Instead, Gauge Equivariant Mesh CNNs have the property that their output known rule the gauge changes.
transforms according
Consider the left hand side of figure 2(a) Given a neighbourhood of vertex p. we want to express each neighbour q in terms of its polar coordinates (ry.0,) on the tangent plane, that the kernel value at that neighbour neiggh (0,) is well defined. This requires choosing a basis on the tangent plane, determined by picking a neighbour as reference neighbour (denoted go), which has the zero angle fgg. = 0. In the top path, we pick QA as reference neighbour. Let us call this gauge A, in which neighbours have angles 0:- In the bottom path, we instead pick neighbour (B as reference point and are in gauge B. We get different basis for the tangent plane and different angles 0B for each neighbour. Comparing the two gauges, we see that they are related by a rotation, so that This change of gauge is called gauge transformation of angle g 6
9 A,1 -
This
In figure 2(a)] we illustrate a gauge equivariant convolution that takes input and output features such gray scale image values the mesh, which called scalar features. The top path represents the convolution in gauge A, the bottom path in gauge B. either case, the convolution can be interpreted as consisting of three steps. First, for each vertex p. the value of the scalar features on the mesh at each neighbouring vertex represented by colors, is mapped to the tangent plane at at angle Og defined by the gauge. Subsequently, the convolutional kernel sums for each neighbour q, the product of the feature at q and kernel K K(O,). Finally the output is mapped back to the mesh. These three steps can be composed into single step, which we could call a geometric convolution, mapping from input features on the mesh to output features on the mesh. The convolution is gauge equivariant this geometric convolution does not depend on the gauge we pick in the interim, so figure 2(a) if the convolution in the top path in gauge A has same result the convolution in the bottom path in gauge B, making the diagram commute. needs to be the same in both gauges, for the convolution to be equivariant. Hence, must have that K(,,,) = K ((B g), as the orientations of the neighbours differ by some angle and the kernel must be isotropic.
this case, however, we see that the convolution output
As we aim to design an anisotropic convolution, the output feature of the convolution at p can, instead of scalar, be two numbers R.2, which can be interpreted coefficients of tangent feature
4

---

Published as a conference paper at ICLR 2021
vector in the tangent space p. visualized in figure 2(b] As shown the right hand side, different gauges induce a different basis of the tangent plane, so that the same tangent vector (shown on the the mesh), is represented by different coefficients in the gauge (shown on the top and
middle right bottom on the right). This gauge equivariant convolution must be anisotropic: going from the top row to the bottom row; if we change orientations of the neighbours by -9. the coefficients of the output vector E IR2 of the kernel must be also rotated by -g. This is written as ),, where R(-9) IR2x2 the matrix that rotates by angle -g.
Vectors and scalars are not the only type of geometric features that can be inputs and outputs of GEM-CNN layer. In general, the coefficients of geometric feature of C dimensions changes by an invertible linear transformation ((-g)= E RCx' if the gauge is rotated by angle The map p: [0, 2x) + RCCC is called the rype of the geometric quantity and is formally known as a group representation of the planar rotation group SO(2). Group representations have the property that p(g + h)) (()) (they are group homomorphisms), which implies in partieular that p(0) and p = (()) For more background group representation theory, we refer the reader to (Serre 1977) and, specifically in the context of equivariant deep learning, to (Lang & Weiler 2020p. From the theory of group representations, we know that any feature type can be composed from "irreducible representations" (irreps). For SO(2), these are the one dimensional invariant scalar representation Pn and for all Nyo. two dimensional representation Pa.
Cos ng sin ng
sin ng cos ng)
po(g) = 1, (n(g)) =
where we write, for example, P Pu E P1 @ p1 to denote that representation p(g) is the direct sum (i.e. block-diagonal stacking) of the matrices pa(g): pi (g): p1 (g). Scalars and tangent vector features correspond to nr and # respectively and we have R(g) = pi (g).
The type of the feature at each layer in the network can thus be fully specified (up to a change of basis) by the number of copies of each irrep. Similar to the dimensionality in a conventional CNN, the choice of type is a hyperparameter that can be freely chosen to optimize performance.
3.1 KERNEL CONSTRAINT
Given an input type Pmn and output type Pout of dimensions Ci and Caut, the kernels are K RCaxCi and miigh 10, 2T) + R Cou Caa However, not all such kernels are equivariant. Consider again examples figure 2(a) and figure 2(b) If we map from scalar to scalar, we get that neiggh (0 g) = Kneigh (0) for all angles 0, g and the convolution is isotropic. If we map from a scalar to a vector, get that rotating the angles 0g results in the same tangent vector as rotating the output vector coefficients, so that Kneigh (f g) = R( (((-g) Kneigh(f).
In general, as derived by Cohen et al. (2019b)
and in appendix B the kernels must satisty for any gauge transformation g E [0, 2x) and angle
Pin Pare
linearly independent solutions for Kangs(()
0 E [0, 2xr), that
+ Pto
(1)
Pr. Pa
[cns nefl sin nd)). (sin nol -cosn8)
K neeigh (f 9) = Poul 9) meigh (f) (in(,,
(3)
(ooss rith sin ma0
(- cas mi)
Po - Pm
sin mB
K = Pour (-)) Ksetf Pin Pin(g).
(4)
(c-
Pn + Pmm
The kernel can be seen as consisting of multiple blocks, where each block takes as input one irrep and outputs one irrep. For example if Pm would be of type Pu @ P1 ① P1 and Pou of type P1 @ ps. we have 4 x 5 matrix
Pio -> Pout
lincarly independent solutions for Kast
Po + Po
(1)
(6 (-90 0
Pr. + Pm
Table 1: Solutions to the angular kernel constraint for kernels that map from Pa. to Pm- We denote c= ===cos((m + n)0) and #+ = =inn((m + m)0).
(i)
(n) (0)
(n(
Kueigh (0) =
(0)
K3i(0)
where e-g. Ky (0) E 2x2 a kernel that takes as input irrep Pi and as output irrep P3 and needs to satisfy Eq. 3 As derived by Weiler & Cesa (2019) and in Appendix C the kernels Kneigh (0) and Keaff mapping from irrep Pn, to urrp Pm can be written as a linear combination of the basis kernels listed in Table
I
The table shows that equivariance requires the self-interaction to only map from
0
Kn
K11)
one irrep to the same irrep. Hence, we have Kgatt
E RA33
0
0
5

---

Published as a conference paper at ICLR 2021
All basis-kemnlls of all pairs of imput irreps and output irreps can be linearly combined to form an arbitrary equivariant kemel from feature of type Pin to Paut- In the above example, we have 2 x 2 + 4 x = 20 basis kernels for neigh and 4 basis kernels for K The layer thus has 24 parameters. As proven in (Weiler & Cesa 2019) and (Lang & Weiler (020), this parameterization of the equivariant kernel space is complete, that is, more general equivariant kernels do not exist.
3.2 GROMETRY AND PARALLEL TRANSPORT
In order to implement gauge equivariant mesh CNNs, we need to make the abstract notion of tangent spaces, gauges and transporters concrete.
As the mesh is embedded in IR3, natural definition of the tangent spaces T.M is as two dimensional subspaces that are orthogonal to the normal vector at p. We follow the common definition of normal vectors at mesh vertices the area weighted average of the adjacent faces' normals. The Riemannian logarithm map logp : N - T represents the one-ring neighborhood of each point p their tangent spaces visualized in figure Specifically, neighbors
NNp are mapped to log,(9) (g) E T,M by first projecting them to A. and then rescaling the projection such that the norm is preserved, i.c. [0) = 19 pl: see Eq. 6 A choice of reference neighbor qp E N uniquely determines a right handed, orthonormal reference frame (ep,1, 6p,2) of TyM by setting cp.1 (40)( (og,(()) and ép.2
cp,1 The polar angle Opg of any neighbor q e N relative to the first frame axis is then given by 0p- = atan2 (ep.2 (g), 6p.1 (og)
Given the reference frame ( a 2-tuple of coefficients (21) E TR2 specifies an (embedded) tangent vector U1 ep, + U2ep,2 T.M 1R3 This assignment formally given by the gauge map Ep: 2 - T C R33 which is vector space isomorphism. In our case, can be identified with the matrix
Cp.2),
Ep
Cp,1
ep,
E Rx32
(5)
Feature vectors fp and f, at neighboring (or any other) vertices p E M and q E N S M live in different vector spaces and are expressed relative independent gauges, which makes it invalid to sum them directly. Instead, they have to be parallel transported along the mesh edge that connects the two vertices. As explained above, this transport given by group elements Jo-p E [0, 2n). which determine the transformation of tangent vector coefficients Ug + R(gy-p)U RR2 and, analogously, for feature vector coefficients as fg + p(Dg-sp)sp Figure 4 in the appendix visualizes the definition of edge transporters for flat spaces and meshes. On flat space, tangent vectors are transported by keeping them parallel in the usual sense on Euclidean spaces. However, if the source and target frame orientations disagree, the vector coefficients relative to the source frame need be transformed to the target frame. This coordinate transformation from polar angles 42 of v to Ppp of R(gg-o)) defines the transporter Jg-op 4p ipg. On meshes, the source and target tangent spaces T T,M and TpMM are not longer parallel. It is therefore additionally necessary to rotate the source tangent space and its vectors parallel to the target space, before transforming between the frames. Since transporters effectively make up for differences in the source and target frames, the parallel transporters transform under gauge transformations Ip and 9, according to 9g-pp + 9p + Jq-p 9q- Note that this transformation law cancels with the transformation law of the coefficients q and lets the transported coefficients transform according to gauge transformations at p. It is therefore valid to sum vectors and features that are parallel transported into the same gauge at p.
A more detailed discussion of the concepts presented in this section can be found in Appendix A
4 NON-LINEARITY
Besides convolutional layers, the GEM-CNN contains non-linear layers, which also need to be gauge equivariant, for the entire network to be gauge equivariant. The coefficients of features built out of irreducible representaions, as described in section 3, do not commute with point- wise non- linearities (Worrall et al. 2017 Thomas et al., 2018 Weiler al.) 2018 Sa Kondor et al.) 2018). Norm non-linean non-linearities and gated non- lineaaitssss (Weiler & Cesa 2019) can be used with such features. but generally perform worse in practice compared to point-wise con-linearities (Weiler & Cesa

---

Published as a conference paper at ICLR 2021
2019). Hence, propose the RegularVonlinearrty, which uses point-wise non- non-linearities and is approximately gauge equivariant.
This non-linearity is built on Fourier transformations. Consider continuous periodic signal, on which we perform a band-limited Fourier transform with band limit 6, obtaining 2b + Fourier coefficients. If this continuous signal is shifted by an arbitrary angle g. then the corresponding Fourier components transform with linear transformation Po.s( g), for 26 + dimensional representation Po.s :- Po + pi ... pb.
It would be exactly equivariant to take a feature of type Poss, take a continuous inverse Fourier transform continuous periodic signal, then apply a point-wise non-linearity that signal, and take the continuous Fourier transform, to recover feature of type Po.b. However, for implementation, we use N intermediate samples and the discrete Fourier transform. This is exactly gauge equivariant for gauge transformation of angles multiple of 2nT /N, but only approximately equivariant for other angles. In App. G we prove that as N oo, the non-linearity is exactly gauge equivariant.
The run-time cost per vertex of the (inverse) Fourier transform implemented as simple linear transformation is ObbN), which is what we use in our experiments. The pointwise non-linearity scales linearly with N, so the complexity of the RegularNonl Linearly is also (bbN). However, one can also use fast Fourier transform, achieving a complexity of O(N log N). Concrete memory and run-time cost of varying N are shown in appendix F.1
5 RELATED WORK
The irregular structure of meshes leads to variety of approaches to define convolutions. Closely related to our method are graph based methods which are often based on variations of graph con- volutional networks (Kipf & Welling, 2017] Defferrard et al., 2016). GCNs have been applied on spherical meshes (Perraudin et al., 2019) and cortical surfaces (Cucurull et al.] 2018 (Zhao et al.) 2019a). Verma et al. (2018) augment GCNs with anisotropic kernels which are dynamically computed via an attention mechanism over graph neighbours.
Instead of operating on the graph underlying a mesh, several approaches leverage its geometry by treating it as a discrete manifold. Convolution kernels can then be defined in geodesic polar coordinates which corresponds a projection of kernels from the tangent space the mesh via the exponential map. This allows for kernels that are larger than the immediate graph neighbourhood and message passing over faces but does not resolve the issue of ambiguous kernel orientation. Masci et al. (2015): Monti et al.) (20T6) and [Sun et al. (2018) address this issue by restricting the network to orientation invariant features which are computed by applying anisotropic kernels in several orientations and pooling over the resulting responses. The models proposed in (Boscaini) ct al. 2016) and (Schonsheck et al. (018)) are explicitly gauge dependent with preferred orientations chosen via the principal curvature direction and the parallel transport of kernels, respectively. Poule. nard & Ovsjanikov| (2018) proposed a non-trivially gauge equivariant network based on geodesic convolutions, however, the model parallel transports only partial information of the feature vectors, corresponding to certain kernel orientations. In concurrent work. Wiersma et al. (2020) also define convolutions on surfaces equivariantly to the orientation of the kernel, but differ in that they use norm non-lincarities instead of regular ones and that they apply the convolution along longer geodesics, which adds complexity to the geometric pre-computation as partial differential equations need be solved, but may result in less susceptibility to the particular discretisation of the manifold.
Another class of approaches defines spectral convolutions on meshes. However, as argued in (Bronstein et al. (207)), the Fourier spectrum of a mesh depends heavily its geometry, which makes such methods instable under deformations and impedes the generalization between different meshes. Spectral convolutions further correspond to isotropic kernels. Kostrikov et al. (2018) overcomes isotropy of the Laplacian by decomposing it into two applications of the first-order Dirac operator.
construction based on toric covering maps of topologically spherical meshes was presented in Maron et al. 2017). An entirely different approach mesh convolutions is to apply a linear map to a spiral of neighbours (Bouritsas et al. 2019: Gong et al. 2019), which works well only for meshes with similar graph structure.

---

Published as a conference paper at ICLR 2021
The
methods operate the intrinsic, 2-dimensional geometry of the mesh. A pop- ular alternative for embedded meshes is to define convolutions in the embedding space R This can for instance be done by voxelizing space and representing the mesh in terms of an occupancy grid (Wu et al.; 2015; ['chapmi et al. 2017; Hanocka et al.) 2018). A downside of this approach the high memory and compute requirements of voxel representations. If the grid occupancy is low, this can partly be addressed by resorting to an inhomogencous grid density (Riegler et al. 2017). Instead of voxelizing space, one may interpret the set of mesh vertices as point cloud and run convolution on those (Qi et al. 2 b). Point cloud based methods can be made equivariant w.r.t. the isometries of I (Zhao et al. 2019b Thomas et al. 2018), which implies in particular the isometry equivariance on the embedded mesh. In general, geodesic distances within the manifold differ usually substantially from the distances in the embedding space. Which approach more suitable depends on the particular application.
On flat Euclidean spaces our method corresponds Steerable CNNs (Cohen & Welling 2017 Weiler al. 2018a Weiler & Cesal 2019 Cohen al., 2019a Lang & Weiler 2020). As our model, these networks process geometric feature fields of types and are equivariant under gauge transformations, however, due the flat geometry, the parallel transporters become trivial. Regular nonlinearities on flat spaces used in group convolutional networks (Cohen & Welling 2016) Weiler et al. al.] 20188)
Hoogeboom al. 2018
Bekkers al. 2018: Winkels & Cohen 2018 Worrall & Brostow
2018
Worrall & Welling
2019
Sosnovik et al. (020)).
6 EXPERIMENTS
6.1 EMBEDDED MNIST
We first investigate how Gauge Equivariant Mesh CNNs perform on, and generalize between, different mesh geometries. For this purpose we conduct simple MNIST digit classification experiments on embedded rectangular meshes of 28 x 28 vertices.
a baseline geometry we consider a flat mesh as visualized in figure 5(a) A second type of geometry is defined as different isometric embeddings of the flat mesh, see figure 5(b] Note that this implies that the intrinsic geometry of these isometrically embedded meshes is indistinguishable from that of the flat mesh. To generate geometries which intrinsically curved, we add random normal displacements to the flat mesh. We control the amount of curvature by smoothing the resulting displacement fields with Gaussian kernels of different widths and define the roughness of the resulting mesh as 3 - a. Figures 5()) show the results for roughnesses of 0.5, 1, 1.5, 2, 2.25 and 2.5. For each of the considered settings we generate 32 different train and 32 test geometries.
To test the performance on, and generalization between, different geometries, we train equivalent GEM-CNN models on a flat mesh and meshes with a roughness of 1, 1.5, 2, 2.25 and 2.5. Each model is tested individually on each of the considered test geometries, which are the flat mesh isometric embeddings and curved embeddings with a roughness of 0.5, 1, 1.25, 1.5, 1.75, 2, 2.25 and 2.5. Figure 3 shows the test errors of the GEM-CNNs on the different train geometries (different curves) for all test geometries (shown on the x-axis). Since our model is purely defined in terms of the intrinsic geometry of a mesh it is expected to be insensitive to isometric changes in the embeddings. This is empirically confirmed by the fact that the test performances on flat and isometric embeddings are exactly equal. As expected, the test error increases for most models with the surface roughness. Models trained more rough surfaces are hereby more robust deformations. The models generalize well from a rough training to smooth test geometry up to training roughness of 1.5. Beyond that point, the test performances on smooth meshes degrades up to the point of random guessing at a training roughness of 2.5.
As baseline, we build an isotropic graph CNN with the same network topology and number of parameters (+= 163k). This model is insensitive to the mesh geometry and therefore performs exactly equal on all surfaces. While this enhances its robustness on very rough meshes, its test error of 19.80 + 3.43% is an extremely bad result on MNIST. In contrast, the use of anisotropic filters of GEM-CNN allows competitive with conventional CNNs on pixel grids, which apply anisotropic kernels as well. More details on the datasets, models and further experimental setup are given in appendix E.
reach a test error of only 0.60 + 0.05% on the flat geometry. It is therefore
8

---

Published as a conference paper at ICLR 2021
Model
Features
Accuracy (%)
ACNN (Boscaini et al. 2016
SHOT
62.4
Geodesic CNN (Masei el al.
2015
SHOT
65.4
MoNet (Monti et al. 2016
SHOT
73.8
FeaStNel (eema al. 2018
XYZ
98.7
ZerNet (Sun et ale 2011
XYZ
96.9
SpiralNett+ (Gong et 2019
XYZ
99.8
Graph CNN
XYZ
1.40:1.55
190
Graph CNN
SHOT
23.80-1-8
Non-equiv. CNN (SIIOT frames)
XYZ
73.00.14.0
Non-equiv. CNN (SHOT frames)
SHOT
75.11:22.4
GEM-CNN
XYZ
99.73.10.00
GEM-CNN (broken symmetry)
XYZ
99.89.10.02
Figure 3: Test errors for MNIST digit classifica- tion on embedded meshes. Different lines denote train geometries, x-axis shows test geometries. Re- gions are standard errors the means over 6 runs.
Table 2: Results of FAUST shape correspondence. Statisties means and standard errors of the mean of over three runs. All cited results are from their respoctive papers.
6.2 SHAPE CORRESPONDENCE
As second experiment, we perform non-rigid shape correspondence on the FAUST dataset (Bogo al. 2014). following Masci et al. 2015 D The data consists of 100 meshes of human bodies in various positions, split into 80 train and 20 test meshes. The vertices are registered, such that vertices on the same position on the body, such as the tip of the left thumb, have the same identifier on all meshes. All meshes have 6890 vertices, making this a 6890-class segmentation problem.
The architecture transforms the vertices' XYZ coordinates (of type 3po). via 6 convolutional layers to features 64pu, with intermediate features 16(po @ P1 @ P2), with residual connections and the RegularNonlinearity with = 5 samples. Afterwards, we use two Xx convolutions with ReLU to map first to 256 and then 6980 channels, after which a softmax predicts the registration probabilities. The 1 X convolutions use a dropout of 50% and 1E-4 weight decay. The network is trained with a cross entropy loss with initial learning rate of 0.01, which is halved when training loss reaches plateau.
As all meshes in the FAUST dataset share the same topology, breaking the gauge equivariance in higher layers can actually be beneficial. As shown in (Weiler & Cesal 2019), symmetry can be broken by treating non- invariant features invariant features as input to the final 1 1x convolution.
As baselines, we compare to various models, some of which use more complicated pipelines, such as (1) the computation of geodesics over the mesh, which requires solving partial differential equations, (2) pooling, which requires finding a uniform sub-selection of vertices, (3) the pre-computation of SHOT features which locally describe the geometry (Tombari al., 2010). or (4) post-processing refinement of the predictions. The GEM-CNN requires none of these additional steps. In addition, we compare SpiralNet+++ (Gong et al. 2019), which requires all inputs to be similarly meshed. Finally, we compare to an isotropic version of the GEM-CNN, which reduces to a conventional graph CNN, as well as a non-gauge- gagge-equivarian CNN based on SHOT frames. The results in table that the GEM-CNN outperforms prior works and non-gauge-equivarinnn CNN, that isotropic graph CNNs are unable to solve the task and that for this data set breaking gauge symmetry in the final layers of the network is beneficial. More experimental details are given in appendix E.2
CONCLUSIONS
Convolutions on meshes are commonly performed as a convolution on their underlying graph, by forgetting geometry, such as orientation of neighbouring vertices. In this paper we propose Gauge Equivariant Mesh CNNs, which endow Graph Convolutional Networks on meshes with anisotropic kernels and parallel transport. Hence, they are sensitive to the mesh geometry, and result equivalent outputs regardless of the arbitrary choice of kernel orientation.
We demonstrate that the inference of GEM-CNNs invariant under isometric deformations of meshes and generalizes well over a range of non-isometric deformations. On the FAUST shape correspondence task, we show that Gauge equivariance, combined with symmetry breaking in the final layer, leads to state of the art performance.
'hheee experiments were executed on QUVA machines.

---

Published as a conference paper at ICLR 2021
REFERENCES
Bekkers, E. J., Lafarge, M. W. Veta, M.. Eppenhof, K. A., Pluim, J. P., and Duits, R. Roto-translation covariant convolutional networks for medical image analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2018.
Bogo, F., Romero, J.. Loper, M., and Black, M. J. Faust: Dataset and evaluation for 3d mesh registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3794 7794-3801, 2014.
Boscaini, D., Masci, J., Rodolá, E., and Bronstein, M. M. Learning shape correspondence with anisotropic convolutional neural networks. In NIPS, 2016.
Bouritsas, G., Bokhnyak, S., Ploumpis, S., Bronstein, M., and Zafeiriou, S. Neural 3d morphable models: Spiral convolutional networks for 3d shape representation learning and generation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 7213-7222, 2019.
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., and Vandergheynst, P. Geometric deep learning: Going beyond Euclidean data. JEEE Signal Processing Magazine, 2017.
Cohen, T. and Welling, M. Group equivariant convolutional networks. In ICML, 2016.
Cohen, S. and Welling, M. Steerable CNNs. In ICLR, 2017.
Cohen, T. S., Geiger, M., and Weiler, M. A general theory of equivariant CNNs on homogeneous spaces. In Conference Neural Information Processing Systems (NeurIPS), 2019a.
Cohen, S., Weiler, M., Kicanaoglu, B., and Welling, M. Gauge equivariant convolutional networks and the Icosahedral CNN. 2019b.
Crane, K., Desbrun, M., and Schröder, P. Trivial connections discrete surfaces. Computer Graphics Forum (SGP), 2 2010.
Crane, K., de Goes, F., Desbrun, M., and Schröder, P. Digital geometry processing with discrete exterior calculus. ACM SIGGRAPH 2013 courses, SIGGRAPH '13, New York, NY, USA, 2013. ACM.
Cucurull, G., Wagstyl, K., Casanova, A., Veličković, P., Jakobsen, E., Drozdzal, M., Romero, A., Evans, A., and Bengio, Y. Convolutional neural networks for mesh-based parcellation of the cerebral cortex. 2018.
Defferrard, M., Bresson, X., and Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering, In Advances in neural information processing systems, pp. 3844-3852, 2016.
Gallier, J. and Quaintance, J. Differential Geometry and Lie Groups: A Computational Perspective, volume 12. Springer Nature, 2020.
Gong, S., Chen, L., Bronstein, M., and Zafeiriou, S. Spiralnet++:: A fast and highly efficient mesh convolution operator. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 0-0, 2019.
Hanocka, R., Fish, N., Wang, Z., Giryes, R., Fleishman, S., and Cohen-Or, D. Alignet: Partial-shape agnostic alignment via unsupervised learning. ACM Transactions on Graphics (TOG), 38(1):1-14, 2018.
Hoogeboom, E., Peters, J. W. T., Cohen, T. S., and Welling, M. HexaConv. In International Conference on Learning Representations (ICLR), 2018.
Kipf, T. N. and Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR, 2017.
Kondor, R., Lin, Z., and Trivedi, S. boobsch-gordan nets: a fully fourier space spherical convolutional neural network. In NIPS, 2018.
10

---

Published as a conference paper at ICLR 2021
Kostrikov, I., Jiang, Z., Panozzo, D., Zorin, D., and Bruna, J. Surface networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2440-2548, 2018.
Lai, Y.K.., Jin, M., Xie, X., He, Y., Palacios, J., Zhang, E., Hu, S.-M., and Gu, X. Metric-driven rosy field design and remeshing. IEEE Transactions on Visualization and Computer Graphics, 16(1): 95-108, 2009.
Lang. L. and Weiler, M. Wigner- Theorem for Group Equivariant Convolution Kernels. arXiv preprint arXiv:2010. 10952, 2020.
Maron, H., Galun, M., Aigerman, N., Trope, M., Dym, N., Yumer, E., Kim, V. G., and Lipman, Convolutional neural networks on surfaces via seamless toric covers, ACM Trans. Graph., 36(4): 71-1, 2017.
Masci, J., Boscaini, D., Bronstein, M. M., and Vandergheynst, P. Geodesic convolutional neural networks riemannian manifolds. ICCVW, 2015.
Monti, F., Boscaini, D., Masci, J., Rodolà, E., Svoboda, J., and Bronstein, M. M. Geometric deep learning graphs and manifolds using mixture model cnns. CoRR, 1.08402, 2016. URL
http:://axtii /ttp:://axxvv../// org/ http:/aaab/1111.///.11
Perraudin, N., Defferrard, M., Kacprzak, T., and Sgier, R. Deepsphere: Efficient spherical convo- lutional neural network with healpix sampling for cosmological applications. Astronomy and Computing, 27:130-146, 2019.
Poulenard, and Ovsjanikov, M. Multi-directional geodesic neural networks via equivariant convolution. ACM Transactions on Graphics, 2018.
Qi. C.C.R. Su, H., Mo, K., and Guibas, L. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652 12-660022 2017a.
Qi, C. R., Yi, L., Su, H., and Guibas, J. Pointnet++: Deep hierarchical feature learning on point sets in metric space. In Advances in neural information processing systems, pp. 5099-5108, 2017b.
Riegler, G., Osman Ulusoy, A., and Geiger, A. Octnet:
Learning deep 3d representations high resolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3177-35866 2017.
Schonsheck, S. C., Dong, B., and Lai, Parallel Transport Convolution: A New Tool for Convolu- tional Neural Networks on Manifolds. 1rXiv:1805.0773 1805.07857 [cs, math, stat], May 2018.
Serre, J.PP. Linear representations of finite groups. 1977.
Sosnovik, I., Szmaja, M., and Smeulders, A. Scale-equivariant steerable networks. In International Conference on Learning Representations (ICLR), 2020.
Sun, Z., Rooke, E., Charton, J., Hc, Y., Lu, J., and Back, Zernet: Convolutional neural networks arbitrary surfaces via zernike local tangent space estimation. arXiv preprint arXiv: 18/2.01082, 2018.
Tchapmi. L., Choy, C., Armeni, I., Gwak, J., and Savarese, S. Segcloud: Semantic segmentation of 3d point clouds. In 2017 international conference on 3D vision (3DV), pp. 537-547. IEEE, 2017.
Thomas, N., Smidt, T., Kearnes, S., Yang, I.., Li, L., Kohlhoff, K., and Riley, P. Tensor Field Networks: Rotation- and Translation-Equine Eqquvaaran Neural Networks for 3D Point Clouds. 2018.
Tombari, F., Salti, S., and Di Stefano, L. Unique signatures of histograms for local surface description. In European conference on computer vision, pp. 356-369, Springer, 2010.
Tu. L. W. Differential geometry: connections, curvature, and characteristic classes, volume 275.
Springer, 2017.
11

---

Published as a conference paper at ICLR 2021
Verma, N., Boyer, E.,, and Verbeek, J. Feastnet: Feature-stecd steered graph convolutions for 3d shape analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2598-2606, 2018.
Weiler, M. and Cesa, G. General E(2)-equivariant steerable CNNs. In Conference Neural Information Processing Systems (NeurIPS), 2019, URL https:
911
08251
Weiler, M., Geiger, M., Welling, M., Boomsma, W., and Cohen, T. 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. In NeurIPS, 2018a.
Weiler, M., Hamprecht, F. A., and Storath, M. Learning steerable filters for rotation equivariant CNNs. In Conference on Computer Vision and Pattern Recognition (CVPR), 2018b.
Wiersma, R., Eisemann, E., and Hildebrandt, K. CNNs on Surfaces using Cotation-Equivarne Features. Transactions on Graphics, 39(4). July 2020. doi: 1L114///386569.33244777
Winkels, M. and Cohen, S. G-CNNs for pulmonary nodule detection. Conference on Medical Imaging with Deep Learning (MIDL), 2018.
Worrall, D. and Welling, M. Deep scale-spaces: Equivariance over scale. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
Worrall, D. E. and Brostow, G. J. Cubenet: Equivariance to 3D rotation and translation. In European Conference on Computer Vision (ECCV), 2018.
Worrall, E., Garbin, J., Turmukhambetov, D., and Brostow, G. Harmonic Networks: Deep Translation and Rotation Equivariance. In CVPR, 2017.
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Zhao, F, Xia, S., Wu, Z., Duan, D., Wang. L., Lin, W., Gilmore, H., Shen, D., and Li, G. Spherical u-net on cortical surfaces: Methods and applications. CoRR, abs/1904.00906, 2019a. URL 00906
Zhao, Y., Birdal, T., Lenssen, J. E., Menegatti, E., Guibas, L., and Tombari, F. Quaternion equivariant capsule networks for 3d point clouds. urXiv preprint arXiv: 1912. 12098, 2019b.
12


Reviews:
[{"review_id": "vBngZDFGLIn", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Graph convolution has been defined to be permutation equivariant to the neighborhood vertices. If one were to define an anisotropic kernel then a reference edge would have to be defined corresponding to edge angle 0. Figure 1 is very illuminating.\n\nEquivariance with respect to this reference can be achieved only with a special mechanism. The authors here proposed message passing via edge transporters.\nThe crux of the approach is in equation (2) and in particular in $\\rho(g_{p \\rightarrow q} \\in [0,2\\pi)$. What happens is that the feature vectors of the adjacent vertices have to be transported to the center node, namely a transformation from the local coordinates in p to the local coordinates in q correcting, thus, the underlying gauge difference. This step includes also an alignment of the tangent planes. I strongly believe that appendix A belongs to the main paper.\n\nThe second crucial point is that equivariance imposes a linear constraint on the kernels $K_{Self}$ and $K_{Neigh}$. This allows a kernel to be written as a linear combination of 20 kernels for $K_{Neigh}$ and $K_{Self}$ resulting in 24 only unknowns for one layer.\n\nThe related work section is comprehensive. \n\nThe experiment on shape correspondence achieves performance comparable to the state of the art spiralNet++. The paper would benefit from other experiments on manifold like performing mesh convolutions for human or object reconstruction from images. \nFaust is pretty standard in the GML community but it is an easy task in terms of feature learning.\n\nThe paper contribution is elegant and significant: Gauge equivariance  is a necessity if you want an anisotropic diffusion.\n\nThe paper is unreadable without the appendix and somehow it would be better to make it self-contained and move the experiments in the appendix.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "l_FhaPC7DGO", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents Gauge Equivariant Mesh CNNs. The method is motivated by the fact that graph convolutions can be modified for meshes to take into account the angular arrangement of local neighborhoods. The result is a Mesh-CNN that is equivalent to GCNs with anisotropic gauge equivariant kernels. \n\nSTRENGTHS\n\n- The problem tackled here is very important and well motivated. The authors identify the issues related to existing networks and devise a sensible approach.\n- The approach is detailed and carefully patches the problems in mesh convolutions.\n- Introducing non-linearity to such networks is far from being trivial. I thank the authors for conveying an analysis on this front. \n\nWEAKNESSES\n\n- I believe that this paper (as well as many other prior works) are missing an important connection to the rich literature of 3D vision. From Ajmal Mian's work to Spin images, from SHOT to the recent deeply learned PPF-FoldNet (and point pair features thereon), there are just too many works that compute local reference frames to canonically orient the local geometry. Many of these works compute some form of a point cloud normal (thus a tangent plane) and choose a direction orthogonal to it (hence fix a gauge). Traditionally, this is known as 'local reference frame' and have set an important milestone in 3D descriptors, learned or not. Simply changing the terminology and rephrasing the established results (e.g. connection of local frames via a rotation) through the lense of Riemannian geometry should not distinguish this paper. I like this principled explanation (only a personal preference) but the ties to all those works should be made concrete. For example, recently, Zhao et al. have used local reference frames on meshes and point clouds to design equivariant point cloud networks: \n* Zhao, Y., Birdal, T., Lenssen, J. E., Menegatti, E., Guibas, L., & Tombari, F. (2020). Quaternion Equivariant Capsule Networks for 3D Point Clouds. European Conference on Computer Vision (ECCV)\n\n- It would then be nice to mention clearly that one would like to omit certain local reference frame choices and directly convolve as there is an ambiguity in the tangent plane.\n\n- There are now many works which can exploit Riemannian geometry to craft convolution operators. Technically, there are subtle differences which makes the contributions of the paper less clearer. I would strongly suggest that the paper is revised such that the contributions are stated very clearly. Otherwise, it is hard to figure out which parts already existed and which are novel.\n\n- I consider Sec. 6.1 to be a synthetic dataset because changing the underlying lattice for MNIST does not have practical use. And the results presented in Tab. 2 have some interesting accuracies such as 1.40. Why is that so low? Can we just fix this by picking a naive baseline: Choose an LRF (see above, use SHOT's frame for instance), canonicalize the points locally and apply convolution. How would this simple approach perform? I think some similar idea is already mentioned in:\n* Yang, Z., Litany, O., Birdal, T., Sridhar, S., & Guibas, L. (2020). Continuous Geodesic Convolutions for Learning on 3D Shapes. arXiv preprint arXiv:2002.02506.\n\n- Would be possible to briefly summarize Mesh-CNN? This paper gives an improvement over that so it could be beneficial to see (even in a supplementary) how it is improved. Or is Mesh-CNN is used just to refer to the category of works? \n\n- There is a form of 'discretization' of the parallel transport going on. Can't we compute the vector heat to do a better and faster discretization:\n* Sharp, Nicholas, Yousuf Soliman, and Keenan Crane. \"The vector heat method.\" ACM Transactions on Graphics (TOG) 38.3 (2019): 1-19.\nIn general it would be great to have an understanding of the proposed transport operator. \n\n- Some more focused explanation of the representation theory could be great.\n\nMy recommendation is positive because the community needs principled ways of convolution on non-Euclidean domains, and this paper seems to make an incremental contribution towards that direction. However, lack of thorough evaluations, the missing links to the literature and the rather inaccessible presentation of the material should definitely be improved.", "extra_scores": {}, "extra_text": {}}, {"review_id": "TX1mhDCFU5s", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Although a mesh embedded in 3D space may be treated as a graph, a graph convolution network uses the same weights for each neighbor and is thus permutation invariant, which is the incorrect inductive bias for a mesh: the neighbors of a node are spatially related and may not be arbitrarily permuted.  CNNs, GCNs, and G-CNNs demonstrate the value of a weight sharing scheme which correctly reflects the symmetry of the underlying space of the data.  The authors argue convincingly that for a signal on a mesh, the appropriate bias is symmetry to local change-of-gauge.  In short, the weights should depend on the relative orientation of a node’s neighbors.  They design a network GEM-CNN which is equivariant to change of gauge.  The design is similar to a GCN but incorporates parallel transport to account for underlying geometry and uses kernels similar to those of $SO(2)$-equivariant $E(2)$-CNN (Weiler & Cesa 2019).  The experiments show the network is able to adapt to different mesh geometries and obtain very high accuracy in the shape correspondence task. \n\nI suggest accepting this paper.  The architecture is an elegant way to incorporate gauge symmetry on meshes and RegularNonlinearity addresses an important issue for equivariant neural nets.  Though I would prefer more interesting experiments, they are sufficient to validate the design. \n\n**Specific Strength, Weaknesses, Points, and Questions:**\n- The symmetry of a graph convolutional network can be broken by including spatial coordinates as features.  If $x_p$ and $x_q$ are inputs, then a function $F(f_q,x_q,x_p)$ can process data in an orientation aware way even though $F$ is isotropic.  How would this compare to the proposed method in the paper?\n-Page 2: One strength of the paper is the argument for why gauge symmetry is necessary in the first place.  Features defined on a mesh may be vector-valued and defined with respect to a frame of reference at each point on the mesh.  However, since the geometry is curved, there is no consistent way to assign a frame of reference.  Thus an arbitrary choice must be made when recording data.  Since this choice is arbitrary, the output of the network should clearly be independent of it.  Gauge equivariance encodes this symmetry.   \n-The argument for encoding the geometry of the mesh is reasonable.  But then why only parameterize $K$ by $\\theta_q$; why not also include the distance $r_q$?  \n-Page 6: **RegularNonlinearity is an important contribution** of the paper.  The authors are correct that non-linearities have been a bottleneck for using equivariant neural networks with representations other than the regular representation.  Transforming to sample space (i.e. embedding in the regular representation) to apply a pointwise non-linearity and then transforming back is a nice idea for addressing this.  Furthermore, Theorem E.1 provides a nice theoretical analysis of the asymptotic error.  It would be nice to include practical non-asymptotic error bounds as well. \n-The experiments are okay, but are a weaker part of the paper.  The argument for a geometry-aware NN is that it processes signals on the geometry better.  It is not clear that embedding MNIST on a mesh illustrates data which is best understood in terms of the underlying mesh.  Far better would be to develop a signal natively on the mesh, for example by solving a PDE directly on the mesh.  Arguably the FAUST shape correspondence data addresses this issue better since the signal is inherently linked to the geometry.  \n-Both experiments also treat only scalar data of type $\\rho_0$.  While it is plausible and reasonable to model such data using vector features of type $\\rho_i$ in the hidden layers, the argument for the necessity of gauge equivariance would be even stronger if the input and/or output signal was itself vector valued, for example a velocity or gradient on the mesh. \n-Changing roughness to the embedded MNIST distorts the signal in the geometry of the manifold (changing distances and angles), so why should we expect generalization across different roughnesses? \n-Page 7: I don’t understand the argument for the value of symmetry breaking.  The gauge equivariant network can be orientation aware by encoding $\\rho_1$ features.  Why is it desirable to be dependent on arbitrary gauge choices?  What breaks down about the original argument for incorporating equivariance in this case?  Is it possible the improvement is due to a different trade off between bias and expressivity at lower layers?\n-Page 7, Para 4: The paper argues other high performing methods in shape correspondence use complicated pipelines.  It is not clear to me (probably from lack of familiarity) which is most complicated.  It seems both this method and the other method contain different complexities and subtleties.        \n\n**Minor Points:**\n- All of the citations in the paper use \\citet, but it would be more readable to use \\citep.\n- Page 3: Should not $\\rho(g_{q\\to p})$ be invertible in $\\mathbb{R}^{c_{in} \\times c_{in}}$?\n- Page 5: The notation $k \\rho_l$ is non-standard, compared to $\\rho_l^k$, but it is more readable.\n- Page 13, $K_{neigh} \\theta_{pq}-g)$ is missing a parenthesis\n- Page 13, “which is true for any features, if”.  if could be if and only if, correct? \n\n**Update From Author Reply**\nI am grateful for the author's replies, edits, and additional evaluation, all performed within limited time. This helps me feel confident in my accept (7) recommendation. My reason for not giving a higher score remains the limited experiments (which is likely not something to be addressed in two weeks), but even so I think the work is quite worthy of being accepted. The methods are a significant contribution and the experiments are sufficient to demonstrate they work.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "57XOAZ_6brf", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The work presents a novel message passing GNN operator for meshes that is equivariant under gauge transformations. It achieves that by parallel transporting features along edges and spanning a space of gauge equivariant kernels. Further, a DFT-based non-linearity is proposed, which preserves the equivariance in the limit of sampling density. The method is evaluated on an MNIST toy experiment and the Faust shape correspondence task.\n\nStrengths:\n- The method is very elegant and novel and seems to be one of the most sophisticated mesh GNN operator so far.\n- At the same time it looks like that the operator stays reasonably efficient, still keeping linear time complexity in number of edges. I would welcome, though, if computational efficiency would be analyzed in the work.\n- It bridges the more theoretical equivariant convolutions with graph neural networks for mesh processing, which are more commonly used in practice.\n- An equivariant non-linearity based on the discrete Fourier transform is presented.\n- The work seems to be technically and mathematically sound.\n- The paper is well written.\n- The figures complement the text well and are helpful for understanding.\n\nWeaknesses:\n- Neither the MNIST nor the FAUST experiment verify the gauge equivariance. I think a toy experiment verifying it would be necessary, especially since equivariance of the non-linearity is only approximated (maybe by showing appropriate feature histograms after the non-linearity for changing reference points).\n- In addition to a missing verification experiment, it is also hard to follow the theoretical reasoning that the whole approach is equivariant. The line of reasoning needs to be gathered from the main text, appendix, and referenced related work. I think with more concrete theorems, clarity could be improved. \n- The Faust shape correspondence task is not sufficient to evaluate a novel mesh operator. From personal experience I know that at least SpiralNet++ (anisotropic, intrinsic, fixed topology) and SplineCNN (anisotropic, extrinsic, arbitrary/varying topology) can be tuned to reach near perfect accuracy on this task. Such small differences in performance can come down to architecture design and might not come from more principled differences in the method. Therefore, I would highly welcome one additional comparison on a different mesh task.\n- Since the experiments are scarse, I wonder if the operator is hard to apply to larger tasks or if the kernel restrictions weaken the approach on other tasks.\n\n\nQuestions and comments:\n- As usual in this area, the kernels are restricted in trade for equivariance. I wonder how expressive the kernels still are. Is there a way to compare the expressiveness (visually or quantitatively) to lets say an MLP kernel function mapping polar or Cartesian coordinates to a full C x C matrix? Is it the/a minimal restriction that ensures equivariance or is it more restrictive?\n- Would it be possible to come up with a space of two-dimensional kernels K_neigh (dependent on the full polar coordinates, including r) while keeping the gauge equivariance?\n- How efficient is the non-linearity based on DFT? I would be interested in a execution time breakdown for the whole method, showing the bottlenecks.\n- In the MNIST experiment, pooling is applied (appendix D.1). How does this interfere with the equivariance property? \n\nMinor/Typo:\n- Appendix page 13, equation above eq 8, parenthesis missing\n\nOverall, I like the paper, it is nice to read and the method is interesting, building on mathematically elegant concepts.  I actually would like to give an accept score. However, in my opinion there are experiments missing: (1, crucial) verifying the gauge equivariance and (2) an additional comparison on a more complex task. Without those experiments (especially 1), I see the paper in a borderline state. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: 
This paper addresses a crucial problem with graph convolutions on meshes. 
The authors identify the issues related to existing networks and devise a sensible approach.
The work presents a novel message passing GNN operator for meshes that is equivariant under gauge transformations.
The reviewers unanimously agree on the both the importance of the problem and the impact the proposed work could have. 

Suggestions for next version:
- The paper is unreadable without the appendix and somehow it would be better to make it self-contained
- Additional references as suggested in the reviews.
- Expanded experiments as suggested by R4, will also improve reader's confidence in the method. 

I would recommend acceptance. I would request the authors to release a sufficiently documented and easy to use implementation. This not only allows readers to build on this work but also increase the overall impact of this method.
Decision: Accept (Spotlight)