Equivariant Mesh Attention Networks
Authors: Sourya Basu
Jose Gallego-Posada
Francesco Viganò
James Rowbottom
Taco Cohen
Venue: Accepted by TMLR ()

Abstract: Equivariance to symmetries has proven to be a powerful inductive bias in deep learning research. Recent works on mesh processing have concentrated on various kinds of natural symmetries, including translations, rotations, scaling, node permutations, and gauge transformations. To date, no existing architecture is equivariant to all of these transformations. In this paper, we present an attention-based architecture for mesh data that is provably equivariant to all transformations mentioned above. Our pipeline relies on the use of relative tangential features: a simple, effective, equivariance-friendly alternative to raw node positions as inputs. Experiments on the FAUST and TOSCA datasets confirm that our proposed architecture achieves improved performance on these benchmarks and is indeed equivariant, and therefore robust, to a wide variety of local/global transformations.

Published in Transactions on Machine Learning Research (08/2022)
Equivariant Mesh Attention Networks
Sourya Basu"
sourya edu
University of Blinois at Urbono-Charn Champaign, USA
Jose "allego-Posada"
gallegoj@rmillo quebee
Mila and DIRO, Université de Montréal,, Canada
Francesco Viganò
Ivigano21 @'mperiallccm atr.iaall..cc. uk
Imperial College London, UK
James Rowbottom'
jadroabbotom&@@mall.com
Independent Scholar
Taco Cohen
lacos@atii crros@@@tiquallcommm.com qualcomuaaccoooommmmmmm
Qualcomm AI Research, The Netherlands'
Reviewed on OpenReview: https:// http:://pppmeeeeene////mmmmttt/
Abstract
Equivariance to symmetries has proven to a powerful inductive bias deep learning research. Recent works on mesh processing have concentrated on various kinds of natural symmetries, including translations, rotations, scaling, node permutations, and gange trans- formations. To date, no existing architecture is equivariant to all of these transformations. In this paper, we present an attention- architecture for mesh data that is provably equivariant to all transformations mentioned above. Our pipeline relies on the use of relative tangential features: a simple, effective, equivariance--rieneee friendly alternative to raw node positions as inputs. Experiments on the FAUST and TOSCA datasets confirm that our proposed architecture achieves improved performance on these benchmarks and is indeed equivariant, and therefore robust,, to wide variety of local /gllballll Iransformations.
1 Introduction
Equivariance to symmetrics has proven to be powerful inductive bias in machine learning tasks ranging across classification, regression, segmentation, and reinforcement learning. Commonly studied symmetries include permutations (Keriven & Peyré, 2019; Zaheer al., 2017), rotations (Cohen & Welling, 2016; Weiler et al., 2018; Li et al., 2018; Veeling et al., 2018), translations (LeCun et al., 1998), scaling (Sosnovik et al., 2020), and gauge transformations (Cohen et al., 2019a; de Haan et al., 2021).
These symmetries lead to different requirements for invariance or equivariance, depending on the learning task. For example, a mesh shape-classification task, we would like to find a classifier that consistently predicts the same label, invoriant to whether the input mesh is rotated, scaled or translated. In a node segmentation task, in addition to these invariances, expect that the model predictions change in concurrence with relabelling of the nodes, thus being permutation equivariant.
Achieving models with guaranteed equivariance is of high practical interest. The equivariance paradigm constitutes a principled way of incorporating task-specific prior information, which allows the model to simultaneously handle "equivalence classes inputs" (i.e., those inputs related by symmetry transformations), and removes the need for data augmentation ation during training (Bronstein al., 2021; Cohen, 2021).
"Eqaal contribution. Our code is available at: /ttps:///iitiee www.coo./aaaao..c www.gggllloossaa//////oo "Qulccomm AT Research is initiative Qualcomm Technologies, Inc.

---

Published in Transactions on Machine Learning Research (08/2022)
In this work we concentrate on learning tasks involving 3-dimensional meshes inputs. The main challenge in realizing equivariance in the case of mesh data lies in adequately handling the (arbitrary) numerical representation associated with the "geometric components" of a mesh. This is because different symmetry transformations have different effects on the mesh components. For example, a rotation modifics the positions of the nodes but leaves the face areas intacl, while scaling leaves angles bet ween nodes unchanged, but affects the positions, edge lengths and face areas.
While it is possible to discard the geometric structure of the mesh and consider it a mere graph or point cloud, retaining this geometric information can be crucial for successful learning. Verma et al. (2018) and de Haan et al. (2021) report poor performance for models which discard connections between nodes (e.g. edges or faces). When used on mesh data, mesh-specific designs outperform models built for less-structured data, like 3D point clouds or embedded graphs. However, employing expressive models on meshes may require the introduction of additional constraints, such as equivariance with respect to gauge transformations (de Haan et al., 2021).
To achieve equivariance, the model predictions should depend only the intrinsic geometry of the mesh, and not on the particular embedding used to represent the mesh computationally. In response to this challenge, the first contribution of our work is the introduction of relative tangential (RELTAN) features. RELTAN features transform absolute node positions into langent vectors, in a way that accounts for the local geometry of the mesh at each node, We prove that this map is equivariant under global rotations, and invariant under translations and scaling of the ambient space R
Moreover, REL TAN features are geometrie features ($3.2): although the numerical representation of these features may change depending on the specific choice of gauge, the geometric quantitics represented by the features remain unchanged. We leverage the gauge equivariant convolutions of de Haan et al. (2021) as a building block to satisfy the gauge equivariance requirement.
The conjunction of RELTAN features and gauge equivariant convolutions allows us to design models that exhibit equivariancee
lo all the symmetry transformations discussed above. Besides these desirable equivariance properties, our experiments demonstrate that relative tangential features provide consistent performance improvements. Thus, RELTAN features constitute simple (yet effective!) alternative to using "raw" node positions inputs.
Our second contribution is an extension of mesh processing algorithms to include a gauge equivariant attention mechanism in a way that provably satisfies the aforementioned requirements. We refer to this architecture as an Equivariant Mesh Attention Network (EMAN). Attention mechanisms have been a groundbreaking innovation in deep learning powering state-of-the-art results natural language processing (Vaswani et al., 2017; Devlin al., 2019; Brown et al., 2020), computer vision (Dosovitskiy et al., 2021), reinforcement lcarning (Chen et al., 2021), multi-modal learning (Jaegle et al., 2021), and graph neural networks (Veličković et al., 2018; Chamberlain et al., 2021). However, attention- based methods remain largely unexplored in the context of mesh data.
We carry out experiments on the FAUST (Bogo et al., 2014) and TOSCA (Bronstein et al., 2008) datasets. We do not apply any form of data augmentation during training. However, we evaluate the performance all models on the test set, while applying transformations to the unscen examples. These transformations consist of rotations, translations, scalings, permulations and local gauge changes. We apply the transformations "one-at-a-time" as an ablation study which allows to systematically verify the equivariance properties of all the compared models. Our results confirm that our proposed model design is the only one to achieve equivariance, and therefore robustness, to all of these local/global transformations
2 Related Work
Equivariance on graphs and manifolds. Equivariance on graphs and manifolds has been studied from numerous perspectives. Cohen et al. (2018; 2019b) propose novel gauge equivariant convolution techniques on manifolds such as spheres and platonic solids. Satorras et al. (2021) propose E(n)-oquivariant graph neural network models, which are applied to equivariant normalizing flows by Köhler et al. (2020). Moreover, de Haan et al. (2021) point out a relation between different types of equivariances: their gauge equivariant.
2

---

Published in Transactions on Machine Learning Research (08/2022)
model is also equivariant to the group of isometrics of the given mesh. The equivariance with respect to this group of isometries has been extensively studied by Weiler et al. (2021) and Cohen (2021). In contrast, our work focuses on equivariance with respect to gauge transformations and global transformations of IR3.
Equivariant attention. Attention has been used to provide more expressive filters over the isotropic convolutions in GCN (Kipf& & Welling, 2017). Veličković et al. (2018); Shi al. (2021) and Chamberlain et al. (2021) implement attention mechanisms give anisotropic filters. Several works propose alternative forms of equivariant attention. Hutchinson et al. (2021) introduce an attention architecture that is equivariant to Lie group actions. SE(3)-Transformm (Fuchs et al., 2020) is roto-translation equivariant attention network, designed for point-cloud data and graphs, and not meshes. Wang et al. (2020) propose a self-supervieed equivariant attention mechanism in image segmentation that generates more consistent class activation maps over rescaling. Romero Cordonnier (2021) propose a general group-equivariant self-attention formulation for processing images.
Equivariant attention on meshes. Gauge Equivariant Transformers (GET), recently proposed by He et al. (2021), also consider gauge equivariant attention meshes. The authors propose features which represent the raw node positions in the local coordinate system of each node. We refer to these GET features. These features are equivariant to global rotations, but are not equivariant to scaling or translation of the mesh. In contrast, RELTAN features are local features which successfully provide invariance to scaling and translations, in addition to equivariance to global rotations (Section 4).
Moreover, the attention mechanism used in GET is only gauge equivariant to multiples of 2T/N, for a certain positive integer hyperparameter N. Only when N is odd, He et al. (2021) provide framework for approzimate cquivariance to arbitrary gauge transformation, along with an estimate of the approximation error incurred. In contrast, EMAN attention mechanismi is exuclly equivariant to arbilrary transformations of gauge (Section 6). A more detailed discussion about the difference between our architecture and choice of input features and those used in GET is provided in Appendix H.
3 Geometry
In this section we describe meshes, geometric features, parallel transport, equivariances and invariances and graph convolutions. Familiar readers may choose to proceed directly to Scction 4.
3.1 Meshes
A mesh M is determined by set of vertices, nodes, set of (undirected) edges, and set of faces. We consider oriented meshes embedded in the ambient space R and require the faces to "properly glue along the edges", so that M is in fact piece-wise linear sub-manifold of R3, One can think of the mesh M as a discretization (e.g., triangulation) of 2-dimensional manifold.
A 2-dimensional sub-manifold of R admits a langent plane at each point. The discrete equivalent for a mesh M, at a point p E M, the plane orthogonal to the normal vector
ΣFep A(F) nF np IIF3, nFll'
(1)
where || denotes the 2-norm. The normal vector np (at a point p) is computed as a weighted sum of the normal vectors to the adjacent faces (F 9 p}, where the contribution of face F proportional to its area A(F). We denote the tangent plane at a point p by T,M.
For the tangent plane T.. at p, consider a gauge, or frame, E, = (cp.1; Cp.2). We only allow orthonormal ganges, for which the triple E, U {np} constitutes a positively oriented orthonormal basis of R3, Therefore, any other admissible orthonormal frame of T is obtained from the previous gauge Ep by a rotation E SO(2). We use g to denote both the rotation and its corresponding angle, modulo 2v.
Finally, we define angles 4pq. which take into account the local orientation of the neighbors of a node p. 0p is the angle between the projection of the vector q - p onto T.- and the reference axis ep.1 The angles 0p are thus gauge-dependent quantities.
3

---

Published in Transactions on Machine Learning Research (08/2022)
3.2 Geometric Features
Geometric features are a central concept in our work. Here we closely follow the presentation and notation of de Haan et al. (2021). Meshes possess richer structure than mere graphs. An important insight in geometrie deep learning is to take the mesh structure into consideration, and allow the features the underlying space to be not just simple functions on the space, but rather sections of vector bundles.
As an example, a tangential feature f on a mesh is given by a choice of tangent vector in the plane TpM, for each point p E M. The langent vector f(p), determined by evaluating f at p, can be represented by a 2-dimensional vector of coordinates fp with respect to the gauge Ez. Note that the coordinate vector fp E TR? is dependent on the choice of gauge, while the tangent vector f(p) Ty-M is independent.
Therefore, change the gauge at p. we should prescribe how the coordinate vector fp gets modified. For tangential features, fp changes as fp J pi(-g)fp where pI(-g) denotes the rotation by the angle -g. This transformation rule precisely characterizes tangential features. In contrast, scalar features are I-dimensional features that do not change when the gauge is transformed, and thus we may write fp + po(-g)fp- where Po( = for all g E (2))).
More generally, for N, 2 1, we can consider 2-dimensional features f, that change fn + pn(lg))n under gauge change g E SO(2). Here pn(-g) denotes the rotation by angle -ng. The representations Pn are
Po(g) = (g) = COOS ng - sin ng sin ng
for n 2 1.
cos ng
We say that a feature is of type if it changes accordingly with the representation p. The {pu} form an exhaustive list irreducible representations, the building blocks of all finite-iieensionll representations of (2).). In other words, there no loss of generality to direct sums of such representations, obtained by concatenation of multiple irreducible components. We consider these types of features throughout the rest of the paper. For example, p 4pu + P1 @3p2 corresponds to Teature type of dimension 4 1 + 1-2 2-243 3.2 2. Note also that these feature types are orthogonal, meaning that p² =p-
considering only geometric features corresponding
3.3 Parallel Transport
Message passing updates involve processing features stored at different nodes of the mesh. However, geometric features present challenge. For instance, tangential features stored at different nodes belong to different tangent planes (i.e. different vector spaces), and thus are not immediately comparable. Parallel transport is a procedure from differential geometry that describes how to "coherently translate" between tangent planes at different points, respecting the curvature of the manifold.
Discrete parallel transport can be intuitively understood as follows": for a node p E M, and a neighbor q e Ny. we first translate the tangent plane T,MM, together with the normal vector ngy to p, along the edge joining and p. Then, we consider the unique rotation of R that maps ny to np, with fixed axis ng Ry- Under this rotation, T is mapped onto T Tr,M, and it is now coherent to compore langend vectors al q with tangent vectors at
However, a feature f on M is represented by coordinate vectors fp and fg with respect to two different gauges. In general, even after rotating T,M onto T,M, the two gauges do not coincide. Therefore, we denote by -9p-p E SO()) the 2-dimensional rotation corresponding to the gauge change from the given gauge at q (after rotating T onto T,MM,, and the given gauge at p. It is now coherent to compare coordinate vectors they both represent coordinates of geometric features of the same type p, at the same point p. with respect to the same gauge Ep. If we denote by Rg-p the unique rotation of 3 that maps ny to nps with fixed axis my x np, we can express (the angle) 9--tp as:
fp with p(gg-p)fgg
Sq- =
ep.t. pp. (pg)))) Tep.)).
(2)
* 'See Fig. 3 in de Haan al. (2021) for a nice illustration.
4

---

Published in Transactions on Machine Learning Research (08/2022)
3.4 Equivariances and Invariances
Throughout this section, we denote by Fin and Fout the spaces of features of type Pin and Pout, respectively, and by F a generic space of features of type Also, given a transformation Vi applied on a mesh M, we use V. to denote the pushforward operator induced by W. We describe specific cases of this pushforward below.
Gauge equi/ivvariance To coherently define a feature mapping K: Fi - Fout; we require its computation to be independent of the choice of the gauge. Consider an arbitrary change of gauge g SO(2). Since features of type Pin transform as fp + pp and similarly for Pout, we demand K. to satisfy the gauge equivariance constraint
Ponl(-)) o K = Kc (ta(g))
If the representation Pout is (direct sum of) scalar feature(s) po, then we talk about gauge invariance, the resulting features are insensitive to the particular choice of gauge.
Global rotation aqui/invariance, Given a gange-equivariant feature mapping K:: Fin + Fout, we study its interaction with a global rotation R E SO(3). Denote by RM the mesh obtained by rotating M. We write KM: F. + F. and KRM FRM FRM to distinguish between the same feature mapping appliod to feature spaces on lao different meshes.
If f E F.M, the rotation R transforms f to a feature R.f E FRMM Formally, if f is represented at p by the coordinate vector fp relative to the gauge Ep. then R.f is represented at Rp by the same coordinate vector (R. lp: relative to the rolated gauge REp For example, when Jp is a tangent vector fp E TMM, the action of R. corresponds geometrically to the rotation of the vector fp by R.
Global rotation equivariance of the feature mapping I. is given by:
Rs I.M KR R..
(4)
Again, if the representation Pout is one, sum of scalar features Po: then we talk about global rotation invariance.
Global translation invariance. A translation Tp - x of R2, with I E RR, trivially pushes features on M to features on TM: if f E J.M is represented by fp in the gauge E, at p. then T.ʃ E JTM is represented by the same coordinate vector (T.f)T> = f in the same gauge E, at Tp. Intuitively, T.f is the same feature as f. "stored" at the translated points. Therefore, we say that a feature mapping is global translation invariant if
T. T. XXTM
(5)
Global scaling invariance, A scaling Sp Ap of IR X O, similarly allows definition of a fush-forward S. JM + JSM, if f E J.M is represented by fp in the gauge E, at p, then S.Jp E J.S is represented by the same coordinates (S.f)sp = fp in the gauge Ep at Sp.
While for rotations R and translations T new gauges are obtained through the differentials dR and dT, this is not the case for scalings S, since the new gauge would not be normalized. With our definition, tangential features do not scale as the mesh scales, and preserve their norm. For this reason, we say that a feature mapping K. is global scaling invariant (and not equivariant) if
(6) Node permutation equi/invariance. Consider a re-labeling of the nodes in a mesh M. We talk about node permulation equivariance (resp. node permutation invariance) when the oulcome of a process computed over rearranged nodes is the rearrangement of the outcome from the original ordering, under the same permutation (resp. does not depend on the ordering of the nodes). We expect a segmentation model to be permutation equivariant, while classification model should be permutation invariant.
S. S. = KMM S..
3.5 Graph Convolution on Meshes
If we ignore the faces of a mesh and its embedding in R we obtain a graph. The works of Scarselli al. (2009); Bruna et al. (2013); Defferrard et al. (2016) and Kipf & Welling (2017) led to Graph Convolutional
5

---

Published in Transactions on Machine Learning Research (08/2022)
Networks (GCNs), which give efficient algorithm for node classification. However, GCNs do not account for the local geometry of the mesh and use the same isotropic kernel to process signals from all neighbors.
Gauge Equivariant Mesh CNNs (GEM-CNNs) (de Haan et al., 2021) were introduced to overcome this geometric obstacle. Their update step uses anisotropic kernels that depend on the spatial arrangement of the neighboring nodes {q E N2). The message passing a GFM-CNN is performed as
fg Keeltfp +
Σ TEA,
The kernel (f) depends on the angle 0pg formed by the edge - q, with respect to the reference gauge at p. Morcover, the kernels Keelf and K neigh satisfy geometric constraints, so that the output feature f transforms accordingly with the change of gauge. Hence, GEM-CNNs take into account the local geometry of the mesh, while also ensuring equivariance to change of gauge. In particular, K and K neiggh (0) satisfy:
Keelt P Kselt Pin(g),
Keeggh (0 g) Pout(-9) (0e Pin(g).
(8)
For details on Keett and Kueigh (0), please see Appendix A, and de Haan et al. (2021).
4 Relative Tangential Features
Given mesh M, we construct relative tangential (RELTAN) features up(r), depending on the local geometry of the mesh M around a node p. We use the adjective relative to underline their dependeney on the relative node positions
of the neighbors {q E N2), rather than the absolute node positions. As shown in Lemma 4.1, RELTAN features provide global rotational equivariance, and invariance under translations and scaling of the ambient space R3. At a node p, we define the 3-dimensional vector Up as:
(r(
Σ Tp
q --P [1g - pll
11g - pll"-1 [ 1191 pll--1
NO
BEN,
where N, = [N denotes the degree of node p, and the projection Ty onto the tangent plane T,MM is I - n The factor 1/N3/2 is included in order to guarantee correct asymptotic behavior of (p(r) as the node degree Np increases; for a detailed discussion this normalizing factor, see Appendix B.1.
RELTAN features provide convenient geometric input, as they satisfy the following properties:
Lemma 4.1. For any r E R, the process of computing relative tangential features vp(r) equivariant under global rotations, and invariaad under translations and scoling of RR3 Namely, if SO(3) is rolation, RR3 is a translation vector, and X >> is a scaling factor, then
[Proof]
nM Rp
R(
u = up
M
M+ = =
u
See Appendix B.2 for a proof of this result. Thanks to their geometric propertics, RELTAN features constitute a simple, equivariance-frien friendll alternative to using raw node positions as input features.
Influence of the relative power r. Each of the neighbors E N, affects the computation of up(r) in two ways. First, there a directional component xq (10-p1) which considers the alignment between the edge connecting to p and the tangent plane at p. Second, the distances between all neighbors and p used to weigh the directional contributions: neighbors with smaller distances contribute more.
Note that the effect of the distances is mediated by the power r - 1. We refer to the real-valued parameler r as the relative power. As the relative power r decreases, the contributions of far-away neighbors are highlighted. In contrast, as the relative power increascs, neighbors close to p become most relevant the computation of (r).. In particular, when r 1, the distances are ignored and only the directional components affect the final value of (r)).. These behaviors illustrated in Fig.

---

Published in Transactions on Machine Learning Research (08/2022)
91
Up (-2)
41
2y (1)
Ep(
20
Up
P
Q2
P
40
20
42
Up (3)
q3
Figure 1: Examples of computation of RELTAN features on planar neighborhoods. Orange, blue and green vectors represent relative tangent features (p(r) for r = 1, r === -2 and 3, respectively. For small relative powers r, neighbors far from p contribute more to the relative tangent feature up(r,, and vice versa.
Selecting relative powers. As discussed above, different values of r provide different perspectives on the local geometry of the mesh M around the node p. Balancing the importance of the directional and distance components may depend on domain-specific properties of the data. Moreover, multiple relative powers can be simultancously used for capturing information of the local neighborhoods "at different scales". Which of these scales is most relevant for the task at hand can be in turn learned as part of the optimization of the model weights during training. Note, however, that this strategy increases the number of parameters in the model. Hence, one should use enough relative powers that can capture rich information about the nodes while not being computationally wasteful. In our experiments, choosing two relative powers simultancously provided desirable performance.
5 Verifiably Equivariant Message Passing
In this section we empirically verify that RELTAN features, coupled with a suitable choice of bias for the convolutional layers, allow us to build models that are in fact equi/invariant to all the transformations mentioned in Section 3.4. We highlight the effect that "small" design choices, such as biases, can have when trying to integrate them towards building fully equivariant pipcline. We also emphasize on the importance of performing a thorough evaluation of the model by applying the transformations of interest to unseen inputs in order to reliably verify the desired equivariance properties.
p1(b)
(I(b)
Designing equivariant biases. The traditional way of including biases in standard convolutional layers involves the addition of a fixed vector across the different channels of the output tensor. However, in the context of mesh data, this procedure is not equivariant to changes of gauge. When consider- ing geometric features, the addition of this "fixed" bias vector would correspond to summing gauge- sensitive quantity to the coordinate vectors repre- senting a feature. As a response to this, we consider angular biases that respect gauge equivariance.
P
fp
Figure 2: Angular bias applied to the tangential fea- tures in the convolution blocks. Original features are coloured in red, bias in purple, new features in blue. Equivariance is preserved by rotating -fettfeatures
Given a general represcntation P, we decompose in its irroducible components {px}. The cumulative bias for is assembled from the biases its irreducible components. Ifn = O, we may add a simple (non-angular) bias b to the 1-dimensional scalar feature f. For > 0, we instead rotate the 2-dimensional coordinate vector fy by an angular bias b, or equivalently we consider (b) See Section 5. Therefore, for feature of type P. the number of involved biases is the same as the number of irreducible components in This choice of bias is therefore suitable for our goal of designing a fully equivariant model.
Verifying model equivariance. Table 1 compares the violation of equivariance to global/local transforma- tions exhibited by randomly initialized models, 1 We consider a SpiralNet++ (Gong et al., 2019) using raw
We untrained models since we are interested assessing the equivariance of the model, rather than its accuracy.
7

---

Published in Transactions on Machine Learning Research (08/2022)
positions as imputs; and GEM-CNN model (de Haan et al., 2021) using raw positions, RELTAN features and GET features (He et al., 2021) as inputs, well as the non-equivariant, and angular biases mentioned above, GET features correspond to raw node positions, represented using the local coordinate system at each mode. A detailed description of GET features is given in Appendix H. Since we are interested in obtaining a given rolation, we do not.
model that is equivariant to arbitrary gange changes, and not only to multiples employ (quantized) regular non-linearities (de Haan et al., 2021, $4).
A GEM-CNN model with RELTAN features and angular bias is the only configuration that achieves equivariance to all the considered transformations. Note that SpiralNet++ is perfectly gauge equivariant as it only employs scalar features. The larger magnitude observed for the minimum error achieved for gauge transformations is explained by the fact that gauge transformations affect every convolutional layer of the model, causing numerical errors to accumulate.
Table 1: Equivariance gap of randomly initialized models for various transformations on FAUST meshes. The gap is computed as the MSE between the logits for the outputs corresponding to the mesh with without given transformation. The reported value is average across all test meshes. Entries with small (resp. high) error are shown in blue (resp. red). Values in this table should be compared within the same column, based on the order of magnitude of the realized errors.
Equivariance Gap
Model
Bias
Initial Features
Gauge
Rot-Tr-Scale
Perm
Spi
XYZ
1.03-10-1
Non-Equiv
XYZ
1.43-10-|
6.55-10*
2.26-10-13
GEM-CNN
XYZ
7.95-10-9
1.19
1.55-10-13
Angular
GET
8.75 -10-6
2.60
1.69-10-13
RELTAN
1.31-10-3
5.57-10-9
1.88-10-13
The importance of tansformations in evaluation. We complete this section by studying the robustness of a similar set of models after training on the FAUST dataset. Experimental details can be found in Appendix E. We do not apply any transfornations to meshes in the training set. The results displayed in Table show that the pattern of robustness these transformations carries out verbatim, from that of untrained models shown in Table 1.
Table 2: Accuracy for models trained FAUST. No data augmentation is applied during training. The last 4 columns represent the performance of the model on the test set under different transformations. Blue entries show robustness to transformations for each column, whereas the red entries correspond to poor performance.
Accuracy (%)
Model
Bias
Initial Features
Train
Test
Gauge
Rot- Tr-Scale
Perm
Net
XYZ
100.0
99.91
99.91
0.30
99.91
Nqniv
XYZ
99.99
99.90
0.06
12.48
99.90
XYZ
99.45
97.97
96.85
0.17
97.97
GEM-CNN
GET
99.18
97.75
97.40
0.61
97.75
Angular
TELTAN (0.
99.68
98.69
98.20
98.69
98.69
TAN 0.7
99.63
98.36
97.84
98.36
98.36
However, we highlight that the shorteomings of the non-equivarianl models cannot be detected by looking only at the un-transformed training and test set accuracies! For example, an inadequate choice a "small" component like the bias used in the convolutional layers can drastically affect the cquivariance of the model: compare the equivariance to gauge transformations for GEM-CNN model with (equivariant) angular bias and with (non-equivariant) additive bias. Therefore, when evaluating equivariance-simed models, applying
8

---

Published in Transactions on Machine Learning Research (08/2022)
the transformations of interest is crucial for successfully validating whether model indeed satisfies the desired equivariance properties.
We do not include accuracies for the SpiralNet+++ model with RELTAN features. However, one finds that this model with RELTAN features is invariant to translations and scalings, as RELTAN features are invariant these transformations. Nonetheless, the SpiralNet+++ model with RELTAN features not global rotation equivariant, since its layers are not designed to be compatible with rotations of the ambient space R We included short note node permutation equivariance of the SpiralNet++ model in Appendix D.
6 Equivariant Mesh Attention Networks
Equipped with "fully equivariant" pipeline comprising base GEM-CNN model with angular biases and the use of RELTAN features, we now proceed to the presentation of our proposed attention mechanism.
The typical update step employed in GCNs considers the information coming from all the neighbors E NNp to be equally imporlant when computingg the updale node p. The similarity or alignment between the features fp and fg is irrelevant. Veličković et al. (2018) introduced Graph Attention Networks (GATss to address this exprossivity issue. In the update step, the mossage passed from neighbors scaled using attention weights dependent on Sp and fg.
Equivariant Mesh Attention Networks (EMAN) are the second contribution of our work. EMAN combines ① anisotropic kernels (de Haan et al., 2021), with @ attention coefficients Org relating neighboring nodes (Veličković et al., 2018). The kernel is gauge equivariant, and the attention coefficients are scalar features (namely, unaffected by gauge transformations).. This combination results gauge equivariant attention model. The convolutional update for EMAN is given by:
S/ E Opg K(0 Pm OEN,
(9)
Equivariant Mesh Attention Layers. Algorithm provides an overview of the design of our allention mechanism. The definitions of the quantities involved are presented below.
Algorithm Convolutional update in Equivariant Mesh Attention Layer
Forward (Up)nEM, Kquery, Keey(0) K Kvalue for p E M:
Qp p <- Kp
K < Concatenate( K key (0pg)) (2g for q E NN))
V < Concatenate Kvalno (0pg) (0pg) )in Pin -p) for q E N₂)
K Q2
f + N₂ Vp soft tmax
Output: (fy)EM
We start by considering an auxiliary representation Patt: SO(2) + BCAt, Let Kquery be a Cast Oi matrix. Consider families Kky((0) (0) and (0) of matrices of size Catt x Cin and Cout x Cin, respectively. Given node on the mesh for every neighbor q Npp compute:
Q Kqueryfps
Km
KKey (0pm) Pmn
V pm = Kvatuo
(00a (0pg) Sp
(10)
To provide gauge cquivariance, we impose constraints on the matrices Kquery, Kaey and K These constraints must. be respected for all g S2))). The solutions to these equations provided in Appendix A.
Kquery Patt (-g) Kquery Pin Pmn(g).
Key (0 g) Patt (-9) KRey (0) Pin (g)
(11)
K Kvalue (0 g) Pout (-9) g) Kvalue (0) (g)
9

---

Published in Transactions on Machine Learning Research (08/2022)
We define Kp and Vp the Catt Np and Cout Np matrices obtained by concatenating columns the vectors K pq. and V pq: respectively, over the neighbors q E Ng:
Kp Concat(K pq for E N
p = Concat (Vo pq for q E Np).
(12)
The value of the updated feature fp (of dimension Cout) is then given by:
asa
K Qp VCatt
K, Qp VCate
S2 Σ
Np softmax
K Kvauuo (ppg)in (gg (gg-sp)gg = Np Vp sofumax
(13)
in GEM-CNNs
Note that the factor Np is not present in the work of Veličković et al. (2018). We introduce it here that Eq. (13) precisely generalizes GEM-CNN convolution (with no self-contribution), in the case where the components of the softmax vector are all equal to I/N, (compare with discussion in Section 3.5).
Equivariance properties of EMAN. Thanks to the constraints satisfied by the kernels k Kkey (0), and vallue (0), we obtain gauge equivariance for EMAN (Lemma 6.1). This illustrated in Fig.
Lemma 6.1. The convolutional update in Eq. (13) is gauge equivariant. [Proof]
gunge change
p( fg)
Q1
fyr
422
O'p4s
axpqz
qo
q
Q2
fp
fgo
Orgo
pl fl
40
p(-9)fp
fqx
pl -9)500
angregation
aggregation
p(-))5
gange change
Figure 3: Message passing mechanism in Equivariant Mesh Attention Networks. For convenience, we represent a planar portion of the mesh and therefore ignore parallel transport. Tangent vectors fp. fy. are aggregated according to the attention coefficients Opg: (on the figure, going from top to bottom). change of gauge reference neighbor (from 4o to q1) determines a rotation p(-g) on tangent vectors of angle -9 (on the figure, going from left to right). Attention coefficients are invariant under gauge change. Pictorially, gauge equivariance can be rephrased as: "go right, go down" is the same "go down, go right".
Lemma 6.2. With the choice of kernels given by Eq. (11), the Equivariant Mesh Attention convolutional update in Eq. (13) (and thus Algorithm 1) is equivariant to global rotations, and invoriant. to translations and scalings of the ambient space R [Proof]
The complete pipeline propose in this work is formed by ① the use of RELTAN features (Section 4) inputs, ② angular biases (Section 5) in the convolutional operators, and 3 EMAN layers (with the update rule from Algorithm 1). In conjunction, all these components contribute to model architecture which is equivariant with respect to the complete range of transformations we are interested in:
Theorem 6.3. With initial relative tangential features, Equivariant Mesh Attention Networks are equivariant to node permutations, and invariant under global rotations, translations and scalings of the ambient space R23, and under arbitrary gauge changes. [Proof]
Self-contribution. The self-contribution Kself fp is not present in Eq. (9). The base implementation of GEM-CNNs (de Haan al., 2021) we used as baseline did not include the self-contribution in the
10

---

Published in Transactions on Machine Learning Research (08/2022)
convolutional step. To perform a fair comparison between attention mechanism and GEM-CNNs, do not include the self-contribution in our implementation either and leave experiments with self-contribution to future work. However, our formulation can be easily extended to include self-contributions. We present the details of this extension in Appendix C.5.
Multi-head attention. Our model can also support multi-head attention. See Appendix C.4 for details. We did not notice an improvement in performance when integrating this factor in our implementation for the considered tasks. The experimental results presented below consider single-head attention only.
7 Experiments
We carry out experiments on the FAUST (Bogo et al., 2014) and TOSCA (Bronstein et al., 2008) datasets for segmentation and classification tasks, respectively. We compare GEM-CNN and EMAN models using raw node XYZ positions, GET and REI.TAN features as inputs. All the models use the angular biases described in Section 5. Details on our experimental settings can found Appendix E.
No transformations are applied to the training meshes. All our experiments involving test accuracy report test results applying different transformations to the unseen meshes.
Since interested in obtaining a model that equivariant to arbitrary gauge changes, and not only to multiples of given rotation, do not employ (quantized) regular non-linearities (de Haan et al., 2021, 54).
In Appendix F we provide performance comparison between equivariant models, and non-equivariant models trained using data augmentation. Appendix G contains a time-complexity comparison between GEM-CNN and EMAN.
Comparison with GET (He et al., 2021). Both GEM-CNN and EMAN are equivariant arbitrary gauge transformations unlike the GET model. From an equivariance perspective, our proposed attention mechanism more powerful than that of GET. For this reason, in the experiments below, the lines labeled GET correspond to retaining the CEM-CNN and EMAN convolution and using GET features inputs.
Relative powers. For models using RELTAN features, we consider two choices of relative powers. Relative langential features allow us to choose a different relative power to be used for each of the channels in the input feature (see Section 4). This increases the expressivity the model as different relative powers induce a processing of the local geometry at point in the mosh. We find that using multiple channels with different. relative powers translates into performance improvements on the (Lransformed) Lest set.
7.1 Segmentation
The FAUST dataset consists of 100 (80 training, 20 test) 3-dimensional human meshes with 6890 vertices each. Nodes in each mesh are numbered so that nodes corresponding to the same "location" on the human meshes are labelled with the same number. The goal of the model is to predict, given an embedded mesh, the label for each of the nodes in the mesh.
Table 3 illustrates the performance of various gauge equivariant models on FAUST. We highlight that, rather than an overwhelmingly better prediction performance, the core advantage of using GEM-CNN or EMAN with REL TAN featuresfeatures is that the models become fully equivariant to wide range of symmetries.
Note that models employing XYZ or GET features as input fail to be equivariant to rolations, translations and scaling transforms (Rot-Tr-Scale), This challenge is reliably overcome by RELTAN features. Using several relative powers along with with attention provides a slight boost in performance. Finally, note that solely evaluating the performance of the models on the un m-transforrmed test set would not have been sufficient detect the lack of equivariance to Rot-Tr-Scal transforms of the models that use XYZ GET features.
11

---

Published in Transactions on Machine Learning Research (08/2022)
Table 3: Means (and standard deviations over 5 seeds) of the segmentation accuracy on the FAUST dataset. No data augmentation is appliod during training. Last columns represent the performance of the model on the test set under different trunsformations. All models use angular biases.
Accuracy (%)
Model
Initial Features
Train
Test
Gauge
Perm
XYZ
99.42 (0.15) 97.92 (0.30)
96.90 (0.25)
2.14 (1.49)
97.92 (0.30)
GET
99.42 (0.15)
98.03 (0.17)
97.15 (0.39)
1.47 (1.60)
98.03 (0.17)
GEM-CNN
[AN[
99.69 (0.05)
98.62 (0.06)
98.04 (0.12)
98.62 (0.06)
98.62 (0.06)
RELTAN 0.55, 0.7]
99.70 (0.09)
98.64 (0.22)
97.99 (0.18)
98.64 (0.22)
98.64 (0.22)
XYZ
99.62 (0.09)
98.46 (0.15)
97.26 (0.34)
0.02 (0.00)
98.46 (0.15)
GET
99.60 (0.08)
98.43 (0.17)
97.32 (0.46)
0.02 (0.00)
98.43 (0.17)
EMAN
RELTAN [0.7]
99.27 (1.01)
98.13 (1.19)
97.44 (1.26)
98.13 (1.19)
98.13 (1.19)
RELTAN[O.5, 0.7]
99.68 (0.00)
98.66
(0.07)
98.41 (0.25)
98.66 (0.07)
98.66 (0.07)
7.2 Classification
TOSCA consists of meshes belonging to nine different classes such as cats, men, women, centaurs, etc. While figures in each class are similarly meshed, each class has varying number of nodes and edges. The dataset consists of 80 meshes, which we uniformly split into a train set of 63 meshes and a test set of 17 meshes. The goal of the model is to predict, given an embedded mesh, the class to which the mesh belongs.
Table 4: Means (and standard deviations over 5 seeds) of the segmentation accuracy on the TOSCA dataset. No data augmentation is applied during training. Last 3 columns represent the performance of the model on the test set under different transformations. All models use angular biases.
Accuracy (%)
Model
Initial Features
Train
Test
Gauge
Rot-Tr-Scale
XYZ
97.78 (2.11)
82.35 (5.88)
82.35 (5.88)
12.94 (2.63)
GET
90.79 (2.84)
82.35 (9.30)
82.35 (9.30)
17.65 (7.20)
GEM-CNN
RELTAN[0.7]
93.97 (4.26)
91.76
(6.71)
91.76 (6.71)
91.76 (6.71)
RELTAN 0.7]
90.16 (8.43)
89.41 (14.65)
89.41 (14.65)
89.41 (14.65)
XYZ
47.30 (4.55)
42.35
(20.55)
44.71
(18.88)
12.94 (2.63)
GET
44.13 (7.39)
42.35
(11.31)
41.18 (9.30)
10.59 (2.63)
EMAN
RFI.TAN(0.7]
92.70
(4.14)
94.12
(4.16)
94.12 (4.16)
94.12 (4.16)
RELTAN 0.7]
97.46 (4.14)
98.82
(2.63)
98.82 (2.63)
98.82 (2.63)
We do not apply permutations to the nodes in the test meshes since the different number of nodes across classes makes the implementation of this transformation cumbersome. In addition to our theoretical guarantees, and the empirical verification of permutation cquivariance in the previous experiments, we do not expect node permutations to significantly alfect the behavior of the models for shape classification task since we use a mean pooling layer for aggregating the information across the nodes for this dataset.
We find that when using XYZ features, GEM-CNNs outperform EMANs. This seems to point to a higher sensitivily of EMANs to un-normalized data. This sensitivity is not present when using RELTAN features. We do not normalize the XYZ data in order to emphasize the fact that finding a good normalization strategy becomes unnocessary when using RELTAN features, given their scaling-inarraace properties. In fact, using RELTAN features (with relative powers [0.5, 0.7]), we find EMAN to achieve the best test performance, which is also robust to all the considered transformations.
12

---

Published in Transactions on Machine Learning Research (08/2022)
8 Conclusion
In this work, we propose Equivariant Mesh Attention Networks (EMAN), attention-based model that is equi/invan squi/ivariiant to node permutations, local gauge transformations, as well as global transformations such as rotations, translations, scalings. Our model consists of two major components: relative tangential features (RELTaN) as imput types and a message passing algorithm based on a gauge equivariant attention mechanism. We also emphasize the importance rigorous lesting of the overall assembled model, since small design choices such as biases can result in an non-equivariant model, damaging its robustness transformations in the data. We verify the equi/invarie cqui/invariance of our overall model thcoretically and empirically. EMANS achieve competitive performance on the FAUST and TOSCA datasets, while maintaining equivariance to all the aforementioned transformations.
Acknowledgments
Experiments on the FAUST and TOSCA datasets were performed using the HAL (Kindratenko et al., 2020)
and Mila compute clusters.
This research is the result of a collaboration initiated at the London Geometry and Machine Learning Summer School 2021 (LOGML). This work utilizes resources supported by the National Science Foundation's 's Major Research Instrumentation program, grant #1725729, as well as the University of Illinois at Urbana-Champaign SB was supported in part by the Department of Energy (DOE) award (DE-SC0012704), JGP is supported by the Canada CIFAR AI Chair Program and by an IVADO PhD Excellence Scholarship. FV is supported by the Engineering and Physical Sciences Research Council IEP/S221500/1 (the EPSRC Centre for Doctoral Training LSGNT - UCL and Imperial College London).
References
Federica Bogo, Javier Romero, Matthew Loper, and Michael Black. FAUST: Dataset and Evaluation for 3D Mesh Registration. In CVPR, 2014.
Alexander M Bronstein, Michael M Bronstein, and Ron Kimmel. Numerical Geomelry of Non-Rigid Shapes. Springer Science & Business Media, 2008
Michael Bronstein, Joan Bruna, Taco Cohen, and Petar Velickovie. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv:2104.13/78 2021.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language Models are Few-Shot Learners. In NewrIPS, 2020.
Joan Bruna, Wojeiech Zaremba, Arthur Salam, and Yann LeCun. Spectral Networks and Locally Connected Networks on Graphs. arXiv:1312. 6203, 2013.
Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emannele Rossi. GRAND: Graph Neural Diffusion. In JCML, 2021.
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Picter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision Transformer: Reinforcement Learning via. Sequence Modeling, In NewrIPS, 2021.
Taco Cohen. Equivariant Convolutional Networks. PhD thesis, University of Amsterdam, 2021.
Taco Cohen and Max Welling. Group Equivariant Convolutional Networks. In ICML, 2016.
Taco Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. In ICLR, 2018.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling, Gauge Equivariant Convolutional Networks and the Icosahedral CNN. In ICML, 2019a.
13

---

Published in Transactions on Machine Learning Research (08/2022)
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge Equivariant Convolutional Networks and the Icosahedral CNN. In ICML, 2019b.
Pim de Haan, Maurice Weiler, Taco Cohen, and Max Welling. Gauge Equivariant Mesh CNNs: Anisotrropic convolutions on geometric graphs. In ICLR, 2021.
Michaäl Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In NewrIPS, 2016.
Jacob Deylin, Ming-Waa Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pro-training of Doep Bidirec- tional Transformers for Language Understanding. In NAACL-HLT, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR, 2021.
Fabian B Fuchs, Daniel E Worrall, Volker Fischer, and Max Welling. SE(3)- Transforrrss: 3D Transsaannonoion Equivariant Attention Networks. In NewrIPS, 2020.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep Sparse Rectifier Neural Networks. In AISTATS, 2011.
Shunwang Gong, Lei Chen, Michael Bronstein, and Stefanos Zafeiriou. Spiralnet+++: A Fast and Highly Efficient Mesh Convolution Operator. In IEEE, 2019.
Lingshen He, Yiming Dong, Yisen Wang, Dacheng Tao, and Zhouchen Lin. Gauge Equivariant Transformer. In NeurlPS, 2021.
Michael J Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Tch, and Hyunjik Kim. Lietransformer: Equivariant Self-Attention for Lie Groups. In JCML, 2021.
Andrew Jacgle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. Perceiver: General Perception with Iterative Attention. In ICML, 2021.
Nicolas Keriven and Gabriel Peyré. Universal Invariant and Equivariant Graph Neural Networks. In NewrIPS, 2019.
Volodymyr Kindratenko, Dawei Mu, Yan Zhan, John Maloney, Sayed Hadi Hashemi, Benjamin Rabe, Ke Xu, Roy Campbell, Jian Peng, and William Gropp. HAL: Computer System for Scalable Deep Learning, pp. 41 48. Association for Computing Machinery, New York, NY, USA, 2020. ISBN 978145036689..
Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In ICLR, 2015.
Thomas N Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR, 2017.
Jonas Köhler, Leon Klein, and Frank No6. Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities. In JCML, 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner Gradient-based learning applied to document. recognition. In IEFE, 1998.
Junying Li, Zichen Yang, Haifeng Liu, and Deng Cai. Deep Rotation Equivariant Network. Neurocomputing, 290:26 33, 2018
David W Romero and fean-Baptiste Cordonnier. Group Equivariant Stand-Alone Self-Attention For Vision. In ICLR, 2021.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. Equivaaan Graph Neural Networks. In ICML, 2021.
14

---

Published in Transactions on Machine Learning Research (08/2022)
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The Graph Neural Network Model. TEEE Transactions on Neural Networks, 80, 2009.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In IJCAT, 2021.
Tvan Sosnovik, Michal Szmaja, and Arnold Smeulders. Scale-Fquivariant Steerable Networks. In ICLR, 2020.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR, 15(1):1999-1958, 2014.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Ti, Kai Kohlhoff, and Patrick Riley. Tensor Field Networks: Rotation- and Translatoon-Equivariatt Neural Networks for 3D Point Clouds. orXiv:1802.082. 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Atlention All You Need. In NewrIPS, 2017.
Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling, Rotation Equivariant CNNs for Digital Pathology. In MICCAI, 2018.
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph Attention Networks. In ICLR, 2018.
Nitika Verma, Edmond Boyer, and Jakob Verbeek. FeaStNet:: Feature-Steered Graph Convolutions for 3D
Shape Analysis. In CVPR, 2018.
Yude Wang, Jie Zhang, Meina Kan, Shiguang Shan, and Xilin Chen. Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation. In CVPR, 2020.
Maurice Weiler and Gabriele Cesa. General E(2)-Equivariant Steerable CNNs. In NeurIPS, 2019.
Maurice Weiler, Fred A Hamprecht, and Martin Storath. Learning Steerable Filters for Rotation Equivariant. CNNs. In CVPR, 2018.
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate Independent Convolutional Networks Isometry and Gauge Equivariant Convolutions on Ricmannian Manifolds. arXiv:2106.066% 060200, 2021.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhshi Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep Sets. In NewrIPS, 2017.
15

---

Published in Transactions on Machine Learning Research (08/2022)
Appendix
A Geometric Kernel Constraints in GEM-CNNs
Table 5: Solutions to the angular kernel constraint for kernels that map from Pn to Pm, where ct cos(mmtn)
and $ = sin((m t n) This table is directly taken from de Haan et al. (2021).
Pin - Pout
linearly independent solutions for K neigh (0)
Po + Po
(1)
Pn + Po
(cos no sin nθ), cos mf sin m0
(sin cos no)
)
sin mf
Po + Pm
cos ml
C-
C+
84 C+
Pn Pm
8+
C+
34
Pin + Pout
linearly independent solutions for Kaett (0)
Po + P9
0 001
0 )
+ Pn
-1 0
Message passing in GEM-CNNs de Haan et al. (2021) is defined by the equation
f = Ksett + gEN,
neiigh (0pg) (9
Gauge equivariance translate on the kernels Keett and Kueigh (0) as
Kaeff Pout(-9) Keelt Pin(g), Kneigh (0 g) Pout(-g) (neigh(O) Pmn(g).
The representation Pin decomposes in irreducible components as @pny and the representation Pout as 0pm: Then, the kernels Ksell and (ee are block matrices obtained by combining possible block kernels (Keelt) and neiigh (f)); from features of type Puy to features of type Pmn.- Linearly independent solutions of the kernel constraint are listed in Table 5 (derived in Weiler & Cesa (2019)). A generic kernel from features of type Ps, to feature of type Pm is a linear combination of learnable parameters of the given basis. Table 5 also provides solutions for the kernel equations in Section 6.
B More Relative Tangential Features
B.1 The exponent 3/2
In this section, discuss the renormalization factor 1/N3/2 present in the expression of RELTAN features. We call unnormalized RELTAN features the same expression, without the renormalization factor. Unnormalized RELTAN features do not scale properly as the number of neighbors grows. In particular, we show that unnormalized RELTAN features at node explode as the node degree increases, under reasonable assumptions. Furthermore, we make the asympthotic behavior of the size of RELTAN features explicit, as a function of the degree of the node. Finally, the rescaling provides an expression for normalized RELTAN features that does not explode nor vanish the node degree increases.
We recall that the expression of REL TAN features is:
119 pl pl
up(r)=
Σ Tp
9- pl
N3/2
ExEN, 1191 -1l
gEN,
16

---

Published in Transactions on Machine Learning Research (08/2022)
Note that may rewrite Up
q q - plr OEN,
Σ
11g pl 1 nEW,
Σ
up Tp
and we focus on the expression inside the square brackets.
Since analyze the asymptotic behavior of the expected size of the relative tangent feature Ups view the vectors y P; y E NN,, as random vectors X;, for i = I,..., N, where N = Np is the degree of The expression for Up becomes therefore
X
Σ
Σ II j=1
We also assume that:
The random vectors X; independent and identically distributed (i.i.d.),
The probability density function of the random vectors X; factors into radial and angular components,
The angular component is a uniform distribution.
Under these assumption, the expected value the squared norm of the vector up
N
N X
N
N Σ f==1
N Σ
|| ||
I | X (x..
E
Σ 1=1
IIIIT
E
Note that the terms with i A 1' in the sum cancel out, as the vectors X, are i.i.d,, and the angular component of the distribution is uniform. The considered expected value then simplifies (only considering = i') as
X
I 1(r-
Σ
E
+J-'==1
This sum presents 3 addends. Notice that these terms are potentially different. As we assume that the variables are ii.d., at least N 3 + 2N terms for which i,j.j' are all distinct are the same. Therefore, the sum scales with a factor of 3 We introduce factor 1/N3/2 into the original vector, so that the considered expected value neither explodes nor vanishes as N becomes large.
B.2 Proof of Lemma 4.1
We recall that RELTAN features the mesh M are given by:
q-P )
[
119 pl p|1--1
D.M
M
q -1]
[ EyeeN, 1191 -pll
N
3/2
gEN,,
where we make explicit the given mesh M.
Equivariance under global rotations of R Let RE SO(3) be a global rotation R and denote by RM the mesh obtained by rotating M according to R. Notice that the set of neighbors é e NRp of the node Rp in the mesh RMM is the set of points Rg, q E Np for the mesh M. Also, the normal vector n mesh RM at the node Rp is nothing but Rn... Consequently,
RM to the
RM T Rp
= --
RM RRM T =I- - -Rn, (m R. Rw.. R. -nRp (n
17

---

Published in Transactions on Machine Learning Research (08/2022)
where we used that RR I. The relative tangential feature at node Rp for the mesh RM is
[1e - Rpll Rppl) )
e- Rp
11e Rpll"
RRM
RM
Rp
N
Thy
E 11 Rppl--3
Rp
BENRs
R(q -p) - [|R(q -DDT)
||[{q - -P)II-
Σ
RM R Rmp
No
3/2
EETEN, 11 -p]ll--
qENp
119 pll"-1
RT
RR p
RM
--p p [[(g - p)ll
Rup
3/2
EV'EN, 1101 - -pli
GEN,
that is equivariance under global rotations.
Invariance under translations of R The argument. is similar. If x E IR3 determines a translation, denole by M + x the translated mesh. The set neighbors E Npt of the node + x in the mesh M + x is the set of points q + I, as q E Np for the mesh M. Also, the normal vector ppt to the mesh M at the node + x. is the original nAM, and therefore M+z mp Hence,
M+x
M+z L (p (-( Tptz
lle - (p + x)]l"-1
ur M+a p+
N 3/22 p+= REN
[1 (p x>)l
EVENtt 11
+ xx)
lla pl|"-1
q - P lqq - pl
M p
Np
p
Eg'EN, 114 - -pl(
QEN,
that is invariance under translations.
Invariance under scaling of R Again, similar argument. Let À > 0 be the scaling factor, determining the map p + Ap for p E R3. The set of neighbors E NAp of the node Ap in the mesh AM is the set of points AM = mp.. Thus, as À > 0, Aq, as q E Np for the mesh M. As above, we doduce that π Ap
8- (Ap)
I1e - (Ap))l-=
AM Xp
Σ
AM
3/22
xx
||E - (1p)
ECENsp 11" (Av)ll--
Ap
BENx.
||(4 - -111) )
Aag A(p-p) p)
11 - -p)ll--1
Σ Tp
ExeEN 11 -)]
Np
--p
ll --pppl1--1
3/2 No
"p
[1(g-p))l) - p)1l
Σ EyEN, ll(g' - -p)l[-- 0p
gEN,
that is invariance under scaling.
C Equivariant Mesh Attention Layer: proofs, multi-head, and self-contribution
C.1 Proof of Lemma 6.1
The proof boils down to two core steps. First, the detmax--rggument is invariant under gauge transformation. In addition, multiplying the matrix V, (that transforms feature of type Pout) with the invariant. softmax-vector produces a feature of type output. Here we provide a detailed proof.
Under gauge transformation g E SO(2), the coordinate vectors Sp and Pmu Pi at p transform as
fp +
Pmn(o--p)f piol pm(-9))pm(0m-p))
18

---

Published in Transactions on Machine Learning Research (08/2022)
Also, the angle 8 changes + 0 - 9 under the same gauge transformation. Using these relations, together with the ones expressed in Equation 11, we see that Qp and the K pq transform as features of type Patt; while the V transform as features of type Pout:
Qp > Paul Pau(-g))pp
py ++ Pare Kg
Vpg + Poue Poucl (-)) Vg.
We see this explicitly, for instance, for the case of Kpy
Kay- = Kkey (m
+ Kkey (0pg 9) (g ()) Pin (go-
Patt! ( Keyy pm) (g) (g (Pp-
Patt -9) Kkey (0) fq
= Paaul (
where made use of the constraint K (0 g) Patr Patr (-g) Kay (0)pmn()) Computations for the other cases are similar. Being obtained by column-concatenation from Kpg and V pgs the matrices Kp and Vp undergo the same transformations as well:
Kp + Patt (-)) Vp + P-ot(-g)Vp
Finally, the convolutional outcome transforms as
KP Patt(gg) g) Patt (at
fo + Pout( Pout(-g) -Np. Vp- softmax
Vasss
KT k VCatt
Powe(-g) Np V, softmax
Pouul f8p
where we used the orthogonality of the representation Patt: namely Patc(-9) (ate(-g)-1 In conclusion, f transforms feature of type Pout, and the proposed method is gauge equivariant.
C.2 Proof of Lemma 6.2
Suppose that R SO(3) is a global rotation of IR, mapping the mesh M to the mesh R.M. Given feature f of type Pmn on M, we represent it at a point by its coordinates fp, with respect to gauge Ep. Then, the rotation R defines a feature f on RM,, with coordinates (R.f) = fp with respect to the gauge REp. Here comes the key remark: the quantities Rg,,2g and gRg-RRp Rp with respect to the gauge REp at Rp for RM are precisely the quantities fy and 9o-pp with respect the gauge E, p for M. Indeed, Iq-op can be computed as
9q-p alan2
pp. (
where Ry is the unique rotation of R that maps ny to up: with fixed axis ny x np:
For ppg instead, we notice that the angle can be written as
Pmg = atan2(e,,2 log, (4), ep,1 (1)),
where log, is the norm por--rreerving discrete logarithmic map
(I np T (g p)
log, /19 pll
[[[1 n (q - p)ll
Therefore, the outcome (R* [) of the convolution at. Rp for RM is equal to the outcome of the convolution f/ at p for M. In other words, the feature mapping defined by the convolutional update is global rotation equivariant.
A similar argument can be applied to global translation T and scaling S: the coordinate vector of the feature do not change under T. or S., the gauge is left unchanged, and the quantities 0yy and 9g-pp are not modified. In conclusion, the convolutional step also translation and scaling invariant. Notice that key property, implicitly used when considering scaling, is the dependence of the kernel only on angles, and not on the radial component.
19

---

Published in Transactions on Machine Learning Research (08/2022)
C.3 Proof of Theorem 6.3
We prove the result for the designed Equivariant Mesh Attention models for segmentation and classification tasks, whose details are described in Appendix E.
Thanks to Lemma 6.1 and Lemma 6.2, each of the three blocks in the convolutional block is gauge and global rolation equivariant, and global translation and global scaling invariant. Therefore, the whole convolutional block satisfies the same properties, and outputs a sum of scalar features. As operations in the dense block are defined on scalar features only, and not involving any quantities related to the geometry of the mesh the same equi/in- varviat properties hold also for the dense block. Finally, thanks to Lemma 4.1, the process of computing RELTAN features is consistent with the above properties, and the whole model is gauge invariant, global rotation equivariant, and global translation and global scaling invariant.
Regarding equivariance under permutation, it is enough to notice that all the operations involved in the convolutional block, and the process of computing RELTAN features, are permutation equivariant. Moreover, the operations in the dense block are defined node-wise, and the same operation on features applied at each node. In conclusion, the whole model is equivariant under permutation (in the segmentation task, and invariant in the classification task).
C.4 Multi-head
It is feasible to incorporate multi-head attention in the Equivariant Mesh Attention layer, and we present here how. However, we did not notice improvement in performance when integrating this factor in our implementation for the considered tasks.
Choose h and d = dmadel such that Cout = dh. For each == == 1:...,, ,,, consider projection matrices of size d Cast, denoted by W querys Wi key: and projection matrix size x Cout, denoted W W value' Also, for each = 1,....h h we fix representation p:: SO(2) - Rd. For these matrices, we require the gauge equivariant conditions
Wi W "uerry = Pi p((-g) Patty)
value Wiey = (e(-g) Why Payt W W - Walte))aaseePPo (g).
Finally, we consider a Cout x Cout matrix WO, We define the representation Pding SO(2) + Rout by block-diagonal concatenation of the h representations Pi- The gauge equivariant condition satisfied by wo is therefore
0 Poun g) w Paing (g).
Then, the multihead attention outcome defined by
MultiHead (Qp, Kp. Vp) = wo Coecat heada);
where
head, = Att(Wi querry query Qp, W key Kpp W W Vp).
C.5 Equivariant Mesh Attention Layer with Cel--oontii
Here provide the details of the convolutional update variant including self-contribntion:
S/ Kaatt +
Σ
Opg (0 Pin (gn-)p)
gENr
In line with Section 6, we consider the quantities
Qp Kqueryfpr
Kpp K'ellf fp: pg Keucigh (ta
V pp
Kaelf& f
V pq Knniigh (9 Pra Pin
Here, Kquery, Kiny and K-neigh (8) are Cat Cin matrices, while Kself and K Kenigh (8) are Cout x Cn matrices. We define K, as the Catt x (Np + 1) matrix obtained by concatenating columns the column vectors K,
20

---

Published in Transactions on Machine Learning Research (08/2022)
and py. as 9 varies. Similarly, Vp is the Cout (Np + 1) matrix obtained via the same procedure from the column vectors V pp and V pq, as q varies. The outcome
K Qp VCau
fp = (N, + 1) v softmax
is column vector of length Cout, and in fact feature of type output. Gauge equivariance conditions for the matrices Kquery; Kkey and Kvalue (0), translates
K querr Patt (- (oggg)KKo querry (g).
Keelf = Patt ( 9) Kseelt Pin (g): key
K Knelgh (0 g) Pattl (- 9) K (0)
K = Pout ( - 9) K Pin
self
K neiggh (0 g) Pout ( 9) K neigh
(0) Ptn (g)
value
D Node permutation equivariance of SpiralNet++
The SpiralNet++ convolution on meshes makes use of spiral sequences around nodes (Gong et al., 2019). Given a node, a spiral length, an orientation, and a preferred starting neighbor, the node sequence that constitutes the spiral is uniquely determined The authors consider a fixed counter-clocwwise orientation for all spiral sequences, and the choice of starting neighbor is arbitrary. With these choices, let S(i, A) denote the indices of the nodes belonging to the spiral sequence of length À starting at node The feature update follows the rule xt,- MLP (I)Es)t.)))))
Section 5, we analyze the response of SpiralNet+++ to different types of transformations, including node permutation. This is equivalent to saying that the indices of the preferred neighbor choices transform according to the permutation. This may represent issue for node permutation equivariance: since the choice of preferred neighbor is arbitrary, unless stored explicitly along with the mesh, it is not possible to guarantee that the "same" choice of starting neighbors will be made after having permuled the nodes. For example, consider common mesh along with two different labelings A and B of its nodes, and an arbitrary choice of starting neighbors under labeling A. It is not possible to infer the arbitrary choice of starting neighbors under labeling B based ouly on the permutation relating the change of labeling + B.
This challenge can be easily resolved by taking into account the geometry of the mesh when choosing starting neighbors: c.g., choosing the closest neighbor in the Euclidean distance preferred neighbor.
E Experimental Details
The input feature type XYZ is 3 x Po: and for RELTAN features it is Po ① P1; where relative tangent features are stored in the PI part of the feature, and the scalar Po is set to zero. GET features also have type Po + Pl, where the Po component stores the projection onto the normal.
Inputs.
Models. For both segmentation and classification tasks, models consist of two blocks: convolution block and a dense block. The convolution block further consists of three sequential gauge equivariant residual blocks. Each residual block consists of two gauge equivariant convolutions followed by a summation of the input to the output of the block. The nat ure of message passing in these convolutions correspond to the choice the model, e.g. GEM-CNN consists of convolutions of the form equation 7, whereas EMAN consists of attention mechanism equation 13.
For each model, the feature type of the input layer matches the feature type of the input features. The final layer of the sequence of residual blocks is of feature type 16 Po, ie., 16 channels with only scalar features.
See Section 3.2 of (Gong et al., 2019) for details the construction of the spirals.
21

---

Published in Transactions on Machine Learning Research (08/2022)
All the intermediate feature types in the model are fixed to 16 x (Pu @ P1 @ P2). The second block consists of two dense layers. The first dense layer is of dimension 16 x 256 and the second dense layer maps to the target dimension followed by softmax function. The output of the first layer is also passed through ReLU Glorot et al. (2011) and a dropout layer with parameter 0.5 Srivastava et al. (2014). The target dimension for the segmentation lask on FAUST is 6890 and for classification task on TOSCA is 9. Further, in the case of classification, we also use mean pooling of the output of the first dense layer over the nodes.
Hyperparameters. We train using learning rate of 0.01 for 100 epochs for FAUST segmentation tasks. In the case of TOSCA, train for 50 epochs and use a learning rate of 2 10-3 for GEM- models, and 7 7-10-A for EMAN models. All tasks use the Adam optimizer Kingma & Ba (2015) and negative log- ligliiklihood loss function.
F Equivariance vs Data Augmentation
Equivariance in machine learning has arisen as a principled alternative to data--ngmentation. Section 5.1 of Thomas et al. (2018) shows that "rotation equivariance eliminates the need for rotational data augmentation" for point cloud data. Here, we show that the same applies to mesh data as well. To this end, we perform experiment the FAUST dataset with data augmentation applied to both the training and test sets and compare the improvements brought by equivariance and data augmentation.
Test Accuracy
RelTan GEMONN
GEMONN
0.95
0.9
0.85
0.8
train/epoch
0.75
20
40
80
100
Figure 4: Effect of RELTAN features test. accuracy during training for GEM-CNN model (averaged over 3 seeds). The results show better performance and lower variance for the model trained using RELTAN features.
Fig. 4 shows the accuracy (over 3 runs) for GEM-CNN models with initial XYZ and RELTAN features trained using rota-translation angmentations on the training set. This experiment confirms that data angmentation improves the generalization of non-equivariant model to unseen voto-translation compare purple line above 90% with 2.14% lest accuracy for GEM-CNN with XYZ features in Table 3.
Despite this improvement, data augmentation is oulperformed by equivariance: the equivariant model (using RELTAN fcatures) learns faster, has better final performance and lower variance during training.
G Time Comparison between EMAN and GEM-CNN
Here we provide a high-level time complexity analysis of EMAN compared to GEM-CNN. The bottleneck computations for both GEM-CNN and EMAN are expressions involving multiple matrix multiplications, of the type K (t computed for every orientation of each edge g - p. For EMAN, additionally, attention coefficients are computed for every orientation of cach edge. Computation of attention coefficients involve the same lype of matrix multiplication as in GEM-CNN. Therefore, we observe an increased time to process features in EMAN than in GEM-CNN. Moreover, both EMAM and GEM-CNN time complexities are proportional to the number of such operations involved in the models, the ratio of runtimes for EMAN and GEM-CNN scales as a constant in the limit of number of edges.
In practice, from Fig. we find that EMAN takes twice as much time as GEM-CNN for the FAUST dataset. However, because of the use of attention mechanism, EMAN surpasses the performance of GEM-CNN within the 30 minutes that GEM-CNN takes complete its 100 epochs. Hence, even though EMAN has higher time-complexity, it can outperform GEM-CNN within a short window of training time.
22

---

Published in Transactions on Machine Learning Research (08/2022)
Test Accuracy
GEMONN
EMAN
0.8
0.7
Time (minutes)
0.6
10
20
30
40
50
60
Figure 5: Wall-time comparison between GEM-CNN and EMAN. EMAN requires twice as long per epoch, due the use of attention. However, given a fixed budget of 30 minutes, the accuracy for EMAN on the FAUST dataset surpasses that of GEM-CNN.
Comparison with Gauge Equivariant Transformers
In this appendix, compare in detail our model with the Gauge Equivariant Transformer (GET) proposed by He et al. (2021). More specifically, we discuss the differences between the choices of initial features, the layer designs, and the effects of the combinations of features and model architectures.
Comparison of features.
We recall the expression of RELTAN features, for node p:
(r(r)=
Σ Tp VEN,
ll9 (
q p-p p pl1
)
11 pl1 EEVEN, l1g' -pll-
N/
where N₂ denotes the degree of node p. The projector Tp onto the tangent plane TpM is I - mppp and the real number is the relative power. The vector Up is 3D vector that belongs to T.MM
On the other hand, GET feature at node p is nothing but the position of the node itself, considered in suitable coordinates. Assume that fep,1, C22) is a frame for T,M, and np is the normal vector to M at p. Then, the coordinates of the GET features with respect to {ep.1, 6p,2 nph. that we denole by Wpp are
up = ((p, ep.)) (p, ep,2) (p. mp))
As noted by the authors in He et al. (2021), GET features are of type Po ① Pl; the po component corresponds to the projection onto the normal vector, and the P1 component to the projection onto the tangent plane. Therefore, GET features are geometric features, and formally constitute a good candidate for initial features of a gauge equivariant model, as discussed in Section 3.2.
We analyze the behavior of Up and Wp when acting with global transformation of the space R3. The vector Up equivariant to rotations of the space R3 as stated in Lemma 4.1. This precisely means that the coefficients of Up with respect to gauge (cp.1.pp.2) are invariant under change of gauge. The same property holds for the coefficients Wpp-
However, the ion looks different for scalings and translations. The vector up is invariant under scalings and translations, as stated in Lemma 4.1. On the other hand, the coefficients up are not invariant under these transformations. scaling of a factor À transforms wa to W.M = > wp and a translation by a vector æ transforms wrM to wpta wyM + ( ex.)) (z, ep.2) (p,np)) (compare with expressions in Lemma 4.1 for the bchavior of TAN features).
Fig. 6 provides a visual interpretation of the differences between RELTAN and GET features. To keep the figure readable, we only show features at node p. RELTAN features remain unchanged compared to Fig. regardless of the choice of global coordinates. On the other hand, translation or scaling of the global coordinates would alter the GET features (not just their coordinate vectors, but the geometric features themselves). Our accompanying code includes notebook with 3D visualization of both types of features.
23

---

Published in Transactions on Machine Learning Research (08/2022)
GET,
Upp 2)
42
(p(1)
Upp 2
GET,
90
P
Up (3)
0/
P Q0
Q2
GET,
Figure 6: Visual comparison between RELTAN and GET features on planar neighborhoods. In contrast. with Fig. 1, we include global coordinate axes in these plots as GET features depend on the absolute node positions (although they are in turn expressed in the local frame node p).
Comparison of the layer designs. The GET layer is only gauge equivariant to multiples of 2x/N, for a certain positive integer N, as He et al. (2021) consider regular representations for the hidden features. They provide a theoretical setting for an extension of regular representations to representations of the whole S0(2) (when N is odd), and develop a framework to deduce the solution to the equivariant constraint. Moreover, they provide am estimation for the error in equviariance for generic rotations in SO(2). In contrast, our model is directly built on top the convolutional kernels of GEM-CNN, for which de Haan et al. (2021) developed an architecture procise oquivariance to arbitrary rotations SO(2).
Comparison of features and models. The GET model architecture applied wp is not designed scale properly for multiplication of factor X, when adding vector (in suitable coordinates). As consequence, the GET model is not scaling and translation equi/in-vrriatt.
A possible solution to this issue is a initial modification of the mesh M, before the computation of the coofficients Upp For instance, we may translate M so that its conter of mass coincides with the origin, and scale so that the average of the norm the nodes p 1. This procedure annihilates the action of potential translations and scalings.
However, we argue that RELTAN features, in contrast with GET features, present another characteristic that makes them favorable for mesh processing. Their expression is local, as it involves the compulation of relative quantities (the vectors 4-242220 and their norms) among the neighbors of node Moreover, discussed in Section 4, different values of the relative power r detect different aspects of the neighboring disposition around p, and therefore the local geometry of the mesh at Opposed to this, mere positions are prescribed in global fashion, as they strictly depend on the embedding of the mesh R3 and are not defined by the local geometry (that is, the neighboring nodes). Our experimental results show that the choice of REL TAN features is preferable to simple node positions.
24


Reviews:
[{"review_id": "HHIkFXnrKX", "reviewer": "Reviewer_AwSq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:  \n1. EMAN seems to be novel. \n2. The computational cost is compatible to GEM-CNN. \n\nWeaknesses: \n1. The mathematical writing can be improved. For example, the notation of a tangent plane should be defined before its first appearance (e.g., in the first sentence of the second paragraph of Section 2.1), $N_p$ is not defined (though can be deduced), the meanings of * in $R_*$ and $T_*$ are unclear, the equation between (9) and (10) should be written in terms of $K_{pq}$ and $V_{pq}$, and \"cat\" in Algorithm 1 lacks a definition. \n2. The last paragraph of Section 3 emphasizes that (7) does not involve self-contribution but does not provide any reason except that the performance is already satisfying. How is the performance if self-contribution is included?  \n3. The insight provided in Appendix B.1 is unclear. What is the reason of the analogy between q-p and a random vector? \n4. I do not see any principled way to rule of thumb to tune the relative power. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a equivariant mesh attention (EMAN) network that is gauge equivariant and invariant with respect to rotation, translation, and scaling. The two main ingredients are the relative tangent feature and a combination of attention and GEM-CNN. The equivariance and invariance properties are verified theoretically and empirically. Experiment results show that the proposed EMAN performs well on FAUST and TOSCA datasets. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Please address the weaknesses above and revise the paper is necessary. ", "broader_impact_concerns": "I do not see any such concerns. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JWiKbnlUS3", "reviewer": "Reviewer_huat", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method to be novel\n- The proposed method doesn't not require modifications (e.g. data augmenation) during training to achieve equivariance\n- The proposed method is empirically better than previous methods in the experimental evaluation. The usage of relative tangential features is very effective. \n\nWeak:\nI do not have major concerns with the submission besides I think the presentation could be made clearer. I detail my suggestions in the next section. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the question of how to design equivariant graph convolutional networks for 3D meshes. \n\nThe proposed method first construct relative tangential features that transform absolute node positions in 3d coordinates to features based on tangential vectors that are not tied with any absolute position. To learn on these relative tangential features, the submission proposes to employ a prior method, gauge equivariant convolution. The submission extends gauge equivariant convolution with a gauge equivariant attention mechanism. Furthermore, the submission designs a better way to include convolution bias (anguilar bias) that ensures gauge equivariance.\n\nThe submission evaluates the proposed method on two benchmarks, FAUST and TOSCA, and showed that the proposed method is more robust to different transformations applied onto the test data. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- Figure 1 is not referred to or explained in the main text and I am also not sure I understand the figure well. Can the authors clarify what this figure is intended to mean?\n- To clarify on the experiment setup, is the parallel transport also applied to the baseline method GEM-CNN? ", "broader_impact_concerns": "I do not have concerns with the broader impact of this paper,", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5QBd18F94a", "reviewer": "Reviewer_s8xe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* I think the idea of the relative tangent features appears to be neat, simple and in the experiments conductive effective.\n* The ablation studies against the baseline show that the use of these features in both convolutional and attention based models improves robustness.\n\nWeaknesses\n* Clarity: The clarity of the idea can in several places in improved. As someone who is aware, but not an expert in neural architectures for meshes, I had to read through several other pieces of related work to understand the core contribution. I think a big part of this is that the visualizations considered in the manuscript do not go far enough in providing intuition to the reader. I think having many more visualizations of how the proposed tangential features look on very simple meshes can go a long way towards improving the readers understanding on the utility of the features. There is a statement in the abstract that this is the only work to look at accuracy on the test set which reads rather strangely. \n* Experimentation and Related work: In terms of related work, the most relevant is that of He. et. al who propose guage equivariant transformers [GET]. It is difficult to attribute contribution (b) (the use of attention) solely to this work since He. et. al has already been published in the literature. In addition, the manuscript states that the key difference between this work and He et. al is that this work designs features equivariant to rotation, scale and translation (as opposed to GETs that only equivariant to rotation). I think this claim is one that can and should be tested empirically since GETs (unless I'm missing something) are the most directly related work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles the problem of creating functions on meshes that are equivariant to transformations of the input such as rotations, scaling and permutations. The starting point of this work are guage equivariant mesh convolutional neural networks (GEM-CNN) which were among the first to design an architecture for learning functions over meshes. In that context, this paper makes two modifications:\n* They propose the use of relative tangential features -- features that by design satisfy the equivariance properties and\n* Propose the use of gauge equivariant attention mechanisms to parameterize the relationships that the relative tangential features have with outcomes of interest.\n\nThey show that the combination of the two results in models that are equivariant to translations, rotations, scaling, and permutations.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "\nThe changes requested are in the form of clarity and writing as well as experimentation:\n\nA] Clarity\n   * I recommend clarifying the statement in the abstract that this work is the only one to look at test accuracies. As far as I can tell, Guage Equivariant CNNs also report numbers on the test set so I'm not sure what this statement is trying to convey.\n   * I recommend creating a figure to visualize relative tangent features (perhaps with a side-by-side visualization against the choices made in GETs for comparison) since this is the core contribution of this work. \n\nB] Experimentation & Related work:\n   * I think the paper could have done a better work of placing the work experimentally in relation to GETs in identifying the strengths and weaknesses. One idea on this front would be to empirically evaluate the claimed rationale showing that this work outperforms GETs when meshes are translated and scaled.", "broader_impact_concerns": "There are no major (to my knowledge) broader impact concerns with this manuscript.\n", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a equivariant mesh attention (EMAN) network that is gauge equivariant and invariant with respect to rotation, translation, and scaling, based on relative tangent feature and a combination of attention and GEM-CNN. Three reviewers provided in-depth reviews of the submission, and they all see the potential value of the work. On the other hand, they also raised a number of minor concerns, including:

1) The mathematical writing can be improved.
2) The principled way of tuning the relative power is lacking
3) The placement of some figures and the necessary explanation of them can be optimized.
4) The description of some experimental details are not cleary enough, and some experimental settings (including the comparison with GET) can be improved.

Overall speaking, the authors have done a good job in addressing the concerns from the reviewers. Some reviewers have explicitly confirmed that the revisions have addressed their concerns. Although some other reviewers have not made their confirmations, as far as I can see, most of the issues have been removed too. In this way, I would recommend ACCEPTING the manuscript subject to some minor revisions. When submitting the final version of the paper, I would like to ask the authors to proof-read the paper carefully to address any remaining writing issue.
Decision: Accept (poster)