Isometric Transformation Invariant and Equivariant Graph Convolutional Networks

1 INTRODUCTION . Graph-structured data embedded in Euclidean spaces can be utilized in many different fields such as object detection , structural chemistry analysis , and physical simulations . Graph neural networks ( GNNs ) have been introduced to deal with such data . The crucial properties of GNNs include permutation invariance and equivariance . Besides permutations , isometric transformation invariance and equivariance must be addressed when considering graphs in Euclidean spaces because many properties of objects in the Euclidean space do not change under translation and rotation . Due to such invariance and equivariance , 1 ) the interpretation of the model is facilitated ; 2 ) the output of the model is stabilized and predictable ; and 3 ) the training is rendered efficient by eliminating the necessity of data augmentation as discussed in the literature ( Thomas et al. , 2018 ; Weiler et al. , 2018 ; Fuchs et al. , 2020 ) .. Isometric transformation invariance and equivariance are inevitable , especially when applied to physical simulations , because every physical quantity and physical law is either invariant or equivariant to such a transformation . Another essential requirement for such applications is computational efficiency because the primary objective of learning a physical simulation is to replace a computationally expensive simulation method with a faster machine learning model . In the present paper , we propose IsoGCNs , a set of simple yet powerful models that provide computationally-efficient isometric transformation invariance and equivariance based on graph convolutional networks ( GCNs ) ( Kipf & Welling , 2017 ) . Specifically , by simply tweaking the definition of an adjacency matrix , the proposed model can realize isometric transformation invariance . Because the proposed approach relies on graphs , it can deal with the complex shapes that are usually presented using mesh or point cloud data structures . Besides , a specific form of the IsoGCN layer can be regarded as a spatial differential operator that is essential for describing physical laws . In addition , we have shown that the proposed approach is computationally efficient in terms of processing graphs with up to 1M vertices that are often presented in real physical simulations . Moreover , the proposed model exhibited faster inference compared to a conventional finite element analysis approach at the same level of accuracy . Therefore , an IsoGCN can suitably replace physical simulations regarding its power to express physical laws and faster , scalable computation . The corresponding implementation and the dataset are available online1 . The main contributions of the present paper can be summarized as follows : • We construct isometric invariant and equivariant GCNs , called IsoGCNs for the specified input and output tensor ranks . • We demonstrate that an IsoGCN model enjoys competitive performance against state-ofthe-art baseline models on the considered tasks related to physical simulations . • We confirm that IsoGCNs are scalable to graphs with 1M vertices and achieve inference considerably faster than conventional finite element analysis . 2 RELATED WORK . Graph neural networks . The concept of a GNN was first proposed by Baskin et al . ( 1997 ) ; Sperduti & Starita ( 1997 ) and then improved by ( Gori et al. , 2005 ; Scarselli et al. , 2008 ) . Although many variants of GNNs have been proposed , these models have been unified under the concept of message passing neural networks ( Gilmer et al. , 2017 ) . Generally , message passing is computed with nonlinear neural networks , which can incur a tremendous computational cost . In contrast , the GCN developed by Kipf & Welling ( 2017 ) is a considerable simplification of a GNN , that uses a linear message passing scheme expressed as Hout = σ ( ÂHinW ) , ( 1 ) where Hin ( Hout ) is an input ( output ) feature of the lth layer , Â is a renormalized adjacency matrix with self-loops , and W is a trainable weight . A GCN , among the variants of GNNs , is essential to the present study because the proposed model is based on GCNs for computational efficiency . Invariant and equivariant neural networks . A function f : X → Y is said to be equivariant to a group G when f ( g · x ) = g · f ( x ) , for all g ∈ G and x ∈ X , assuming that group G acts on both X and Y . In particular , when f ( g · x ) = f ( x ) , f is said to be invariant to the group G. Group equivariant convolutional neural networks were first proposed by Cohen & Welling ( 2016 ) for discrete groups . Subsequent studies have categorized such networks into continuous groups ( Cohen et al. , 2018 ) , three-dimensional data ( Weiler et al. , 2018 ) , and general manifolds ( Cohen et al. , 2019 ) . These methods are based on CNNs ; thus , they can not handle mesh or point cloud data structures as is . Specifically , 3D steerable CNNs ( Weiler et al. , 2018 ) uses voxels ( regular grids ) , which though relatively easy to handle , are not efficient because they represent both occupied and non-occupied parts of an object ( Ahmed et al. , 2018 ) . In addition , a voxelized object tends to lose the smoothness of its shape , which can lead to drastically different behavior in a physical simulation , as typically observed in structural analysis and computational fluid dynamics . Thomas et al . ( 2018 ) ; Kondor ( 2018 ) discussed how to provide rotation equivariance to point clouds . Specifically , the tensor field network ( TFN ) ( Thomas et al. , 2018 ) is a point cloud based rotation and translation equivariant neural network the layer of which can be written as H̃ ( l ) out , i = w ll H̃ ( l ) in , i + ∑ k≥0 ∑ j 6=i W lk ( xj − xi ) H̃ ( k ) in , j , ( 2 ) W lk ( x ) = k+l∑ J=|k−l| φlkJ ( ‖x‖ ) J∑ m=−J YJm ( x/‖x‖ ) QlkJm , ( 3 ) where H̃ ( l ) in , i ( H̃ ( l ) out , i ) is a type-l input ( output ) feature at the ith vertex , φ lk J : R≥0 → R is a trainable function , YJm is the mth component of the J th spherical harmonics , and QlkJm is the ClebschCordan coefficient . The SE ( 3 ) -Transformer ( Fuchs et al. , 2020 ) is a variant of the TFN with selfattention . These models achieve high expressibility based on spherical harmonics and message passing with nonlinear neural networks . However , for this reason , considerable computational resources 1https : //github.com/yellowshippo/isogcn-iclr2021 are required . In contrast , the present study allows a significant reduction in the computational costs because it eliminates spherical harmonics and nonlinear message passing . From this perspective , IsoGCNs are also regarded as a simplification of the TFN , as seen in equation 14 . Physical simulations using GNNs . Several related studies , including those by Sanchez-Gonzalez et al . ( 2018 ; 2019 ) ; Alet et al . ( 2019 ) ; Chang & Cheng ( 2020 ) focused on applying GNNs to learn physical simulations . These approaches allowed the physical information to be introduced to GNNs ; however , addressing isometric transformation equivariance was out of the scope of their research . In the present study , we incorporate isometric transformation invariance and equivariance into GCNs , thereby , ensuring the stability of the training and inference under isometric transformation . Moreover , the proposed approach is efficient in processing large graphs with up to 1M vertices that have a sufficient number of degrees of freedom to express complex shapes . 3 ISOMETRIC TRANSFORMATION INVARIANT AND EQUIVARIANT GRAPH CONVOLUTIONAL LAYERS . In this section , we discuss how to construct IsoGCN layers that correspond to the isometric invariant and equivariant GCN layers . To formulate a model , we assume that : 1 ) only attributes associated with vertices and not edges ; and 2 ) graphs do not contain self-loops . Here , G = ( V , E ) denotes a graph and d denotes the dimension of a Euclidean space . In this paper , we refer to tensor as geometric tensors , and we consider a ( discrete ) rank-p tensor field H ( p ) ∈ R|V|×f×dp , where |V| denotes the number of vertices and f ∈ Z+ ( Z+ denotes the positive integers ) . Here , f denotes the number of features ( channels ) of H ( p ) , as shown in Figure 1 ( a ) . With the indices , we denote H ( p ) i ; g ; k1k2 ... kp , where i permutes under the permutation of vertices and k1 , . . . , kp refers to the Euclidean representation . Thus , under the permutation , π : H ( p ) i ; g ; k1k2 ... kp 7→ H ( p ) π ( i ) ; g ; k1k2 ... kp , and under orthogonal transformation , U : H ( p ) i ; g ; k1k2 ... kp 7→ ∑ l1 , l2 , ... , lp Uk1l1Uk2l2 . . . UkplpH ( p ) i ; g ; l1l2 ... lp . 3.1 CONSTRUCTION OF AN ISOMETRIC ADJACENCY MATRIX . Before constructing an IsoGCN , an isometric adjacency matrix ( IsoAM ) , which is at the core of the IsoGCN concept must be defined . The proof of each proposition can be found in Appendix B . An IsoAM G ∈ R|V|×|V|×d is defined as : Rd 3 Gij ; ; : : = ∑ k , l∈V , k 6=l Tijkl ( xk − xl ) , ( 4 ) where Gij ; ; : is a slice in the spatial index of G , xi ∈ Rd is the position of the ith vertex ( rank1 tensor ) , and Tijkl ∈ Rd×d is an untrainable transformation invariant and orthogonal transformation equivariant rank-2 tensor . Note that we denote Gij ; ; k to be consistent with the no- tation of H ( p ) i ; g ; k1k2 ... kp because i and j permutes under the vertex permutation and k represents the spatial index while the number of features is always 1 . The IsoAM can be viewed as a weighted adjacency matrix for each direction and reflects spatial information while the usual weighted adjacency matrix can not because a graph has only one adjacency matrix . If the size of the set { Gij ; ; : 6= 0 } j is greater than or equal to d , then it can be deemed to be a frame , which is a generalization of a basis . For the simplest case , one can define Tijkl = δilδjkAijI ( Figure 1 ( b ) ) , where δij is the Kronecker delta , A is the adjacency matrix of the graph , and I is the identity matrix that is the simplest rank-2 tensor . In another case , Tijkl can be determined from the geometry of a graph , as defined in equation 16 . Nevertheless , in the bulk of this section , we retain Tijkl abstract to cover various forms of interaction , such as position-aware GNNs ( You et al. , 2019 ) . Here , G is composed of only untrainable parameters and thus can be determined before training . Proposition 3.1 . IsoAM defined in equation 4 is both translation invariant and orthogonal transformation equivariant , i.e. , for any isometric transformation ∀t ∈ R3 , U ∈ O ( d ) , T : x 7→ Ux+ t , T : Gij ; ; k 7→ ∑ l UklGij ; ; l. ( 5 ) Based on the definition of the GCN layer in the equation 1 , let G ∗ H ( 0 ) ∈ R|V|×f×d denote the convolution between G and the rank-0 tensor field H ( 0 ) ∈ R|V|×f ( f ∈ Z+ ) as follows : ( G ∗ H ( 0 ) ) i ; g ; k : = ∑ j Gij ; ; kH ( 0 ) j ; g ; . ( 6 ) With a rank-1 tensor field H ( 1 ) ∈ R|V|×f×d , let G H ( 1 ) ∈ R|V|×f and G G ∈ R|V|×|V| denote the contractions which are defined as follows : ( G H ( 1 ) ) i ; g ; : = ∑ j , k Gij ; ; kH ( 1 ) j ; g ; k , ( G G ) il ; ; : = ∑ j , k Gij ; ; kGjl ; k. ( 7 ) The contraction of IsoAMs G G can be interpreted as the inner product of each component in the IsoAMs . Thus , the subsequent proposition follows . Proposition 3.2 . The contraction of IsoAMs G G is isometric transformation invariant , i.e. , for any isometric transformation ∀t ∈ R3 , U ∈ O ( d ) , T : x 7→ Ux + t , G G 7→ G G. With a rank-p tensor field H ( p ) ∈ R|V|×f×dp , let G⊗H ( p ) ∈ R|V|×f×d1+p . and G⊗G ∈ R|V|×|V|×d2 denote the tensor products defined as follows : ( G⊗ H ( p ) ) i ; g ; km1m2 ... mp : = ∑ j Gij ; ; kH ( p ) j ; g ; m1m2 ... mp , ( 8 ) ( G⊗G ) il ; ; k1k2 : = ∑ j Gij ; ; k1Gjl ; ; k2 . ( 9 ) The tensor product of IsoAMs G⊗G can be interpreted as the tensor product of each of the IsoAMs components . Thus , the subsequent proposition follows : Proposition 3.3 . The tensor product of the IsoAMs G⊗G is isometric transformation equivariant in terms of the rank-2 tensor , i.e. , for any isometric transformation ∀t ∈ R3 , U ∈ O ( d ) , T : x 7→ Ux + t , and ∀i , j ∈ 1 , . . . , |V| , ( G⊗G ) ij ; ; k1k2 7→ Uk1l1Uk2l2 ( G⊗G ) ij ; ; l1l2 . This proposition is easily generalized to the tensors of higher ranks by defining the pth tensor power of G as follows : ⊗0 G = 1 , ⊗1 G = G , and ⊗p G = ⊗p−1 G⊗G . Namely , ⊗p G is isometric transformation equivariant in terms of rank-p tensor . Therefore , one can see that ( ⊗p G ) ⊗ H ( q ) = ( ⊗p−1 G ) ⊗ ( G ⊗ H ( q ) ) . Moreover , the convolution can be generalized for ⊗p G and the rank-0 tensor field H ( 0 ) ∈ R|V|×f as follows : [ ( p⊗ G ) ∗ H ( 0 ) ] i ; g ; k1k2 ... kp = ∑ j ( p⊗ G ) ij ; ; k1k2 ... kp H ( 0 ) j ; g ; . ( 10 )

Summary: The paper proposes a network that operates on features of graphs that are embedded in a d-dim Euclidean space. The paper considers equivariance to a group G that is the direct product of permutations of N points and Euclidean transformations. The features they consider are tensor products of the N-dimensional natural representations of permutations and the d-dimensional standard representation of O(d). From the coordinates, an “isometric adjacency matrix” is created, which is such a tensor. This matrix is combined in various G-equivariant ways with the features and then linearly combined with learnable weights to create new features. These operations are interleaved with non-linearities to form the network. The authors compare to several graph network methods and show competitive performance on several tasks.