Graph Deformer Network

1 INTRODUCTION . Graph is a flexible and universal data structure consisting of a set of nodes and edges , where node can represent any kind of objects and edge indicates some relationship between a pair of nodes . Research on graphs is not only important in theory , but also beneficial to in wide backgrounds of applications . Recently , advanced by the powerful representation capability of convolutional neural networks ( CNNs ) on grid-shaped data , the study of convolution on graphs is drawing increasing attention in the fields of artificial intelligence and data mining . So far , Many graph convolution methods ( Wu et al. , 2017 ; Atwood & Towsley , 2016 ; Hamilton et al. , 2017 ; Velickovic et al. , 2017 ) have been proposed , and raise a promising direction . The main challenge is the irregularity and complexity of graph topology , causing difficulty in constructing convolutional kernels . Most existing works take the plain summation or average aggregation scheme , and share a kernel for all nodes as shown in Fig . 1 ( a ) . However , there exist two nonignorable weaknesses for them : i ) losing the structure information of nodes in the local neighborhood , and ii ) causing signal entanglements of nodes due to collapsing to one central node . Thereby , an accompanying problem is that the discriminative ability of node representation would be impaired , and further non-isomorphic graphs/subgraphs may produce the same responses . Contrastively , in the standard convolutional kernel used for images , it is important to encode the variations of local receptive fields . For example , a 3 × 3 kernel on images can well encode local variations of 3× 3 patches . An important reason is that the kernel is anisotropic to spacial positions , where each pixel position is assigned to a different mapping . However , due to the irregularity of graphs , defining and operating such an anisotropic kernel on graphs are intractable . To deal with this problem , Niepert et al . ( Niepert et al. , 2016 ) attempted to sort and prune neighboring nodes , and then run different kernels on the ranked size-fixed nodes . However , this deterministic method is sensitive to node ranking and more prone to being affected by graph noises . Furthermore , some graph convolution methods ( Velickovic et al. , 2017 ; Wang et al. , 2019 ) introduce an attention mechanism to learn the importances of nodes . Such methods emphasize on mining those significant struc- tures/features rather than designing anisotropic convolution kernels , so they can not well represent local variations of structures in essence . In this work , we propose a novel yet effective graph deformer network ( GDN ) to implement anisotropic convolutional filtering on graphs as shown in Fig . 1 ( b ) , exactly behaving like the standard convolution on images . Inspired by image-based convolution , we deform local neighborhoods of different sizes into a virtual coordinate space , implicitly spanned by several anchor nodes , where each space granularity corresponds to one anchor node . In order to perform space transformation , we define the correlations between neighbors and anchor nodes , and project neighboring nodes into the regular anchor space . Thereby , irregular neighborhoods are deformed into the anchorcoordinated space . Then , the image-like anisotropic convolution kernels can be imposed on the anchor-coordinated plane , and local variations of neighborhoods can be perceived effectively . Due to the importance of anchors , we also deform anchor nodes with adaptive parameters to match the feature space of nodes . As anisotropic convolution kernels are endowed with the fine-grained encoding ability , our method can better perceive subtle variations of local neighborhood regions as well as reduce signal confusion . We also show its connection to previous work , and theoretically analyze the stronger expressive power and the satisfactory property of the isomorphism test . Extensive experiments on graph/node classification further demonstrate the effectiveness of the proposed GDN . 2 OUR APPROACH . In this section , we elaborate on the proposed graph deformer method . Below we first give an abstract formulation for our method and then elaborate on the details . Denote G = ( V , E ) as an undirected graph , where V represents a set of nodes with |V| = n and E is a set of edges with |E| = e. According to the link relations in E , the corresponding adjacency matrix can be defined as A ∈ Rn×n . And X ∈ Rn×d is the feature matrix . To state conveniently , we use Xi· or xi to denote the feature of the i-th node . Besides , for a node vi , the first-order neighborhood consists of nodes directly connected to vi , which is denoted as N 1vi = { vj | ( vj , vi ) ∈ E } . Accordingly , we can define s-order neighborhood N svi as the set of s-hop reachable nodes . 2.1 A BASIC FORMULATION . Given a reference node vr in graph G , we need to learn its representation based on the node itself as well as its contextual neighborhood Nvr . However , the irregularity causes difficulty in designing anisotropic spatial convolution . To address this problem , we introduce anchor nodes to deform the neighborhood . All neighboring nodes are calibrated into a pseudo space spanned by anchors . We denote the set of anchor nodes by V = { v0 , v1 , ... , vm−1 } . The convolution onNvr is formulated as : x̃r = ( G ∗ f ) ( vr ) = C ( F ( r ) , K ) , ( 1 ) F ( r ) i = ∑ vt∈Nvr Dvt→vi ( xi , xt , Θ ) , ( 2 ) where • v· , x· : an anchor node and a pseudo coordinate vector ( a.k.a . feature vector ) . Please see Section 2.2 for anchor generation . • D : the deformer function . It transforms node vt into a virtual coordinate space spanned by anchors . Θ is the deformer parameter to be learned . Please see Section 2.3.1 for details . • F ( r ) ∈ Rm×d : the deformed multi-granularity feature from the neighborhood of node vr . Each granularities F ( r ) i corresponds to an anchor node vi . • C , K : the anisotropic convolution operation on anchor space and convolution kernel . G ∗ f represents filter f acting on graph G. The relationship between anchor nodes can be built by some metrics such as Cosine distance , and anchor nodes may be format as a pseudo 2-D grid just like the patch in images . Please see the details in Section 2.3.2 . 2.2 ANCHOR GENERATION . Anchor nodes are crucial to the graph convolution process , because neighborhood regions are unitedly calibrated with them . Rigid anchors will not adapt to the variations of the feature space during convolution learning . Thus we choose to optimize anchor nodes as one part of the entire network learning . In the beginning , we cluster some nodes randomly sampled from the graph as initial anchors . When enough anchors cover the space of neighborhood nodes , the anchors can be endowed with a strong expressive ability to encode neighborhoods like a code dictionary . Formally , we use the K-means clustering to generate initial anchors , V ← Clustering { ( vi , xi ) |vi ∈ Vsampling } , ( 3 ) where Vsampling are the sampled node set , in which each node is randomly sampled from the graph , V = { ( vk , xk ) } |m−1k=0 is the initial anchor set generated by clustering , in which vk represents kth anchor node and xk represents its feature vector , m is the number of anchor nodes . Note that when given anchor nodes , the response of our method will be invariant however to permute nodes of one graph during the training stage as well as testing stage . The clustering algorithm might affect the final anchors due to random sampling for initialization , but it can not affect the property of permutation invariance , which just like random initialization on the network parameters . A larger m could increase the expression capacity of anchors , but causes more redundancy and a larger computational cost . Due to the sparsity of graphs , in practice , several anchors are sufficient to encode each neighborhood region . To better collaborate with node/feature variations during graph convolution learning , we transform the initial anchors into a proper space by parameterizing them : ak = ReLU ( WAxk + bA ) , k = 0 , 1 , · · · , m− 1 , ( 4 ) where WA , bA are the learnable parameters , and ReLU is the classic activation function . Besides , other flexible multi-layer networks may be selected to learn deformable anchors . 2.3 DEFORMER CONVOLUTION . 2.3.1 SPACE TRANSFORMATION . Now we define the deformer function D in Eqn . ( 2 ) , which transforms neighborhood nodes to the anchor space . For each node vj ∈ Nvr , we derive the anchor-related feature ( also query feature ) and value feature vectors as qj = ReLU ( WQxj + bQ ) , j = 0 , 1 , · · · , nr − 1 , ( 5 ) uj = ReLU ( WUxj + bU ) , j = 0 , 1 , · · · , nr − 1 , ( 6 ) where WQ , WU are the learnable weight matrices , and bQ , bU are the biases . The query feature qj indicates how to transform vj to the anchor space by interacting with anchors , and the value vector uj is the transformable component to the anchor space . For the neighborhood Nvr , the correlation to anchors defines a set of weights α = { α1,1 , · · · , α1 , m , · · · , αnr,1 , · · · , αnr , m } , which measures the scores of all nodes within the neighborhood projected onto the directions of anchor nodes . Formally , αj , k = exp ( 〈qj , ak〉 ) ∑ k′ exp ( 〈qj , ak′〉 ) , k′ = 0 , 1 , · · · , m− 1 , ( 7 ) where 〈· , ·〉 denotes the inner production , then normalization is done by softmax function . αj , k may be viewed as the attention score of the node vj w.r.t . the anchor vk . After obtaining the attention score , the irregular neighborhood can be transformed into the anchor-coordinated space , ũk = ∑ j αj , kuj , j = 0 , 1 , · · · , nr − 1 . ( 8 ) The deformed components are accumulated on each anchor , and form the final deformed features . Thus , any neighborhood with different sizes can be deformed into the virtual normalized space coordinated by anchors . In experiment , for simplicity , the query feature and value feature are shared with the same parameters in Eqns . ( 5 ) and ( 6 ) . 2.3.2 ANISOTROPIC CONVOLUTION IN THE ANCHOR SPACE . Afterward , s-hop neighborhood of node vr is deformed into the size-fixed anchor space , i.e. , N svr → { ũ0 , ũ1 , · · · , ũm−1 } . The anisotropic graph convolution can be implemented by imposing different mapping on each anchor as x̃ ( s ) r = ReLU ( ∑ i Kᵀi ũi + b ) , i = 0 , 1 , · · · , m− 1 , ( 9 ) where x̃ ( s ) r ∈ Rd ′ , the matrix Ki is a d × d′ weight parameter imposed on the features w.r.t . an anchor , and b is the bias vector . In the convolution process , different filter weights are imposed on different features of anchor nodes , which is an anisotropic filtering operation . For an intuitive description , we assume the simplest case of 2-D space , which of course can be extended higher dimension . Assuming in 2-D , we project all neighborhood nodes onto anchor nodes , then employ different filters on different anchor nodes in 2-D plane , which likes the standard convolution , so called anisotropic convolution . In contrast to the traditional aggregation method , the deformer convolution has two aspects of advantages : i ) well preserving structure information and reducing signal entanglement ; ii ) transforming different-sized neighborhoods into the size-fixed anchor space to well advocate anisotropic convolution like the standard convolution on images .

Summary: This work proposes the Graph Deformer Network (GDN), whose key component is the proposed Graph Deformer Convolution (GDC). The GDC is based on the attention mechanism, where a fixed number of query vector comes from a clustering process of some randomly sampled nodes (In my opinion, the q vector in the paper should be named the key vector instead query vector, according to [1]). The attention thus yields a fixed number of ordered vectors as outputs, where an anisotropic convolution can be applied on. Experimental results on both node and graph classification tasks demonstrate the effectiveness of the proposed GDC.