Learning Discrete Adaptive Receptive Fields for Graph Convolutional Networks

1 INTRODUCTION . After a series of explorations and modifications ( Bruna et al. , 2014 ; Kipf & Welling , 2017 ; Velickovic et al. , 2017 ; Xu et al. , 2019 ; Li et al. , 2019 ; Abu-El-Haija et al. , 2019 ) , Graph Convolutional Networks ( GCNs ) 1 have gained considerable attention in the machine learning community . Typically , a graph convolutional model can be abstracted as a message-passing process ( Gilmer et al. , 2017 ) – nodes in the neighborhood of a central node are regarded as contexts , who individually pass their messages to the central node via convolutional layers . The central node then weighs and transforms these messages . This process is recursively conducted as the depth of network increases . 2 Neighborhood convolutions proved to be widely useful on various graph data . However , some inconveniences also exist in current GCNs . While different nodes may yield different importance in the neighborhood , early GCNs ( Kipf & Welling , 2017 ; Hamilton et al. , 2017 ) did not discriminate contexts in their receptive fields . These models either treated contexts equally , or used normalized edge weights as the weights of contexts . As a result , such implementations failed to capture critical contexts – contexts that pose greater influences on the central node , close friends among acquaintances , for example . Graph Attention Networks ( GATs ) ( Velickovic et al. , 2017 ) resolved this problem with attention mechanisms ( Bahdanau et al. , 2015 ; Vaswani et al. , 2017 ) . Soft attention weights were used to discriminate importance of contexts , which allowed the model to better focus on relevant contexts to make decisions . With impressive performances , GATs became widely used in later generations of GCNs including ( Li et al. , 2019 ; Liu et al. , 2019 ) . However , we observe that using soft attention weights in hierarchical convolutions does not fully solve the problem . Firstly , we will show as Proposition 1 that under common conditions , soft attention weights almost surely approach 0 as the neighborhood sizes increase . This smoothness 3 hinders the discrimination of context importance in large neighborhoods . Secondly , we will show by experiments 1We use the name GCN for a class of deep learning approaches where information is convolved among graph neighborhoods , including but not limited to the vanilla GCN ( Kipf & Welling , 2017 ) . 2We use the term contexts to denote the neighbor nodes , and receptive field to denote the set of contexts that the convolutions refer to . 3The smoothness discussed in our paper is different to that in ( Li et al. , 2018 ) , i.e . the phenomenon that representations of nodes converge in very deep GNNs . in Section 4.2 that GATs can not well distinguish true graph nodes from artificial noises : attention weights assigned to true nodes and noises are almost identical in distribution , which further leads to a dramatic drop of performance . Meanwhile , an ideal GCN architecture is often expected to exploit information on nodes with various distances . Most existing GCNs use hierarchical convolutional layers , in which only one-hop neighborhoods are convolved . As a result , one must increase the model depth to detect long-distance dependencies ( informative nodes that are distant from the central nodes ) . This is particularly an issue in large graphs , as the complexity of the graph convolutions is exponential to the model depth . 4 In large graphs , the model depths are often set as 1 , 2 or 3 ( Hamilton et al. , 2017 ; Velickovic et al. , 2017 ) . Accordingly , no dependencies longer than 3 hops are exploited in these models . Motivated by the discussions above , we propose the idea of adaptive receptive fields ( ARFs ) . Figure 1 illustrates the differences between hierarchical convolutions and convolutions with ARFs . An ARF is defined as a subset of contexts that are most informative for a central node , and is constructed via selecting contexts among the neighborhood . Nodes in an ARF can be at various distances from the central node . The discrete selection process of contexts gets rid of the undesired smoothness of soft weights ( see Section 2 ) . In addition , by allowing ARFs to choose contexts on different hops from the central node , one can efficiently explore dependencies with longer distances . Experiments also show that ARFs are more robust to noises ( see Section 4 ) . We further propose GRARF ( GCNs with Reinforced Adaptive Receptive Fields ) as an instance for using ARFs in node-level tasks . In GRARF , an optimal policy of constructing ARFs is learned with reinforcement learning ( RL ) . An RL agent ( constructor ) successively expands the ARF via a two-stage process : a contact node in the intermediately-constructed ARF is firstly selected ; a context among the direct neighbors of the contact node is then added to the ARF . The reward of the constructor is defined as the performance of a trained GCN ( evaluator ) on the constructed ARF . GRARF is validated on datasets from different domains including three citation networks , one social network , and an inductive protein-protein interaction dataset . GRARF matches or improves performances on node classification tasks compared with strong baselines . 5 Moreover , we design two tasks to test the models ’ abilities in focusing on informative contexts and leveraging long-distance dependencies by injecting node noises in graphs with different strategies . 2 PRELIMINARIES AND THEORIES . Notations . In our paper , we consider node-level supervised learning tasks on attributed graphs . An attributed graph G is generally represented as G = ( V , A , X ) , where V = { v1 , · · · , vn } denotes the set of nodes , A ∈ { 0 , 1 } n×n denotes the ( binary ) adjacency matrix , and X ∈ Rn×d0 denotes the input node features , xv ∈ Rd0 the features of node v. E is used as the set of edges . We use N ( vi ) to denote the one-hop neighborhood of node vi , with vi itself included . We use H ( l ) ∈ Rn×dl as the matrix containing dl-dimensional hidden representations of nodes in the l-th layer , h ( l ) v that of node 4With sparse adjacency matrices , the average complexity of graph convolutions is O ( dL ) , where L is the model depth and d is the graph degree ( or the neighborhood-sampling sizes in ( Hamilton et al. , 2017 ) ) . 5We mainly show the results of node classification tasks in our paper , whereas GRARF is intrinsically adapted to all node-level supervised learning tasks . v. Â denotes the symmetrically normalized adjacency matrix with Â = D−1/2 ( A+ In ) D−1/2 and D = diag ( d ) , di = ∑ j ( A+ In ) ij . We use bold letters for neural network parameters . The smoothness of Graph Attention Networks . As a pioneering work of simplifying architectures of graph neural networks , the vanilla GCN layers in ( Kipf & Welling , 2017 ) were defined as H ( l+1 ) = σ ( ÂH ( l ) W ( l ) ) = σ ∑ j∈N ( vi ) Âijh ( l ) j W ( l ) , l = 0 , 1 , · · · ( 1 ) In each layer , the node representations in one-hop neighborhoods were transformed with W and averaged by normalized edge weights Âij . Graph Attention Networks ( GATs ) ( Velickovic et al. , 2017 ) elaborated the average scheme in Eq . ( 1 ) with attention mechanisms ( Bahdanau et al. , 2015 ; Vaswani et al. , 2017 ) . Instead of using Âij , an attention weight αij between node vi and vj was calculated in GAT layers as eij = fθ ( hi , hj ) , αij = softmaxj ( eij ) = exp ( eij ) ∑ k∈N ( vi ) exp ( eik ) , ( 2 ) where fθ ( · ) is often called the energy function with parameter θ. GATs implicitly enabled specifying different weights in a neighborhood . However , under some common assumptions and as the neighborhood size increases , these attention weights , normalized with the softmax function , suffer from over-smoothness : all attention weights approach 0 as the neighborhood size increases . We formally introduce and prove this claim as Lemma 1 and Proposition 1 : Lemma 1 ( the smoothness of softmax ) . If random variables X1 , X2 , · · · are uniformly bounded with probability 1 , that is , for any i and some C , P ( |Xi| > C ) = 0 , then the softmax values taking { Xi } ni=1 as inputs approach 0 almost surely when n→∞ , i.e . eXi / n∑ j=1 eXj → 0 a.s. ( 3 ) Proof . The proof is simple noting that eXi/ ∑n j=1 e Xj > 0 , and that with probability 1 , eXi / n∑ j=1 eXj < eC / ne−C → 0 , n→∞ . Proposition 1 ( the smoothness of attention weights ) . If the representation of nodes ( random vectors ) H1 , H2 , · · · ∈ Rd are uniformly bounded with probability 1 ( for any i and some C , P ( ‖H1‖ > C ) = 0 ) , and for any ( fixed ) node vi , the energy function fθ ( hi , · ) is continuous on any closed set D ∈ Rd , then the attention weights in the neighborhood of vi approach 0 almost surely when n→∞ , i.e . αij = exp ( fθ ( hi , Hj ) ) ∑ k∈N ( vi ) exp ( fθ ( hi , Hk ) ) → 0 a.s. ( 4 ) Proof . Following the a.s. boundedness { Hi } s and the continuity condition on fθ ( · ) , the random energies Eij = fθ ( hi , Hj ) are also bounded a.s .. The desired result then follows Lemma 1 . Note that the continuity condition on fθ ( hi , · ) in Proposition 1 can be satisfied with almost any commonly used non-linear functions and ( regularized ) parameters in deep learning , specifically , those in the official version of GATs ( eij = aT [ Whi‖Whj ] , where ‖ is the operator of concatenation ) . Also , the boundedness of inputs is trivial in deep learning . GCNs with ARFs overcome smoothness . What Proposition 1 shows is that in large neighborhoods , attention weights are smoothed to 0 , thus hindering the discrimination of context importance . In addition , such smoothness can be immediately generated to any other form of normalized weights as long as αij > 0 uniformly and ∑ j∈N ( vi ) αij = 1 . We alleviate the smoothness with ARFs by incorporating discreteness . Specifically , let us denote the convolution in the evaluator as h′i = σ ∑ j∈Na ( u ) ηijhjW = σ ∑ j∈Nk ( u ) η̃ijhjW , η̃ij = { ηij , j ∈ Na ( u ) ,0 , j /∈ Na ( u ) , ( 5 ) where Na ( u ) is an ARF , k is the maximum hop that the ARF explores , and Nk is the entire k-hop neighborhood . Accordingly , η̃ij is not subjected to smoothness : if ηij has a uniform lower bound D > 0 , then for p ∈ Na ( u ) and q /∈ Na ( u ) , we have η̃ip − η̃iq > D , regardless of the sizes of N ( k ) . Note that ηij > 0 uniformly can be guaranteed in most cases when the maximum ARF size is limited , for example , with uniform weights or softmax weights of bounded energies . It should be noted that the discrimination of nodes in ARFs is different to that in MixHop ( Abu-ElHaija et al. , 2019 ) , which directly takes multi-hop nodes as inputs . MixHop can specify different parameters 6 for contexts on different hops ( hop-level discrimination ) , while it CAN NOT specify different weights for contexts on the same hop ( node-level discrimination ) as they were uniformly treated ( averaged ) . The two levels of discrimination are orthogonal , and we focus on the latter . Deep reinforcement learning on graphs . As the discrete context selection process is nondifferentiable , we apply deep reinforcement learning approaches to learn the policy of construting ARFs in GRARF , specifically , the Deep Q-Learning ( DQN ) ( Mnih et al. , 2015 ) algorithm . DQN uses deep neural networks to approximate the action value function ( Q-function ) , and chooses the action that maximizes it in each step . The Q-function is defined iteratively as Q∗ ( st , at ) = R ( st , at , st+1 ) + γmax a∈A Q∗ ( st+1 , a ) , ( 6 ) where s is the state , R ( · ) is the reward function , A is the action space , and γ is a discount factor . A reward shaping technique ( Ng et al. , 1999 ) is also used in GRARF to alleviate the sparsity of rewards , which decorates the original reward R ( · ) with a potential energy F ( · ) , yielding an immediate reward R̂ ( · ) . Denoted in formula , F ( s , a , s′ ) = Φ ( s′ ) − Φ ( s ) , R̂ ( s , a , s′ ) = R ( s , a , s′ ) + F ( s , a , s′ ) , ( 7 ) where Φ ( · ) is a fixed potential function of states that does not change during training . ( Ng et al. , 1999 ) proved that the optimal policies of MDPs remain invariant if R ( · ) is replaced by R̂ ( · ) . There are other recent papers implementing reinforcement learning on graphs . For example , GCPN ( You et al. , 2018 ) proposed an RL agent for generating graph representations of biomedical molecules , and DGN ( Jiang et al. , 2020 ) introduced a multi-agent reinforcement learning approach where the agents in the system formed a dynamic network . The successive molecule generation process in GCPN inspired us in designing the ARF constructor in GRARF , whereas the two models are of different motivations and applications . ARFs and neighborhood sampling . It should be noted that GRARF can also be interpreted as a neighborhood sampling approach . Neighborhood sampling was proposed as a necessary process to apply GCNs to large graphs with arbitrarily large neighborhoods . GraphSAGE ( Hamilton et al. , 2017 ) proposed a general framework of neighborhood sampling and aggregation , where contexts were uniformly sampled . Later work improved the sampling strategy with importance sampling ( Chen et al. , 2018 ) and explicit variance reduction ( Huang et al. , 2018 ; Hamilton et al. , 2017 ) . Sub-graphs instead of subsets of neighborhoods were directly sampled in ( Zeng et al. , 2020 ) . Indeed , selecting ARF nodes takes a specific form of neighborhood sampling . However , the aim of constructing ARFs is to ignore trivial information and to focus on critical contexts , rather than to estimate the neighborhood average as is the primary target of neighborhood sampling . Therefore , despite the similarity , the two approaches are in different directions .

Summary: The authors theoretically and empirically show that soft-attention mechanism uses in GCNs suffers from over-smoothness in large neighborhoods. For addressing this shortcoming, they propose a neighborhood sampling approach called adaptive receptive fields (ARFs) which discretely select nodes among the multi-hop neighborhood and allow to efficiently explore long-distance dependencies in graphs. The authors also propose GRARF (GCN with Reinforced ARF) which learns optimal policy of constructing ARFs with reinforcement learning. For a given node, an RL agent successively expands ARF via a two-stage process. Firstly, a contact node in an intermediately-constructed ARF is selected and then a context among the direct neighbors of the contact node is added to ARF. The reward is the performance of the trained GCN on constructed ARF. Overall, the results demonstrate the effectiveness of the approach on benchmark datasets. The authors also demonstrate that the method is quite effective at handling noise in the graph compared to GCN and GAT by evaluating them on the Cora dataset with synthetically added noise.