Polynomial Graph Convolutional Networks

1 INTRODUCTION . In the last few years , the definition of machine learning methods , particularly neural networks , for graph-structured input has been gaining increasing attention in literature ( Defferrard et al. , 2016 ; Errica et al. , 2020 ) . In particular , Graph Convolutional Networks ( GCNs ) , based on the definition of a convolution operator in the graph domain , are relatively fast to compute and have shown good predictive performance . Graph Convolutions ( GC ) are generally based on a neighborhood aggregation scheme ( Gilmer et al. , 2017 ) considering , for each node , only its direct neighbors . Stacking multiple GC layers , the size of the receptive field of deeper filters increases ( resembling standard convolutional networks ) . However , stacking too many GC layers may be detrimental on the network ability to represent meaningful topological information ( Li et al. , 2018 ) due to a too high Laplacian smoothing . Moreover , in this way interactions among GC parameters at different layers pose a bias on the flow of topological information . For these reasons , several convolution operators have been defined in literature , differing from one another in the considered aggregation scheme . We argue that the performance of GC networks could benefit by increasing the size of the receptive fields , but since with existing GC architectures this effect can only be obtained by stacking more GC layers , the increased difficulty in training and the limitation of expressiveness given by the stacking of many local layers ends up hurting their predictive capabilities . Consequently , the performances of existing GCNs are strongly dependent on the specific architecture . Therefore , existing graph neural network performances are limited by ( i ) the necessity to select an appropriate convolution operator , and ( ii ) the limitation of expressiveness caused by large receptive fields being possible only stacking many local layers . In this paper , we tackle both the issues following a different strategy . We propose the Polynomial Graph Convolution ( PGC ) layer that independently considers neighbouring nodes at different topological distances ( i.e . arbitrarily large receptive fields ) . The PGC layer faces the problem of selecting a suitable convolution operator being able to represent many existing convolutions in literature , and being more expressive than most of them . As for the second issue a PGC layer , directly considering larger receptive fields , can represent a richer set of functions compared to the linear stacking of two or more Graph Convolution layers , i.e . it is more expressive . Moreover , the linear PGC design allows to consider large receptive fields without incurring in typical issues related to training deep networks . We developed the Polynomial Graph Convolutional Network ( PGCN ) , an architecture that exploits the PGC layer to perform graph classification tasks . We empirically evaluate the proposed PGCN on eight commonly adopted graph classification benchmarks . We compare the proposed method to several state-of-the-art GCNs , consistently achieving higher or comparable predictive performances . Differently from other works in literature , the contribution of this paper is to show that the common approach of stacking multiple GC layers may not provide an optimal exploitation of topological information because of the strong coupling of the depth of the network with the size of the topological receptive fields . In our proposal , the depth of the PGCN is decoupled from the receptive field size , allowing to build deep GNNs avoiding the oversmoothing problem . 2 NOTATION . We use italic letters to refer to variables , bold lowercase to refer to vectors , and bold uppercase letters to refer to matrices . The elements of a matrix A are referred to as aij ( and similarly for vectors ) . We use uppercase letters to refer to sets or tuples . Let G = ( V , E , X ) be a graph , where V = { v0 , . . . , vn−1 } denotes the set of vertices ( or nodes ) of the graph , E ⊆ V × V is the set of edges , and X ∈ Rn×s is a multivariate signal on the graph nodes with the i-th row representing the attributes of vi . We define A ∈ Rn×n as the adjacency matrix of the graph , with elements aij = 1 ⇐⇒ ( vi , vj ) ∈ E. With N ( v ) we denote the set of nodes adjacent to node v. Let also D ∈ Rn×n be the diagonal degree matrix where dii = ∑ j aij , and L the normalized graph laplacian defined by L = I−D− 12AD− 12 , where I is the identity matrix . With GConvθ ( xv , G ) we denote a graph convolution with set of parameters θ . A GCN with k levels of convolutions is denoted as GConvθk ( . . . GConvθ1 ( xv , G ) . . . , G ) . For a discussion about the most common GCNs we refer to Appendix A . We indicate with X̂ the input representation fed to a layer , where X̂ = X if we are considering the first layer of the graph convolutional network , or X̂ = H ( i−1 ) if considering the i-th graph convolution layer . 3 POLYNOMIAL GRAPH CONVOLUTION ( PGC ) . In this section , we introduce the Polynomial Graph Convolution ( PGC ) , able to simultaneously and directly consider all topological receptive fields up to k − hops , just like the ones that are obtained by the graph convolutional layers in a stack of size k. PGC , however , does not incur in the typical limitation related to the complex interaction among the parameters of the GC layers . Actually , we show that PGC is more expressive than the most common convolution operators . Moreover , we prove that a single PGC convolution of order k is capable of implementing k linearly stacked layers of convolutions proposed in the literature , providing also additional functions that can not be realized by the stack . Thus , the PGC layer extracts topological information from the input graph decoupling in an effective way the depth of the network from the size of the receptive field . Its combination with deep MLPs allows to obtain deep graph neural networks that can overcome the common oversmoothing problem of current architectures . The basic idea underpinning the definition of PGC is to consider the case in which the graph convolution can be expressed as a polynomial of the powers of a transformation T of the adjacency matrix . This definition is very general , and thus it incorporates many existing graph convolutions as special cases . Given a graph G = ( V , E , X ) with adjacency matrix A , the Polynomial Graph Convolution ( PGC ) layer of degree k , transformation T of A , and size m , is defined as PGConvk , T , m ( X , A ) = Rk , TW , ( 1 ) where T ( A ) ∈ Rn×n , Rk , T ∈ Rn×s∗ ( k+1 ) , Rk , T = [ X , T ( A ) X , T ( A ) 2X , .. , T ( A ) kX ] , and W ∈ Rs∗ ( k+1 ) ×m is a learnable weight matrix . For the sake of presentation , we will consider W as composed of blocks : W = [ W0 , . . . , Wk ] > , with Wj ∈ Rs×m . In the following , we show that PGC is very expressive , able to implement commonly used convolutions as special cases . 3.1 GRAPH CONVOLUTIONS IN LITERATURE AS PGC INSTANTIATIONS . The PGC layer in equation 1 is designed to be a generalization of the linear stacking of some of the most common spatially localized graph convolutions . The idea is that spatially localized convolutions aggregate over neighbors ( the message passing phase ) using a transformation of the adjacency matrix ( e.g . a normalized graph Laplacian ) . We provide in this section a formal proof , as a theoretical contribution of this paper , that linearly stacked convolutions can be rewritten as polynomials of powers of the transformed adjacency matrix . We start showing how common graph convolution operators can be defined as particular instances of a single PGC layer ( in most cases with k = 1 ) . Then , we prove that linearly stacking any two PGC layers produces a convolution that can be written as a single PGC layer as well . Spectral : The Spectral convolution ( Defferrard et al. , 2016 ) can be considered the application of Fourier transform to graphs . It is obtained via Chebyshev polynomials of the Laplacian matrix . A layer of Spectral convolutions of order k ? can be implemented by a single PGC layer instantiating T ( A ) to be the graph Laplacian matrix ( or one of its normalized versions ) , setting the PGC k value to k ? , and setting the weight matrix to encode the constraints given by the Chebyshev polynomials . For instance , we can get the output H of a Spectral convolution layer with k ? = 3 by the following PGC : H = [ X , LX , L2 , L3X ] W , where W = W0 −W2W1 − 3W32W2 4W3 , Wi ∈ Rs×m . ( 2 ) GCN : The Graph Convolution ( Kipf & Welling , 2017 ) ( GCN ) is a simplification of the spectral convolution . The authors propose to fix the order k ? = 1 of the Chebyshev spectral convolution to obtain a linear first order filter for each graph convolutional layer in a neural network . By setting k = 1 and T ( A ) = D̃− 12 ÃD̃− 12 ∈ Rn×n , with Ã = A + I and d̃ii = ∑ j ãij , we obtain the following equivalent equation : H = [ X , D̃− 1 2 ÃD̃− 1 2X ] W , where W = [ 0 W1 ] , ( 3 ) where 0 is a s ×m matrix with all entries equal to zero and W1 ∈ Rs×m is the weight matrix of GCN . Note that the GCN does not consider a node differently from its neighbors , thus in this case there is no contribution from the first component of Rk , T . GraphConv : In Morris et al . ( 2019 ) a powerful graph convolution has been proposed , that is inspired by the Weisfeiler-Lehman graph invariant . In this case , T ( A ) = A ( the identity function ) , and k is again set to 1 . A single GraphConv layer can be written as : H = [ X , AX ] W , where W = [ W0 W1 ] , and W0 , W1 ∈ Rs×m . ( 4 ) GIN : The Graph Isomorphism Network ( GIN ) convolution was defined in Xu et al . ( 2019 ) as : H′ = MLP ( ( 1 + ) X̂ + AX̂ ) . Technically , this is a composition of a convolution ( that is a linear operator ) with a multi-layer perceptron . Let us thus decompose the MLP ( ) as f ◦ g , where g is an affine projection via weight matrix W , and f incorporates the element-wise non-linearity , and the other layers of the MLP . We can thus isolate the GIN graph convolution component and define it as a specific PGC istantiation . We let k = 1 and T ( ) the identity function as before . A single GIN layer can then be obtained as : H = [ X , AX ] W , where W = [ ( 1 + ) W1 W1 ] . ( 5 ) Note that , differently from GraphConv , in this case the blocks of the matrix W are tied . Figure 1 in Appendix B depicts the expressivity of different graph convolution operators in terms of the respective constraints on the weight matrix W. The comparison is made easy by the definition of the different graph convolution layers as instances of PGC layers . Actually , it is easy to see from eqs . ( 3 ) - ( 5 ) that GraphConv is more expressive than GCN and GIN .

Summary: This work proposes the Polynomial Graph Convolutional Networks (PGCNs), which is built upon the Polynomial Graph Convolution (PGC). The PGC is able to aggregate k-hop information in a single layer and comes with the hyper-parameter k. The PGCNs are composed of a PGC with k=1, followed by a PGC with a chosen k (usually > 1), and a complex readout layer using avg, max, and sum over all nodes. Theoretically, the proposed PGC has two major benefits as claimed: 1) Common graph convolution operators can be represented as special cases of the PGC; 2) A PGC with k = q (q > 1) is more expressive than linearly stacked q PGCs with k=1. The PGCNs are thus more general, expressive, and efficient than existing GNNs. Experimental studies are conducted on common graph classification benchmarks, showing the improved performances of the PGCNs.