Graph Convolutional Networks via Adaptive Filter Banks

1 INTRODUCTION . In many application domains , structured data is supported by graphs or networks . To deal with such data , a collection of graph convolutional networks ( GCNs ) , including MPGCNs and spectral GCNs , have been proposed by generalizing architectures used for data in the Euclidean domain , to the irregular graphs . Unfortunately , both types of architectures have implicit limitations when it comes to efficiently representing graph data information in terms of frequency filtering . MPGCNs ( Gilmer et al. , 2017 ; Velikovi et al. , 2018 ; Hamilton et al. , 2017 ; Xu et al. , 2019 ) built on diverse message passing ( MP ) schemes are much prevalent with their flexible and intuitive formulations of graph convolution . The message passing schemes in classical MPGCNs , however , mostly focus on the low frequency characteristics of the data . For example , GCN ( Kipf & Welling , 2017 ) performs as Laplacian smoothing ( Li et al. , 2018 ) , and the mean aggregator adopted in GraphSage ( Hamilton et al. , 2017 ) has naturally low-pass properties . However , in addition to low-frequency components , graph data may also carry rich information in the middle- and high-frequency ranges . There have been some recent attempts to complement low-pass features , either with predefined filters ( Gao et al. , 2019 ; Chang et al. , 2021 ) or with other aggregation schemes ( Abu-El-Haija et al. , 2019 ; Zhu et al. , 2020 ) . Yet , they still have some limitations , such as the nonadaptiviy of predefined filters . The importance of different frequency components may vary with the target tasks , it is thus beneficial to be able to adapt the representation accordingly . Spectral graph convolutional networks ( Defferrard et al. , 2016 ; Levie et al. , 2018 ; Bianchi et al. , 2021 ) are designed in the graph frequency domain directly and are much more powerful alternatives to process diverse spectral characteristics of graphs . The lack of proper sharing schemes between filters , however , renders these models redundant and even prone to overfitting . We go beyond the above limitations and propose to build effective adaptive representations for graph data with diverse spectral properties . We design a novel graph convolution operator termed BankGCN that utilizes an adaptive filter bank to process graph data in the frequency domain , as presented in Fig . 1 . BankGCN provides an effective implementation to simplify spectral methods with a proper sharing scheme between filters with the help of signal decomposition . Firstly , we decompose multi-channel graph signals into a collection of subspaces through projection , in order to adaptively separate input data according to signal characteristics . Components in a subspace will then share a learnable filter that captures their particular frequency properties . Notably , representations are built on finite impulse response ( FIR ) filters with a universal design ( Tremblay et al. , 2018 ) that correspond to local message passing schemes in the spatial domain . Filters of all the subspaces together form a filter bank , and they are simultaneously learned from data together with subspace decomposition . The filter bank representation is regularized to favour diversity in the frequency responses , in order to properly capture various spectral components in the graph signals . The proposed convolution operator is stackable , and can be optimized together with other modules in the GCNs like graph pooling . We validate the proposed architecture through extensive experiments on various graph classification tasks . BankGCN is more powerful than most MPGCNs , such as GCN ( Kipf & Welling , 2017 ) , GraphSage ( Hamilton et al. , 2017 ) , and GIN ( Xu et al. , 2019 ) , in that it is able to exploit more diverse spectral characteristics than ‘ low-pass ’ features in the data and adapts to its heterogeneous properties with a group of learnable multi-hop message passing strategies . Furthermore , it outperforms its counterpart ChebNets ( Defferrard et al. , 2016 ) with a more compact architecture and achieves better generalization , and is superior to the most recent spectral method ARMA ( Bianchi et al. , 2021 ) . It would be interesting to extend BankGCN to tasks like link prediction and applications into non-Euclidean data like 3-D point clouds . 2 RELATED WORK . We briefly overview the main architectures for graph representation learning . We describe first existing message passing schemes , and then discuss several spectral methods . Message Passing Graph Convolutional Networks . MPGCNs design graph convolution with a variety of message passing schemes in the spatial domain . For instance , messages are aggregated with the node-wise mean or max operation in a localized neighborhood in GraphSage ( Hamilton et al. , 2017 ) , or based on attention scores in GAT ( Velikovi et al. , 2018 ) . A more expressive scheme , GIN , is further proposed with summing features in a neighborhood followed by a multi-layer perceptrons ( MLP ) to approximate any injective function on multiset ( Xu et al. , 2019 ) . However , these methods are typically constrained to a single one-hop message passing strategy that mostly captures ‘ lowpass ’ characteristics in the graph data . Klicpera et al . ( 2019 ) expand the MP range to multi-hop neighborhoods with graph diffusion operators . Besides , several methods are proposed to implement an equivalent high-order ( polynomial ) filter with the whole network to alleviate over-smoothing representation , including SGC ( Wu et al. , 2019 ) , LGC ( Navarin et al. , 2020 ) , GPR-GNN ( Chien et al. , 2021 ) , SGF ( NT et al. , 2021 ) , and S2GC ( Zhu & Koniusz , 2021 ) . However , a single equivalent filter may be limited when representing multi-channel signals with diverse frequency characteristics . Alternatively , a group of works attempt to complement the ‘ low-pass ’ features using different strategies , such as graph wavelet filters ( Gao et al. , 2019 ; Chang et al. , 2021 ) , a one-order high-pass filter ( Bo et al. , 2021 ) and an additional quadratic middle-pass filter ( Wu et al. , 2021 ) , and other aggregation schemes ( Abu-El-Haija et al. , 2019 ; Zhu et al. , 2020 ) . Contrary to these methods , we employ a learnable filter bank with various frequency responses in a single layer to adaptively capture diverse spectral characteristics of signals rather than merely adding ‘ high-pass ’ features or specific components via predefined graph wavelets . Finally , in contrast with some works that further consider adaptive filters ( Li et al. , 2021 ; Dong et al. , 2021 ; Pasa et al. , 2021 ) , BankGCN is a more powerful and compact design with polynomial filter banks to avoid eigen-decomposition and a further sharing scheme between filters to reduce free parameters . More detailed comparisons in App . F. Filtering on Graphs and Spectral GCNs . Filtering and Graph Fourier Transform have been generalized to graph data via spectral graph theory ( Shuman et al. , 2013 ) . Correspondingly , several spectral GCNs are derived from filtering in the graph frequency domain directly , as pioneered by Bruna et al . ( 2014 ) . However , they rely on computationally expensive eigen-decomposition of the graph Laplacian . To avoid the eigendecomposition , similarly to dictionary learning methods ( Thanou et al. , 2014 ) in graph signal processing ( GSP ) , the filters are built as polynomial or rational functions of eigenvalues of the graph Laplacian , such as Chebyshev polynomials in ChebNets ( Defferrard et al. , 2016 ) and TIGraNet ( Khasanova & Frossard , 2017 ) , Cayley polynomials in CayleyNets ( Levie et al. , 2018 ) , and auto-regressive moving average ( ARMA ) filters ( Bianchi et al. , 2021 ) . In contrast with these spectral convolution methods that focus on the implementation of filters with desirable prop- erties , like localization and narrowband specialization , this paper focuses on the design of the filter bank in order to construct a simplified architecture with improved generalization . Finally , there are several papers in graph signal processing literature about the design of graph filter banks for graph signal decomposition ( Narang & Ortega , 2012 ; Tanaka & Sakiyama , 2014 ) and multiscale analysis ( Hammond et al. , 2011 ; Narang & Ortega , 2013 ) . However , they usually work under a rigid constraint , e.g. , perfect reconstruction , and thereby their produced ( sparse ) representations may not be flexible enough to adapt to diverse tasks in machine learning . In contrast , our method relaxes the perfect reconstruction condition and rather regularizes filters in a filter bank to be sufficiently different in the graph frequency domain . 3 PRELIMINARIES AND PROBLEMS . In this paper , we consider data that is represented on undirected graphs . Each vertex of graphs is attributed a d-channel signal or feature . The node signal is typically multi-channel , i.e. , d > 1 , and with different spectral statistics along different dimensions , which we refer to heterogeneous signals in this paper . Notations and preliminaries on Graph Fourier Transform and filtering are introduced in App . A . We first show how filtering can be implemented by messaging passing , which is used in many stateof-the-art graph representation learning methods . Filtering in the graph frequency domain and MP in the spatial domain is closely related ( Shuman et al. , 2013 ) . Taking the most popular GCN ( Kipf & Welling , 2017 ) as an example , according to its initial formation , the j-th channel of a filtered signal is hj = d∑ i=1 ( I +D− 1 2AD− 1 2 ) Θxi = d∑ i=1 ( 2I − L ) Θi , jxi = d∑ i=1 Θi , jUĝ ( Λ ) U ∗xi ( 1 ) where Θ denotes learnable parameters , d indicates the dimension of signals , and ĝ ( Λ ) = 2I−Λ is a low-pass filter ( NT & Maehara , 2019 ; Min et al. , 2020 ) . The renormalized version of GCN and the GIN can be formulated similarly . Please refer to ( Balcilar et al. , 2021 ) for their specific equivalent supports of ĝ ( Λ ) . Besides , GraphSage uses the mean or max operator to aggregate information in neighborhoods . Although it is hard to explicitly formulate its corresponding frequency response , the element-wise mean operation is a low-pass operation by nature and the element-wise max operator performs as an envelope extraction that suppresses the high-frequency components . Another line of works , spectral GCNs , are directly derived from spectral filtering . For the representative ChebNets and CayleyNets , the filters are defined as ĝ ( λ ) = ∑K k=0 θ ( k ) Tk ( λ ) in the graph frequency domain with the Chebyshev polynomial or Cayley polynomial basis { Tk ( · ) } and correspondingly hj = d∑ i=1 U K∑ k=0 Θ ( k ) i , j Tk ( Λ ) U ∗xi = d∑ i=1 Uĝij ( Λ ) U ∗xi , ( 2 ) where K is the order of the polynomial filters and { Θ ( k ) } are learnable parameters . Notably , the above MPGCNs employ just a single kind of filter to handle all the channels of signals , and then takes different ( linear ) combinations of the filtered signals to obtain different channels of the output signals . Thereby , most MPGCNs are restricted in the number and types of filters and have a limited capacity in frequency-domain filtering . On the contrary , ChebNets and CayleyNets employ a different filter for each mapping from an input channel to an output channel and adopt a total of d × d′ K-order polynomial filters ( d and d′ denote the respective number of channels of input and output signals ) , as presented in Eq . 2 . This naive design neglects the potential relationship between filters and fails to reduce redundancy . Consequently , they explore such a large number of learnable filters to process diverse frequency components of graph data at the expense of numerous free parameters , which sometimes leads to overfitting .

Summary: The authors introduce BankGCN, a graph convolutional network that learns (chebyshev) polynomial filters over the graph. BankGCN includes an initial subspace projection, and a cosine similarity regularization to encourage diverse filters. BankGCN is compared to a number of other graph networks on standard whole graph classification tasks.  The authors claim that most architectures of MPGCNs are limited in that they (1) focus on low frequency information and (2) lack a proper sharing scheme between filters, and that BankGCN addresses these two issues.