Graph Convolutional Networks via Adaptive Filter Banks
Authors: Xing Gao
Wenrui Dai
Chenglin Li
Junni Zou
Hongkai Xiong
Pascal Frossard
Venue: ICLR 2022 Submitted (2022)

Abstract: Graph convolutional networks have been a powerful tool in representation learning of networked data. However, most architectures of message passing graph convolutional networks (MPGCNs) are limited as they employ a single message passing strategy and typically focus on low-frequency information,  especially when graph features or signals are heterogeneous in different dimensions.  Then, existing spectral graph convolutional operators lack a proper sharing scheme between filters, which may result in overfitting problems with numerous parameters. In this paper, we present a novel graph convolution operator, termed BankGCN, which extends the capabilities of MPGCNs beyond single `low-pass' features and simplifies spectral methods with a carefully designed sharing scheme between filters. BankGCN decomposes multi-channel signals on arbitrary graphs into subspaces and shares adaptive filters to represent information in each subspace. The filters of all subspaces differ in frequency response and together form a filter bank. The filter bank and the signal decomposition permit to adaptively capture diverse spectral characteristics of graph data for target applications with a compact architecture. We finally show through extensive experiments that BankGCN achieves excellent performance on a collection of benchmark graph datasets. 

Under review as a conference paper at ICLR 2022
GRAPH CONVOLUTIONAL N VIA ADAPTIVE FILTER BANKS
Anonymous authors Paper under double-blind review
ABSTRACT
Graph convolutional networks have been powerful tool in representation learning of networked data. However, most architectures of message passing graph convo- lutional networks (MPGCNs) are limited as they employ a single message pass- ing strategy and typically focus on low-frequency information, especially when graph features or signals are heterogeneous in different dimensions. Then, exist- ing spectral graph convolutional operators lack proper sharing scheme between filters, which may result in overfitting problems with numerous parameters. In this paper, we present a novel graph convolution operator, termed BankGCN, which extends the capabilities of most MPGCNs beyond single 'low-pass' features and simplifies spectral methods with a carefully designed sharing scheme between filters. BankGCN decomposes multi-channel signals on arbitrary graphs into sub- spaces and shares adaptive filters to represent information in each subspace. The filters of all subspaces differ in frequency response and together form filter bank. The filter bank and the signal decomposition permit to adaptively capture diverse spectral characteristics of graph data for target applications with a compact archi- tecture. We finally show through extensive experiments that BankGCN achieves excellent performance on a collection of benchmark graph datasets.
INTRODUCTION
In many application domains, structured data is supported by graphs or networks. To deal with such data, a collection of graph convolutional networks (GCNs), including MPGCNs and spectral GCNs, have been proposed by generalizing architectures used for data the Euclidean domain, to the irregular graphs. Unfortunately, both types of architectures have implicit limitations when it comes to efficiently representing graph data information in terms of frequency filtering.
MPGCNs (Gimmer et al.) 2017] Velikovi et al. 2018 Hamilton et al. 2017 Xu al.) (019) built on diverse message passing (MP) schemes are much prevalent with their flexible and intuitive formu- lations of graph convolution. The message passing schemes in classical MPGCNs, however, mostly focus on the low frequency characteristics
the data. For example, GCN (Kipf & Welling 2017)
performs Laplacian smoothing (Li et al. 2018). and the mean aggregator adopted in GraphSage (aamilton et al.) 2017) has naturally low- pass properties. However. in addition to low-frequency components, graph data may also carry rich information in the middle- and
ranges. There have been some recent attempts to complement low-pass features, either with predefined fil-
ters (Gao et al. 2019 Chang al. 2021) or with other aggregation schemes (Abu-EI-Haijy et al, 2019 Zhu et al. 2020). Yet, they still have some limitations, such as the nonadaptiviy of predefined filters. The importance of different frequency components may vary with the target tasks, is thus beneficial to be able to adapt the representation accordingly.
Spectral graph convolutional networks (Defferrard et al., 2016 Levie et al., 2018 Bianchi et al.) 2021) are designed in the graph frequency domain directly and are much more powerful alternatives to process diverse spectral characteristics of graphs. The lack of proper sharing schemes between filters, however, renders these models redundant and prone to overfitting.
We go beyond the above limitations and propose to build effective adaptive representations for graph data with diverse spectral properties. We design novel graph convolution operator termed BankGCN that utilizes an adaptive filter bank to process graph data in the frequency domain, as presented in Fig. | BankGCN provides effective implementation simplify spectral methods

---

Under review as a conference paper at ICLR 2022
with a proper sharing scheme between filters with the help of signal decomposition. Firstly, decompose multi-channel graph signals into a collection of subspaces through projection, in order to adaptively separate input data according to signal characteristics. Components in subspace will then share a learnable filter that captures their particular frequency properties. Notably, represen- tations are built on finite impulse response (FIR) filters with universal design (Tremblay et al.) 2018) that correspond to local message passing schemes in the spatial domain. Filters of all the subspaces together form filter bank, and they are simultaneously learned from data together with subspace decomposition. The filter bank representation is regularized to favour diversity in the fre- quency responses, in order to properly capture various spectral components in the graph signals. The proposed convolution operator is stackable, and can be optimized together with other modules in the GCNs like graph pooling. We validate the proposed architecture through extensive experi- ments on various graph classification tasks. BankGCN is more powerful than most MPGCNs, such as GCN (iipf& & Welling 2017). GraphSage (Hamilton et al. 2017), and GIN (Xu et al. 2019). in that it is able to exploit more diverse spectral characteristics than 'lowppass' features in the data and adapts to its heterogeneous properties with a group of learnable multi-hop message passing strate- gies. Furthermore, it outperforms its counterpart ChebNets (Defferrard et al.] (016)) with a more compact architecture and achieves better generalization, and is superior to the most recent spectral method ARMA (Bianchi al. 2021). It would be interesting to extend BankGCN tasks like link prediction and applications into non- Euclidean data like 3-D point clouds.
2 RELATED WORK
We briefly overview the main architectures for graph representation learning. We describe first existing message passing schemes, and then discuss several spectral methods.
Message Passing Graph Convolutional Networks. MPGCNs design graph convolution with a va- ricty of message passing schemes in the spatial domain. For instance, messages are aggregated with the node-wisc mean or max operation in a localized neighborhood in GraphSage (Hamilton et al. 2017). or based on attention scores in GAT (Velikovi et al., 2018). A more expressive scheme, GIN, further proposed with summing features in a neighborhood followed by a multi-layer perceptrons (MLP) to approximate any injective function on multiset (Xu et al.] 2219).. However, these methods are typically constrained to a single one-hop message passing strategy that mostly captures 'low- pass' characteristics in the graph data. Klicpera et al. (2019) expand the MP range to multi-hop neighborhoods with graph diffusion operators. Besides, several methods are proposed implement an equivalent high-order (polynomial) filter with the whole network to alleviate over-smoothing rep- resentation, including SGC (Wu et al., 2019). LGG (Navarin al. 2020). GPR-GNI (Chien et al., 2021]. SGF (NT et al. 2021). and SCGC (Zhu & Koniusz 2021). However, single equivalent fil- ter may be limited when representing multi-channel signals with diverse frequency characteristics. Alternatively, group of works attempt to complement the 'low-pass' features using different strate- gies, such as graph wavelet filters (Gao et al. 2009 Chang et al. (20T), a one-order high-pass filter (Bo et al.] 202I) and an additional quadratic middle pass filter (Wu et al. (02T). and other aggrega- tion schemes (Abu-El -aija et al.] 2019 Zhu et al.] 2020). Contrary to these methods, we employ a learnable filter bank with various frequency responses in single layer to adaptively capture diverse spectral characteristics of signals rather than merely adding 'high-pass' features or specific com- ponents via predefined graph wavelets. Finally, in contrast with some works that further consider adaptive filters (Li et al. [021] Dong et al. [2021: Pasa et al. [022I) BankGCN is a more powerful and compact design with polynomial filter banks avoid eigen-decomposition and a further sharing scheme between filters to reduce free parameters. More detailed comparisons in App.
Filtering on Graphs and Spectral GCNs. Filtering and Graph Fourier Transform have been gener- alized to graph data via spectral graph theory (Shuman et al. 2013). Correspondingly, several spec- tral GCNs are derived from filtering in the graph frequency domain directly, as pioneered by Bruna et al.:(22000 (2014). However, they rely on computationally expensive eigen rigen-decompositionn of the graph Laplacian. To avoid the eigendesomposiiionn similarly to dictionary learning methods (Thanou et al.) 2014) in graph signal processing (GSP), the filters are built as polynomial or rational functions of eigenvalues of the graph Laplacian, such as Chebyshev polynomials in ChebNets (Defferrard et al, f 2017), Cayley polynomials in CayleyNets (Levie et al. 2018), and auto regressegeee moving average (ARMA) filters (Bianchi et al. 2021). In contrast with these spectral convolution methods that focus on the implementation of filters with desirable prop-
2016) and TIGraNet (Khasanova& & Frossard
2

---

Under review as a conference paper at ICLR 2022
KFT
Rell
Alrwi
Concalemat
(b) Filter bank.
M
Haali
Mule
(E.E.))
(a) Framework.
(c) Message passing.
Figure 1: Illustration of the BankGCN operator. In each layer, the graph signal æ ((mm) is decom- posed into a collection of subspaces and processed by an adaptive filter bank to capture distinct frequency properties. A filter in the filter bank corresponds to an adapted K K-hop neighborhood message passing scheme in the spatial domain, as illustrated in (b) and (c).
erties, like localization and narrowband specialization, this paper focuses on the design of the filter bank in order to construct a simplified architecture with improved generalization.
Finally, there are several papers in graph signal processing literature about the design of graph fil- ter banks for graph signal decomposition (Narang & Ortega 2012 Tanaka & Sakiyama 2014) and multiscale analysis (Hammond et al. 2011 Narang & Ortega (013)). However, they usually work under rigid constraint, e.g., perfect reconstruction, and thereby their produced (sparse) represen- tations may not be flexible enough to adapt to diverse tasks in machine learning. In contrast, method relaxes the perfect reconstruction condition and rather regularizes filters in a filter bank to be sufficiently different in the graph frequency domain.
PRELIMINARIES AND PROBLEMS
In this paper, we consider data that is represented on undirected graphs. Fach vertex of graphs is attributed a d-channel signal or feature. The node signal is typically multi-channel, i.e., d > 1, and with different spectral statistics along different dimensions, which refer to heterogeneous signals in this paper. Notations and preliminaries on Graph Fourier Transform and filtering are introduced in App-
We first show how filtering can be implemented by messaging passing, which is used in many state- of-the-art graph representation learning methods. Filtering in the graph frequency domain and MP in the spatial domain is closely related (Shuman et al.] 2013). Taking the most popular GCN (Kipf & Welling 2017 an example, according to initial formation, the j-th channel of filtered signal is
d E - L)),
hj Σ + 1=1
Σ ",,AU)U
where & denotes learnable parameters, d indicates the dimension of signals, and ((A) 2/ - A is low-pass filter (NT & Machara 2019 Min al. 2020). The renormalized version of GCN and the GIN can be formulated similarly. Please refer to (allcilar et al. (021)) for their specific equivalent supports of (A). Besides, GraphSage uses the mean or max operator to aggregate information in neighborhoods. Although it is hard to explicitly formulate its corresponding frequency response, the element-wise mean operation a low-pass operation by nature and the element-wise max operator performs as an envelope extraction that suppresses the high-frequeng components.
Another line of works, spectral GCNs, are directly derived from spectral filtering. For the repre- sentative ChebNets and Cetl the filters are defined as ğ(A) = >k frequency domain with the Chebyshev polynomial Cayley polynomial basis (Tx(-)) and corre- spondingly
k g())T, in the graph
d
K
h2 = Σ
(k)
Tx(AUU x Σ (A)U"A) (A)
(2)
1=1
3

---

Under review as a conference paper at ICLR 2022
where K is the order of the polynomial filters and { are learnable parameters.
Notably, the above MPGCNs employ just a single kind of filter to handle all the channels of signals, and then takes different (linear) combinations of the filtered signals to obtain different channels of the output signals. Thereby. most MPGCNs are restricted the number and types of filters and have limited capacity in frequency-domain filtering. On the contrary, ChebNets and CayleyNets employ a different filter for each mapping from an input channel to an output channel and adopt total of d x d' K-order polynomial filters (d and d' denote the respective number of channels of input and output signals). as presented in Eq 2 This naive design neglects the potential relationship between filters and fails reduce redundancy. Consequently, they explore such a large number of learnable filters to process diverse frequency components of graph data at the expense of numerous free parameters, which sometimes leads to overfitting.
THE BANKGCN ALGORITHM
In this section, we first outline the framework of BankGCN, then present its main elements, and finally discuss connections with spectral methods.
4.1 GRAPH CONVOLUTION WITH ADAPTIVE FILTER BANKS
Signals (features) supported on nodes of a graph are usually nigh-dimensional and composed of multiple spectral patterns. In other words, different signal channels vary differently over nodes, leading to diverse spectral characteristics. To handle the heterogeneous signals, we employ filter bank composed of a set of filters with different frequency responses the design of BankGCN. In order to further facilitate information processing, signal decomposition is adopted to explore latent relationships in the data.
For multi-channel input node signal ((um), we first adaptively decompose it into different sub- spaces and then employ different filters to deal with the signal components in each subspace sepa- rately. The dccomposition aims to map the components of signals with similar characteristies into the same subspace in order to facilitate the subsequent adaptive filtering. This is implemented with learnable subspace projections. Mathematically, the input signal (r(m)) is projected into s subspaces with group of projection functions denoted by (-))
"[p] (Mm) = (w)((a (m)()))) p = 1, 2, .,..,,,, Vrm E.
Here, the subseript "[v]" indicates the terms belonging the p-th subspace and Mpj (Um) is the pro- jected signal. The choices for projection functions will be introduced in Section 4.2 Subsequently, an adaptive filter gip) (-) designed within each subspace to represent the spectral characteristics of the corresponding signal components,
pnj (Um) (g)n) Tip (Um),
(4)
where we reuse ** to denote the convolution between each channel of signal Tipl and the filter 9ppl-
The filtered signals hip (Um), p = 1.2,.... of all the subspaces are concatenated to produce the output features. A Rectified Linear Unit (ReLU) is then used as non-linear activation function,
h(vm,) = Concat Concat(hpnj (h hug (t..)).... (v.,))) (
x(v[+1,m) ReLU(h(v,,.))
(6)
where the subscript / indicates variables or parameters in the 1-th layer to describe the forward propagation between different layers in hierarchical architecture.
Specially, considering that some filters are only supported on medium-to-bi to-hghh frequency compo- nents that usually mean large signal variations in the spatial domain, we further introduce shortcut in each subspace that corresponds to full-pass in the graph frequency domain in order to make the filtered signals stable. Correspondingly, the filtered signal h (mm) in each subspace becomes
hipl (vm) (ppp r'li) (vm) + (Tp] (Um)
The shortcut can further facilitate back propagation of gradients, as for CNNs He et al. 2016).
The proposed graph convolution operator is stackable, and the subspace mapping function in the following layers will further combine features from different subspaces in the preceding layers to enable information interaction between different channels.
4

---

Under review as a conference paper at ICLR 2022
4.2 SUBSPACE PROJECTION
For the projection function /pj there is a variety of design choices. Here, we take the linear mapping as an example, since it is simple yet able decompose different channels of signals. Specifically, the d-channel | signal x((m)) is projected into different subspaces with learnable matrices pWi For the sake of simplicity, all the subspaces have the same dimension in this paper. To increase flexibility, we further introduce learnable bias bip) for each subspace.
(1m) [w) (2((m)) Wh (v(m))
bbo) p ........6
The introduction of subspace projection brings in two advantages: (i) it simplifies the learning pro- cess. Since the multi-channel graph signals have been decomposed, the filter in each subspace just needs to learn to capture spectral characteristics of the corresponding signal components; (ii) with low-dimnnsional subspaces, it limits the dimension of features output by the filter bank, and thereby reduces the number of free parameters and computation in the following layers.
4.3 FILTERS
In order to handle graphs with arbitrary topologies and diverse signals, we adopt universal and adaptive filters to construct the filter bank.
For any function t = [0, 2] + R. we can obtain a corresponding filter whose frequency response is 9 (A) t(.), and its spatial construction is computed through IGFT (defined as Eq. S2 in App.
[lp) (mm)
Σ Biol (N.))
with ug i = 1, , n, the eigenvectors of the symmetric normalized graph Laplacian of graph. Notably, we directly design the frequency response of the filter for the continuous range D = [0, in which the spectrum of an arbitrary graph locates as introduced in Section A In other words, given the discrete spectrum {Yo, A } of an arbitrary graph, we have A; E D and obtain the corresponding filter value on these specific discrete values (A,) t(A,), for i 1.2...... n.
Thereby, the filter Uip! (-) is adaptable any graph even with different topologies, Le., is universal form (Tremblay et al. 2018 Levie et al. 2019).
Furthermore, constrain the filter the K-order polynomial function space in order avoid the computation-intense eigendecomposition of the graph Laplacian, similarly to parametric dictionary learning (Thanou al. 2014) in GSP. It corresponds to FIR filter. Mathematically, the frequency response of a filter can be represented as
K
(A) = Σ
1 TA(),
(10)
k=
where } denotes a specific polynomial basis such as Chebyshev polynomials, and {a indi- (8))
cates the corresponding coefficients. With {a[?) learnable, we obtain an adaptive filter whose fre- quency response adapts to the data and to the target task. Correspondingly, for the signal projected
to the p-th subspace Tip (Um) and Rip) = (v)), (1))), mpll (v,))"" the filtered signal is
calculated
hil (wm) (rip) * (v.)
(k) [
(T((L)
(11)
Equivalently, according to the relationship between frequency filtering and localized linear trans- forms in the spatial domain (Shuman et al.) 2013). the filtering strategy corresponds to a message passing scheme within a K -hoop neighborhood
hpn (vm) (mm?"yy (Um)
Cmo (p) (v)),
(12)
BoENK (em)
For the input signal with d = 1, we set the feature channels in the neural network in the first layer so that the signal is scaled differently in each subspace. In the following hidden layers, it can he used as the other cases handle multi-channel features produced by previous layers in GCNs.
5

---

Under review as a conference paper at ICLR 2022
with
CYPL
(L)) Vm, (1,2,...,7) (p)
(K)
(13)
Specially, the induced K-hop message passing scheme learned from the data and exploits the multi-hop topological information of graphs through polynomials of the graph Laplacian. Signal information is also taken into consideration through the learnable parameters {d J as well in
(k)
the subspace projection step. More importantly, it permits to represent features that do not only have "low-pass" properties and explores the frequency components in a data-driven manner.
4.4 DIVERSITY REGULARIZATIO FOR FILTER BANKS
The filters constituting a filter bank should ideally have diverse frequency responses so that the signal decomposed through filtering into series of signals with different frequency characteristies. In GSP, the filters are usually band-pass and divide the spectrum in different bands. Considering that strict band-pass filters difficult to fit through polynomial functions, we relax this strict band- pass requirement and rather target filters with diverse frequency responses, which we call "diversity condition". With the filter 91p (A) given as a K-order polynomial function, the regularization on
(k) the filter is imposed on the respective polynomial coefficients (a p.. in Eq. 10 To achieve the diversity condition, we regularize the polynomial coefficients to be well distributed in the parameter space. Considering that the distances of the coefficient vectors of two scaled filters may still be large in terms of the Euclidean distance, we thereby take the cosine distance measure the distance between the polynomial coefficients of filters. Specifically, the regularization term is:
[pl. [4] >
(aa)= max poo
(14)
||< 12 |2||o l2
K K The max function reflects the maximum similarity between [p] the polynomial coefficients of any pair of filters in the filter bank. Through minimizing Eq. [4] the most similar filters will have different orientations in the parameter space. Thus, all the pairs of filters tend to be different. When (Tx(L) an orthogonal basis such as Chebyshev polynomials, the diversity of polynomial coefficients (a bp implies that filters defined as Eq. [0 are different in the graph frequency domain. More intuitively, the message passing schemes in the spatial domain induced by the filters are different with diverse [0' Ip. presented in Eq. 12 and Eq. 13
where ((pp] [a (0) [p]
(1)
We can note that, if the filter is defined on the basis composed of rectangular pulse functions i.e.,
{
2k
< X
2(k+1) K+I
(A)
K+T
(15)
others
the ideal subband filter banks, whose filters have different passbands in GSP, just corresponds to the optimal solution the regularization with ?(a=) in Eq. 14 when K 2
With Te(g,Y) Y) generally representing a target function, the overall objective function is then for- mulated as
min (G + M(x). 8
(16)
where Y indicates ground truth labels, O denotes the parameter set including {a( in) bin } 1=1 and 7 is a hyperparameter to adjust the contribution of regularization term. Like most popular graph convolution operators, BankGCN can be optimized via gradient- gradient-based methods together with modules such as graph pooling operators in GCNs. It achieves linear computational complexity with O(KI (d) and constant learning complexity, similarly most existing MPGCNs.
4.5 DISCUSSION
BankGCN actually provides an effective simplification of existing polynomial-base spectral meth- ods with parameter decomposition. With θ 1O:: : E dd x(K+1) representing the
Note that this is the case in this paper.

---

Under review as a conference paper at ICLR 2022
free parameters of the spectral methods adopting corresponding polynomial filters, BankGCN with a linear mapping as projection function actually provides a decomposition based approximation of ④ with
W1n @ [qj) Wly @ a..)........ Wi @ asj]]
(17)
where @ indicates outer product. This decomposition strategy allows for filter sharing to reduce potential redundancy existing spectral methods. Multi-channel signals are first adaptively decom- posed into various subspaces with [Wip] pp and signal components within subspace share filter defined with ppi to process corresponding characteristics. In addition, it permits to further control the relationship between filters with the diversity regularization as introduced in Section 4.4
We further study the capacity of BankGCN in handling the spectrum of signals with a compact architecture.
Proposition BankGCN can capture and preserve diverse frequency components of input graph signals, like its counterpart introduced in Eq. 2
Proof. Please refer App.
As demonstrated in Prop. T its proof, a single filter Рp] in BankGCN together with signal projec- tion is capable explore and represent diverse frequency components of graph signals, as done in its counterpart with a group of filters (gighs This theoretically validates the effectiveness of BankGCN in reducing the number of filters and thereby potential redundancy existing polynomial-based spectral architectures, while maintaining the capability of representing diverse spectral information.
5 EXPERIMENTS
In this section, in order to evaluate models in learning representation of diverse spectral information of graph data, we resort to node classification and graph classification tasks.
Baselines. We compare BankGCN with several of-te--rtt graph convolution methods, includ- ing MPGCNs and spectral methods, as listed in Table and Table 2 Furthermore, two simplifica- tions of the proposed model (BankGCN-Dir and BankGCN-NR) are considered for ablation study. Please refer to App. & D for details.
5.1 NODE CLASSIFICATION
We first evaluate the proposed model in node classification tasks on several heterophilic (ie., neighbor nodes are dissimilar in la- bels/fcatures) graph datasets, in order to vali- date the advantages of diverse frequency rep- resentation with filter banks. We follow the data split and evaluation procedure of (Pei et al. 2020 Zhu et al. 2020; Li et al. 2021), 1), and de- scribe the detail information of datasets and ex-
Table 1: Results (mean accuracy + stdev) on
node classification with different data splits (* and denoting the results cited from (Zhu al. 2020 and
Lii Liel al. 202T) respectively).
Wasomsin 0.16
Cunell
GCX+
99.46 = 5.25
59.80 6.90
57.05
4:4.7
30.26 :: 0.79
SUC
38.9 = &.1
51.8
5M
58.1
2.4.5
DAY
51.38 = 4:45
55 09-00-11
871
5892.30 3.32
36:08 in 1.73
perimental settings in App. C Results of base- lines are cited from their respective publications if available, or recent public works (Zhu et al.
GEOM-GCN
67.57
64.12
60.81
31.63
Misdsop"
77.64 = 7.73
75.85 4.90
73.51 6.34
32.22 d: 2.54
83.41
81.18 5.96
CheisNes
79.7
76.5 9.4
x22
2020 Li al. 202T). The metric B introduced by Pei et al. (2020) reveals the degree of ho- mophily of graph data in datasets (Small 3 cor- responding to strong heterophily). As presented in Table 0 BankGCN achieves the best performance on all the datasets. In contrast, the MPGCNs that rely on low-pass features, like GCN and GAT, per- form worse. Notably, BankGCN still outperforms several of the most recent MPGCNs that improve on the over-smoothing issues, including a simplified GCN (SGC), improved aggregation schemes MixHop, GEOM-GCN and H2GCN, and adaptive filter based methods SGF and ASGAT.
HDONN
ASGAT Club
82.7
BaskGNN
86.49 4.19
87.06
3.19
$3.78
2:
4.68
36.57 :: 8.95
7

---

Under review as a conference paper at ICLR 2022
Table 2: Results graph classification with 20 runs for different datasets (#P/I. denotes the number of free parameters per hidden layer).
ENZY
DD
NCI1
PRCT
NC1109
MUTA
FRAN
--PPIL
GCN
62.75 4 5.83
77.75 1: 3.55
79.00
1,93
74.87
4,08
78.90
1.52
81.34 =: 1.61
62.21 2,41
4.2k
GraphSage
66.75 6.31
2.72
75.13
4.04
79.54
2.24
82.30
1,48
GIN
75.42+3.31
81.19
14.91
3.88
80.71
81.66
2.48
68.11
2.09
GAT
62.67 1 7.52
17.50
17.50:22.14
79:43
2 2.38
75.09
1: 4.05
79.16
1.85
81.28
:1 2.20
63.89
1.53
4.3k
ChebNets (K 2)
66.75 4: 4.79
17.67 1: 2.91
81.80
2.35
74.64 11 4.75
81.27 1.89
82.50 -: 1.58
68.35
2.65
12.4k
ARMA(K
3.04
80.
1.89
1.81
BunkGCN-DiI'G
66.92 +: 5.71
77.88 +: 2.81
80.07
2.03
74.87
79.23 +: 2.29
82.27
2,00
64.63
1.96
4.2k
BankCK NR - 2,5
65.83 1: 6.66
77.03
1: 4.08
81.59
1: 1.95
75.36
4.68
81.03 4: 1.95
82.41 -: 1.69
67.82
2.30
4.2k
BankGCN - 2.5
68.00 1: 5.23
78.14
1: 2.81
82.06
1.75
75.67
4,19
81.54 2.13
82.89
1,61
67.82
2.30
4.2k
BankGCN
66.83
3,50
1.87
1.61
Table 3: Study on the order K of filters and the number of subspaces per layer.
ENZY
DD
NC11
PROT
NC1109
MUTA
FRAN
63.58 6,31
76.40 +: 2.34
80.46
2.34
74.38
4,80
79.23
2.29
82.09
1.11
65.52
2.44
66,75 + 5.61
78.14 + 2.81
81.62
+ 1.84
75.67
+ 4.61
81.19 + 2.08
82.70 = 1.63
67.48
2.09
=
8
68.00 + 5.23
78.14 +: 2.81
82.06 + 1.75
75.67
+ 4.19
81.54 = 2.13
82.89
161
67.82 2.30
3.50
81.93
+ 2.15
76.12
5.08
81.62 = 1.87
82.57
1.61
68.43 L98
K =
67.17 4: 5.68
76.99 +: 2.99
81.02 +: 1.88
75.89 + 5.07
80.92 + 1.66
82.40
1.89
66.95
1.91
68.00 + 5.23
78.14 + 2.81
82.06 + 1.75
75.67
+ 4.19
81.54 2.13
82.89
1.61
67.82
2.30
65.75 4 5.54
77.75
2.66
81.85
1.92
74.96
5.77
80.82 L84
82.53
68.35
2.13
K =
65,17 + 6.62
77.75 + 3.01
82.46 + 1.98
75.09 + 4.94
81.17 = 2.11
82.26
1.71
68,35
1.92
5.2 GRAPH CLASSIFICATION
We further evaluate the different models in graph classification tasks and study the components of BankGCN. Specially, for fair and complete comparison, we consider two cases. (i) the same number of features, i.e., all of models with the same number of feature maps per hidden layer, and (ii) the same number of parameters, i.e., models with nearly the same number of free parameters per hidden layer, in Table 2 and Table respectively. Experimental settings are presented in App.[]
As presented in Table BankGCN outperforms all the MP baselines with nearly the same or even fewer number of parameters on TU-benchmarks.
further achieves better performance than its counterpart ChebNets with much fewer free parameters, iee, about 1/(K + 1), and is also superior to the most recent spectral method ARMA on most datasets. As presented in Fig. [ the learned filters have different frequency responses rather than only low-pass, some of them being high-pass and some focusing on muddle-frequencier With such a bank of filters, BankGCN handles the multi- channel signals flexibly and thereby achieves the best performance almost all the datasets.
We then go one step further to evaluate the adaptive filtering capabilities. As presented in Fig. 2 the learned filters have different frequency responses on various datasets as they adapted to the data characteristics. As listed in Tables and 3 the BankGCN (s 1) employing one single adaptive filter still outperforms GCN with 'low-pass' filtering on most datasets; Furthermore, BankGCN is superior to its variant BankGCN-Diff that uses predefined filter banks on all the datasets. These validate the benefits of adaptive filtering to flexibly capture the spectral characteristics of data.
Study on the number of filters. Furthermore, we evaluate the adoption of filter banks rather than a single filter, and study the impact of the number of filters the filter bank (equivalently s, the num- ber of subspaces) on the classification performance. With group of filters, the ability of convolution operators handle information is enhanced. As presented in Table 2 BankGCN. -Dff outperforms GCN with additional band-pass and high-pass filters on almost all the datasets, and BankGCN fur- ther improves the performance with adaptive filters. Furthermore, as 8 increases from to 8, the performance of BankGCN is improved benefits of using more than one filter. With s further increased into 16, the performance is degraded on several datasets. Given that the total dimension of all the subspaces is fixed, the dimension of each space decreases and the representation capacity of each subspace probably declines with the further growth of
most datasets, as listed in Table 3 These validate the
Study on the order of filters. The order of polynomials determines the function space of filters. In the graph frequency domain, demonstrated in Fig. 2 it can better realize the bandpass property
8

---

Under review as a conference paper at ICLR 2022
(a) K =2
(b) = 2 0.1.
(c) K = 2,y 10.
(d) K 3,y 10.
(e) K 2,y
Figure 2: Illustrations of the frequency responses of the learned filters of BankGCN (s 8) in the first layer of networks. (a) ~ (d) are on NC1109 and (e) on FRANKENSTEIN.
of filters using larger K but the cost of greater risk of overfitting the spectrum of training data. In the spatial domain, the value of K corresponds to the message passing range, and large K will affect the locality of signals. Thereby, tradeoff is needed. As shown Table BankGCN with K = 2 achieves the best performance on most datasets. We notice that for the cases with complex node signals, like FRANKENSTEIN with 780-channel node attributes, a relatively large K' is needed to exploit their various spectral characteristics; and the small K' is preferred on simple datasets, such as the PROTEINS dataset with 3-channel node category features.
Study on the regularization. We further evaluate the effect of the proposed diversity regularization. The comparison of BankGGN-NR and BankGCN Table 2 shows that the regularization improves the classification performance almost all the datasets. On the FRANKENSTEIN dataset whose signal is composed of 780-channel attributes, the regularization information in such high-channe signal is complex enough to induce different filters, as presented in Fig. ee). Fig. ([a)(c)) show that the learned filters in a filter bank with regularization present better diversity in frequency response, than those without regularization. For example, the filters denoted by blue and red in Fig. ([a) are with similar frequency responses, while they are more diverse in Fig. bbb and Fig. [c... This is further verified by the maximum similarity scores of the polynomial coefficients that define the filters M(a) 0.997, 0.744, and 0.649 (computed as Eq. 14] for Fig. [a)-(c), respectively. More results and analysis are presented in App.
not helpful. We infer that the
BankGCN still achieves the best perfor- mance on both CIFAR-10 and Ogbg- molhiv with all the models having sim- ilar number of free parameters per hidden layer, as presented in Table Furthermore, to evaluate the generalization of models, construct reduced CIFAR-10 (1000) dataset by taking 100 graphs per category to form the training set, while maintaining the validation and testing sets. BankGCN still performs best on the reduced CIFAR- 10 dataset and is among the models with the least performance loss compared with the full dataset. Together with ROC-AUC being mea- sure of the generalization ability of model, BankGCN performs well in the sense of generalization, especially when compared with its counterpart ChebNets. These further validate the effect of the proposed sharing scheme between filters for model simplification on improving generalization.
Table 4: Classification accuracy on CIFAR-10 10 and Oghg-molhin (no edge attributes) datasets.
Method
CIPAR-10
CIFAR-10
Oghe-molliiv ROC-AUC
GCN GraphSage
$5.64
36.47 0.31
1.85
40.03
50.0 : 0.06
31.97
60 0.19
36.08 0.04
ChebNees (K
0.75
-38.7
ChebNens CK
63.62 0.23
37.91
0.40
-0.6%
73.17
1.57
ARMA (K
61.06 -: 0.35
32
0.09
45.0%
75.73 1: 1.15
ARMA CK -3
61.79 - 0.28
33.47
0.32
451 9/6
74.61 + 0.58
BinkGCNI 2,a 16)
66.17
0.34
42.82
0.33
-15
77.95 + 1.96
BankCCNIK - 3.6 16)
66.00
-1 0.51
43.95
0.49
34.9%
75.72 1.45
6 CONCLUSION
In this paper, we propose a novel graph convolution operator, termed BankGCN, constructed on an adaptive filter bank for graph representation learning. The filter bank is equivalent to a group of learnable message passing schemes in K-hopp neighborhoods. Together with subspace decom- position, BankGCN explores a sharing scheme between filters to adaptively handle information of diverse spectral characteristics with significantly fewer parameters than its competitors, and achieves excellent performance on node classification and graph classification tasks. An interesting direction for future research resides in discussing the capacity of the proposed graph convolution operator in terms of graph isomorphism test. may also be promising to employ BankGCN in a variety of tasks on non-Fuclidean data like 3-D point cloud classification and segmentation.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Sami Abu-EI-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In Proc. 36th Int. Conf. Mach. Learn., pp- 21-29, Long Beach, California, USA, 2019.
Muhammet Balcilar, Renton Guillaume, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. Analyzing the expressive power of graph neural networks in spectral perspective. In 9th Int. Conf. Learn. Rep.. 2021.
Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. Graph neural networks with convolutional arma filters. IEEE Trans. Pattern Anal. and Mach. Intell., 2021.
Deyu Bo, Xiao Wang. Chuan Shi, and Huawei Shen. Beyond low freuureyy information in graph convolutional networks. In Proc. 35th AAAI Conf. Artif. Intell., pp. 3950-3957, Virtual Event, 2021.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. In 2nd Int. Conf. Leamn Rep., Banff, AB, Canada, April 2014.
Heng Chang, Yu Rong, Tingyang Xu, Wenbing Huang, Somayeh Sojoudi, Junzhou Huang, and Wenwu Zhu. Spectral graph attention network with fast eigen-approximation In 30th ACM Int. Virtual Event, 2021. ACM.
Conf. Inf. Knowl. Manag,, pp. 2905-290000
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In Int. Conf. Learn. Rep., 2021.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Adv. Neural Inf. Process. Syst. 29, pp. 3844-3852, Barcelona, Spain, December 2016.
Yushun Dong, Kaize Ding, Brian Jalaian, Shuiwang Ji, and Jundong Li. Adagnn: Graph neural networks with adaptive frequency response filter. In 30th ACM Int. Conf. Inf. Knowl. Manag, pp. 392-401, Virtual Event, 2021. ACM.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv preprint arXiv:2003.0098) 2020.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with Py PyTorch Geometric. In ICLR Workshop Representation Learning on Graphs and Manifolds, 2019.
Fernando Gama, Alejandro Ribeiro, and Joan Bruna. Diffusion scattering transforms on graphs. In 7th Int. Conf. Learn. Rep., 2019.
Feng Gao, Guy Wolf, and Matthew Hirn Geometric scattering for graph data analysis. In Proc. 36th Int. Conf. Mach. Learn., Long Beach, Califormia, USA, 2019.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proc, 34th Int. Conf. Mach. Learn., pp. 1263-1272, Sydney, NSW, Australia, August 2017.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Adv. Neural Inf. Process. Syst. 30, pp. 1024-1034, Long Beach, CA, USA, December 2017.
David Hammond, Pierre Vandergheynst, and Rémi Gribonval. Wavelets on graphs via spectral graph theory. Appl. Comput. Harmon. Anal., 30(2):12--150, 2011.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 770-778, 2016.
Weihua Hu, Matthias Fey, Marinka Zimik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Adv. Neural Inf. Process. Syst. 33, 2020.
10

---

Under review as a conference paper at ICLR 2022
Kristian Kersting, Nils M. Kriege, Christopher Morris, Petra Mutzel, and Marion Neumann. Bench- mark data sets for graph kernels, 2016. UR//ittp///grapbkaeeeels cs tu tu-dortmund.
de
Renata Khasanova and Pascal Frossard. Graph-based isometry invariant representation learning. In Proc. 34th Int. Conf. Mach. Learn pp. 1847-1856, Sydney, NSW, Australia, August 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Int. Conf.
Learn. Rep., San Diego, CA, USA, May 2015.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In 5th Int. Conf. Learn. Rep.. Toulon, France, April 2017.
Johannes Klicpera, Stefan Weißenberger, and Stephan Günnemann. Diffusion improves graph learn- ing. In Adv. Neural Inf. Process. Syst. 32, pp. 13354-1336,, 2019.
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling, In Proc. 36th Int. Conf. Mach. Learn., pp. 3734 3734-3743, Long Beach, CA, USA, June 2019.
Ron Levie, Federico Monti, Xavier Bresson, and Michael M. Bronstein. CayleyNets: Graph convo- lutional neural networks with complex rational spectral filters. IEEE Trans. Signal Process., 67 (1):97-109, January 2018.
Ron Levie, Wei Huang, Lorenzo Bucci, Michael M Bronstein, and Gitta Kutyniok. Transferability of spectral graph convolutional neural networks. arXiv preprint arXiv:1907.12972, 2019.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proc. 32nd AAAI Conf. Artif. Intell., pp. 3538-3545, New Orleans, LA, USA, February 2018.
Shouheng Li, Dongwoo Kim, and Qing Wang. Beyond low-pass filters: Adaptive feature propaga- tion on graphs. In ECML PKDD 2021 Joint Eur. Conf. Mach. Learn. Knowl. Discov. Databases, volume 12976, pp. 465. Springer, 2021.
Yimeng Min, Frederik Wenkel, and Guy Wolf. Scattering GCN: overcoming oversmoothness in graph convolutional networks. In Adv. Neural Inf. Process. Syst. 33, Virtual Event, 2020.
Sunil K Narang and Antonio Ortega. Perfect reconstruction two-channel wavelet filter banks for graph structured data. IREE Trans. Signal Process., 6066)2788- 2012.
Sunil K Narang and Antonio Ortega. Compact support biorthogonal wavelet filterbanks for arbitrary undirected graphs. IEEE Trans. Signal Process., 61(19):4633-668 2013.
Nicolò Navarin, Wolfgang Erb, Luca Pasa, and Alessandro Sperduti. Linear graph convolutional networks. In 28th Eur. Symp. Artif. Neural Netw., Comput. Intell. Mach. Leam.. pp. 151-156, Bruges, Belgium. 2020.
Hoang NT and Takanori Maehara. Revisiting graph neural networks: All we have low-pass filters. arXiv preprint arXiv: 1905 095009 2019.
Hoang NT, Takanori Maehara, and Tsuyoshi Murata. Adaptive stacked graph filter, 2021. URL httpss://ppereville net ftttt/oormmm1ddd00 9kh
Luca Pasa, Nicolò Navarin, and Alessandro Sperduti. Polynomial graph convolutional networks,
2021. [tt[ittt:://opeeeviieee wwtp://wwttt..de...eeeoo
Mzd-
Adam Paszke et al. Automatic differentiation in PyTorch. In Adv. Neural Inf. Process. Syst., Autodiff Workshop, Long Beach, CA, USA, December 2017.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. In 8th Int. Conf. Learn. Rep. OpenReview.net, 2020.
David I. Shuman, Sunil K. Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag., 30(3):83-98, May 2013.
11

---

Under review as a conference paper at ICLR 2022
Yuichi Tanaka and Akie Sakiyama. m-channel oversampled graph filter banks. IEEE Trans. Signal Process., 62(14))3578-33 3590, 2014.
Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social influence analysis in large-scale networks, In Proc. 15th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., pp. 807-816, 2009.
Dorina Thanou, David I Shuman, and Pascal Frossard. Learning parametric dictionaries for signals on graphs. IEEE Trans. Signal Process., (115))338493338 2014.
Nicolas Tremblay, Paulo Gonçalves, and Pierre Borgnat, Design of graph filters and filterbanks. In Cooperative and Graph Signal Processing, pp. 299-3244 Elsevier, 2018.
Petar Velikovi al. Graph attention networks. In 6th Int. Conf. Learn. Rep., Vancouver, BC, Canada, May 2018.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In Proc. 36th Int. Conf. Mach. Learn., pp. 6861-6871. PMLR, 2019.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Beyond low-pass filter- ing: Graph convolutional networks with automatic filtering. arXiv preprint arXiv:2107.04755 2021.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanic Jegelka. How powerful are graph neural networks? In 7th Int. Conf. Learn. Rep., New Orleans, LA, USA, May 2019.
Keyulu Xu et al. Representation learning graphs with jumping knowledge networks. In Proc. 35th Int. Conf. Mach. Learn., pp. 5449-5458, Stockholm, Sweden, July 2018
Hao Zhu and Piotr Koniusz. Simple spectral graph convolution. In 9th Int. Conf. Learn. Rep., 2021.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Adv. Neural Inf. Process. Syst. 33, virtual, 2020.
A NOTATIONS AND PRELIMINARIES
In this paper, we consider data that is represented on undirected graphs. An input graph is described as G = (V,E). with V and E respectively denoting the set of vertices and the set of edges. We generally use capital letters for matrices and bold lowercase letters for vectors. The graph topology is characterized by the adjacency matrix A with a non-zero value (A)is indicating the weight of the edge connecting vertices U; and uj. If available Fl the d-channel signals or attributes of nodes are represented as (² E I2 for Vum E V, and X = [x((1)) ), (v22),... (T, is for the whole graph 9 with = |V] nodes. Specially, for vector features, 2. (1m) indicates the i-th channel of the signal on vertex Um. and a; [r:((r) (12)),, , Iz (r,,) represents this signal on the whole graph. For the hierarchical structure of GCNs, we employ subscript 7 indicate variables or parameters belonging to the 1-th layer. In most cases, we ignore / for clarity without causing confusion.
The Graph Fourier Transform (GFT) is defined based the graph Laplacian matrix L (Shu- man et al. 2013). We adopt the symmetric normalized graph Laplacian operator, i.e., D- AD- with the diagonal degree matrix D defined as (D);: = (A()) The eigen- decomposion of the Laplacian matrix is denoted by L UAU", where U = [M1 uz...., un] composed of the eigenvectors uk. k 1.... n and A is a diagonal matrix with eigenvalues Y1 ,,...,,,,, An, with 0 = \ < Aa S ... - Xn < 2. The eigenvalues } construct the spectrum
IIf node attributes are not available, we adopt the structural information like the onc-hot encoding represen- tation of node degree and the local clustering coefficient node signals.
12

---

Under review as a conference paper at ICLR 2022
of the graph g and the GFT of signal æ is caleulated the inner-product between each of its component 24 and the eigenvectors { (Shuman et al. 2013):
(1a))
(S1)
where indicates conjugate transpose and uz (mm) denotes the m-th element in uz- The Inverse Graph Fourier Transform (IGFT) is defined as
vr( = Σ 3z (xx (ax)) (Um)-
(S2)
k=1
According to the convolution theorem, the convolution between the i-th channel of the signal x (vm) and the corresponding filter g(v.)) (with frequency response O(Ax) ) defined as:
(z: (Um)
(S3)
Let us define ((A) diag((((A))) ((2)))))(A22..... ((n))) We have for æs (Um) that (x) g) UA(A)U*ar wi.
((n)))
(S4)
Finally, graph filter bank is composed of set of filters (ū:(A)) to decompose graph signal into a series of signals with different frequency components (Narang & Ortega 2012).
B PROOF OF PROPOSITION ]
Proof. For spectral methods with polynomial filters ğ(A) Ek-o (()T(A), the j-th channel
(j == 1,2,, .1,,...,dh d') of the output signal is
hj
9ig
1=1
(S5)
Σ TA)U'
Thereby, the spectrum of output signal a linear transform of the K-order polynomial expansion of input signals 2A)
h
o (k) TA
(S6)
k=
For BankGCN, we assume that the j-th channel (j = 1,2...., d') of the output signal corresponds to the q-th channel of the filtered signal in the p-th subspace, and have
hj = (gp) Tppla) = (g[p)
(ii))oo bbbga)))
(S7)
= U
(W Te) (A)U 24 + +(p]g)
0=
where T[p]a denotes the q-th channel of the projected signal R in the p-th subspace, binj, indicates the q-th element b[p]. 1 denotes constant vector with value 1, and Eip]g represents vector independent on input signals. Correspondingly, the spectrum of h, is
h
(A)) Elply'
(S8)
According to Eq. S6 and Eq. S8 BankGCN can capture and preserve diverse frequency components of input graph signals, like its counterpart introduced in Eq. 2
In analogy to classical Fourier analysis, eigenvalues provide corresponding notion of frequency and lead frequency filtering.
13

---

Under review as a conference paper at ICLR 2022
C EXPERIMENTAL SETTINGS ON NODE CLASSIFICATION
We evaluate the models on four heterophilic graph datasets (Pei et al. 2020 Tang ct al. 2009), including Texas, Wisconsin, Cornell, and Actor. Texas, Wisconsin, Comell are three subdatasets from WebKB. In these datasets, nodes represent web pages with their 1703-dimensional bag-of- words representation
node features, and hyperlinks between the web pages form edges. Actor is a dataset preprocessed by Pei et al. (2020) from the fim--irecttoractoo--iiii 2009). The graph in the dataset represents the co-oceurrence (connection) of actors (nodes) on the same Wikipedia page. There are five categories on all these datasets.
network (Tang et al.)
We follow the data split and evaluation procedure of previous methods (Pei et al. 2020 Zhu al.) 2020 Li et al. 2021). Specifically.
(0::32%:2090 of nodes per class are split into training, vali- dation and test sets, and 10 such random splits on each dataset are provided by Pei al. (2020); Fey & Lenssen (2019). The network architecture consists of two layers. We select the following hyper-parameters through grid-search: number of feature maps of the hidden layer E {32, 64, 128), ). learning rateE (0.02, 0.04]. weight decay {5e 1eee- "), and E {4, 8, 16). In addition, the dropout is 0.5 only in the hidden layer, and is set as for the output layer.
We compare BankGCN with several state-of-the-an graph convolution methods. For the MPGCNs, we consider GCN (Kipf& & Welling 2017). GraphSage (Hamilton et al. (017), GAT (Vlikovi et al. 2018), Hix (Abu-El Haija et al. 20199) SGC (Wu et al.) 2019), GEOM-GCN (Pei et al. 2020), H2GCN (Zhu et al. 2020). SGF (NT et al. 2021). and ASGAT (Li et al. 2021]. Regarding spec- tral GCNs, we compare with its counterpart ChebNets (Defferrard al.) 2016) that also adopts
Chebyshev polynomial filters, and the most recent spectral method ARMA (Bianchi et al. 2021
D EXPERIMENTAL SETTINGS ON GRAPH CLASSIFICATION
Datasets and data splits. For TU datasets (Kesttng et al. 2016), we conduct experiments on seven widely used public benchmark graph classification datasets, including ENZYMES, D&D, PRO- TEINS, NCII, NCI109, MUTAGENICITY, and FRANKENSTEIN We adopt node categorical features (one-hot encoding) and node attributes as node signals, depending on availability on the datasets, which are usually heterogeneous in different dimensions. Specifically, node attributes are adopted on FRANKENSTEIN, node categorical features and node attributes (normalized in range [0, 1J) on ENZYMES, and node categorical features on the other datasets. The statistics and prop- erties of the datasets are summarized in Table ST According (Lee al., 2019), we use stratified sampling to randomly split each dataset into training, validation and test sets with a ratio of 8:1:1. The trained model with the best validation performance is selected for test. In order to alleviate the impact of data partition and network initialization, we conduct 20 random runs with different data splits and network initializations on each dataset, and report the mean accuracy with standard deviation of these 20 test results.
We further adopt two large benchmark datasets, CIFAR-10 (Dwivedi et al., 2020) and Ogbg-molhiv (Hu et al. 2020). in the experiments. The graph version of CIFAR-10 is composed from the super- pixels of images, and RGB intensities and normalized coordinates form node signals. Ogbg-molhiv is molecule graph dataset, with 9-dimensional node features including atomic number, chirality, and additional atom features. We divide data on these two datasets in accordance with (Dwivedi) let al. 2020) and (Hu al. 2020), respectively. Similarly, results are achieved with 3 runs on CIFAR-10 10 and 10 runs on Ogbg-molhiv in order to alleviate the impact of network initialization.
Network architectures. In the experiments, we adopt similar architecture (Xu et al. 2018). and the network consists of four convolution layers, one graph-level readout module and one prediction module for all the datasets. Specifically, the graph convolution layer is designed as introduced in Section 4 or defined by various baseline models. An le normalization function is utilized in each convolution layer to stabilize and accelerate the training process, as in (Hamilton et al., 2017). A graph-level readout module then aggregates the graph features from all the convolution layers to generate the graph representation hgg consists of node-wise mean and maximum operators (or mean operator on Ogbg-molhiv as in (Hu et al. 2020)), represented by w(-).
hg (on 1,2,3.4).
(S9)
Datasets could be downloaded from http:://h......aaaooooooc/oo///a
14

---

Under review as a conference paper at ICLR 2022
Table S1: Dataset statistics and properties (I. indicates node categorical features and A denotes node attributes).
ENZ
D&D
PROT
NCI1
NC1109
MUTA
FRAN
CIFAR-10
Ogbg-molhiv
Avg [V]
32.63
284.32
39.06
29.87
29.68
30.32
16.90
117.63
25,51
Avg [E]
62.14
715.66
72.82
32.30
32.13
30.77
17.88
564.86
27.47
Node feature
L+A
1.
1.
1.
1.
A
A
Dim(feat)
3+18
89
3
37
38
14
780
5
#Classes
2
2
2
2
2
2
10
#Graphs
600
1,178
1,113
4,110
4,127
4,337
4,337
60,000
41,127
Finally, prediction module composed of linear fully connected layer and softmax layer makes the prediction of the category of the input graph. The cross-entropy (represented as (e. Y)) is adopted as the loss function, which together with the regularization U?(a) for diversity condition composes the objective function:
Tol9, Y) + ((()
(S10)
where Q denotes all the free parameters and Y indicates ground truth categorical labels.
In more details, we consider two versions of the architectures in the experiments in order to evaluate the models under the same number of features and the same number of parameters, respectively. For the first case, all of models have the same number feature maps per hidden layer, whereas models composed of nearly the same number of parameters per hidden layer for the latter case. We consider the first version with TU datasets Table 2 with 64 feature maps per hidden layer, and the second version in Table 4 with nearly 16,500 and 65,800 free parameters per layer for all the models on CIFAR-10 10 and Ogbg-molhiv, respectively.
Configurations. We implement the proposed models in Pytorch (Paszke al., 2017 with geometric package (Fey & Lenssen 2019), and optimize all the models with the Adam optimizer (Kingma & Ba 2015) workstations with GPU GeForce GTX 1080 Ti for TU datasets as well as CIFAR- 10 and with GPU RTX 2080 Ti for Ogbg-molhiv. In more details, for TU datasets, the learning rate is 0.001 and the batch size is 64. The number of training epochs is set as 500, and early stopping is employed with patience 30. Finally, we obtain the following optimal htyer-parametees through grid search: weight decay E {0, 16-5, 1e-1) and 7 E (0, 10).
For CIFAR-10 10 and booggmo the batch size is increased to 256 due to their large scale. Similarly to (Dwivedi et al. 2020). we adopt dynamic learning rate that is initialized as 0.001 and decays by 0.1 when validation loss is not improved for 20 epochs until the minimum learning rate 1e-5, -5 The number training epochs 500 with early stoping (patience 50). The other settings are the same as TU datasets.
For the polynomial basis in the filter design of BankGCN, we adopt the widely used Chebyshev polynomials.
To(A) 1, (A) = X, Ta (A) - T
(S11)
The graph Laplacian is adopted as = L - I for numerical stability, similarly to Defferrard et al. 2016).
Baselines. We compare BankGCN with several state-of-the-art graph convolution methods. For the MPGCNs, we consider GCN (Kipf & Welling 2017). GraphSage (Hamilton al. 2017) with mean aggregation, GAT (8-heads) (Velikovi et al. 2018), and GIN using SUM-MLP (2 layers) that achieves the best performance (Xu al.) 2019). Regarding spectral GCNs, we compare with its counterpart ChebNets (Defferrard et al. (016) that also adopts Chebyshev polynomial filters, and the most recent spectral method ARMA (I-stack) (Bianchi al. (02T)) For a fair comparison, the results of baseline models are obtained with the same configurations as BankGCN using the public versions provided in the optimal hyper- parametrsss (weight decay) selected by grid search.
package (Fey & Lenssen 2019), with their respective
Ablation Study. Furthermore, we consider two simplifications of the proposed model for ablation study. First, BankGCN-Diff adopts predefined filter bank. The filter bank consists of low- pass filter, a high-pass filter, and band-pass filters from graph diffusion wavelets as used by Gama et al. 2019); Min et al. (200).. The specific frequency responses of these filters presented in
15

---

Under review as a conference paper at ICLR 2022
Table S2: Ablation study on the diversity regularization with BankGCN (K = = =8).
ENZY
DD
NCI1
PROT
NC1109
MUTA
FRAN
7 0
65.83 1 6.66
77.03 :: 4.08
81.89 +: 1.95
75.36
1: 4,68
81.03 + 1.95
82.44 +: 1.69
67.82 = 2.30
? 0.01
+ 5.23
77.54 - 3.21
81.63
+
75.49
1: 4.07
81.27 + 2.16
82.87 +: 1.94
67.71
= 1.96
7 0.1
68.00
78.09 + 2.18
82.06
+ 1.75
75.67
+ 4.19
81.34
+ 1.92
82.83 + 1.87
67.68
2.10
1
66.42
+ 6.20
77.97
+ 3.57
81.81
+ 1.97
76.29
+: 4.84
81.00 + 2.21
82.95 + 1.44
67.76
= 1.65
7
66.75
+ 5.90
78.14
+ 2.81
82.06
2.10
75.31
+ 4.64
81.54
+ 2.13
82.89
+ 1.61
67.20
1.67
7 100
67.08
5.42
77.75
3.26
61.11221
75.98
+ 4.40
81.14
1.94
82.44
1.71
67.52
1.83
Appendix
D Second, BankGCN-NR removes the diversity regularization from BankGCN with
y 0.
BankGCN-Diff' adopts predefined filter bank that consists of a low-pass pass filter, high-pass righ-pass filter, and band-pass filters from graph diffusion wavelets as used (Gama et al. 2019 Min et al. 2020). The specific frequency responses of these filters are
(1) = 1 - T,
1h. (7) = 1 - T"
(7) = T(I -))
Nh.1 (7) = (' - T")
(T) = (I - -T²,,
2 = ( -T
(T) - ((-)).))
(n, T T74)
(S12)
with two diffusion operators:
T (1-1 - D- AD-
T' (1+ D ADD /
(S13)
Their frequency responses can be easily derived with the relationship between the diffusion operators and the graph Laplacian. Specifically, t2,, and thoo are respective low-pass and high-pass filters, and the others are band-pass filters.
E MORE EXPERIMENTAL RESULTS AND ANALYSIS
More Ablation Results about the Diversity Regularization. Table S2 we consider the values of y {0, 0.01, 0.1, 1, 10, 100} to adjust its contribution in the objective function (Eq. [6]. The regularization improves the classification performance on almost all the datasets. Mostly, the best performance is achieved with y 0.1 7 10, and thereby in adjusting the contribution of regularization in the experiments. As illustrated in Fig. aa) the learned filters in filter bank with regularization present better diversity in terms of frequency response than those without regularization on NCI109 and ENZYMES datasets. For example, the filters denoted by blue and red in Fig. SI a) are with similar frequency responses, while they are more diverse in Fig. (b) and Fig. ST Besides, the filters indicated by gray and purple in Fig. (f)) show similar results. These are further verified with the maximum similarity scores of the polynomial coefficients that define the filters (2a) = 0.997, 0. 044,, and 0.649 (computed as Eq. (18)) for Fig. SI (C)),, and M(x) = 0.899, 0.714, and 0.705 for Fig. SI ()))), respectively.
consider y f0, 0.1, 10)
[l
More illustration results. We provide more illustration results in Fig. S2 to demonstrate adaptive filter banks with different numbers filters. With the increase of the 8, BankGCN more flexible to decompose the signal into components with different spectral characteristics. Furthermore, as shown in Fig. S3 the frequency responses of the learned filters in filter bank vary with the datasets and the layers the neural networks. These demonstrate the necessity and benefit of adopting learnalle rather than predefined filters.
F MORE COMPARISONS WITH RELATED WORKS
Message Passing Graph Convolutional Networks. MPGCNs design graph convolution with a va- riety of message passing schemes in the spatial domain. For instance, messages are aggregated with the node-wise mean or max operation in a localized neighborhood in GraphSage (Hamilton et al, 2017), or based attention scores in GAT (Velikovi et al. 2018). A more expressive scheme, GIN,
16

---

Under review as a conference paper at ICLR 2022
NC1109
(8) 0.
(b) 7 0.1.
(c) 7 10.
ENZYMES
(d) 0.
(e) 7 0.1.
(1) 10.
FRANKENSTEIN
(g) 7
7 0.1.
i Y 10.
Figure $1: Comparisons of the frequency responses of the learned filters of BankGCN (K 2, 8) under different regularization strengths on different datasets.
120
020
098
30
135
16
15
1.58
1.0
(a)
1.
(b) 4.
2.08
1.59
8.75
0.25
0.30
170
1.50
170
1.70
(c) 8.
(d) 16.
Figure S2: Illustrations of the frequency responses of the learned filters of BankGCN (K = 2,7 =
10) in the first layer of networks with different number of subspaces (s) NCI109.
17

---

Under review as a conference paper at ICLR 2022
NC1109
(a) Layer-1. 1.
(b) Layer-2.
(c) Layer-3.
ENZYMES
(d) Layer-1.
(c) Layer-2.
(f) Layer-3.
TRANKENSTEIN
(g) Layer-1.
(h) Layer-2.
(i) Layer-3.
Figure S3: Illustrations of the frequency responses the learned filters BankGCN (K = 2,8
in the different layers of networks different datasets.
18

---

Under review as a conference paper at ICLR 2022
further proposed with summing features in neighborhood followed by multi-layer perceptrons (MLP) to approximate any injective function on multiset (Xu et al. 2019). However, these methods typically constrained to a single one-hop message passing strategy that mostly captures 'low- pass' characteristics in the graph data. Klicpera et al. (2019) expand the MP range to multi-hop neighborhoods with graph diffusion operators. Besides, several methods are proposed imple- ment high-order (polynomial) filter with the whole network. For example, SGC (Wu et al. (2119) corresponds fixed polynomial filter, and LGC (Navarin et al. 2020), GPR-GNN (Chien al. 2021). SGF (NT et al. 2021), and S2GC (Zhu & Koniusz [2021) further realizes an adaptive filter with diverse motivations. Alternatively, group of methods attempt to complement the 'low-pass' features of GCN using different strategies. For instance, Scattering GCN (Min et al. 2020) resorts to geometric scattering transform (Gao et al. 2019) to complement the 'low-pass' features GCN (Kipf & Welling, 2017) with 'band-pass' Teatures, one-order high-pass filter is adopted as the com- plement (Bo ct al. 2021), and SpGAT (Chang et al. 2021] employs predefined graph wavelet filters. Besides, MixHop (Abu-El-Hail et 2019) explores linear mixing of neighborhood information to further realize difference operators, which can capture high-frequency information, and H2GCN (Zuu et al., (020)) additionaly aggregates information from high-order neighborhoods. Contrary to these methods, we employ a learnable filter bank with various frequency responses to adaptively capture diverse spectral characteristics of signals rather than merely adding 'high-pass' features or specific components via predefined graph wavelets. Finally, some works further consider adaptive
filters (Li al. 2021 Dong et al. 2021 Pasa al. 2021 but BankGCN is a more powerful and compact design with polynomial filter banks to avoid ingen-dccomposition and a further considera-
tion of the sharing scheme between filters to reduce free parameters. For instance, AdaGNN (Dong et al. [221]] implements an equivalent filter for each channel of input signals through the whole network, while BankGCN can realize it within a single convolution operator. Furthermore, the filter space of BankGCN with K Kooodd polynomial filters is larger than AdaGNN, since the K K-order filter in AdaGNN is obtained by multiplying one-order filters. The expressive capacity can be further enhanced by stacking multiple BankGCN layers.
19


Reviews:
[{"review_id": "ggUqP3ciihO", "reviewer": "Reviewer_1kMb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong points: The ablation studies are quite comprehensive and give a good sense of the effects of the K, s, and \\lambda parameters. \n\nThe Figures are clear and informative, the notation and writing is clear.\n\nWeak points: \n\nBankGCN tackles a by now well characterized problem, that of oversmoothing in the original formulations of graph convolutional networks. I find the claim that BankGCN is more powerful than \"most\" MPGCNs to be somewhat misleading, while true for those mentioned in the introduction, there are many new MPGCN architectures that address this problem, (mentioned below) as well as Scattering GCN and GPR-GNN (as mentioned in section 2). Comparison to at least some of these methods, or justification as to why these methods are not comparable is a must in an empirically driven paper like this.\n\nWhile the experiments are well designed, although code is not provided, the improvements are relatively minor as compared to the variation inherent in these datasets (as seen in similar work). I would encourage the authors to try different datasets where more substantial differences between methods may emerge, as on these benchmarks improvement is difficult to show.\n\nThe \"sharing scheme\" amounts to a linear layer before message passing, I believe this has been used in a number of earlier works (although I could be convinced otherwise).\n\nQuestions:\n\nIt is mentioned that section 5.2 compares models with the same number of free parameters per hidden layer, how is this done and why was this done on different datasets (Cifar / molhiv)?\n\nOther points:\n\nI find the way the authors use the term MPGCNs a bit confusing, as claims are made about BankGCN in contrast to MPGCNs where BankGCN. is a message passing GCN architecture. Could this be clarified further in the writing?\n\nWhile the notation is well explained, some of it is a bit unusual, for example \\mathbf{\\alpha} is a vector of \\alpha coeffs, but R is a vector of \\mathbf{r} features. T_k are chebyshev polynomials while T_\\Omega(G, Y) is the cross-entropy target.\n\nI believe the reference to Johannes Klicpera et al. (2019) in section 2 paragraph 2 is incorrect, you might have meant to reference the work:\n\nJohannes Klicpera, Aleksandar Bojchevski, and Stephan Gunnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In International Conference on Learning Representations, 2019\n\nRather than \n\nJohannes Klicpera, Stefan Weißenberger, and Stephan Gunnemann. Diffusion improves graph learn- ing. In Adv. Neural Inf. Process. Syst. 32, pp. 13354–13366, 2019.\n\nIn fact the work that you reference employs a learnable filter bank contrary to your claim in section 2.\n\nAdditional recent work in \n\nHao Zhu and Piotr Koniusz. Simple Spectral Graph Convolution. In ICLR 2021.\n\nAlso tackles the problem of oversmoothing and learns polynomial filters of the Laplacian using a slightly different formulation.", "summary_of_the_paper": "The authors introduce BankGCN, a graph convolutional network that learns (chebyshev) polynomial filters over the graph. BankGCN includes an initial subspace projection, and a cosine similarity regularization to encourage diverse filters. BankGCN is compared to a number of other graph networks on standard whole graph classification tasks.\n\nThe authors claim that most architectures of MPGCNs are limited in that they (1) focus on low frequency information and (2) lack a proper sharing scheme between filters, and that BankGCN addresses these two issues. ", "summary_of_the_review": "\nWhile the experiments are well designed, given that there are very few comparisons to similar works that claim to solve the problems of oversmoothing, and limited theoretical contribution I lean towards rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bhFCTJlXZlR", "reviewer": "Reviewer_84WF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nGraph neural networks are timely and of great interest to researchers.\n\nWeaknesses:\n\nIntegrating filter banks in graph neural networks is not new. There has been many studies, such as the work of Tremblay et al. (design of graph filters and filterbanks) and Bianchi et al. (ARMA-based filtering in graph neural networks), as well as related work like Thanou et al. (parametric dictionary learning for signals on graphs). Besides these cited papers, there are other papers on the same topic (in filter-adaptation for graph convolutional networks), such as:\n- Li, S., Kim, D., & Wang, Q. (2021). Beyond Low-Pass Filters: Adaptive Feature Propagation on Graphs. ECML/PKDD.\n- Wu, Z., Pan, S., Long, G., Jiang, J., & Zhang, C. (2021). Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering. ArXiv, abs/2107.04755.\n- Bo, D., Wang, X., Shi, C., & Shen, H. (2021). Beyond Low-frequency Information in Graph Convolutional Networks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 5, pp. 3950-3957).\nOnly the latter was cited in the submitted paper. While these papers are new, they do cite older papers on the same topic as this work.\n\nRoughly, the proposed method consists of integrating FIR filter banks, with the ARMA-based filtering of Bianchi et al. considers IIR filter banks.\n\nExperiments need to be extensive. While the authors consider several conventional graph convolutional networks in the comparative analysis, only the ARMA-based filtering of Bianchi et al. seems to be comparable in terms of adaptive filter banks. The authors need to provide extensive experiments by comparing to other related methods, considering at least the aforementioned ones.\n\nThe authors need to provide a thorough analysis of the computational complexity of the proposed method. While there is one sentence on computational complexity \"It achieves linear computational complexity with O(K|E|d) and constant learning complexity, similarly to most existing MPGCNs\", this needs to be demonstrated. In practice, the proposed method requires including a filter bank per layer and estimating its parameters can be cumbersome for large-scale graphs and multiple channels. The authors need to provide some experimental analysis of both the memory usage and the computational time, and comparing them to other methods in the literature.\n-------------------------\nWe thank the authors for addressing our concerns.", "summary_of_the_paper": "This paper proposes a graph convolutional networks where, at each layer, the graph signal is decomposed with an adaptive filter bank.", "summary_of_the_review": "Interesting work. However, there are several related methods in the literature.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "TL8G3oIFOC4", "reviewer": "Reviewer_nrop", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n(1) This paper addresses the limited flexibility of message passing convolutional networks (MPGCNs) because of their low-pass properties, which are one of the most important issues of Graph Neural Networks.\n\n(2) This paper is well written and structured.\n\n(3) This paper provides comprehensive experiments. In particular, the discussion of the number/order of filters and regularization is well presented to understand.\n\n**Weakness**\n\n(1) The related work is not detailed. There are many graph neural networks to deal with low-pass properties of MPGCNs [1], [2]. \n\n(2) I think the novelty of this paper is limited. The adaptive graph filter is a not novel method. For example,  GPR-GNNs [3] proposes adaptive polynomial graph filtering, which is similar to the proposed method. Please comment the difference.\n\n(3) The evaluation for the proposed methods is not sufficient. It is highly recommended to conduct node classification experiments to validate BankGCN. There are node classification datasets, which are dependent on high-frequency information not low-frequency information. *i.e.,* texas, wisconsin, and cornell. [2],[4]\n\n*[1] Scattering gcn: Overcoming oversmoothness in graph convolutional networks, Min, Yimeng et al., NeurIPS 2020.*\n\n*[2] Beyond low-frequency information in graph convolutional networks, Bo, Deyu, et al., AAAI 2021.*\n\n*[3] Adaptive universal generalized pagerank graph neural network, Chien, Eli, et al., ICLR 2021.*\n\n*[4] Geom-gcn: Geometric graph convolutional networks, Pei, Hongbin, et al., ICLR 2020.*", "summary_of_the_paper": "This paper proposes BankGCN, which simplifies spectral graph neural networks by utilizing an adaptive filter bank to extend the capabilities of GCNs beyond low-pass features. Compared to existing spectral graph convolutional networks, which have numerous free parameters, the proposed method reduces parameters by sharing learnable filters. The empirical study validates that BankGCN shows good performance on the graph classification task.", "summary_of_the_review": "Overall, I recommend marginally below the acceptance score. My concern is about the limited novelty and not enough experimental results. Hopefully, the authors can address it.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JfFG1rpsyTM", "reviewer": "Reviewer_rmns", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper is easy to follow with clear motivation and is well written. The design of BankGCN is closely rooted in Graph Signal Processing, which has merits.\n- Due to the common assumption on the homophily of underlying graphs, it is of great importance to go beyond low-pass filter types of GNNs.\n- Empirical observations on the filter bank and its response to the proposed model are interesting.\n\nWeaknesses:\n- The novelty of the proposed BankGCN is limited. BankGCN is closely related to polynomial GNNs with regularized and learnable coefficients. Thus, several designs towards simplifying polynomial GNNs that were proposed recently [1,2,3] might have relatively the same power of expressiveness, since the learned filters could be very similar. The authors are suggested to mention them, add discussions, and especially have an empirical comparison with them to demonstrate the superiority of BankGCN. Meanwhile, several other attempts that try to integrate adaptive frequency response filters into the design of GCNs such as [4,5] are missed and should be added in the discussion of Related Work.\n- As one main highlight of the design of BankGCN is to use the filter of K-order polynomial function space instead of computational-intense eigendecomposition, this sacrifice should be discussed more in detail, such as the comparison of the flexibility of learned filters and the degree of acceleration through computational analysis.\n- For experiments, the concerns include:\n    - The diversity spectral information of the chosen benchmark datasets, such as the average homophily ratio of all graphs, should be quantitatively analyzed since this is the main contribution of the proposed model, and how the learned filters correspond to specific graphs would demonstrate more convincingly.\n    - Most graphs in the benchmarks seem to be small and it is a known problem with those small and noisy datasets. Thus, instead of simply averaging several runs, cross-validation should be conducted and report the confidence interval.\n    - More recent baselines that also aim to use K-order polynomial function to go beyond low-frequency in graphs should be added. I understand that the design of BankGCN is inspired by the graph signal processing, but the proposed architecture is so similar to these methods, and only comparing with spectral-based GNNs is not adequate.\n\n[1] Polynomial Graph Convolutional Networks, OpenReview 2020\n\n[2] Stacked Graph Filter, arXiv 2021\n\n[3] Simple Spectral Graph Convolution, ICLR 2021\n\n[4] Graph Neural Networks with Adaptive Frequency Response Filter, CIKM 2021\n\n[5] Spectral Graph Attention Network with Fast Eigen-approximation, CIKM 2021\n", "summary_of_the_paper": "This paper proposes a new variant of GCN model, namely BankGCN, based on a novel graph convolutional operator. The main idea of BankGCN is to the sharing scheme among different filters to adaptively capture information from different frequencies. Through detailed discussion, BankGCN is claimed to be equivalent to the learnable message passing mechanism across K-hop neighborhood. Experiments on graph classification tasks over a collection of graph benchmark datasets demonstrate that BankGCN could outperform many state-of-the-art spectral-based GNNs.", "summary_of_the_review": "My concorns are mainly from two aspects:\n- The novelty of the proposed method is limited. The contribution of this paper needs further discussion between BankGCN and other efforts on designing filters from K-order polynomial function space. \n- The current experimental analysis is insufficient and can be further improved to make it more convincing.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6X5UAypczuU", "reviewer": "Reviewer_yK19", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The manuscript proposes an interesting point of view on graph convolutional operators. The proposed idea is clearly defined and theoretically grounded. The empirical evaluation of the operator shows interesting results, but some points about the experimental methodology have to be clarified since the comparison with the baseline models seems not completely fair.\n\nIn the introduction, the authors discuss the spectral convolution networks, and they claim that the lack of proper sharing schemes between filters makes these models redundant and even prone to overfitting. To me, this point is not completely clear, and in particular the criticism about the sharing scheme. Since it is one of the motivations on which BankGCN relies, in my opinion, this point it should be extended and further discussed.\n\nIn section 4.2 the authors state that f_{[p]} are defined as linear projections. Moreover, all the functions project the input in subspaces that have the same dimension. In the performed experiments how similar are the projections computed by the various f_{[p]} have the same dimension? Does their similarity influence the expressiveness of the model?\n\nThe definition of the filters (section 4.3) recalls the one defined by Wu, Felix, et al. in  \"Simplifying graph convolutional networks.\" (SGC) and the LGC proposed in (Navarin, Erb, et al. “Linear Graph Convolutional Networks.”), that exploits a normalized Laplacian instead of the Chebyshev polynomial. Considering that these models are very similar to the BankGCN with p=1, they should be also considered in the discussion and the experimental comparison.\n\nFor what concerns the experimental comparison to me it is not clear how the various models were validated, and so it is difficult to assess whether the reported comparison is fair or not. Indeed in section 5, the authors state:\n“The network in the experiment consists of four convolution layers, one graph-level readout module and a linear classier.” \nwhile in appendix C they state: “The trained model with the best validation performance is selected for test.” and on the next page: “...the learning rate is 0.001 and the batch size is 64. The number of training epochs is set as 500, and early stopping is employed with patience 30. Finally, we obtain the following optimal “hyper-parameters through grid search: weight decay ∈{0,1e−5,1e−4}and \\gamma ∈{0,0.1,10}.”\nThe fact that the model with the best performance on the validation set was selected is of course correct, but my understanding only the weights decay and \\gamma were selected through a grid search process while the other hyper-parameters seem to be set without performing any validation procedure.  Moreover, it is not clear whether the validation procedure was also used to select the hyper-parameters of the baselines models. Considering models that are all validated following the same validation procedure is crucial to perform a fair comparison. Indeed, as shown in (Errica et al.  “A Fair Comparison of Graph Neural Networks for Graph Classification”') the policy used to validate the hyperparameters of a model highly impacts the model's performance. The authors also state that “For a fair comparison, the results of baseline models are obtained with the same configurations as BankGCN using the public versions provided in the pytorch-geometric package”. In my opinion, it is not a fair way to compare the models using the configuration selected for the BankGCN,  mainly because each model has to be validated independently, in order to select the hyper-parameters that ensure the model to achieve the best performance (on the validation set). \nIn section 5 “For a fair and complete comparison, we consider two cases, (i) the same number of features, i.e., all of the models with the same number of feature maps per hidden layer, and (ii) the same number of parameters, i.e., models with nearly the same number of free parameters per hidden layer”. Comparing the models that exploit the same number of parameters is a very interesting analysis and highlights the expressiveness of the models but it does not show how good each model is in the graph classification task (as said before, this evaluation would require validating each model independently).\n\nThe results reported in Table 1 show that, in many cases, the accuracy gains obtained by the proposed method seem within noise levels due to high variance. Therefore it would be better to perform a statistical significance test,  to identify which improvements are significant. Note that the methodology followed to obtain the results and validation policy influence also the variance of the accuracy detected for each method.\n\nMinors:\neq (6): the last parenthesis is missing\nthe acronym IGFT is used without a prior definition of its meaning\nFor what concerns the paper organization, putting the notation in the appendix makes it a bit complex for the reader to follow the mathematical definition of the various components. in my opinion, the notation definition has to be present in the paper before the start of the theoretical discussion.\n", "summary_of_the_paper": "The manuscript proposes a novel convolutional operator dubbed BankGCN, that given a node input signal, first projects it in different subspaces using a group of projection functions (one for each subspace), and then applies to each projection an adaptive filter. \nAs projection functions, the authors use Linear projections. The model was tested on the Graph classification task.", "summary_of_the_review": "The manuscript proposes an interesting point of view on graph convolutional operators. The proposed idea is clearly defined and theoretically grounded. The empirical evaluation of the operator shows interesting results, but some points about the experimental methodology have to be clarified since the comparison with the baseline models seems not completely fair.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a graph convolution operator (BankGCN) to be used in graph neural networks. The reviewers mainly raised concerns about the limited of novelty in the light of numerous previous works that are similar or address similar problems as well as lacking evaluation. While the rebuttal addressed some of the concerns, the overall impression is that the paper is not of sufficient methodological or experimental significance for the conference.
Decision: Reject