Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited
Abstract
Designing spectral convolutional networks is a challenging problem in graph learning.
ChebNet, one of the early attempts, approximates the spectral graph convolutions using Chebyshev polynomials.
GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets.
GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral graph convolutions.
Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function.
In this paper, we revisit the problem of approximating the spectral graph convolutions with Chebyshev polynomials.
We show that ChebNet’s inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting.
We then propose ChebNetII, a new GNN model based on Chebyshev interpolation , which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomenon.
We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph convolutions and achieve superior performance in both full- and semisupervised node classification tasks.
Most notably, we scale ChebNetII to a billion graph ogbn-papers100M, showing that spectral-based GNNs have superior performance.
Our code is available at https://github.com/ivam-he/ChebNetII .
Introduction
Graph neural networks (GNNs) have received considerable attention in recent years due to their remarkable performance on a variety of graph learning tasks, including social analysis [31, 24, 38], drug discovery [49, 19, 32], traffic forecasting [26, 3, 7] and recommendation system [42, 46].
Spatial-based and spectral-based graph neural networks (GNNs) are the two primary categories of
GNNs.
To learn node representations, spatial-based GNNs [21, 15, 39] often rely on a message propagation and aggregation mechanism between neighboring nodes.
Spectral-based methods [8, 12] create spectral graph convolutions or, equivalently, spectral graph filters, in the spectral domain of the graph Laplacian matrix.
We can further divide spectral-based GNNs into two categories based on whether or not their graph convolutions can be learned.
• Predetermined graph convolutions: GCN [21] employs a simplified first tow Chebyshev polynomials as the graph convolution, which is a fixed low-pass filter [1, 41, 43, 54].
APPNP [22] and GDC [12] set the graph convolution with Personalized PageRank (PPR) and also achieve a low-pass filter.
S 2 GC [52] derives the graph convolution from the Markov Diffusion Kernel, which is a low- and high-pass filter trade-off.
GNN-LF/HF [54]
designs the graph convolutions from the perspective of graph optimization that can imitate low- and high-pass filters.
• Learnable graph convolutions: ChebNet [8] approximates the graph convolutions using Chebyshev polynomials and, in theory, could learn arbitrary filters [1].
CayleyNet [23] learns the graph convolutions with Cayley polynomials and generates various graph filters.
GPR-GNN [6] uses the Monomial basis to approximate graph convolutions, which can derive low- or high-pass filters.
ARMA [2] learns the rational graph convolutions through the Auto-Regressive Moving Average filters family [28].
BernNet [17] utilizes the Bernstein basis to approximate the graph convolutions, which can also learn arbitrary graph filters.
Despite the recent developments, two fundamental problems with spectral-based GNNs remain unsolved.
First of all, it is well-known that GCN [21] outperforms ChebNet [8] on real-world datasets (e.g., semi-supervised node classification tasks on citation datasets [21]).
However, it is also established that GCN is a simplified version of ChebNet with only the first two Chebyshev polynomials and that ChebNet has more expressive capability than GCN in theory [1].
Consequently, a natural question is: Why is ChebNet’s performance inferior to GCN’s despite its better expressiveness?
Secondly, as shown in [17], the real-world performance of ChebNet is also inferior to that of GPRGNN [6] and BernNet [17], which use Monomial polynomial basis and Bernstein polynomial basis to approximate the spectral graph convolutions.
Such a conclusion is counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves near-optimum error when approximating a function [13].
Therefore, the second question is: Why is ChebNet’s filter inferior to that of GPR-GNN and BernNet, despite the fact that Chebyshev polynomials have a higher approximation ability?
In this paper, we attempt to tackle these problems by revisiting the fundamental problem of approximating the spectral graph convolutions with Chebyshev polynomials.
First of all, according to the theory of the Chebyshev approximation, we observe that the coefficients of the Chebyshev expansion for an analytic function need to satisfy an inevitable convergence.
Consequently, we prove that
ChebNet’s inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting.
Furthermore, we propose ChebNetII, a new GNN model based on Chebyshev interpolation , which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomenon [10].
Our ChebNetII model has robust scalability and can easily cope with various constraints on the learned filters via simple reparameterization, such as the non-negativity constraints proposed in [17].
Finally, we conduct an extensive experimental study to demonstrate that ChebNetII can achieve superior performance in both full- and semi-supervised node classification tasks and scale to the billion graph ogbn-papers100M .
Revisiting ChebNet
Notations .
We consider an undirected graph G =
( V, E ) with node set V and edge set E .
Let n = | V | denote the number of nodes.
We use x ∈ R n to denote the graph signal, where x ( i ) denotes the signal at node i .
Note that in the general case of GNNs where the input feature is a matrix X , we can treat each column of X as a graph signal.
Let A denote the adjacency matrix and D denote the diagonal degree matrix, where D ii = (cid:80) nj A ij .
For convenience, we use P =
D − 1 / 2 AD − 1 / 2 and
L = I − D − 1 / 2 AD − 1 / 2 to denote the normalized adjacency matrix and the normalized Laplacian matrix of G , respectively.
We use L = UΛU T to represent the eigendecomposition of L , where U denotes the matrix of eigenvectors and Λ = diag [ λ 1 , ..., λ n ] is the diagonal matrix of eigenvalues.
2.1 Spectral-based GNNs and ChebNet
Spectral-based GNNs
create the spectral graph convolutions in the domain of Laplacian spectrum.
Recent studies suggest that many popular methods use the polynomial spectral filters to achieve graph convolutions [8, 21, 17].
We can formulate this polynomial filtering operation as
[Equation]
where y denotes the filtering results of x , and h ( λ ) is called the spectral filter, which is a function of eigenvalues of the Laplacian matrix L .
The w k denote the polynomial filter weights, and the
[Table]
[Table]
polynomial filter can be defined as h ( λ ) =
(cid:80) Kk {+=
+}
w k λ k , λ ∈ [0 , 2] .
ChebNet [8] is a remarkable attempt in this field, which uses Chebyshev polynomial to approximate the filtering operation.
[Equation]
where ˆ L = 2 L /λ max − I denotes the scaled Laplacian matrix.
λ max is the largest eigenvalue of L and w k denote the Chebyshev coefficients.
The Chebyshev polynomials can be recursively defined as
T k ( x ) = 2 xT k − 1 ( x ) − T k − 2 ( x ) , with T 0 ( x ) = 1 and T 1 ( x ) = x .
ChebNet’s structure is:
[Equation]
[Equation]
with the trainable weights W k .
The Chebyshev coefficients w k of the filtering operation (2) are implicitly encoded in the weight matrices W k .
We list more spectral-based GNNs’ details in the supplementary materials.
2.2 The motivation of revisiting ChebNet
ChebNet versus GCN.
Even though GCN is a simplified form of ChebNet, it is well known that
ChebNet is inferior to GCN for semi-supervised node classification tasks [21].
Table 1 shows the results of ChebNet and GCN for semi-supervised node classification tasks on Cora, Citeseer and
Pubmed datasets (see the Appendix for experimental details) .
We find that ChebNet is inferior to
GCN, especially when we increase the polynomial order K from 2 to 10 in Equation (3).
On the other hand, existing research [1] has shown that ChebNet is more expressive than GCN in theory.
In particular, ChebNet can approximate arbitrary spectral filters as K increases, while GCN is a fixed low-pass filter.
If we set K = 1 and w 0 = w 1 in the Equation (2), ChebNet corresponds to a high-pass filter; if we set K = 1 and w 0 = − w 1 , ChebNet becomes a low-pass filter which is essentially the same as GCN.
Consequently, a natural question is: Why is ChebNet’s performance inferior to GCN’s despite its better expressiveness?
Chebyshev basis versus other bases.
Chebyshev polynomials are widely used to approximate various functions in the digital signal processing and the graph signal filtering [36, 37].
The truncated
Chebyshev expansions are demonstrated to produce a minimax polynomial approximation for the analytic functions [13].
Consequently, the spectral filters can be well-approximated by a truncated expansion in terms of Chebyshev polynomials T k ( x ) up to K -th order [16].
[Equation]
where ˆ λ is the eigenvalue of the scaled Laplacian matrix ˆL .
ChebNet [8] then defined the graph convolutions using the Chebyshev approximated filters, while recent works were inspired by ChebNet and used Monomial (i.e., GPR-GNN [6]) and Bernstein (i.e., BernNet [17]) bases to approximate filters.
In order to evaluate the approximation ability of Chebyshev basis, we propose ChebNet with explicit coefficients, ChebBase, which simply replaces the Monomial basis of GPR-GNN and
Bernstein basis of BernNet with the Chebyshev basis.
The expression of ChebBase is
[Equation]
where f θ ( X ) denotes Multi-Layer Perceptron (MLP).
Table 2 reveals the results of ChebBase,
GPR-GNN and BernNet for node classification tasks on three citation graphs.
We can observe that
[Figure]
ChebBase has the worst performance, which is inconsistent with the fact that the Chebyshev basis can approximate minimax polynomial in theory.
Therefore, the second question is: Why is ChebNet’s filter inferior to that of GPR-GNN and BernNet, despite the fact that Chebyshev polynomials have a higher approximation ability?
2.3 Coefficient Constraints
We now demonstrate that ChebNet’s suboptimal performance is due to the illegal coefficients learned, which results in over-fitting.
Given an arbitrary continuous function f ( x ) in the interval [ − 1 , 1] , the Chebyshev expansion is defined as f ( x ) = (cid:80) ∞ k =0 w k T k
( x ) with the Chebyshev coefficients w k .
The following theorem establishes that in order to approximate an analytic function, the Chebyshev expansion’s coefficients must be constrained.
Theorem 2.1.
[48] If f ( x ) is weakly singular at the boundaries and analytic in the interval ( − 1 , 1) , then the Chebyshev coefficients w k will asymptotically (as k → ∞ ) decrease proportionally to 1 /k q for some positive constant q .
Here, "weakly singular" means that the derivative of f could vanish at the boundaries, and "analytic" means f can be locally given by a convergent power series in the interval ( − 1 , 1) . Intuitively,
Chebyshev polynomial T k ( x ) with larger k corresponds to higher frequency oscillation (see the Appendix for more details).
Theorem 2.1 essentially demonstrates that high frequency polynomials should be constrained in the Chebyshev expansion to approximate an analytic function.
Figure 1(a) depicts the Chebyshev expansion’s coefficients of the analytic function exp (ˆ λ ) used as a spectral filter in GDC [12] and shows that the coefficients are convergent.
[Table]
The ability to approximate an analytic function is crucial in the task of approximating the spectral filters, since non-analytic filters are more difficult to approximate by polynomials and may result in over-fitting.
In particular, ChebNet and ChebBase learn the coefficients w k by gradient descent without any constraints.
The coefficients may not satisfy
Theorem 2.1, leading to their poor performance.
To validate this conjuncture, we conducted an empirical analysis of ChebBase with difference coefficient constraints.
Inspired by Theorem 2.1, we use the following propagation process for the ChebBase /k .
[Equation]
where w k /k denote the Chebyshev coefficients implemented by reparameterizing the learnable parameters w k .
Table 3 shows the experimental results of the semi-supervised node classification performed on the citation graphs.
We can observe that with a simple penalty on w k , ChebBase /k outperforms
ChebNet, ChebBase, and GCN.
Figure 1(b) plots the absolute value of the Chebyshev coefficients learnt by ChebBase and ChebBase/ k on Cora.
We can observe that the coefficients of ChebBase/ k could more readily satisfy the convergence constraint.
These results validate Theorem 2.1.
ChebNetII model
Although ChebBase /k appears to be a promising approach, it still has some drawbacks: 1) Imposing the penalty on the coefficients is not mathematically elegant, as Theorem 2.1 only provides a necessary condition for the coefficients; 2) It is hard to impose further constraints on the learned spectral filters.
For example, it is unclear how we can modify Equation (6) to obtain non-negative filters, a requirement proposed in [17].
In this section, we describe ChebNetII, a GNN model based on Chebyshev interpolation that resolves the above two issues.
We also discuss the advantages and disadvantages of various polynomial interpolations as well as the Runge phenomenon.
3.1 Chebyshev interpolation
Consider a real filter function h (ˆ λ ) that is continuous in the interval [ − 1 , 1] .
When the values of this filter are known at a finite number of points ˆ λ k , one can consider the approximation by a polynomial
P K with K degree such that h (ˆ λ k ) =
P K (ˆ λ k ) , which is the general polynomial interpolation.
We give an explicit expression of the general polynomial interpolation in the supplementary materials.
We generally sample the K + 1 points ˆ λ 0 < ˆ λ 1 < ... <
ˆ λ K uniformly from [ − 1 , 1] to construct the interpolating polynomial P K (ˆ λ ) .
Intuitively, increasing K should improve the approximation quality.
However, this is not always the case due to the Runge Phenomenon [10] (The details are discussed in section 3.3).
The popular approach to this problem in the literature [14] is Chebyshev interpolation, having superior approximation ability and faster convergence.
Instead of sampling the interpolation points uniformly, Chebyshev interpolation uses Chebyshev nodes as the interpolation points, which are essentially the zeros of the ( K + 1) -th
Chebyshev polynomial.
Definition 3.1.
( Chebyshev Nodes )
The Chebyshev polynomial T k ( x ) satisfies the closed form expression T k ( x ) =
cos ( k arccos( x )) .
The Chebyshev Nodes for T k ( x ) are defined as x j = cos (cid:0) 2 j +12 k π (cid:1) , j = 0 , 1 , ..., k − 1 , which lie in the interval ( − 1 , 1) and are the zeros of T k ( x ) .
Definition 3.1 suggests that each Chebyshev polynomial T k ( x ) has k zeros, and we can define
Chebyshev interpolation by replacing the equispaced points with Chebyshev nodes in the general polynomial interpolation (see the Appendix for details).
More eloquently, definition 3.2 efficiently defines the Chebyshev interpolation via their orthogonality properties.
Definition 3.2.
( Chebyshev Interpolation )
[14] Given a continuous filter function h (ˆ λ ) , let x j = cos (cid:16) j +1 / 2 K
+1 π (cid:17) , j = 0 , . .
, K denote the Chebyshev nodes for T K +1 and h ( x j ) denotes the function value at node x j .
The Chebyshev interpolation of h (ˆ λ ) is defined to be
[Equation]
where the prime indicates the first term is to be halved, that is, c ′
=
c
[Equation]
[Equation]
3.2 ChebNetII via Chebyshev Interpolation
Inspired by Chebyshev interpolation, we propose ChebNetII, a graph convolutional network that approximates an arbitrary spectral filter h (ˆ λ ) with an optimal convergence rate.
ChebNetII simply reparameterizes the filter value h ( x j ) in Equation (7) as a learnable parameter γ j , which allows the model to learn an arbitrary spectral filter via gradient descent.
More precisely, the ChebNetII model can be formulated as
[Equation]
where x j = cos (( j + 1 / 2) π/ ( K + 1)) are the Chebyshev nodes of T K +1 , f θ ( X ) denotes an MLP on the node feature matrix X , and γ j for j = 0 , 1 , ..., K are the learnable parameters.
Note that similar to APPNP [22], we decouple feature propagation and transformation.
[Table]
Consequently, the filtering operation of ChebNetII can be expressed as
[Equation]
It is easy to see that compared to the filtering operation of the original ChebNet (2), we only make one simple change: reparameterizing the coefficient w k by w k = 2 K +1 (cid:80) Kj =
γ j T k ( x j ) .
However, this simple modification allows us to have more control on shaping the resulting filter, as Chebyshev interpolation suggests that γ j directly corresponds to the filter value h ( x j ) at the Chebyshev node x j .
The coefficients w k = 2 K +1 (cid:80)
Kj =
h ( x j ) T k ( x j ) are fundamentally guaranteed to satisfy the constraints of Theorem 2.1 since we directly approximate the filter h .
Furthermore, Chebyshev interpolation also provides the ChebNetII with several beneficial mathematical properties.
3.3 Analysis of ChebNetII
ChebNetII has several advantages over existing GNN models due to the unique nature of Chebyshev interpolation.
From the standpoint of polynomial approximation and computational complexity, we compare ChebNetII with current related approaches such as GPR-GNN [6] and BernNet [17].
Near-minimax approximation.
First of all, we examine ChebNetII’s capabilities in terms of filter function approximation.
Theorem 3.1 exhibits that ChebNetII provides an approximation that is close to the best polynomial approximation for a spectral filter h .
Theorem 3.1.
A polynomial approximation P ∗ K ( x ) for a function f ( x ) is said to be nearbest/minimax approximation with a relative distance ρ if
[Equation]
where ρ is the Lebesgue constant, P ∗ B ( x ) is a best polynomial approximation, and || · || represents the uniform norm (i.e., || g
|| =
max x ∈ [ − 1 , 1] | g ( x ) | ).
Then, we have ρ ∼ 2 K as K → ∞ for the general polynomial interpolation, and ρ ∼ log( K ) as K → ∞ for the Chebyshev interpolation.
Convergence.
In comparison to BernNet [17], which uses the Bernstein basis, ChebNetII has a faster convergence rate for approximating a filter function.
Specifically, we have the following Theorem:
Theorem 3.2.
[14, 29] Let P K ( x ) be the polynomial approximation for a function f ( x ) .
Then the error is given as || f ( x ) − P K ( x ) || ≤ E ( K ) .
If P K ( x ) is obtained by Bernstein approximation, then E ( K ) ∼ (1 + (2 K ) − 2 ) ω
( K − 1 / 2 ) ; if P K ( x ) is obtained by Chebyshev Interpolation, then
E ( K ) ∼ Cω ( K − 1 ) log( K ) with a constant C , where ω is the modulus of continuity.
Runge phenomenon.
In comparison to GPR-GNN [6], which uses the Monomial basis, ChebNetII has the advantage of reducing the Runge phenomenon [10].
In particular, when we use the general polynomial interpolation to approximate a Runge filter h (ˆ λ ) with a high degree over a set of equispaced interpolation points, it will cause oscillation along the edges of an interval.
Consequently, as the degree of the polynomial increases, the interpolation error increases.
Following [14], we define the error of polynomial interpolation as
[Equation]
where π K +1 (ˆ λ ) = (cid:81)
Kk =
(ˆ λ − ˆ λ k ) denotes the nodal polynomial and ζ is the value depending on ˆ λ .
The terrible Runge phenomenon is caused by the values of this nodal polynomial, which have very
[Table]
high oscillations around the interval endpoints.
In particular, for high-degree polynomial interpolation at equidistant points in [ − 1 , 1] , we have
[Table]
On the other hand, we have the following Theorem 3.3 that explains that Chebyshev nodes can minimize and quantify this error caused by the nodal polynomial, meaning Chebyshev interpolation minimizes the problem of the Runge phenomenon.
Theorem 3.3.
[14] Consider the Chebyshev nodes x j = cos
( j + 1 / 2) π/ ( K + 1)) , j = 0 , 1 , ..., K .
Then the nodal polynomial ˆ T K +1 ( x ) = (cid:81) K k
=0 ( x − x j ) has the smallest possible uniform norm,
i.e., || ˆ T K
+
( x ) ||
= 2 − K .
Computational complexity.
Compared to BernNet [17], which has a time complexity quadratic to the order K in the forward process, ChebNetII can be computed in time linear to K .
Specifically, we first compute the ChebNetII’s coefficients
[Equation]
) in time linear to K as we can precompute T k ( x j ) , and then plug the coefficients into Equation (8) for propagation, which also takes the time linear to K , the same as that of ChebNet [8] and GPR-GNN [6].
Experiments
In this section, we conduct experiments to evaluate the performance of ChebNetII against the state-of-the-art graph neural networks on a wide variety of open graph datasets.
Dataset and Experimental setup.
We evaluate ChebNetII on several real-world graphs for the Semiand Full-supervised node classification tasks.
The datasets include three homophilic citation graphs:
Cora, Citeseer, and Pubmed [35, 45], five heterophilic graphs: the Wikipedia graphs Chameleon and Squirrel [34], the Actor co-occurrence graph, and webpage graphs Texas and Cornell from
WebKB * [30], two large citation graphs: ogbn-arxiv and ogbn-papers100M [18], as well as six large heterophilic graphs: Penn94, pokec, arXiv-year, genius, twitch-gamers and wiki [25].
We measure the level of homophily in a graph using the edge homophily ratio H ( G ) =
|{ ( u,v ):(
u,v ) ∈ E ∧
y v = y u }| | E | [53], where y v denotes the label of node v .
We summarize the dataset statistics in Tables 4 and 5.
All the experiments are carried out on a machine with an NVIDIA RTX
GPU (48GB memory), Intel
Xeon CPU (2.20 GHz) with 40 cores, and 512 GB of RAM.
4.1 Semi-supervised node classification with polynomial-based methods
Setting and baselines.
For the semi-supervised node classification task, we compare ChebNetII to 7 polynomial approximation filter methods, including MLP, GCN [21], ARMA [2], APPNP [22],
ChebNet [8], GPR-GNN [6] and BernNet [17].
For dataset splitting, we employ both random and fixed splits and report the results on random splits.
The results of fixed splits will be discussed in the Appendix.
Specifically, we apply the standard training/validation/testing split [45] on the three homophilic citation datasets (i.e., Cora, Citeseer, and Pubmed), with 20 nodes per class for training, 500 nodes for validation, and 1,000 nodes for testing.
Since this standard split can not be used for very small graphs (e.g. Texas), we use the sparse splitting [6] with the training/validation/test sets accounting for 2.5%/2.5%/95%, respectively, on the five heterophilic datasets.
For ChebNetII, we use Equation (8) as the propagation process and use the ReLu function to reparametrize γ j , maintaining the non-negativity of the filters [17].
We set the hidden units as
[Table] [Table]
and K = 10 for the all datasets as the same as GPR-GNN [6] and BernNet [17].
We employ the
Adam SGD optimizer [20] with an early stopping of 200 and a maximum of 1000 epochs to train
ChebNetII.
We use the officially released code for GPR-GNN and BernNet and use the Pytorch
Geometric library implementations [11] for other models (i.e., MLP, GCN, APPNP, ARMA, and
ChebNet).
More details of hyper-parameters and baselines’ settings are listed in the Appendix.
Results.
We utilize accuracy (the micro-F1 score) with a 95% confidence interval as the evaluation metric.
Table 6 reports the relevant results on 10 random splits.
Boldface letters indicate the best result for the given confidence interval, and underlinings denote the next best result.
We first observe that ChebNet is inferior to GCN even on heterophilic graphs, which concurs with our theoretical analysis that the illegal coefficients learned by ChebNet lead to over-fitting.
ChebNetII, on the other hand, outperforms other methods on all datasets excluding Pubmed, where it also achieves topclassification accuracy.
This quality is due to the fact that the learnable parameters γ j of ChebNetII directly correspond to the filter value h ( x j ) at the Chebyshev node x j , effectively preventing it from learning an illegal filter.
4.2 Full-supervised node classification
Setting and baselines.
For full-supervised node classification, we compare ChebNetII to the baselines in the prior semi-supervised node classification.
We also include GCNII [5], TWIRLS [44],
EGNN [51] and PDE-GCN [9] four competitive baselines for full-supervised node classification.
For all datasets, we randomly split the nodes into 60%, 20%, and 20% for training, validation and testing, and all methods share the same 10 random splits for a fair comparison, as suggested in [30, 6, 17].
For ChebNetII, we also set the hidden units to be 64 and K = 10 for all datasets, and employ the same training manner as in the semi-supervised node classification task.
For GCNII, TWIRLS,
EGNN and PDE-GCN we use the officially released code.
More details of hyper-parameters and baselines’ settings are listed in the Appendix.
Results.
Table 7 reports the mean classification accuracy of each model.
We first observe that, given more training data, ChebNet starts to outperform GCN on both homophilic and heterophilic datasets, which demonstrates the effectiveness of the Chebyshev approximation.
However, we also observe that ChebNetII achieves new state-of-the-art results on 7 out of 8 datasets and competitive results on Pubmed.
Notably, ChebNetII outperforms GPR-GNN and BernNet by over 10% on the
[Table]
Squirrel dataset.
We attribute this quality to the fact that Chebyshev interpolation achieves nearminimax approximation of any function with respect to the uniform norm, giving ChebNetII greater approximation power than GPR-GNN and BernNet do.
4.3 Scalability of ChebNetII
For ChebNetII, if we calculate and save T k (ˆ L ) X for k ∈ 0 , · · · , K in the preprocessing, we can scale it to large graphs.
Specifically, we use the below propagation expression.
[Equation]
The pre-computed ˆ L k X allow us to train γ j and f θ ( · ) in a mini-batch manner.
This approach also works for GPR-GNN [6] and BernNet [17], so we report their results in this manner when ChebNetII does.
We evaluate the scalability of ChebNetII on the large heterophilic graphs [25] and the widely used OGB datasets [18].
[Table]
On the large heterophilic graphs, we compare ChebNetII to eight competitive baselines, including MLP,
LINK [50], LINKX [25], GCN [21], GCNII [5],
ChebNet [8], GPR-GNN [6] and BernNet [17].
For
ChebNetII, we use Equation (12) as the propagation process on pokec and wiki and Equation (8) on the four remaining datasets.
We establish the experimental setting following [25] and use the published baselines’ results, excluding GPR-GNN.
More details are listed in the Appendix.
Table 8 reports the mean results of each method over 5 runs.
ChebNetII outperforms all other methods on 4 out of 6 datasets and the polynomial-based methods on arXiv-year and twitch-gamers.
Notably, LINK and LINKX outperform ChebNetII on arXiv-year because they use a directed graph on this dataset.
Using the directed graphs to the spectral-based GNNs is a future meaningful work because the current spectral graph theory only applies to undirected graphs.
For the largest heterophilic graph wiki, ChebNetII obtains a new state-of-the-art result.
We attribute that ChebNetII can precompute ˆ L k X without the graph sampling used in LINK and LINKX and has a strong filter approximation ability.
On ogbn-arxiv and -papers100M, we compare ChebNetII to polynomial-based GNNs and state-ofthe-art scalable GNNs, SIGN [33], GBP [4], and NDLS ∗ [47].
We follow the standard splits [18] and use Equation (12) as the propagation process.
More details of settings are listed in Appendix.
Tablereports the mean accuracy of each model over 10 runs.
Note that we do not include the result of BernNet on ogbn-papers100M as BernNet has a time complexity quadratic to the order K and fails to finish the preprocessing in 24 hours.
We can observe that ChebNetII outperforms both datasets,
[Figure]
which we attribute to Chebyshev Interpolation’s superior approximation quality.
These results also show that ChebNetII has lesser complexity and greater scalability than BernNet.
4.4 Comparison of Different Polynomial Bases
We perform numerical studies comparing the Chebyshev basis to the Monomial and Bernstein bases to demonstrate ChebNetII’s approximation power.
Considering a Runge filter h (ˆ λ )
=
1 / (1+25ˆ λ 2 ) , ˆ λ ∈ [ − 1 , 1] , Figures 2(a) and 2(b) depict the approximation results and errors for several polynomial bases, with the polynomial degree K denoted by the numbers in brackets.
We find that the Chebyshev basis has a faster convergence rate than the Bernstein basis and does not exhibit the Runge phenomenon compared to the Monomial basis.
Notably, JacobiConv [40] investigated different polynomial bases at the same period and discovered that orthogonal polynomial bases could learn graph filters more effectively.
These findings provide empirical motivations for designing GNNs with Chebyshev interpolation.
Conclusion
This paper revisits the problem of approximating spectral graph convolutions with Chebyshev polynomials.
We show that ChebNet’s inferior performance is primarily due to illegal coefficients learned by approximating analytic filter functions, which leads to over-fitting.
Moreover, we propose
ChebNetII, a new GNN model based on Chebyshev interpolation, enhancing the original Chebyshev polynomial approximation while reducing the Runge phenomenon.
Experiments show that ChebNetII outperforms SOTA methods in terms of effectiveness on real-world both homophilic and heterophilic datasets.
For future work, a promising direction is further to improve the performance of ChebNetII on large graphs and investigate the scalability of spectral-based GNNs.
Acknowledgments and Disclosure of Funding
The work was partially done at Gaoling School of Artificial Intelligence, Peng Cheng Laboratory,
Beijing Key Laboratory of Big Data Management and Analysis Methods and MOE Key Lab of Data
Engineering and Knowledge Engineering.
This research was supported in part by the major key project of PCL (PCL2021A12), by National Natural Science Foundation of China (No. 61972401, No. 61932001, No. 61832017, No. U2001212), by Beijing Natural Science Foundation (No. 4222028), by
Beijing Outstanding Young Scientist Program No.
BJJWZYJH012019100020098, by Alibaba Group through Alibaba Innovative Research Program, by CCF-Baidu Open Fund (NO.2021PP15002000) and by Huawei-Renmin University joint program on Information Retrieval.
We wish to acknowledge the discussion with Professor Zhouchen Lin from Peking University.
We also wish to acknowledge the support provided by Engineering Research Center of Next-Generation Intelligent Search and
[-δ 0 ( λ ) where δ 0 ( λ ) = 
-]
[-= δ 1 ( λ ) where δ 1 ( λ ) = 1 if λ = 1 and δ 1 ( λ ) = 
 otherwise.-]Recommendation, Ministry of Education.
[-with the nodal polynomial π K +1 (ˆ λ ) = (cid:81) Kj =
-]
Additionally, we acknowledge the support from Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary
Platform for the “Double-First Class” Initiative, Public Policy and Decision-making Research Lab,
[-we set w 0 = 1 . 0 , w k = 0 . 
-]
Public Computing Cloud, Renmin University of China.