Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles
Authors: Rui Duan
Mingjian Guang
Junli Wang
Chungang Yan
Hongda Qi
Wenkang Su
Can Tian
Haoran Yang
Venue: NeurIPS 2024 poster (2024)
Keywords: graph neural networks; filter ensemble; homophily and heterophily
Primary area: graph_neural_networks
TL;DR: Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles

Abstract: Polynomial-based learnable spectral graph neural networks (GNNs) utilize polynomial to approximate graph convolutions and have achieved impressive performance on graphs. Nevertheless, there are three progressive problems to be solved. Some models use polynomials with better approximation for approximating filters, yet perform worse on real-world graphs. Carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints leaded to overfitting, which diminishes the generalization of the models. How to design a model that retains the ability of polynomial-based spectral GNNs to approximate filters while it possesses higher generalization and performance? In this paper, we propose a spectral GNN with triple filter ensemble (TFE-GNN), which extracts homophily and heterophily from graphs with different levels of homophily adaptively while utilizing the initial features. Specifically, the first and second ensembles are combinations of a set of base low-pass and high-pass filters, respectively, after which the third ensemble combines them with two learnable coefficients and yield a graph convolution (TFE-Conv). Theoretical analysis shows that the approximation ability of TFE-GNN is consistent with that of ChebNet under certain conditions, namely it can learn arbitrary filters. TFE-GNN can be viewed as a reasonable combination of two unfolded and integrated excellent spectral GNNs, which motivates it to perform well. Experiments show that TFE-GNN achieves high generalization and new state-of-the-art performance on various real-world datasets.

Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles
Rui Duan', Mingjian Guang', Junli Wang', Chungang Yan', Hongda Qi+,
Wenkang Su Can Tian'; Haoran Yang²
School of Computer Science and Cyher Engineering, Guangzhou University, China, Donghua University, China, 'Tongii University, China 'Shanghai Normal University, China *[duan, swk1004, tiancan) 0gzhu. edu. en; 2 breangmingjianeene edu. cn; [junliwang, yanchungang, (1.0498)@tonge edu. en; "hongda_ _iciaa__iieee @shnu. edu.
Abstract
Polynomial-bee based learnable spectral graph neural networks (GNNs) utilize polyno- mial to approximate graph convolutions and have achieved impressive performance graphs. Nevertheless, there are three progressive problems to be solved. Some models use polynomials with better approximation for approximating filters, yet perform worse real-world graphs. Carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints leaded overfitting, which diminishes the generalization of the models. How to design a model that retains the ability of polynomial-ban -based spectral GNNs to approximate filters while it possesses higher generalization and performance? In this paper, we propose spectral GNN with triple filter ensemble (TFE-GNN), which extracts ho- mophily and heterophily from graphs with different levels of homophily adaptively while utilizing the initial features. Specifically, the first and second ensembles are combinations of set of base low-pass and high-pass filters, respectively, after which the third ensemble combines them with two learnable coefficients and yield a graph convolution (TFE-Conv) Conv). Theoretical analysis shows that the approximation ability of TFE- GNN is consistent with that of ChebNet under certain conditions, namely it can learn arbitrary filters. TFE-GNN can be viewed as a reasonable com- bination of two unfolded and integrated excellent spectral GNNs, which motivates it to perform well. Experiments show that TFE-GNN achieves high generalization and new state-of-the -htt--rtt performance on various real-world datasets. The source code of GEN is publicly available https: //gpttbtttttt///////////gt com/ graphhN//TEGGNN
Introduction
Graph neural networks (GNNs) are competitive in graph-related tasks (Scarselli et al., 2008; Yang et al., 2023; Shirzad et al., 2023; Duan et al., 2024) and can be divided into two main categories: spatial-based (Xu et al., 2019) and ppectral-based (Shirzad et al., 2023; Tao et al., 2023; He et al., 2022; Bo al., 2023b; Guo et al., 2023) GNNs. Spectral graph convolutions in the spectral domain of the graph Laplace matrix, i.e., the spectral graph filters, are the core component of spectral- based GNNs. We further classify spectral-based GNNs into two categories based whether their graph convolutions can be learned or not.
The first class of spectral-basee GNNs' graph convolutions predetermined, i.c., they filter the graph signals (features) in fixed way. Graph convolutional networks (GCNs) (Kipf & Welling, 2016)
*oorrespndiigg authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and their variants (Rong et al., 2020; Chen et al., 2020; Yang et al., 2021) utilize only the first two Chebyshev polynomials to simplify ChebNet (Defferrard et al., 2016), and their graph convolutions are the fixed low-pass filters.
The second class of spectral-based based GNNs' graph convolutions are learnable, i.e., their filters variable and they filter the graph signals in learnable way. ChebNet (Defferrard et al., 2016) utilizes Chebyshev polynomials to approximate the graph convolutions and it can learn arbitrary filters in theory (Balcilar et al., 2021; He et al., 2022). CayleyNet (Levie et al., 2018) utilizes Cayley polynomials to learn the graph convolutions, and ARMA (Bianchi al., 2021a) learns the rational graph convolutions by using the 2013). GPR-GG GPN (Chien et al., 2021) and BernNet (He et al., 2021a) use Monomial and Bernstein polynomials to approximate the graph convolutions. ChebNetll (He et al., 2022) revisits ChebNet and makes learned coefficients more legal through Chebyshev interpolation. EvenNet (Lei et al., 2022) ignores odd-hop neighbors and improves the robustness of GNNs by using the even-poly wven-polynomial graph filter. PCNet (Li et al., 2023) uses the Possion-Chariinn polynomials to approximate the graph filter and constrain the coefficients. Just like EvenNet, the heterophilic graph heat kernel provided by PCNet pushes odd-hop neighbors away and aggregates even-hop neighbors, FavardGNN (Guo & Wei, 2023) learns polynomial basis from the space of possible orthonormal bases and OptBasisGNN (Guo & Wei, 2023) computes the optimal basis for a given graph structure and signal. Specformer (Bo et al., 2023a) encodes the set of eigenvalues and performs self-attention in the spectral domain, which leads to learnable set-to-set spectral filter.
Moving Average filters family (Narang et al.,
Despite volynomial-based learnable spectral GNNs have achieved impressive performance on graphs. there are three progressive problems to be solved. First, some polynomial-ban models use polyno- mials with better approximation than some other models when approximating filters, but the former's ss performance is lagging behind that of the latter on real-world graphs. For example, GPR-GNN and BernNet outperform ChebNet, even though they use polynomials that are weaker than Chebyshev polynomials in approximation theory (He et al., 2022). ChebNetll, an enhanced version of ChebNet, whose performance still lags behind that of PCNet using Possion-Charlier polynomials. The important factors influencing the real-world performance of such GNNs are graph learning methods, polynomial approximation and coefficient constraints, but of course there are others well, in any case not only the polynomial approximation ability. The following two facts exist: ChebNetll outperforms GPR-GNN and BernNet through refined coefficient constraints; and PCNet outperforms ChebNetll through the carefully crafted graph learning method, i.e., pushing odd-hop neighbors away to match the structural properties of heterophilic graph.
The second problem was raised based on the answer to the first problem. Carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints leaded to overfitting while improving models' performance, which diminishes the generalization of the models. FFKSF (Zeng et al., 2023) attributes the degradation of polynomial filters' performance to the overfitting of polynomial coefficients. ChebNetlI (He et al., 2022) further constrains the coefficients to enable them easier to be optimized. ARMA (Bianchi et al., 202 suggests that the filter will overfit the training data when aggregating high-order neighbor information. Whereas spectral GNNs is usually large to increase the approximation of the
the order of polynomial-basee polynomials, which directs them to obtain high-order neighborhood information, and then leads to overfitting. Therefore, it is reasonable to assume that carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints lead to overfitting of the models, which diminishes generalization of the models. Desired learnable spectral GNNs can learn various graph convolutions, which motivates them to extract homophily from heterophilic graph and vice versa. Instead of ignoring odd-hop or even-hop neighbors, which makes the model miss important neighbor information.
Finally, the third problem was raised. How to design a model that retains the ability of polynomial- based spectral GNNs to approximate filters while it possesses higher generalization and performance? In this paper, inspired by ensemble learning (Schapire, 1990; Hansen & Salamon, 1990; Zhou, 2012) (see in section 3.1), we design triple filter ensemble (TFE) mechanism to adaptively extract homophily and heterophily from graphs with different levels of homophily while utilizing the initial features, where the first and second ensembles combine a set of base low-pass and high-pass graph filter, respectively, and the third ensemble combines them by two learnable coefficients. The third ensemble of TFE will yield graph convolution (TFE-Conv) used to filter the graph signal. The filtered signal is
2

---

fed into fully connected linear neural network (NN), whose output is then passed through a softmax layer to obtain the prediction.
TFE-GNN does not impose refined constraints on the coefficients and does not design very complex learning methods, which possesses higher generalization. The key difference between TFE-GNN and prior models is that TFE-GNN retains the ability of polynomial- based spectral GNNs while getting rid of polynomial computations, coefficient constraints, and specific scenarios. We describe the differences between TFE-GNN and several other recent methods (Li et al., 2024; Huang et al., 2024b,a) in Appendix B.7. TFE-GNN also offers the following three advantages. We theoretically demonstrate that the approximation ability of TFE-GNN agrees with that of ChebNet under certain conditions, as outlined in Theorem 1, i... can learn arbitrary filters. Theorem 2 shows that IFE-GNN is a reasonable combination of two excellent polynomial-bas -based spectral GNNs, which motivates it to perform well. TFE extract the initial information, homophily and heterophily from graphs adaptively, which allows TFE-GNN to be applied to various homophily level cases. Experiments show that TFE-GNN achieves new state-of-the-art performance on datasets, and the homophily levels measured by the edge homophily ratio (Zhu al., 2020) for these datasets is 0.06, 0.21, 0.23, 0.30, 0.57, 0.74, 0.80, 0.81, and 0.93.
G
TFE
NN
(Hpp)*
=
Wn
TFE;
Base low-pass filter (Hap) HKpp
(11p))
W
First
ensemble
11
TFE3
2'
(Hlp)) Kkip
Third ensemble
fmip sof smax
(Hap)) =
a/b
TFE2
02
Base bigb-pass filter (H Nap
(Hhp))
Second ensemble
(Hnp) Knp
Figure 1: An illustration of TFE-GNN.
2 Preliminaries
Notations. Given a graph G = (E,, V) with node set V and edge É' CVVV Let |V] denote the size of the node set, i.e., the number of nodes. This paper paper uses x E R² to denote the graph signals, and xd) to denote the signal at node Y E {0, 1] 1jnXC denotes label matrix of G. and Y; the label vector of node where C is the number of classes. We denote X Raxdo
the initial feature matrix of G, denote the adjacency matrix of G as A E (0, 1 and denotes the degree matrix D, where D, Σラ= Aij. and do is the initial feature dimension. The (combinatorial) graph Laplacian is defined as L = - A, and its eigendecomposition is L = UAUT". The columns u, of U E Rexn orthonormal eigenvectors, namely the graph Fourier basis, and A diag((>),...,, An.)) is the diagonal matrix of eigenvalues. We also call these eigenvalues frequencies.
2.1 Metrics Homophily
The homophily metrics are used to define the homophily level of a graph by considering the different relationships between node labels/features and graph structures, including edge homophily (Zhu et al., 2020), node homophily (Pei et al., 2020), class homophily (Lim et al., 2021b), etc. In this paper, we use the edge homophily ratio ehr to measure the ratio of intra-class edges contained in graph as the homophily level:
(u, v) (u, FAY, Y
ehr
(I)
E

---

where u, are nodes, Y, is the label vector of u. and Y, is the label vector of v. The value of ehr close to I corresponds strong homophily, while the value of ehr close to 0 indicates strong heterophily.
2.2
Graph Filter
The graph filters are core components of spectral GNNs. We classify the graph filters into three categories based on the type of signal they filter: low-pass, high-pass and full-pass filters. The high-pass filter Hyp are more suitable for extracting high-frequency signals (Bo et al., 2023a; Yang et al., 2022). Empirically, the commonly used high-pass filters are the symmetric normalized Laplacian Laym = D LD = I-D-1/2 1/221/22D AD- and the random walk normalized Laplacian Lrw = D- L = I D The low-pass filter Hip are more suitable for extracting low-frequency signals and it is the affinity (transition) matrix of Hap ie., Hip corresponding to Hyp above are Layy [ Laym = D D-112 AD-1/2 and L I Low D-1. The full-pass filter Hp the identity matrix I and retains all graph initial signals.
=
2.3 Learnable Spectral GNNs
Spectral-based GNNs create the spectral graph convolutions (filters) in the domain of Laplacian spectrum and many methods use the polynomial spectral filters to achieve graph convolutions, such as ChebNet Defferrard et al. (2016), GPR-GNN Chien et al. (2021), BernNet He al. (2021a), ChebNetlI He et al. (2022), PCNet (Li et al., 2023), FavardGNN (Guo & Wei, 2023), etc. We begin by describing how the signal æ is filtered by h:
y = (L.)x A(UAUT) xx Ul(A)UTs = ([iA))"==Udd((((())))) M.,
where y denotes the filtering results of and h denotes the spectral filter, which is function on the eigenvalues of the graph Laplacian matrix L. Existing studies have replaced nonparametric filters with polynomial filters:
K-1 M(A) Σ A
(3)
where 02 E IRK is vector of polynomial coefficients. We bring Equation 3 into Equation 2:
y - Σ OAAUUT= Σ EOUNUUT - R=
K-1
k=0
We describe polynomial-bone based spectral GNNs in terms of ChebNet (Defferrard et al., 2016) and its enhanced version ChebNetll (He et al., 2022), including how they approximate graph convolution with polynomials and how constraining the coefficients. Other related methods are similar, such as EvenNet (Lei et al., 2022) and PCNet (Li et al., 2023). ChebNet uses Chebyshev polynomial to approximate the filtering operation, which is remarkable attempt:
K-1
== Σ (L)x,
(5)
where i 2L.gmm Xmaz I denotes the scaled graph Laplacian matrix, Amaz is the largest
eigenvalue of and is a vector of Chebyshev coefficients. Chebyshev polynomial (x) of order k can be recursively defined as (w 2x)x-1 Tk 22x) with Tn( 1 and Ti (zz) w.
ChebNet's structure is:
R-1
Σ Ta(L)xW.. R=U
(6)
where Wi the trainable weights, which contain the Chebyshev coefficients 4g. ChebNetll (He et al., 2022) proposes ChebBase to determine who is more competitive, Chebyshev basis or other bases.
K
0 (L)(())

---

where f is a Multi-Layer Perceptron (MLP). ChebNetll believes that ChebNet learns the illegal coefficient by analyzing a series of polynomial filters. Therefore, it then proposes ChebBase/k; which is an improvement on ChebNet and constrains the coefficients with 0s: k (Fquation 7). Finally, ChebNetll is proposed.
K K Σ (T(()) K+1 k=
2
(8)
where 7j is the learnable coefficient, and it links Equation 7 and Equation 8 with On K+1 Σド= (xy), ie., the learnable coefficients constrain.
3 Methodology
We propose a spectral GNN with triple filter ensemble (TFE-GNN) to solve three progressive problems in polynomial-based learnable spectral GNNs. In this section, describe the methodology of TFE-GNN in detail and theoretically prove its approximation capabilities, including motivations, TFE-Conv, TFE-GNN, time complexity and scalability and theoretical analysis.
3.1 Motivations
Polynomial-basee learnable spectral GNNs (He et al., 2022; Li al., 2023; Guo Wei, 2023) performs well on homophilic and heterophilic graphs, because their graph convolutions are flex- ible and variable. However, carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints leaded to overfitting, which diminishes GNNs' generalization.
Inspired by the following properties of ensemble learning (Schapire, 1990; Hansen & Salamon, 1990; Zhou, 2012): the strong classifier determined by the base classifiers can be more accurate than any of them if the base classifiers are accurate and diverse; and this strong classifier retains the characteristics of the base classifier to some extent. First, we combine a set weak base low-pass filter to determine strong low-pass filter that can extract homophily. Then, we use the same method to extract heterophily. Finally, TFE-Conv generated by combining the above two strong filters with two learnable coefficients, which retains the characteristics of both two strong filters, i.e. it can extract homophily and heterophily from graphs adaptively. TFE-Conv and TFE-GNN are shown in Figure 1.
3.2 TFE-CoRY
We design triple filter ensemble (TFE) mechanism for combining low-pass and high-pass filters yield a graph convolution (TFE-Conv), which can match various graph structures adaptively without carefully crafted graph learning methods, i.e., can extract homophily and heterophily adaptively from graphs with different homophily level. The first ensemble of TFE is formalized as follows:
(9) where TFE, denotes the result of combining a set of base low-pass filters, which is a graph convolution that can extract homophily while utilizing the initial signals, EM, denotes the ensemble method of the first ensemble, E (la, hı, kx) denotes the combination of elements ha. hy, ..., h with EMI, w are the learnable coefficients, and Ki, is the order of the first ensemble. The second ensemble is similar to the first ensemble and is formalized as follows:
TFE: EM1 (WoT, wi Hip- W2 (2
WK,, (H1p)
(10) where FFE, denotes the result of the second ensemble, which can extract heterophily while utilizing the initial signals, EM, denotes the ensemble method of the second ensemble, w' are the learnable coefficients, and Kip is the order of the second ensemble. The third ensemble combines TF E, and TF E2 with two learnable coefficients 02 and Uz:
TF = EM2 I I w/ Hhp: Wh (Hnp)?
w''k. (Hhp) Knp)
FF FM& 02TF22)
(11)
where TFE3 denotes the result of the third ensemble, i.e., TFE-Cony, and EM, denotes the ensemble method of the third ensemble. The learnable coefficients 02 and i2 used to combine the two strong graph convolutions TF E; and E2 guarantee TFE3's adaptivity.
5

---

3.3 TFE-GNN
TFE- CEnv matches various graph structures adaptively, which facilitates the filtering of the graph signals (features) X. The filtered signal is fed into an MLP, whose output is then passed through a softmax layer for obtaining the prediction. This is the forward propagation of TFE-GNN and is coupled with the cross-entropy loss and the backpropagation mechanism to form the complete TFE-GNN. The formalization of X being filtered by TFE-Conv is:
Z = TFE3 X = -MM (,TFE X = EM,(0,IFE: X, 2,FFFFF2 X), (12)
where TFE and TF'2 are the graph convolutions obtained from the first and second ensembles, respectively, which are similar to TFEE in that they filter the graphical signal as follows: Mn TFE- X and Zhp = TFEE X. Thus, Equation 12 can be rewritten in the following form:
Z EM Zip, 02Znpt.
(13)
We decouple transformation/precciitio and feature propagation (Rong al., 2020; He et al., 2022) for TFE-GNN and formalize TFE-GNN used for node classification:
Ż fmip(Z)
7" = soʃ soflmar 2.)
(14)
L = Σ
log((
rEVL
where L denotes the cross-entropy loss, Yt denotes the training set with labels, and T denotes the vector transpose.
3.4 Time Complexity and Scalability
Similar to some spectral GNNs (He al., 2022; Li et al., 2023), the triple filter ensemble mechanism does not influence the time complexity magnitude of the TFE-GNN, i.e., the time complexity of TFE-GNN is linear Kip when EM,,, EM2,, and E.M take summation. Specifically, the time complexity of message propagation is O((Kip + Kip) E|C), the time complexity of the combination of Hot with respectively w and w' (Equations 9 and 10) is O((Kip Knn)nC),, and the time complexity of the coefficient calculation is not greater than OKKpp + Khp). We report the training time overhead of the different spectral GNNs in Appendix B.
We scale TFE-GNN by exchanging the order of message propagation and feature dimensionality reduction: 7 = EM Zip, 32Znp} = 0 E F, (m)p(X),
We use a sparse form of the adjacency matrix of large graphs, which greatly reduces the space required for feature spaces.
TFE-GNN. Therefore, TFE-GNN scales well to large graphs and
3.5 Theoretical Analysis
ChebNet has been shown to learn arbitrary filters in theory (Balcilar et al., 2021; He al., 2022). We find a connection between TFE-GNN and ChebNet and then analyze the conditions under which they transform into each other. We prove that TFE-GNN is equal to ChebNet under certain conditions, i.e., they have the same approximation ability, so TFE-GNN can also learn arbitrary filters.
Theorem 1. TFE-GNN and ChebNet can be transformed inlo each other under the following conditions, (1) learning the proper coefficients w, w', O and ChebNet' coefficient 0, (2) the ensemble methods EM,, EM₂ and E.3 take summation, the base high-pass filter Hip takes the symmetric normalized Laplacian Laym and the base low-pass filter Hip takes the affinity (transition) matrix of Leym- and (3) Kip Kip- K- 1. Thus, TFE-GN can also learn arbitrary filters.
Theorem (proof Appendix A. shows that TFE-GNN matches various graph structures adaptively while learning arbitrary filters under certain conditions. The proof of Theorem shows that ChebNet can be unfolded into a combination of high-pass or low www-pass filters, or combination of high-pas pass and www-pass.T TFE-GNN, on the other hand, is a combination of two different unfoldings of ChebNet. Condition (1) of Theorem I shown in its proof, the EM, and E.2 in condition (2) are taken to summation to correspond to the expansion of the Chebyshev polynomials, while EM, is an ensemble method capable of preserving the properties of the model, such as summation and concatenation, and condition (3) motivates the agreement between the orders of TFE-GNN and ChebNet.

---

Polynomial-bise based spectral graph GNNs can all be unfolded into combinations of filters, which is similar the idea of filter ensemble in TFE-GNN, TFE-GNN a combination of two polynomial- based GNNs under certain conditions by displacing its base filters and setting Kip and Kep-
Theorem 2. TFE-GNN can be rewritten in the following form, with certain conditions to be satisfied, which is a combination of two polynomial-bases learnable spectral GNNs: Z = so Immax[Smmpp (1: Ex=o 10PP gf
(F x ① 02 Ex ( X)), where Pk denote polyno- mials used for approximation, 0 are the learnable coefficients, Hot denote graph filters, and + denotes EM3. Conditions are (1) learning the proper coefficients w, w', 9. 02, and 02, (2) the ensem- ble methods EM,, EM2 take summation and E.M takes ensemble method cupable of preserving the properties of the model, such as summation and concatenation, the base high-pass filter Hap takes H? of. and the base low-pass filter Hip takes H, g)> and (3) Kip K' and Khp K" Thus, TFE-GNN can match various graph structures adaptively.
The proof of Theorem 2 is reported in Appendix A. Theorems and 2 show that polynomial -bssed learnable spectral GNNs are able to leam different filters, and thus perform well on both homophilic and heterophilic graphs. In contrast, txed-filter GNNs can only filter graph signals based on their filter forms. TFE-GNN combines different filters directly, preserving the ability of the different filters while reducing overfitting problem. Theorem 2 also shows that TFE-GNN will perform well on real-world datasets: it is reasonable combination of two excellent polynomial- based spectral GNNs. We discuss the limitations of TFE-GNN in Appendix A.2.
4 Experiments
In this section, we conduct experiments to evaluate the proposed TFE-GNN against the state-of-the- art (SOTA) GNNs real-world datasets and conduct ablation study, generalization, visualization (Appendix B.3), hyper-parameters (Appendix B.5), and time efficiency (Appendix B.6) analysis to verify TFE-GNN's excellent.
Datasets and experimental setup. We evaluate TFE-GNN on several real-world datasets for supervised node classification and chose 11 graphs with various levels of homophily, including 4 citation graphs (Kipf & Welling, 2016) Cora, Citeseer, Pubmed, and Cora-Full (Bojchevski & Günnemann, 2017). 2 Co-authorship graphs (Shchur et al., 2018) Coauthor CS and Coauthor Physics, 2 Wikipedia graphs (Rozemberezki et al., 2021) Chameleon and Squirrel, and WebKB graphs (Pei et al.. 2020) Texas, Cornell, and Wisconsin. The dataset statistics summarized in Table 1. In addition, we choose four additional datasets, i.c. roman-empire (Platonov et al., 2023), amazon-rating (Platonov et al., 2023), 1b100-Penn94 (Lim et al., 2021a) and genius (Lim et al., 2021a), further validate the classification performance, generalization and scalability of TFEGNN. Detailed dataset statistics and experimental results reported in the Appendix B.1. All experiments are carried out on the machine with Linux system, two NVIDIA Tesla V100 and twelve Intel(R) Xcon(R) Gold 5220 CPU @ 2..00GH
Table I: Dataset statistics.
DATASHTS
CUBA
CrrESIUR
PURMIn
CURA-PULL.
Puysics
CHAMMILION
SQUIREE
WISCONSIN
TEXAS
CORNELA
Nonns
2708
3527
19717
19793
18333
$4493
2227
185
185
BoGas
10556
9104
88648
62421
81894
247962
36051
216953
466
309
295
FEATURES
1433
3703
500
8710
6805
8415
1703
2059
951
1703
1705
CLASSES
7
3
70
15
5
S
ehr 0.81
0.74
0.80
0.57
0.81
0.93
0.23
0.22
0.21
0.06
0.30
TFE-GNN Settings. There are many options available for Hip, Hnp. EM,, EM2, and EM3 in TFE-GNN, and we choose common and frequently used options for them to make a broad and fair comparison between TFE-GNN and other SOTA GNNs. Hip and Hip take the symmetric normalized Laplacian L Lagm and Laym respectively. We add self-loops to the graph in practice, so Leym 1- 1-2 We use generalized normalization about Laym to alleviate the overcorrelation issue in spectral GNNs (Yang et al., 2022; et al., 2023) and keep T.gym. unchanged, namely Laynm I - Đ-n Åñ-r, EM1 and EM, take summation, and EM3 takes summation concatenation. Thus, TFE-Cony
1/2 AD AD21 and L D 1/2 AD-1/2 where À = A 1, and D. = ΣØ= Agg.
7

---

Table 2: Mean accuracy of different models datasets for full full-supervised node classification.
DATASETS
CORA
CITESEER
PUBMED
CS
PHYSICS
chr
0.81
0.74
0.80
0.81
0.93
MLP
0.98
76.52+0.89
86. 0.5
94.76+0..51
96.52+0.66
GCNS
87.18+1.12
79.85+0.78
86.79-t.31
93.11+0.19
96.66+0.77 74
ARMA
87.1310.80
80.0410.55
86.93-1.24
92.1410.35
95.1110.19
APPNP
88.1610..74
80.4710.73
88.1310.33
92.6111.288
95.8110.11
CHEBNET
87.3211.92
79.33=0.57
87.8210.24
91.63-1..39
94.2110.26
GPR-GNN
88.54:0.67
80.13+0.84
88.46=0.31
95.67=0.16
96.80+0.08
BERNNET
88.51#0.92
80.08=0.75
88.51=0.39
95.81+0.13
96.81+0.07
CHEBNETII
88.71=0.93
80.53=0.79
88.93=0.29
96.03=0.11
97.23+0.07
SPECFORMER
88.5711.01
81.49
89.1310.35
95.92-10.19
97.44
EVENNET
8..25
78.65
89.52+10.11
94.6611.23
9
FAVARD
1..35
81.89-00.33
90.90-10.27
95.77=00.15
97.58-10.08
PCNET
90.02=#0.62
81.7620.78
91.30=0.38
96.33+0.15
97.62+0.08
HALF-HOP
88.73+1.22
80.33+0.66
89.86-+0.36
95.13+0.21
95.75+0..1
GCNTI
88.46::082
79.97=0.65
89.9440.31
96.58+0.07
97.27+0.12
TWIRLS
88.57:0.91
80.07=0.94
88.8720.43
95.43+0.04
97.1740.07
PDE-GCN
88.62+1.03
79.98=0.97
89.92-0.38
95.35=0.19
96.8910.08
EGNN
87.4
80.51
88.7420.46
95.22:10.20
96.6110.08
TFE- TFE-GNN eon
90.11E1.27
82.3910.96
90.9410.29
93.55H1.56
97.62.0.233
90.7311.11
8.28
91.6610.51
96.9610.17
98.8510.13
TFE-GNNSTFE,
89.49+1.30
61.00=1.19
90.80+0.37
96.96+0.17
97.24+0.10
90.15+1.75
8..83+1.24
91.66+0.51
96.57+0.16
97.19+0.19
TFE-GNNrnctavve
89.57
81.9211.14
90.96.10.9
95.7010..3
97.7310.16
filters the graph signal X in the following way:
Kip
Kky
01 7ip + 02 = -0,TFE + (2TFF2) X= = (01 Σ wi (Hip)* ) + + Σ (Hpp))) X
Z
1=0
j=f
Kip
Kny
01 1111011022np - (01TEE - = (8)
wi (Hip) |1
> w) X,
1=0
5=
(15)
where || denotes concatenation and has lower arithmetic priority than addition, subtraction, multipli- cation and division.
4.1 Supervised Node Classification
Setting and baselines. We compare TFE-GNN to series of SOTA models for full-supervised node classification on datasets with random splits, including 11 polynomial approximation filter methods GCNs (Kipf & Welling, 2016), ARMA (Bianchi et al., 202 APPNP (Klicpera et al., 2018), ChebNet (Defferrard et al., 2016), GPR-GNN (Chien et al., 2021). BernNet (He et al., 2021a), ChebNetll (He al., 2022), SPECFORMER (Bo et al., 2023a). EvenNet (Lei et al., 2022), PavardGNWWOUBBssiSGNNN (Guo & Wei, 2023), and PCNet (Li et al., 2023). We also add 5 competi- tive SOTA models Half-Hop (Azabou al., 2023), GCNII (Chen et al., 2020), TWIRLS (Yang et al., 2021), PDE-GCN (Eliasof et al., 2021), )), and EGNN (Zhou et al., 2021). We randomly split each class of nodes into 60%, 20%, and 20% training, validation, and testing sets for full-supervised node classification and all models share the same ten random splits for a fair comparison.
For TFE-GNN, set the hidden units to be 64 or 512 (Squirrel, Chancleon, roman-empire, and amazon-rating). the number of early stoppings is 200 and the number of epochs is 1000 for all datasets. We employ the ReLu as activation function for fmip- We use the officially released code for GCNII, GPR-GNN BernNet, etc and use the Deep Graph Library implementations for other models, such GCNs, APPNP, ChebNet, etc. More experimental details of hyper- parameters and code URLs are listed Appendix C.
Results. Tables 2 and 3 report the results of the different models on all datasets and gives mean classification accuracy and standard deviation over ten random splits, where the bolded numbers indicate the best results, "Favard" denotes FavadGNNVOOBBaisNNNN TFE-GNN, sum denote that

---

Table 3: Mean accuracy of different models datasets for full-supervised node classification.
DATASETS
CORA-FULL
CHAMELEON
SQUIRREL
WISCONSIN
TEXAS
CORNELL
chr
0.57
0.23
0.22
0.21
0.06
MLP
52.45+0.64
46.59
31.01=1.18
86.55=2.36
86.81=2.24
84.15=3.05
GCNS
66.0410.38
60.81+2.95
4..87=0.88
74.19=3.15
76.97=3.97
65.7814.16
ARMA
63.53110.66
60.21:11.00
36.27=0.62
8..25=1.63
83.97=3.77
85.62::2.13
APPNP
59.8510.54
$2.15.1.779
35.71
91.08
90.6411.70
91.52.1..11
CHEBNET
58.65+0074
19.51=1.25
40.81=0.42
84.19-2.58
86.28=2.62
83.91+2.17
GPR-GNN
71.86=00.29
67.49-1.38
50.43=1.89
91.71=1.62
92.91=1.32
1.957=1.96
BERNNET
72.01=0.26
68.53-1166
51.39=0.92
92.45=1.22
92.62-1.37
92.13=1.64
CHEBNETII
72.1110..24
71.3711.01
57.72-0.59
93.72-1.27
93.28-1.47
92.3011.44
SPECFORMER
71.84+0.26
74.72+0.11
64.64-0.81
92.9
92.77=2.37
91.86+2.69
EVENNET
70.041:0.47
6..57:11.52
50.36-0.93
93.55
93.77=1.73
12.1311.71
FAVARD
72.39=0.34
74.26+0.74
63.62=0.76
93.33=1.95
91.87=3.11
92.06=2.96
PCNET
72.35+00.26
13.55+1.26
63.53-0.26
94.26- 1.85
92.78-1.80
1.983+1191
HALF-HOP HOP
72.35+0.31
62,98+3.35
4..25=+1552
87.59 1.77
85.95=6.42
74.60-6.06
GCNII
66,70=0.85
63.44+0.85
41.96=1.02
85.66=1.95
80.46=5.91
84.26=2.15
TWIRLS
68.88+0.22
50.21+2.97
39.63=1.02
91.53-2.81
91.31=3.36
29.83+2.29
PDE-GCN
71.37-0035
66.0111.56
1..73=1.06
92.85=1.67
93.24-2.03
89.7341.335
EGNN
71.51+0.277
11.55
35.81=0.91
83.76=1.64
1.53
1..0
TFE- OFE-GNN.co
74.12=0.40
17.03+1.47
7..47=1.55
96.00=1.73
93.11=4.26
94.26=2.77
TFE-GNN.uum
73.6010.21
77.1611.41
7..27-1.32
97.38-1.42
94.67-2.66
93.1112..96
TFE- GNN,
64.09+0.41
76.63+2.20
57.06=1.00
92.97-1.38
93.44- 1.80
93. 93.11
"FE-GNN.TFE
73.60=0.21
61.05+2.45
41.91=0.69
81.62=8.40
68.36=7.10
03.11
TFE-GNNrusts sum
72.8110.51
69.26.14.84
56.93
1.34
12.69
90.9813.38
EM3 of THE-GNN is summation, and THE-GNN con denote that EM3 of THE-GNN is concatenation. The experimental results are taken from ChebNetll, Half-Hop, SPECFORMER, EvenNet, and WaarGGNN/OpdBasssSNNN. when they report relevant results, and the remaining results reproduced by us. Tables 2 and 3 illustrate that ChebNet starts to outperform GCNs when there is more training data available, which suggests the validity the Chebyshev approximation. TFE-GNN achieves new state-of-ther totth--r-the-rrt results on all datasets. TFE-GNN outperforms some SOTA models with similar results on Cora and Citeseer, including GPR-GNN (Chien al., 2021), BernNet (He et al., 2021a), ChebNetll (He et al., 2022) and Half-Hop (Azabou et al., 2023). Notably. TFE-GNN outperforms ChebNetll on Chameleon and Squirrel by 5.79% and 14.85% respectively, which amounts to performance improvements of 8% and 26%. TFE-GNN achieves exciting results on Physics (strong homophily) and Wisconsin (strong heterophily) for full-supervised node classification, already close to 100% accuracy. We put the relevant settings and results of TFE-GNN for semi-supervised node classification in Appendix B.2, due to space limitations.
4.2 Ablation Study
We conduct experiments to investigate the (joint) contributions of TFE-GNN' components. The last three rows of Tables 2 and 3 report the results of the ablation experiments. The symbol "TFE-
GNN TFE, means D1 0 Kip O, "TFE-GNN,TFFE F means 12 = 0 or Kap 0. and "TFE-GNN rw- indicates that THE-GNN uses the random walk normalized Laplacian Tyne D. T- D-1 A as the high-pass graph filter Hng. Partial ablation experiments yielded the same results due to the choice of hyperparameters Kip and Knp- We observe that TFE-GNNY FFE, performs worse under strong homophily, while (FF-GNN,TFE,, performs worse under strong heterophily. TFE-GNN with all graph filters achieves the best results, which suggests that it can match various graph structure.
4.3 Generalization Analysis
We verify the generalization of TFE--NNY by analyzing its cross-entropy loss in the training and validation sets on Cora, and smaller gap between the two losses indicates better generalization of the model (Feng et al., 2020). Figure 2 shows the significant gap between the training and validation losses for ChebNet, ChebNetll, and PCNet, which indicates a possible overfitting and diminishes their generalization. The validation loss of TFE-GNN is much closer to its training loss and the early stopping mechanism allows the model to carry less stable losses. More generalization analyses are reported in Appendix B.4.

---

OCN
Bo
Training
Trining Loss
Training
Loss
Eparhe
Training Loss
Taining Loss
Training Loss
Validation Loss
idation Loss
Validation Loss
10:00
180
500
280
Figure 2: Generalization on Cora.
4.4 Loss Oscillation Analysis
We try to explain the reason for the oscillation in TFE-GNN's losses in Figure 2. The learning rate controls the step size which in turn affects the loss optimization. The large learning rate (= 0.1) is responsible for the oscillations of the validation loss in Figure 2. Figure 3 shows that the loss is stable when the learning rate is 0.001. The early stopping mechanism allows TFE-GNN to carry less stable losses and losses do not fall into unacceptable local minimum.
TRE-CNN
TRE-GNN
Validation Loss
Validation Loss]
Validation Loss2
Validation Loss2
3
Epecte
(a) learning rate 0.1
(b) learning rate 0.001
Figure 3: Verification loss at different learning rates, keeping the rest of the parameters constant. There two validation loss curves each subfigure, and each loss is the average of five experiments.
5 Conclusions
We propose TFE-GNN with triple filter ensembles (TFE) to solve three progressive problems. TFE- GNN extracts homophily and heterophily from graphs with different homophily levels adaptively while utilizing the initial features, which motivates it to match various graph structure. We theoreti- cally prove that TFE-GNN can learn arbitrary filters and is combination two polynomial-based spectral GNNs. Experiments show that TFE-GNN achieves new state-of-the-art performance on various real-world datasets. the future, we will dig deeper into ensemble methods triple filter ensembles and expect further improve the performance of TFE-GNN
10

---

Acknowledgments and Disclosure of Funding
We would like to thank the School of Computer Science of Guangzhou University and the School of Telecommunication of Tongji University for their help, including the experimental environment, office location, and writing guidance. This work was supported in part by the Education Bureau of Guangzhou Municipality under Grant 2024312243 and in part by the National Natural Science Foundation of China under Grant 62202507.
References
Azabou, M., Ganesh, V., Thakoor, S., Lin, C.-H., Sathidevi, I.., Liu, R., Valko, M., Veličković, P., and Dyer, E. L.. Half-hop: graph upsampling approach for slowing down message passing. International Conference Machine Learning, pp. 1341-1360, 2023.
Balcilar, M., Guillaume, R., Héroux, P., Gaüzère, B., Adam, S., and Honeine, P. Analyzing the expressive power of graph neural networks in spectral perspective. Proceedings of the International Conference on Learning Representations, 2021.
Bianchi, F. M., Grattarola, D., Livi, L.,, and Alippi, C. Graph neural networks with convolutional arma filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):349--3577 2021a.
Bianchi, F. M., Grattarola, D., Livi, L., and Alippi, C. Graph neural networks with convolutional arma filters. IEEE transactions on pattern analysis and machine intelligence, 44(7):3496-3577, 2021b.
Bo, D., Wang, X., Shi, C., and Shen, H. Beyond low-frequency information in graph convolutional networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5):3950-33) 2021.
Bo, D., Shi, C., Wang, L., and Liao, R. Specformer: Spectral graph neural networks meet transformers. In International Conference on Learning Representations, 2023a.
Bo, D., Wang, X., Liu, Y., Fang, Y., Li, Y., and Shi, C. A survey on spectral graph neural networks. arXiv preprint 232x:23022.5631,,, 2023b.
Bojchevski, A. and Günnemann, S. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. International Conference Learning Representations, 2017.
Chen, M., Wei, Z., Huang, Z., Ding, B., and Li, Y. Simple and deep graph convolutional networks. International Conference on Machine Learning, pp. 1725-1735, 2020.
Cheng, D., Yang, F., Xiang. S., and Liu, J. Financial time series forecasting with multi-modality graph neural network. Puttern Recognition, 121:108218. 2022.
Chien, E., Peng. J., Li, P., and Milenkovic, O. Adaptive universal generalized pagerank graph neural network. arXiv preprint arXiv:2006.07958 077988 2020.
Chien, E., Peng, J., Li, P., and Milenkovie, O. Adaptive universal generalized pagerank graph neural network. International Conference on Learning Representations, 2021.
Defferrard, M., Bresson, X., and Vandergheynst, Convolutional neural networks graphs with fast localized spectral filtering. Advances in Neural Information Processing Systems, 29, 2016.
Duan, R., Yan, C., Wang. J., and Jiang, Path-aware multi-hop graph towards improving graph learning. Neurocomputing, 494:13-22, 2022.
Duan, R., Yan, C., Wang, J., and Jiang. C. Class-hompphilic-bor based data augmentation for improving graph neural networks. Knowledge Based Systems, 269: 10518, 2023.
Duan, R., Yan, C., Wang, J., and Jiang, C. Graph ensemble neural network. Information Fusion, 1 110: 102461, 2024. ISSN 1566-2535.

---

Eliasof. M., Haber, E., and Treister, E. Pde-gen: Novel architectures for graph neural networks motivated by partial differential equations. Advances Neural Information Processing Systems, 34:3836-3849 2021.
Feng, W., Zhang, J., Dong, Y., Han, Y., Luan, H., Xu, Q., Yang, Q., Kharlamov, E., and Tang, J. Graph random neural network for semi supervise learning on graphs. NeurIPS, 2020.
Guang, M., Yan, C., Xu, Y., Wang, J., and Jiang, C. A multichannel convolutional decoding network for graph classification. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-11, 2023. doi: 10. 200.9/TNNN..222.300000
Guang, M., Yan, C., Xu, Y., Wang, J., and Jiang, C. Graph convolutional networks with adaptive neighborhood awareness. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.
1-13, 2024. doi: 10. 10.100/PPAMI.222.3931333
Guo, J., Huang, K., Yi, X., and Zhang, R. Graph neural networks with diverse spectral filtering. pp- 306-316, 2023.
Guo, Y. T. and Wei, Z. Graph neural networks with learnable and optimal polynomial bases. International Conference on Machine Learning, 2023.
Hansen, L. K. and Salamon, P. Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence, 12(10))993-1000 1990.
He, M.. Wei, Z., Xu, H., al. Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. Advances in Neural Information Processing Systems, 14259-1425,, 2021a.
He, M.. Wei, Z., Xu, H., et al. Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. Advances in Neural Information Processing Systems, 14:14239-14251, 2021b.
He, M., Wei, Z., and Wen, J.-R. Convolutional neural networks on graphs with chebyshev ap- proximation, revisited. Advances in Neural Information Processing Systems, 35:7264-7276. 2022.
Huang, C., Wang, Y., Jiang, Y., Li, M., Huang, X., Wang, S., Pan, S., and Zhou, C. Flow2gnn: Flexible wwo-way flow message passing for enhancing gnns beyond homophily. IEEE Transactions on Cybernetics, 2024a.
Huang, K., Wang, Y. G.. Li, M., et al. How universal polynomial bases enhance spectral graph neural networks: Heterophily, over-s weo-smoothing, and over-squashing ICML, 2024b.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2016.
Klicpera, J., Bojchevski, A., and Günnemann, Predict then propagate: Graph neural networks meet personalized pagerank. International Conference Muchine Learning, 2018.
Lei, R., Wang, Z., Li, Y., Ding, B., and Wei, Z. Evennet: Ignoring odd-hop neighbors improves robustness of graph neural networks. In Advances in Neural Information Processing Systems, volume 35, pp. 4694-4706. 2022.
Levie, R., Monti, F., Bresson, X., and Bronstein, M. M. Cayleynets: Graph convolutional neural networks with complex rational spectral filters. JEEE Transactions on Signal Processing, 67(1): 97-109, 2018.
Li, B., Pan, E., and Kang, Z. Pe-conv: Unifying homophily and heterophily with two-fold filtering. arXiv preprint 2xXiv22312.14433 2023.
Li, J., Zheng, R., Feng, H., Li, M.. and Zhuang, X. Permutation equivariant graph framelets for heterophilous graph learning. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., and Lim, S. N. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods, Advances Neural Information Processing Systems, 34:200 34:20887-20902 2021a.
12

---

Lim, D., Li, X., Hohne, F., and Lim, S.-N. New benchmarks for learning non-homophilous graphs. arXiv preprint ar 20X4:22004.0144,, 2021b.
Luan, S., Zhao, M., Hua, C., Chang, X.-W., and Precup, D. Complete the missing half: Augmenting aggregation filtering with diversification for graph convolutional neural networks. arXiv preprint arXiv:2212.10822. 2022.
Narang, S. K., Gadde, A., and Ortega, A. Signal processing techniques for interpolation in graph structured data. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 5445-5449, 2013.
Pei, H., Wei, B., Chang, K. C.-C., Lei, Y., and Yang, B. Geom-gen Geometric graph convolutional networks. International Conference on Machine Learning, 2020.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at the evaluation of gnns under heterophily: Are we really making progress? ICLR, 2023.
Rong. Y., Huang, W., Xu, T., and Huang, J. Dropedge: Towards deep graph convolutional networks node classification. International Conference Learning Representations, 2020.
Rozemberezki, B., Allen, C., and Sarkar, Multi-scale attributed node embedding. Journal of Complex Networks, 9()xxnnab014, 2021.
Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M.. and Tarantola, S. Variance based sensitivity analysis of model output, design and estimator for the total sensitivity index. Computer Physics Communications, 181(2):259-275 27000 2010.
Scarselli, F., Gori, M., Tsoi, A. C.,, Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks and Learning Systems, 20(1 20(1):61-88) 2008.
Schapire, R. E. The strength of weak learnability. Machine learning, 5(2):197-227, 1990.
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. arXiv preprint urXiv:1811.05800 2018.
Shirzad, H., Velingker, A., Venkatachalam, B., Sutherland, D. J., and Sinop, A. K. Exphormer: Sparse transformers for graphs. International Conference on Machine Learning, 2023.
Tang, J., Sun, J., Wang, C., and Yang, Z. Social influence analysis in large-scale networks. Proceed- ings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 807-816, 2009.
Tao, Q., Wang, Z., Yu, W., Li, Y., and Wei, Z. Lon-gnn: Spectral gnns with learnable orthonormal basis. ArXiv, 2023.
Traud, A. L., Mucha, P. J., and Porter, M. A. Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, 391(66))416--4188 2012.
Wang, M. Y. Deep graph library: Towards efficient and scalable deep learning on graphs. International Conference on Learning Representations Workshop representation learning graphs and manifolds, 2019.
Weston, J., Ratle, F., Mobahi, H., and Collobert, R. Deep learning via semi-supervised embedding. pp. 639-655, Springer, 2012.
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. Simplifying graph convolutional networks. International Conference Machine Learning, pp. 6861-6871, 2019.
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? International Conference on Learning Representations, 2019.
Xu, Y., Wang, J., Guang, M., Yan, C., and Jiang, C. Multistructure graph classification method with attention- pooling. IEFE Transactions on Computational Social Systems, 10(2):602-613, 2022.
13

---

Xu, Y., Wang. J., Guang, M., Yan, C., and Jiang, C. Graph contrastive learning with min min-max mutual information. Information Sciences, pp. 120378. 2024.
Yang, M., Shen, Y., Li, R., Qi. H., Zhang, Q., and Yin, B. new perspective on the effects of spectrum in graph neural networks. In International Conference on Machine Learning, pp. 25261-25279 PMLR, 2022.
Yang. M., Feng, W., Shen, Y., and Hooi, B. Towards better graph representation learning with parameterized decomposition & filtering. International Conference Machine Learning, 202:
39234-39251, 23-29 Jul 2023.
Yang. Y., Liu, T., Wang. Y., Zhou, J., Gan, Q. Wei, Z., Zhang, Z., Huang, Z., and Wipf. D. Graph neural networks inspired by classical iterative algorithms. International Conference on Learning Representations, 139:1 1773-11783, 18 24 Jul 2021.
Zeng, 7., Peng, Q. Mou, X., Wang, Y., and Li, R. Graph neural networks with high-order polynomial spectral filters. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Zhou, K., Huang, X., Zha, D., Chen, R., Li, L., Choi, S.-H., and Hu, X. Dirichlet energy constrained learning for deep graph neural networks. Advances in Neural Information Processing Systems, 34: 21834-21846, 2021.
Zhou, Z.-H. Ensemble methods: foundations and algorithms. 2012.
Zhu, H. and Koniusz, P. Simple spectral graph convolution. International Conference on Learning Representations, 2020.
Zhu, J., Yan, Y., Zhao, L.., Heimann, M., Akoglu, I.., and Koutra, Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 2020.
14

---

A Proofs and Limitations
A.1 Proofs of Theorems 1 and 2
Theorem 1. TFE-GNN and ChebNet can be transformed into each other under the following conditions, (1) learning the proper coefficients w, w', 0 and ChebNet' coefficient 0, (2) the ensemble methods EM, EM2 and EM3 take summation, the base high-pass filter Hig takes the symmetric normalized Laplacian Layy and the base low-pass filter Hip takes the affinity (transition) matrix of Laym and (3) Kip-
Kip K K-1 1. Thus, TFE-GNN can also learn arbitraar,fitters.
proof: We first prove that ChebNet can be transformed into TFE-GNN under conditions (1), (2), and (3). Instead of ChebNet, we use its excellent version Chebyshev basis (Equation 7). We change Equation 7 to the following form that does not affect approximation ability:
y =
softmax
Pg (b))x))
(16)
Equation 16 can be rewritten in the form of equal approximation ability using the learnable parameters (constants) 91 and 122
K
K TxTx(()) + 02 Σ (T)x(()) = softmax (0, + 02))). k=
γ = soʃ faattmaXSSmtt (0)
0g Σ k=
(17)
We parse and reconstruct the key parts of Equation 17 by unfolding and integrating technologies, i.e., analyzing Σ (LTx(L)X = Y. We get the follow equation by unfolding the summation and Chebyshev polynomial (L))
Y =OoTo(L) + (L) + +K-
(18)
(0p1 + 0, 00 1) + 0k (2L 2(L) Tk ((i))),
where (T) = (i)) i., T2(T) 22.2 I, and on. Equation 18 can be viewed
(K 1)-th order polynomial of Ý with respect L. We then integrate the coefficients of different
orders Equation 18:
Y = ((0g 02 + 04 0 +
+...
0x 2
(19)
1 + 8 L +
0 x 1
where the first coefficient of the polynomial 0g = (0p 02 + 02 0s +...) in Equation 19, the coefficient of the highest order 0'x-1 0K 0k-12 and
We can calculate the relationship between coefficients 8% and (0 < k < K 1) based on the generalized form of the Chebyshev polynomials.
Ln/2]
Ln/2|
Tx(x) (xx = Σ 1(-1)* Σ
K-
(20)
k
where [n/2] denotes downward rounding, and (2) denotes combinatorial numbers. We get the connection between Y; and Znp according to the given conditions (1), (2), and (3):
Y (w61 + w/ (Hhp)) + ..... W'knp (Hhp) Knp) ) = TFFE2 X = Znp
(21)
where the conversion of the coefficients 0' to w/ are realized through equation Ż = 2L Amoz I. We continue to unfold Equation 19:
2L
2L - 1) 13 x
Y = (0/1 + +0'( Amaz
1)
+ 0xx-1( Amaz
2
Ammax
2 (D 1/2 AD AD- 1)) + +..
(067 + 0
Amaa
Xmar
(22)
8x-11
Xmmar
2
(D AD k))
Amaz
Amax
= 10'1 + 0D(
+
0% 1D
(D- AD AD-= -
15

---

where 6 = 86 + ~xma== 04 + +... + (2>dmax) 1 1- and 001 = (2 )k-1 6
)
K-1- We get
Amos
the connection between Y and Zip according to the given conditions (1), (2), and (3):
Y (6) 101++6110 (Hip) +... 0 1 (Hp))
(23)
(wol + wi (Hip)) + +... + wKip (Hhp) Khp) ).x, = TFEF X Zip.
where w 0". We get a new conclusion about Y:
y = softmaz (0) Zip + 32Zzp) = so (f (en)((Z)) = softmar( Z"
(24)
ChebNet and its excellent version Chebyshev basis are essentially the same (Defferrard et al., 2016),
except that Chebyshev basis implements the decoupling (He et al., 2022). Therefore, we proved that ChebNet can be transformed into TFE-GNN under conditions (1), (2), and (3), and vice versa.
Theorem 2. TFE-GNN can be rewritten in the following form, with certain conditions to be satisfied, which is combination of two polynomial-basee based learnable spectral GNNs: Z" woftmax(ffm Ek=o (
① Ex ( X)), where Ps denote polyno- mials used for approximation, 8 are the learnable coefficients, Hof denote graph filters, and b denotes EM3. Conditions are (I) learning the proper coefficients w', 0. 0¹, and 02, (2) the ensem- ble methods EM,, EM2 take summation and EM3 takes ensemble method capable of preserving the properties of the model, such summation and concatenation, the base high-pass filter Hup takes H2, and the base low-pass filter Hip takes H and (3) Kip K' and Khp K" Thus, TFE-GNN can match various graph structures adaptively.
proof: The proof of Theorem 2 is similar to that of Theorem 1. First, we unfold the polynomials Pk and P in Z" to get a combination of K" order polynomial with respect to Hy, and K" order polynomial with respect to H22 which is similar to Equation 18. Pa can be the Chebyshev, Bernstein, even, Possion-Chaalic polynomials in ChebNet (Defferrard et al., 2016), BernNet (He et al., 2021a), EvenNet (Lei et al., 2022), PCNet (Li al., 2023), and so on. We then sum their coefficients using (Hr (I E [0, K')) and (H2 (k E [0, K")) as variables respectively, which is similar Equations 19 and 22. Finally, we replace 0 and @2 with w and w', which is similar to Equations 21 and 23. Therefore, TFE-GNN is combination of two polynomial-based learnable spectral GNNs. Theorems 1 and state that TFE-GNN can learn any filter and matches various graph structure and extracts homophily and heterophily from graphs adaptively.
Table 4: Mean classification accuracy of models at different Hip and Hap-
DATASETS
(Hip. Hap)
(5. 0)
5).
(5. 5)
(10, 10)
CITESEER
12.36=1.33
77.89+1.00 .03
82.32E1.00 95.75+2.81
81.68=1.41 96.00=2.49
WISCONSIN
96.38-2.20
94.88+2.05
A.2 Limitations
We design triple filter ensembles to generate TFE-Conv which can match various graph structures and extract the initial information, homophily and heterophily adaptively. The learnable coefficients 0, and d2 and customized hyperparameters Kt, and KΩp enhance the performance of TFE-GN while creating some limitations. In particular, Kip and Kap rely on one's experience, and it is solved by selecting the appropriate Kip and Kip according to the homophily level of datasets. For example, we choose larger Kip for datasets with strong homophily while choosing larger Ktpp for datasets with strong heterophily. However, the real-world dataset cannot determine its homophily level due to the presence of unlabeled nodes. We are able to determine its approximate homophily level based on training set labels or dataset knowledge, which may be in error.
It is conceivable that there are other limitations of the TFE-GNN, such as the choice of the base filter, which affect the performance of the model. In fact, none of Ktp. Kip- Hip and Hap is fatally weakening the performance of TFE-GNN, as long as we make a suitable choice based on the general phenomenon of polynomial normalized Laplacian Leym or the random walk normalized Laplacian Lrw. Hap taking their affinity
spectral GNNs. For example, Hap taking the symmetric
16

---

(transition) matrix L or L Kip and Khp taking the interval [1,10]. Table 4 show mean classification accuracy TFE-GNN (Hhp taking Lyym) different Hip and Hay. where n, and n2 in (f1, are the values of Hip and Hap respectively.
Table 5: Mean classification accuracy of different models for semi-superrised node classification.
DATASETS
CORA
CITESHER
PUBMED
WISCONSIN
TEXAS
CORNELL
MLP
58.88=0.62
56.97=0.54
73.15=0.22
47.09=4.225
45.03+2.45
46.18+5.10
GCNS
81.32+0.18
71.77+0.21
19.15+0.18
35.34+3.72
32.42+2.23
35.57+3.55
ARMA
83.15=0.54
71.31=0.36
78.75=0.14
45.81=3.46
47.84=3.35
30.8944.23
APPNP
83.5210..24
12.0910.22
80.2310.15
47.31
46.3113.01
45.73.14.85
CHEBNET
80.54+0.38
00.35+0.33
75.52+0.75
28.62+4.28
28.55#3.28
25.54+3.42
GPR-GNN
83.95=0.22
70.9210.57
78.9710.27
45.9114.12
45.76-13.78
43.42:14.95
BERNNET
92.15=0.32
72.24+0.25
79.65=00225
49.93-3.115
48.31=3.17
46.6445.62
CHEBNETII
83.6710.33
72.7510.16
80.4810.23
53.1613.17
$4.68.3.87
$0.92.15.49
EVENNET
82.85+0.332
72.57+0.42
79.36+0.31
54.55+2.68
53.77+3.73
50.64+5.71
FAVARD
83.6120.30
73.1610.22
80.5210.26
54.1342.99
55.23:3.41
51.4314.96
TFE-GNN COV
83.6240.39
72.77=0.64
80.41=0.49
63.12:4.991
64.3443.03
57.9242.32
TFE-GNN, saom
84.11=0.20
74.53+0.25
80.78+0.21
58.71+2.97
67.24=1.19
58.21+4.27
TFE-GNN, pou-t--oe 4a0---
83.1:=0.62
74.21+0.22
79.73+0.48
43.33+3.25
61.27+2.57
55.66+4.22
B More Experimental Results
B.1 Additional experiments
Datasets and Setting. We illustrate the generalization and scalability of TFE-GNN with more intuitive experiments. We select four new datasets (Platonov et al., 2023; Lim et al., 2021a) and conduct relevant experiments to further explain why TFE-GNN has better generalization, including emmaneempire, amazon-racs amazon-rating, and two large graphs (Lim et al., 2021a) whose edge homophily are 0.05, 0.38, 0.47, and 0.62, respectively. Experimental results show that THE-GNN achieves competitive performance 84.76% and 89.32%, respectively. TFE-GNN achieves competitive rankings on all datasets, with the best performance fb100-Penn94, outperforming most spectral GNNs on roman-empire, and topping the rest of the datasets. These additional experimental results further validate the generalization ability of TFE-GNN both homophilic and heterophilic datasets: TFE-GNN can generalize well on graphs with different edge homophily levels. Dataset statistics are reported in Table 6 and the hyperparameters are reported in Table Note that we use the same dataset splits in the article (Platonov et al., 2023; Lim et al., 2021a).
and genius,
these datasets with results of 75.87%, 52.21 %,
B.2 Semi-supervised node classification
Setting and baselines. We compare TFE-GNN to a series of SOTA models for semi-supervised node classification on real-world datasets, including GCNs (Kipf & Welling, 2016), ARMA (Bianchi al., 2021a), APPNP (Klicpera et al., 2018), ChebNet (Defferrard al., 2016), GPR-GNN (Chien et al., 2021), BernNet (He et al., 2021a), ChebNetll (He et al., 2022), EvenNet (Lei et al., 2022), and FavardGNN//OuBasssSNNN (Guo Wei, 2023). We employ the standard and popular train- htggaaliiatiion/estiing split method (Kipf & Welling, 2016; Klicpera et al., 2018) on three citation networks for semi-supervised node classification,, i.e., Cora, Citeseer, and Pubemd, with 20 nodes per class for training, 500 and 1,000 nodes for validation and testing. We split each class of nodes into 2.5%, 2.5%, and 95% as training. validation, and testing sets for other datascts and all models share the same ten random splits for fair comparison. The seed used for dataset splitting is 42 or 1941488137 (He et al., 2022).
Results. Table 5 reports the results of the different models on all datasets for semi-supervised node classification and gives mean classification accuracy and standard deviation over ten foxed splits. Ta- ble 5 shows that TFE-GNN achieves state-of-the-as performance on 6 datasets, where TFE-GNN con and TFE-GNN, swm also show strong competitiveness, which similar to the experimental results of full-supervised node classification. TFE-GNN outperforms FavardGNW/OpBBasisCCC Wei, 2023) on Wisconsin, Texas, and Corenell (strong heterophily) by 8. 8.99% 12.01 %, and 6. 6.78%, respectively, which amounts to performance improvements of 17%. 22%, and 13%.
(Guo &
17

---

Table 6: Additional dataset statistics.
DATASETS
ROMAN-EMPIRE
AMAZON-RATING
FB PB100-PNNN94
GENIUS
NODES
22662
24492
41554
421961
EDGES
32927
93050
1362229
984979
FEATURES
300
300
5
12
CLASSES
18
5
2
2
ehr
0.05
0.38
0.47
0.62
Table 7: The hyper-parameters of TFE-GNN for additional datasets.
DATASETS
ROMAN-EMPIRE
AMAZON-RATING
FB 1B100-PENN94
GENIUS
optim
ADAM
RMSPROP
ADAMW
ADAMW
Kip
4
0
8
hp
6
droppro
0.0
0.0
droptin
0.3
0.8
0.5
0.9
n
0.4
0.5
0.5
0.4
truda
0.5
0.5
0.01
0.05
wdada
0.05
0.0005
0.0005
0.0
Iradue
0.05
0.05
0.01
0.005
unddde
0.1
0.05
0.0
0.0
Irin
0.05
0.005
0.01
0.005
uddin
0.0005
0.005
0.1
0.05
B.3 Visualization Analysis
The I-SNE visualization in Figure 4 demonstrates that the graph convolution TFE,()) can extract meaningful patterns on Citeseer (strong homophily), which the graph convolution TF E2(c) is not able capture. The t-SNE visualization Figure demonstrates that TF better than TF Eg at extracting the information used for categorization on Squirrel (strong heterophily). The output of TFE(GNN(d) shows clearer boundaries among classes (colors) than that T FE, and TF TFE2. These visualization findings validate the competitiveness of TFE-GNN and echo its leading performance on node classification.
Input Features
(b) TFE, Outputs
(c) TFF, Outputs
(d) TFE-GNN Outputs
Figure 4: Visualization of different models Citeseer, which uses dimensionality reduction method t-SNE with 1000 iterations.
B.4 Complementary Generalization Analysis
We add some experiments to analyze TFE-GNN generalization. These newly designed generaliza- tion experiments use the early stopping mechanism, i.e., no longer train for 1000 epochs as fixed as in Figure 2. Figure 6 reports the results for ChebNet(a), GCN(b), BernNet(c), (he) PCNet(e) and TFE-GNN((), and smaller gap between the two losses indicates a better generalization of the model. Figure 2, the gap between the training and validation losses for GCN(b) is less than ChebNet(a) and ChebNetll(c) and larger than TFE-GNN(f). We can observe that the validation loss
18

---

(a) Input Features
(b) TFE, Outputs
(c) TFF2 Outputs
(d) TFE-GNN Outputs
Figure 5: Visualization different models on Squirrel, which uses dimensionality reduction method
t-SNE with 1000 iterations.
of (FE-GNN(() is much closer to its training loss in the six subplots of Figure 6, although its losses more volatile. This observation demonstrates that TFE-GNN's generalization is best when achieves state-of of-the-art classification performance because the early stopping mechanism allows TFE-GNN to carry less stable losses.
Chetdict
GCH
Benile
Training
Training Loss Validation Loss
Training
Valdation
Validation
i-
200
1553
240
Coeche
Coocre
TT8-0
Training Validation
Training
Soce
Figure 6: Generalization of models on Cora.
B.5 Hyper-parameters Analysis
TFE-GNN yields two hyper-parameters Kip and Khp in addition those associated with the neural network, since we parameterize w, w', and 9 as learnable coefficients. We conduct 200 experiments to obtain the interactions between Kip and Kap and the objective values corresponding to model accuracy. We draw their contour plots on Cora and Chameleon. Figure 7 shows parametric contour, where the horizontal coordinate "hopl" denotes the hyper-parameter Kip and the vertical coordinate "hop2" denotes the hyper-paramett there is a difference in the distributions of the objective values (model accuracy) on Cora(a) and Chameleon(b), i.e., there is large difference in the influence of "hopl" 1" and "hop2" on homophily and heterophily graphs.
Kap- The parametric contour in Figure 7 demonstrates that
B.6 Time Efficiency Analysis
We conduct time efficiency experiments to count training times for ChebNet (Defferrard et al., 201 6), BernNet (He et al., 2021a), ChebNetll (He al., 2022) and TFE-GNN. All experiments are carried out on the machine with Linux system, two NVIDIA Tesla V100 and twelve Intel(R) Xeon(R) Gold
19

---

Contour Plot
Contour Plot
6.0
50
5.5
80
68
5.0
60
4.5-
hop2
hop2
52
-00
4.0
5
40
3.5
44
4-
3
28
2.0
20
10
2.0
3.0
3.5
4.0
4.5
5.0
5.5
6.0
hop 1
hopp
(a) Cora
(b) Chameleon
Figure 7: Parametric contours of Cora and Chameleon.
5220 CPU @2.20GHz. Table shows the results time efficient experiments on Cora, where LeN_
TFE-GNN zero indicates that Kip or Kap has and only value of O, TFE-GNN _een indicates that hyper-parameter Kip or Khp has and only a value of 10, the other hyper- has the value K and TFE-GNN all indicates that Kip and Khp have the same value K. We take the average of the times ten training (100 epochs each) as time overheads (unit: second).
The columns in Table 8 imply the change in indee-iivvarintt training time as K increases, and the rows imply the change K-invariant training time as the model varies. Table 8 shows the variation in training time and the comparison between the different models. We observe that the training time of TFN-GNN significantly smaller than that of ChebNedl when the value of is relatively large. Although BernNet and ChebNetll start with a low training time, they quickly catch up and surpass other models as increases. Therefore, TFE-GNN has a clear advantage when larger receptive field (K) is required.
Table 8: Time overheads (s) on Cora.
CHEBNET
BERNNET
CHEBNETIl
TFE-GNN GEROO
TFE-GNN_TEN
TFE-GNN_AN GNN_ALL
K=10
10.88
1.89
3.27
3.03
5,60
5.60
K=20
13.34
5.16
9.75
5.67
8.12
10.98
K=40
22.30
17.19
35.77
10.85
13.44
21.82
K=60
35.94
36.48
79.52
16.02
18.57
31.62
K=80
56.38
64.28
137.60
21.07
23.89
42.00
K=100
80.74
98.76
211.36
26.32
29.16
52.27
B.7 TFE-GNN Versus Other GNNs with Similar Paradigms
TFE-GNN is more suitable full-supervised node classification than semi-supervised node classifi- cation. The key technical differences: TFE-GNN is free from polynomial computation, coefficient constraints, and specific scenarios, compared to polynomial-base spectral GNNs and heterophily- specific Models (Platonov et al., 2023), such ChebNetll (He et al., 2022) and PCNet (Li et al., 2023). We also describe the differences between TFE-GNN and several other methods (Li et al., 2024; Huang et al., 2024b,a).
PEGFAN (Li et al., 2024) sets up three optional feature matrices for the homophilic and heterophilic graphs, respectively, and performs a concatenation operation for the selected feature matrices. PEG-
FAN performs row normalization on the feature matrices that message passing and dimensionality- reducing outputs, and adds a new linear layer before so [mar after the activation function ReLU.
These components increase network complexity of PEGFAN, whereas TFE-GNN is more concise and requires less space.
UniFilter (Huang et al., 2024b) establishes a link between the estimated graph homophily rate and the propagation matrix P by r/2(1 h), and forms heterophily bases up U1.... Ug and
20

---

the rotation matrix Pe. in which learn coefficient wo discloses the significance of each frequency component in graph. The performance of UniFilter is affected by the estimated from the training data labels, which seems to require that the training data label distribution is similar to the graph true global label distribution. The difference between TFE-GNN and UniFilter is that TFE-GNN rarely relies on such priori knowledge and conditions, which ensures its better adaptability and classification performance.
Flow2GNN (Huang al., 2024a) is an interesting and novel attempt, which decomposes the original adjacency matrix into two matrices via random binary matrix G with elements that follow the Bemoulli distribution. Information flows within the nodes in two respective disentangled graphs with reduced heterophily, and then adaptively aggregates them with the strength estimation vector p of information flow. TFE-GNN combines the well-performing and well-tested graph filters, while Flow2GNN is highly dependent on G(!) and p/)
C More Experimental Details
Hyper-parameters. We provide more experimental details for reproducing the experiments. Table shows the hyper-parameters of IFE-GNN on datasets for full-supervisee node classification. Table 10 shows the parammtereesters of TFE-GNN on datasets for semi-supervised node classification. The symbol optim denotes the optimizer, Kip denotes the order of the low-pass graph filter, Kap denotes the order the high-pass graph filter, droppro denotes the dropout rate of input features, droptin denotes the dropout rate of intermediate features, lrada denotes the learning rate of the learnable coefficients w and w/ wdoda denotes the weight decay of w and w', Ir !addae denotes the learning rate of 0, wdadae denotes the weight decay of 1) Irtin denotes the learning rate of MLP fmips and wdin denotes the weight decay of fmip-
Note that different DGL. and Py PyTorch versions can affect model performance, and that fine-tuning of parameters, including but not limited to hyperparameters and seeds, may be required in order to reproduce the effect in this paper. For datasets roman-empire amazon-rating, genius, this paper uses DGL 0.5.2 and PyTorch 1.5.1, and for other datasets, this paper uses DGL 0.9.0 and Py Torch 1.12.1.
and
Baseline implementations. We use the officially released code for GCNII, TWIRLS, GPR-GNN, BernNet, ChebNeil,, H,GCN, Haf-Hop, ARMA, EGNN, PDE-GCN, SPECFORMER, EvenNet, PCNet. And we use the Deep Graph Library implementations for other models, such as GCNs, APPNP, ChebNet, etc. We did not spend lot of time tuning parameters for these models. The code URLs are as follows.
BavardGNN//OttBssssiiiNN
Table The hyper-parameters of TFE-GNN for full-supervised node classification.
DATASETS
CS
CORA
CHAMELSON
COBNELL
RMSPRRP
RMSmRoP
RMSraaP
RMSPRuP
RMSrsoP
droppro
0.2
0.4
0.2
0.2
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.001
0.01
0.01
0.01
0.01
0.01
0.05
0.005
0.005
0.5
0.5
0.05
0.1
0.0
0.0
0.1
0.05
0.0005
0.01
0.05
Iraker
0.1
0.005
0.01
0.05
0.0
0.0
0.0
0.001
0.01
0.0
0.05
0.0
0.0
0.0005
frus
0,1
0.001
0.001
0.001
0.005
0.01
0.0005
0.0005
0.0005
0.0005
0.01
0.01
0.0005
0.0005
0.0005
0.005
GCNII: CCNNUlaattrr//ttthabb..o///e/e//
GPR-GNN: GRPGGN/ittpp/gghhubooo/////haaaa
BernNet: http:://ggttth.ooo///a/aaa///aaa
ChebNetlI: htelkddtt.tttrrggggaaaa..c/////s
HCG http:////gittab..ommmemmt..aaaa
21

---

Table 10: The Syper-parameters of TFE-GNN for semi-supervised node classifications.
DATASETS
CORA
CITESEER
PUBMED
WISCONSIN
TEXAS
CORNELL
optim
ADAM
ADAM
RMSPROP
ADAM
RMSPROP
RMSPROP
Kip
9
10
9
1
Kop
2
1
9
0
1
1
droppro
0.8
0.1
0.9
0.3
0.4
0.6
dropiin
0.5
0.4
0.9
0.7
0.1
0.5
0.5
0.4
0.4
0.4
0.3
0.5
Ir addu
0.01
0.01
0.05
0.05
0.005
0.001
wdada
0.1
0.0005
0
0.05
0.1
0.0005
wdadae
0.01
0.0
0.1
0.01
0.005
0.001
Irades
0.005
0.01
0.1
0.01
0
0.0005
Irin
0.005
0.001
0.005
0.05
0.01
0.05
wdtin.
0.01
0.1
0.01
0.01
0.005
EvenNet: Sht.we/wtrrrregggttt.ooooo//ooaa
PCNet: http:///e/ggggaooooo/////o//ooao
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: we discuss the limitations of TFE-GNN in Appendix A.2.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
23

---

Justification:
the proofs of Theorem I and 2 are reported in Appendix A.1.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: all information in section 4 and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
24

---

Answer: [Yes]
Justification: we provide detailed code in the supplemental material, which can be run directly after installing the required packages.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. ec/
olicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips... public/guider
sssissiin wwwssssoiicyyyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: all details are in section 4 and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: all information section and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
25

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: all information in section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http://eeeuu..eeeeeee httprrpps.cpppppc/eteeeee
cttppuulic//hhhcccueeeeeeee.eeee wtw.idheeeeelllleccccccc ines?
Answer: [Yes]
Justification: full paper.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: articles are fundamental research.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
26

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: we cite the papers covered in the public datasets in the experimental section 4.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
27

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: provide code in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: this paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: this paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "Y5tHvmz2cW", "reviewer": "Reviewer_eCxX", "summary": "This paper proposed a spectral GNN with triple filter ensemble (TFE-GNN) that aims to retain both the ability of polynomial based spectral GNNs to approximate filters and the higher generalization and performance on graph learning tasks, when most existing GNNs can only retain one of the two capabilities. Theoretical analysis shows that the approximation ability of TFE-GNN is consistent with that of ChebNet under certain conditions, namely it can learn arbitrary filters. Experiments show that TFE-GNN achieves high generalization and best performance on various homophilous and heterophilous real-world datasets.", "questions": "It would be great if the authors can address the points discussed in weakness during the rebuttal.", "limitations": "The paper includes discussion regarding the limitations in Appendix A.2, most notably the reliance on setting hyperparameters $K_{lp}$ and $K_{hp}$ that are suitable for the homophily level of datasets.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The experiments are thoroughly conducted with 10 datasets and 17 baselines, which show that the proposed TFE-GNN achieve the best performance over all baseline models.\n- The introduction of learnable spectral GNNs and the proposed TFE-GNN is clear and in details.", "weaknesses": "- My main concern is that the heterophilous datasets that are used in the experiments are not up-to-date. Specifically, the five heterophilous datasets in the experiments are a subset of those first adopted by Pei et al. 2020 (Geom-gcn).  However, recent works like Platonov et al. 2022 has identified several drawbacks of these datasets, including leakage of test nodes in the training set in squirrel and chameleon, which can render the results obtained on these datasets unreliable. I recommend the authors to run additional experiments on two more up-to-date heterophilous datasets, such as those introduced in the papers below:\n    - Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. 2022. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. In The Eleventh International Conference on Learning Representations.\n    - Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao,\n    and Ser Nam Lim. 2021. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems 34 (2021), 20887–20902.\n- The authors discussed that for existing GNN models, “some polynomial-based models use polynomials with better approximation than some other models when approximating filters, but the former’s performance is lagging behind that of the latter on real-world graphs.” The proposed TFE-GNN models seem to have overcome that limitation, but it remains unclear to me in a high-level, what are the key differences between TFE-GNN compared to prior models which make TFE-GNN able to achieve a better trade-off here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We appreciate your valuable suggestions and we will explain them line by line.**\n\n> **Question 1:** I recommend the authors to run additional experiments on two more up-to-date heterophilous datasets, such as those introduced in the papers  [1, 2].\n\n**A1:** Thank you for your valuable comment. We think that adding relevant experiments is necessary to make our paper more complete after discussing it. We validate our model with additional experiments on two large graphs [2] --fb100-Penn94 and genius-- whose edge homophily are 0.47 and 0.62, respectively. Experimental results on these two datasets were 84.76\\% and 89.32\\% for node classification. **The performance of TFE-GNN on fb100-Penn94 dataset exceeds that of all models mentioned in the article [2].** TFE-GNN tops the performance on genius dataset, which demonstrates its high competitiveness. All dataset statistics and preliminary experimental parameters are reported in Tables 1 and 2, respectively, of the author rebuttal’s one-page PDF. Note that we use the same dataset splits as in the article [2].\n\nWe will add relevant experiments and their results analysis to the final version. We will modify our publicly available code accordingly and follow up with sufficient experiments and report the experimental results and parameters in the final version.\n\n> **Question 2:** It remains unclear to me in a high-level, what are the key differences between TFE-GNN compared to prior models which make TFE-GNN able to achieve a better trade-off here?\n\n**A2:** Thanks for your constructive feedback. The key difference between TFE-GNN and prior models is that **TFE-GNN retains the ability of polynomial-based spectral GNNs while getting rid of polynomial computations, coefficient constraints, and specific scenarios.** The first problem we describe in line 52 of the introduction shows that it is not only the approximation ability of the polynomials affects the performance of GNNs, but also coefficient constraints and graph learning methods influence the performance. For example, ChebNetII [3], a prior model, constrains learnable coefficients by Chebyshev interpolation to boost ChebNet’s performance, which brings overfitting [6] for it. EvenNet [4] and PCNet [5] elaborate graph learning methods, namely even-polynomials and heterophilic graph heat kernel to ignore odd-hop neighbors, whcih makes them to accommodate the structure nature of heterophilic graph. This elaborate design is similar to the heterophily-specific models [7]. These models achieve great node classification performance for specific problems or scenarios through specific tricks.\n\nIn this paper, our proposed model is free from polynomial computations, coefficient constraints, and specific scenarios. Specifically, Equation 13 and its predecessors in our paper show that TFE-GNN is free from polynomial computations, unlike ChebNetII and PCNet which require the computation of Chebyshev polynomials $T_k(x)$(Eq. 8 in [3]) and Possion-Charlier polynomials $C_n(k, t)$ (Eq. 15 in [5]), respectively. TFE-GNN does not impose refined constraints on the coefficients and does not design very complex learning methods, which helps it avoid overfitting as much as possible. TFE-GNN extracts homophily and heterophily from graphs with different levels of homophily adaptively while utilizing the initial features, which helps it not be limited to specific scenarios. Furthermore, Theorems 1 and 2 prove the approximation ability and expressive power of TFE-GNN, and experimental results verify that TFE-GNN achieves our purpose. In summary, TFE-GNN achieves a better trade-off in approximation ability and classification performance.\n\nWe will add the following sentences from line 80 of the introduction: The key difference between TFE-GNN and prior models is that TFE-GNN retains the ability of polynomial-based spectral GNNs while getting rid of polynomial computations, coefficient constraints, and specific scenarios. \n\nWe appreciate the advice! We will gladly answer any additional questions you may have.\n\n**References**\n\n[1] Oleg Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. ICLR 2022.\n\n[2] Derek Lim et al. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. NeurIPS 2021.\n\n[3] He, M et al. Convolutional neural networks on graphs with chebyshev approximation, revisited. NeurIPS 2022.\n\n[4] Lei, R et al. Evennet: Ignoring odd-hop neighbors improves robustness of graph neural networks. NeurIPS 2021.\n\n[5] Li, B et al. Pc-conv: Unifying homophily and heterophily with two-fold filtering. AAAI 2024.\n\n[6] Z. Zeng et al. Graph Neural Networks With High-Order Polynomial Spectral Filters. IEEE Transactions on Neural Networks and Learning Systems, 2023.\n\n[7] Oleg Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? ICLR 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VMy9EC0m5b", "reviewer": "Reviewer_pzPy", "summary": "This paper proposes a spectral GNN with triple filter ensemble (TFE-GNN). As the ensemble of two diverse and simple low-pass and high-pass graph filters, TFE-GNN achieves good generalization ability and SOTA performance on both homophilic and heterophilic datasets.", "questions": "Questions: \n\nQ1. In the introduction section, could you provide some references for the second problem, that the existing methods for solving problem one reduce the generalization ability of GNNs? \n\nQ2. In Table 5 in Appendix A.2, is there a typo (93.13±0.62) in the last row of the second column? It significantly outperforms all other settings but is not marked in bold as the best result. \n\nQ3. In Figure 5 in Appendix B.3, there is a large oscillation of the validation loss of the TFE-GNN, compared to other models. Could you account for this oscillation?  \n\n \n\nSuggestions: \n\nS1. The idea of combining low-pass and high-pass filters is explored by many works (PCNet[1], GBK-GNN[2]). It would be interesting to explore the generalization ability between this class of two-fold filters versus general polynomial filters. \n\nS2. The statement of the second problem (starting from line 64) that “Carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints led to overfitting” needs clarification or references. \n\n\n[1] Li, Bingheng, Erlin Pan, and Zhao Kang. \"Pc-conv: Unifying homophily and heterophily with two-fold filtering.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 12. 2024. \n\n[2] Du, Lun, et al. \"Gbk-gnn: Gated bi-kernel graph neural networks for modeling both homophily and heterophily.\" Proceedings of the ACM Web Conference 2022. 2022.", "limitations": "-", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1. This work is motivated reasonably by leveraging the diversity of graph filters to create an ensemble GNN. It is well-explained with clear definitions, theories, and results. The results are further presented with visual tools to create an easy-to-understand intuition.  \n\nS2. The conclusions are supported by extensive experiments with a variety of baseline models and datasets.", "weaknesses": "W1. One major statement that TFE-GNN has better generalization then other adaptive filter GNNs is not fully explained - only empirical results on the loss curves are shown. As a major contribution emphasized in the abstract, more theoretical and empirical results are usually expected. \n\nW2. The relevance of the abstract and introduction and the later sections can be enhanced. For example, most theoretical analyses are about the expressiveness of TFE-GNN, which is not stressed in the introduction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely appreciate your valuable comments and suggestions and we answer your concerns point to point!**\n\n> **Weakness 1:** More theoretical and empirical results are expected to explain why TFE-GNN has better generalization.\n\n**W-A1:** Thanks for your meaningful comments. We further explain why TFE-GNN has better generalization by introducing two new references [5, 6]. FFKSF [5] and ARMA [6] attribute the model overfitting to the overfitting of polynomial coefficients and the aggregation of high-order neighborhood information. TFE-GNN does not impose refined constraints on the coefficients and achieves better performance with a smaller order, whcih possesses higher generalization.\n\nWe select four new datasets [2, 3] and conduct relevant experiments to further explain why TFE-GNN has better generalization, including roman-empire, amazon-rating, and two large graphs [3] fb100-Penn94 and genius, whose edge homophily are 0.05, 0.38, 0.47, and 0.62, respectively. Experimental results show that TFE-GNN achieves competitive performance on these datasets with results of 75.87\\%, 52.21\\%, 84.76\\% and 89.32\\%, respectively. TFE-GNN achieves competitive rankings on all datasets, with the best performance on fb100-Penn94, outperforming most spectral GNNs on roman-empire, and topping the rest of the datasets. These additional experimental results further validate the generalization ability of TFE-GNN on both homophilic and heterophilic datasets: **TFE-GNN can generalize well on graphs with different edge homophily levels.** We will add the above relevant content to Section 4.3 and Appendix B.3 in the final version.\n\n> **Weakness 2:** The relevance of the abstract and introduction and the later sections can be enhanced.\n\n**W-A2:** Thanks for pointing out our current shortcomings. To stress the expressiveness of TFE-GNN, we add the following sentences from line 83 of the paper: Theorem 2 show that TFE-GNN is a reasonable combination of two excellent polynomial-based spectral GNNs, which motivates it to perform well. TFE-GNN preserves the ability of the different filters while reducing overfitting problem.\n\n> **Question 1:** Could you provide some references for the second problem?\n\n**Q-A1:** We appreciate your valuable question and find that the second problem, which builds on the first, is really not sufficiently described. We add the following sentences to clarify it from line 67 of the introduction: FFKSF [5] attributes the degradation of polynomial filters’ performance to the overfitting of polynomial coefficients. ChebNetII [4] further constrains the coefficients to enable them easier to be optimized. ARMA [6] suggests that the filter will overfit the training data when aggregating high-order neighbor information. Whereas the order of polynomial-based spectral GNNs is usually large to increase the approximation of the polynomials, which directs them to obtain high-order neighborhood information, and then leads to overfitting. Therefore, it is reasonable to assume that carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints lead to overfitting of the models, which diminishes generalization of the models. TFE-GNN discards these factors to improve generalization, while retaining the approximation ability. \n\n> **Question 2:** In Table 5 in Appendix A.2, is there a typo (93.13±0.62) in the last row of the second column?\n\n**Q-A2:** Thanks for your responsible, careful and valuable comment! We are sorry for our carelessness. Actually, its correct value should be 83.13±0.62, we will revise it in the final version.\n\n> **Question 3:** Could you account for this oscillation in Figure 5 in Appendix B.3?\n\n**Q-A3:** Thanks for your valuable comment. The learning rate controls the step size which in turn affects the loss optimization. The large learning rate (= 0.1) is responsible for the oscillations of the validation loss in Figure 5 in Appendix B.3. Figure 1 in the one-page PDF in the author rebuttal shows that the loss is stable when the learning rate is 0.001. The early stopping mechanism allows TFE-GNN to carry less stable losses and losses do not fall into unacceptable local maximum.\n\n> **Suggestion 1:** It would be interesting to explore the generalization ability between this class of two-fold filters versus general polynomial filters.\n\n**S-A1:** Thanks for your valuable ideas. We will add the following to Section 5: We are eager to explore different types of combinations of low-pass and high-pass filters, as well as global combining methods such as pooling.\n\n> **Suggestion 2:** The statement of the second problem needs clarification or references.\n \n**S-A2:** Thank you for your helpful suggestion. In addition to PCNet [1] pointing out that carefully crafted graph learning methods lead to low model generalization, we add two new references [5, 6] to state the second problem of this paper. FFKSF [5] and ARMA [6] attribute the overfitting to the overfitting of polynomial coefficients and the aggregation of high-order neighborhood information. We add the relevant analysis to the introduction starting at line 67 of the introduction\n\nWe appreciate the advice! We will gladly answer any additional questions you may have.\n\n**References**\n\n[1] Li, B et al. Pc-conv: Unifying homophily and heterophily with two-fold filtering. AAAI 2024.\n\n[2] Oleg Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. ICLR 2022.\n\n[3] Derek Lim et al. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. NeurIPS 2021.\n\n[4] He, M et al. Convolutional neural networks on graphs with chebyshev approximation, revisited. NeurIPS 2022.\n\n[5] Z. Zeng et al. Graph Neural Networks With High-Order Polynomial Spectral Filters. IEEE TNNLS, 2023.\n\n[6] F. M. Bianchi et al.. Alippi, Graph Neural Networks With Convolutional ARMA Filters. IEEE TPAMI, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DdYHWZ0NmG", "reviewer": "Reviewer_zzRJ", "summary": "This paper proposes a novel spectral GNN architecture, the TFE-GNN, which integrates homophily and heterophily using a triple filter ensemble (TFE) mechanism. The model aims to overcome the limitations of existing polynomial-based learnable spectral GNNs by adaptively extracting features from graphs with varying levels of homophily. The TFE-GNN combines low-pass and high-pass filters through three ensembles, enhancing the generalization and performance of the model. Theoretical analysis and experiments on real-world datasets demonstrate the effectiveness and state-of-the-art performance of TFE-GNN.", "questions": "* Can the ensemble methods (EM1, EM2, EM3) be further optimized or replaced with alternative techniques to improve performance?\n* What is the computational overhead of the triple filter ensemble mechanism compared to traditional spectral GNNs? How does the model scale with larger graphs and higher-dimensional feature spaces?\n* The authors mention that TFE-GNN has higher generalization. Why? Any theoretical results for charactering the generalization ability of TFE-GNN?\n* (I am quite curious about it) Are there specific scenarios or types of graphs where TFE-GNN might not perform as well?\n*Authors may consider discussing the key technical differences between the proposed work and recent methods on heterophilous graph learning, such as: [1] Permutaion equivariant graph framelets for heterophilous graph learning; [2] How universal polynomial bases enhance spectral graph neural networks: Heterophily, over-smoothing, and over-squashing; [3] Flow2GNN: Flexible two-way flow message passing for enhancing GNNs beyond homophily", "limitations": "The authors have discussed the limitations of their work adequately.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper provides a fresh perspective on integrating homophily and heterophily in graph learning tasks;\n* The presentation is easy to follow;\n* Some experiments covering various datasets with different levels of homophily show significant improvements over state-of-the-art models.", "weaknesses": "* The paper does not address potential computational overhead introduced by the triple filter ensemble mechanism;\n* The descriptions of the theorems are too lengthy and not concise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We are grateful for your comments and suggestions, and we will explain and answer your concerns point by point!.**\n\n> **Weakness 1:** The potential computational overhead introduced by the triple filter ensemble.\n\n**W-A1:** Thanks for pointing out what we need to strengthen. The triple filter ensemble (TFE) is the core component of our proposed TFE-GNN, which combines filters to form a graph convolution. Similarly to ChebNetII [4] and PCNet [5], the time complexity of TFE-GNN is linear to $K_{lp}$+$K_{hp}$ when $EM_1$, $EM_2$, and $EM_3$ take summation. This shows that TFE does not influence the time complexity magnitude of the model.\n\n> **Weakness 2:** The descriptions of the theorems are too lengthy and not concise.\n\n**W-A2:** Thanks for your valuable and helpful comment. We simplify condition (2) of Theorem 1 as \"$EM_1$, $EM_2$, and $EM_3$ take summation, $H_{hp}$ take $L_{sym}$ and $H_{lp}$ take $L_{sym}^a$\". We simplify condition (2) of Theorem 2 as \"$EM_1$, $EM_2$ takes summation and $EM_3$ takes ensemble method capable of preserving the model's properties, $H_{hp}$ takes $\\bar{H}^2_{gf}$ and $H_{lp}$ takes $\\bar{H}^1_{gf}$\".\n\n> **Question 1:** Can the ensemble methods be further optimized or replaced?\n\n**Q-A1:** This is a insightful question that helps us analyze the TFE-GNN. There exist some other ensemble methods or alternative techniques to replace $EM_1$, $EM_2$, and $EM_3$, such as mean, maximization, pooling, etc., however, these methods do not perform as good as summation according to our experimental results.\n\n> **Question 2:** What is the computational overhead of the triple filter ensemble? How does the model scale with larger graphs and higher-dimensional feature spaces?\n\n**Q-A2:** Thanks for your insightful questions. Similar to most spectral GNNs [4, 5], the triple filter ensemble mechanism does not influence the time complexity magnitude of the TFE-GNN, i.e., the time complexity of TFE-GNN is linear to $K_{lp}$+$K_{hp}$ when $EM_1$, $EM_2$, and $EM_3$ take summation. Specifically, the time complexity of message propagation is $O((K_{lp}+K_{hp})|E|C)$, the time complexity of the combination of $H_{gf}$ with respectively $\\omega$ and $\\omega'$ (Equations 9 and 10) is $O((K_{lp}+K_{hp})nC)$, and the time complexity of the coefficient calculation is not greater than $O(K_{lp}+K_{hp})$, where $C$ denote the number of classes and $n$ denotes the number of nodes. We report the training time overhead of the different spectral GNNs in Table 6 of Appendix B.5.\n \nFor graphs with higher dimensional features, we scale TFE-GNN by exchanging the order of message propagation and feature dimensionality reduction:\n$\\widetilde{Z}=EM_3\\lbrace\\vartheta_1Z_{lp}, \\vartheta_2Z_{hp}\\rbrace = EM_3\\lbrace \\vartheta_1TFE_1f_{mlp}(X), \\vartheta_2TFE_2f_{mlp}(X) \\rbrace$. We use a sparse form of the adjacency matrix of large graphs, which greatly reduces the space required for TFE-GNN. Therefore, TFE-GNN scales well to large graphs and high-dimensional feature spaces.\n\nWe chose two large graphs [6] --fb100-Penn94 and genius-- to verify the scalability of TFE-GNN, and the experimental results are 84.76\\% and 89.32\\%, respectively. **The performance of TFE-GNN on fb100-Penn94 exceeds that of all models mentioned in the article [6].** TFE-GNN tops the performance on genius dataset, which demonstrates its high competitiveness. All dataset statistics and experimental parameters are reported in Tables 1 and 2, respectively, of the author rebuttal’s one-page PDF. We will add a new section 3.5 to analyze and study the scalability of TFE-GNN in the final version.\n\n> **Question 3:** Why TFE-GNN has higher generalization.\n\n**Q-A3:** Thanks for your valuable and meaningful questions. TFE-GNN is free from polynomial computation, coefficient constraints, and specific scenarios. These factors, especially the coefficient constraints, are responsible for model overfitting [7]. We illustrate the generalization ability of TFE-GNN with more intuitive experiments. In addition to the two large graphs --fb100-Penn94 and genius--, we add two additional datasets [9] --roman-empire and amazon-rating-- whose edge homophily are 0.05 and 0.38 respectively, to verify the generalization of TFE-GNN. Experimental results on the two datasets are 75.87\\%, and 52.21\\%, respectively. These additional experimental results further validate the generalization ability of TFE-GNN on both homophilic and heterophilic datasets: **TFE-GNN can generalize well on graphs with different edge homophily levels.** We will follow up with sufficient experiments and report the experimental results and parameters in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Global Response**

We sincerely thank all reviewers for their thoughtful and constructive feedback and their recognition of our work! We have made significant improvements to our initial results in response to their comments. We add a new figure and two tables in the one-page PDF and update the paper and supplementary PDFs with revisions accordingly. We welcome any follow-up discussions!

**Time Complexity and Scalability.** We further investigate the time complexity of TFE-GNN and add relevant results to Appendix B.5 in the final version. Similarly to ChebNetII [1] and PCNet [2], the time complexity of TFE-GNN is linear to $K_{lp}$+$K_{hp}$ when $EM_1$, $EM_2$, and $EM_3$ take summation. Specifically, the time complexity of message propagation is $O((K_{lp}+K_{hp})|E|C)$, the time complexity of the combination of $\omega$ and $H_{gf}$ (Equations 9 and 10) is $O((K_{lp}+K_{hp})nC)$, and the time complexity of the coefficient calculation is not greater than $O((K_{lp}+K_{hp}))$, where $C$ denote the number of classes and $n$ denotes the number of nodes.

For graphs with higher dimensional features, we scale TFE-GNN by exchanging the order of message propagation and feature dimensionality reduction:
$\widetilde{Z}=EM_3\lbrace\vartheta_1Z_{lp}, \vartheta_2Z_{hp}\rbrace = EM_3\lbrace \vartheta_1TFE_1f_{mlp}(X), \vartheta_2TFE_2f_{mlp}(X) \rbrace$. We use a sparse representation of the adjacency matrix of large graphs, which greatly reduces the space required for TFE-GNN. The new form and the time complexity of TFE-GNN show that **TFE-GNN scales well to large graphs and high-dimensional feature spaces.**

**Generalization Analysis.** We further illustrate the generalization ability of TFE-GNN by introducing new references. PCNet [2] points out that many carefully crafted graph representation learning methods are incapable of generalizing well across real-world graphs with different levels of homophily. FFKSF [3] attributes the degradation of polynomial filters’ performance to the overfitting of polynomial coefficients, and some spectral GNNs, such as ChebNetII [1], further constrains the coefficients to enable them easier to be optimize. ARMA [6] suggests that the filter will overfit the training data when aggregating high-order neighbor information. Whereas the order of polynomial-based spectral GNNs is usually large to increase the approximation of the polynomials, which directs them to obtain high-order neighborhood information, which in turn leads to overfitting. Therefore, it is reasonable to assume that carefully crafted graph learning methods, sophisticated polynomial approximations, and refined coefficient constraints lead to overfitting of the models, which diminishes generalization of the models. TFE-GNN discards these factors to improve generalization, while retaining the approximation ability.

**More Experiments.** We conduct more experiments for verifying the performance, generalization and scalability of TFE-GNN on four additional datasets. We select two heterophilic [4] --roman-empire and amazon-rating-- whose edge homophily are 0.05 and 0.38 respectively, and select two large graphs [5] --fb100-Penn94 and genius-- whose edge homophily are 0.47 and 0.62 respectively. Note that we use the same dataset splits as the provenance of these datasets. Experiments results on these datasets are 75.87\%, 52.21\%, 84.76\% and 89.32\%, respectively. TFE-GNN achieves competitive rankings on all datasets, with the best performance on fb100-Penn94, outperforming most spectral GNNs on roman-empire, and topping the rest of the datasets. It is worth noting that TFE outperforms all SOTA models in the article [4] on homophilic datasets. These additional experimental results further validate the generalization ability of TFE-GNN on both homophilic and heterophilic datasets: **TFE-GNN can generalize well on graphs with different edge homophily levels.** All dataset statistics and experimental parameters are reported in Tables 1 and 2, respectively, of the one-page PDF. The very competitive results shown by TFE-GNN on these datasets demonstrate its good generalization ability and scalability. 

**Loss Oscillation.** The learning rate controls the step size which in turn affects the loss optimization. The large learning rate (= 0.1) is responsible for the oscillations of the validation loss in Figure 5 in Appendix B.3. Figure 1 in the one-page PDF shows that the loss is stable  when the learning rate is 0.001.

**TFE-GNN Versus Other GNNs with Similar Aggregation Paradigms.** The key technical differences: our proposed TFE-GNN is free from polynomial computation, coefficient constraints, and specific scenarios, compared to polynomial-based spectral GNNs and heterophily-specific Models [4], such as ChebNetII [1] and PCNet [2]. TFE-GNN achieves higher performance through a simpler network structure compared to PEGFAN [7] and UniFilter[8].

**References**

[1] He, M et.al. Convolutional neural networks on graphs with chebyshev approximation, revisited. NeurIPS, 2022.

[2] Li, B et.al. Pc-conv: Unifying homophily and heterophily with two-fold filtering. AAAI, 2024.

[3] Z. Zeng et al. Graph Neural Networks With High-Order Polynomial Spectral Filters. IEEE Transactions on Neural Networks and Learning Systems, 2023.

[4] Oleg Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. ICLR 2022.

[5] Derek Lim et al. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. NeurIPS 2021.

[6] F. M. Bianchi et al.. Alippi, Graph Neural Networks With Convolutional ARMA Filters. IEEE TPAMI, 2022.

[7] Jianfei Li et al. Permutaion equivariant graph framelets for heterophilous graph learning. TNNLS, 2024.

[8] Keke Huang et al. How universal polynomial bases enhance spectral graph neural networks: Heterophily, over-smoothing, and over-squashing. ICML, 2024.

Meta-review: In this paper, the authors proposed a new GNN model based on a triple filter ensemble mechanism (TFE-GNN), which aims to learn a spectral GNN with theoretically guaranteed filter approximation power and strong generalizability in practical node classification tasks. In theory, the authors demonstrated that the proposed TFE-GNN is equivalent to ChebNet under some specific settings, so they have the same filter approximation power. Moreover, this model leads to the combination of two spectral GNNs that correspond to heterophilic and homophilic graphs, respectively, which provides a unified GNN framework. Experiments show that the proposed method achieves promising performance in various node classification tasks. 

All three reviewers scored this paper positively, and their concerns about this work's motivation, complexity, and solidness were resolved in the rebuttal phase. In addition, although both SAC and I agree that the merits of this paper seem to outweigh the issues, we believe that more generalizability tests are required, e.g., testing model performance with increased data noise or in some OOD scenarios, and that comparisons with baselines like ChebNet in such scenarios are necessary. Otherwise, the claims in the abstract and introduction are not convincing to some extent.

In summary, I decided to accept this work and strongly suggest the authors polish this paper according to the reviewers' comments.
Decision: Accept (poster)