Original: G RAPH I NFORMATION M ATTERS : U NDERSTANDING G RAPH F ILTERS FROM I NTERACTION P ROBABILITY
Revised: G RAPH I NFORMATION M ATTERS : U NDERSTANDING G RAPH F ILTERS FROM I NTERACTION P ROBABILITY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph Neural Networks (GNNs) have received extensive affirmation for their promising performance in graph learning problems.
Revised: Graph Neural Networks (GNNs) have received extensive affirmation for their promising performance in graph learning problems.

Original: Despite their various neural architectures, most are intrinsically graph filters that provide theoretical foundations for model explanations.
Revised: Despite their various neural architectures, most are intrinsically graph filters that provide theoretical foundations for model explanations.

Original: In particular, low-pass filters show superiority in label prediction in many benchmarks.
Revised: In particular, low-pass filters show superiority in label prediction in many benchmarks.

Original: However, recent empirical research suggests that models with only low-pass filters do not always perform well.
Revised: However, recent empirical research suggests that models with only low-pass filters do not always perform well.

Original: Although increasing attempts to understand graph filters, it is unclear how a particular graph affects the performance of different filters.
Revised: Although increasing attempts to understand graph filters, it is unclear how a particular graph affects the performance of different filters.

Original: In this paper, we carry out a comprehensive theoretical analysis of the synergy of graph structure and node features on graph filters’ behaviors in node classification, relying on the introduction of interaction probability and frequency distribution.
Revised: In this paper, we carry out a comprehensive theoretical analysis of the synergy of graph structure and node features on graph filters’ behaviors in node classification, relying on the introduction of interaction probability and frequency distribution.

Original: We show that the homophily degree of graphs significantly affects the prediction error of graph filters.
Revised: We show that the homophily degree of graphs significantly affects the prediction error of graph filters.

Original: Our theory provides a guideline for graph filters design in a data-driven manner.
Revised: Our theory provides a guideline for graph filters design in a data-driven manner.

Original: Since it is hard for a single graph filter to live up to this, we propose a general strategy for exploring a data-specified filter bank.
Revised: Since it is hard for a single graph filter to live up to this, we propose a general strategy for exploring a data-specified filter bank.

Original: Experimental results show that our model achieves consistent and significant performance improvements across all benchmarks.
Revised: Experimental results show that our model achieves consistent and significant performance improvements across all benchmarks.

Original: Furthermore, we empirically validate our theoretical analysis and explain the behavior of baselines and our model.
Revised: Furthermore, we empirically validate our theoretical analysis and explain the behavior of baselines and our model.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph Neural Networks (GNNs) have continuously attracted interest as their promising performance in various graph learning problems.
Revised: Graph Neural Networks (GNNs) have continuously attracted interest as their promising performance in various graph learning problems.

Original: It is known that most of GNNs are intrinsically graph filters (Kipf & Welling, 2017; Defferrard et al., 2016; Ortega et al., 2018;
Revised: It is known that most of GNNs are intrinsically graph filters (Kipf & Welling, 2017; Defferrard et al., 2016; Ortega et al., 2018;

Original: Nt & Maehara, 2019).
Revised: Nt & Maehara, 2019).

Original: With the theoretical foundation of filters, there is an increasing attempt at model explanation, e.g. explaining the behavior of various GNNs in node classification.
Revised: With the theoretical foundation of filters, there is an increasing attempt at model explanation, e.g. explaining the behavior of various GNNs in node classification.

Original: Nt & Maehara (2019) investigated the superiority of low-pass filters backed up with theoretical arguments while recent research (Balcilar et al., 2020;
Revised: Nt & Maehara (2019) investigated the superiority of low-pass filters backed up with theoretical arguments while recent research (Balcilar et al., 2020;

Original: Chang et al., 2020; Bo et al., 2021) empirically revealed the weakness of GNNs with only low-pass filters in certain datasets.
Revised: Chang et al., 2020; Bo et al., 2021) empirically revealed the weakness of GNNs with only low-pass filters in certain datasets.

Original: These contradictory views on low-pass filters pose a significant problem:
Revised: These contradictory views on low-pass filters pose a significant problem:

Original: Why does a filter work on one dataset but not on another?
Revised: Why does a filter work on one dataset but not on another?

Original: More precisely, for a given filter, what kinds of structure and features are useful for prediction?
Revised: More precisely, for a given filter, what kinds of structure and features are useful for prediction?

Original: This makes it clear to us that in order to solve this problem, it is necessary to take into account graph information, including the graph structure, features, and labels.
Revised: This makes it clear to us that in order to solve this problem, it is necessary to take into account graph information, including the graph structure, features, and labels.

Original: Existing theoretical research is mostly restricted to the investigation of filters themselves such as exploring their expressive power (Oono & Suzuki, 2020; Balcilar et al., 2020), without considering their inconsistency of performance on different graphs.
Revised: Existing theoretical research is mostly restricted to the investigation of filters themselves such as exploring their expressive power (Oono & Suzuki, 2020; Balcilar et al., 2020), without considering their inconsistency of performance on different graphs.

Original: It is clear that structural and feature information lead to the possible inconsistency.
Revised: It is clear that structural and feature information lead to the possible inconsistency.

Original: However, there has been little explicit analysis of how graph information influences the performance of graph filters.
Revised: However, there has been little explicit analysis of how graph information influences the performance of graph filters.

Original: For instance, GNNs have formulated a variety of graph filters in a heuristic manner under a suppressed homophily assumption, i.e., nodes with similar attributes/labels tend to have connections.
Revised: For instance, GNNs have formulated a variety of graph filters in a heuristic manner under a suppressed homophily assumption, i.e., nodes with similar attributes/labels tend to have connections.

Original: There remains a paucity of quantitative description of homophily until Pei et al.
Revised: There remains a paucity of quantitative description of homophily until Pei et al.

Original: (2020) designed a rough index to measure it.
Revised: (2020) designed a rough index to measure it.

Original: In this paper, we establish a comprehensive theoretical analysis of the effect of structure and feature information on node label prediction to fill the gap and provide deep insights into the explanation of graph filters.
Revised: In this paper, we establish a comprehensive theoretical analysis of the effect of structure and feature information on node label prediction to fill the gap and provide deep insights into the explanation of graph filters.

Original: We first establish a systematic investigation on graphs with an indicator in terms of homophily - the interaction probability and a distributional representation of input information - the frequency distribution .
Revised: We first establish a systematic investigation on graphs with an indicator in terms of homophily - the interaction probability and a distributional representation of input information - the frequency distribution .

Original: The interaction probability derived from random walk theory relates node labels with its local topology and quantifies the degree of clustering of nodes in the same/different class.
Revised: The interaction probability derived from random walk theory relates node labels with its local topology and quantifies the degree of clustering of nodes in the same/different class.

Original: We argue that interaction probability reflects the difficulty in identifying one class from others.
Revised: We argue that interaction probability reflects the difficulty in identifying one class from others.

Original: In terms of feature information, we draw on spectral analysis representing features as frequency distributions.
Revised: In terms of feature information, we draw on spectral analysis representing features as frequency distributions.

Original: Furthermore, we consider the moment of frequency and build an explicit relation with graph structure.
Revised: Furthermore, we consider the moment of frequency and build an explicit relation with graph structure.

Original: Interestingly, we find that the moment of label frequency (noting that a one-hot label vector can be regarded as a special node feature) is determined by interaction probability.
Revised: Interestingly, we find that the moment of label frequency (noting that a one-hot label vector can be regarded as a special node feature) is determined by interaction probability.

Original: The aforementioned preparations underpin our deep understanding of graph filters.
Revised: The aforementioned preparations underpin our deep understanding of graph filters.

Original: We validate the prediction error of a graph filter under two settings: a. fixed graph structure, unravel the influence of input (original or transformed node features) ; b. given input, show how structure matters , and provide analysis utilizing frequency distribution and interaction probability.
Revised: We validate the prediction error of a graph filter under two settings: a. fixed graph structure, unravel the influence of input (original or transformed node features) ; b. given input, show how structure matters , and provide analysis utilizing frequency distribution and interaction probability.

Original: The main conclusions are: 1.
Revised: The main conclusions are: 1.

Original: given structure, the frequency response of an ideal graph filter should be consistent with the main frequency band of label frequency, that is, a matched frequency response is the premise of success; 2.
Revised: given structure, the frequency response of an ideal graph filter should be consistent with the main frequency band of label frequency, that is, a matched frequency response is the premise of success; 2.

Original: given input, a graph filter essentially tunes the weight of edges - failing to make a homophily degree large enough may cause an unsatisfactory prediction accuracy.
Revised: given input, a graph filter essentially tunes the weight of edges - failing to make a homophily degree large enough may cause an unsatisfactory prediction accuracy.

Original: These interpretations of graph filters imply a data-driven filter design principle.
Revised: These interpretations of graph filters imply a data-driven filter design principle.

Original: In addition, we apply these theoretical results to three types of filters - low-pass, high-pass, and band-pass filters with specified form.
Revised: In addition, we apply these theoretical results to three types of filters - low-pass, high-pass, and band-pass filters with specified form.

Original: It shows that a single graph filter is hard to comply with the principle of ideal filters, especially when the homophily degree and label frequency distribution of different classes are very different.
Revised: It shows that a single graph filter is hard to comply with the principle of ideal filters, especially when the homophily degree and label frequency distribution of different classes are very different.

Original: For example, when frequency distributions of labels are far from each other, it is hard to find a single filter whose frequency response can cover all the main frequency bands well.
Revised: For example, when frequency distributions of labels are far from each other, it is hard to find a single filter whose frequency response can cover all the main frequency bands well.

Original: In this paper, we leverage a combination of band-pass graph filters to overcome this problem and develop a simple yet effective framework to show how to learn multiple filters depending on datasets.
Revised: In this paper, we leverage a combination of band-pass graph filters to overcome this problem and develop a simple yet effective framework to show how to learn multiple filters depending on datasets.

Original: We empirically validate our theoretical analysis and investigate structure and feature information of benchmarks.
Revised: We empirically validate our theoretical analysis and investigate structure and feature information of benchmarks.

Original: We verify our model on a variety of datasets and explain the behavior of baselines and our model.
Revised: We verify our model on a variety of datasets and explain the behavior of baselines and our model.

Original: Experimental results show that our model achieves a consistent and significant performance improvement across all benchmarks.
Revised: Experimental results show that our model achieves a consistent and significant performance improvement across all benchmarks.

Original: Our main contributions are: 1.We develop a theoretical analysis of graph information based on the introduction of interaction probability and frequency distribution ; 2.We provide a deep understanding of the performance of graph filters illustrating how graph structure and input information matter; 3.We indicate the weakness of GNNs with a single graph filter and propose a general framework to learn a data-specified filter bank which contributes to significant improvement.
Revised: Our main contributions are: 1.We develop a theoretical analysis of graph information based on the introduction of interaction probability and frequency distribution ; 2.We provide a deep understanding of the performance of graph filters illustrating how graph structure and input information matter; 3.We indicate the weakness of GNNs with a single graph filter and propose a general framework to learn a data-specified filter bank which contributes to significant improvement.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: In this paper, we focus on the analysis of graph filters in the context of graph neural networks.
Revised: In this paper, we focus on the analysis of graph filters in the context of graph neural networks.

Original: Since Bruna et al.
Revised: Since Bruna et al.

Original: (2014) defined spectral graph filters and extended convolutional operations to graphs, various spectral graph neural networks have been developed.
Revised: (2014) defined spectral graph filters and extended convolutional operations to graphs, various spectral graph neural networks have been developed.

Original: For example, ChebNet (Defferrard et al., 2016) defines the Chebyshev polynomial filter which can be exactly localized in the k-hop neighborhood.
Revised: For example, ChebNet (Defferrard et al., 2016) defines the Chebyshev polynomial filter which can be exactly localized in the k-hop neighborhood.

Original: Kipf & Welling (2017) simplified the Chebyshev filters using a first-order approximation and derived the well-known graph convolutional networks (GCNs).
Revised: Kipf & Welling (2017) simplified the Chebyshev filters using a first-order approximation and derived the well-known graph convolutional networks (GCNs).

Original: Bianchi et al.
Revised: Bianchi et al.

Original: (2021) proposed the rational auto-regressive moving average graph filters (ARMA) which are more powerful in modeling the localization and provide more flexible graph frequency response, however more computationally expensive and also more unstable.
Revised: (2021) proposed the rational auto-regressive moving average graph filters (ARMA) which are more powerful in modeling the localization and provide more flexible graph frequency response, however more computationally expensive and also more unstable.

Original: Very recently, Min et al.
Revised: Very recently, Min et al.

Original: augmented conventional GCNs with geometric scattering transforms which enabled band-pass filtering of graph signals and alleviated the oversmoothing issue.
Revised: augmented conventional GCNs with geometric scattering transforms which enabled band-pass filtering of graph signals and alleviated the oversmoothing issue.

Original: In addition, most graph neural networks originally defined in the spatial domain are also found essentially connected to the spectral filtering (Balcilar et al., 2020).
Revised: In addition, most graph neural networks originally defined in the spatial domain are also found essentially connected to the spectral filtering (Balcilar et al., 2020).

Original: By bridging the gap between spatial and spectral graph neural networks, Balcilar et al.
Revised: By bridging the gap between spatial and spectral graph neural networks, Balcilar et al.

Original: (2020) further investigated the expressiveness of all graph neural networks from their spectral analysis.
Revised: (2020) further investigated the expressiveness of all graph neural networks from their spectral analysis.

Original: However, their analysis is limited to the spectrum coverage of a graph filter itself and lacks deeper insights into the graph-dependent performance of these filters.
Revised: However, their analysis is limited to the spectrum coverage of a graph filter itself and lacks deeper insights into the graph-dependent performance of these filters.

Original: Another related topic is the measurement of graph homophily.
Revised: Another related topic is the measurement of graph homophily.

Original: Beyond the interaction probability that we define in this paper, there are some other heuristic metrics for homophily.
Revised: Beyond the interaction probability that we define in this paper, there are some other heuristic metrics for homophily.

Original: Pei et al.
Revised: Pei et al.

Original: (2020) defined a node homophily index to characterize their datasets and help explain their experimental results for Geom_GCN
Revised: (2020) defined a node homophily index to characterize their datasets and help explain their experimental results for Geom_GCN

Original: [Equation]
Revised: [Equation]

Original: Zhu et al.
Revised: Zhu et al.

Original: (2020) defined edge homophily ratio instead and identified a set of key designs that can boost learning from the graph structure in heterophily: h
Revised: (2020) defined edge homophily ratio instead and identified a set of key designs that can boost learning from the graph structure in heterophily: h

Original: = # edges whose end nodes have same labels # edge .
Revised: = # edges whose end nodes have same labels # edge .

Original: This edge homophily definition is sensitive to the number of classes and size of each class, and Lim et al.
Revised: This edge homophily definition is sensitive to the number of classes and size of each class, and Lim et al.

Original: (2021) made a modification to alleviate this problem.
Revised: (2021) made a modification to alleviate this problem.

Original: Our work differentiates from these works in that we not only use our definition to characterize the graph but also directly relate it to the performance of graph filters (or GNNs).
Revised: Our work differentiates from these works in that we not only use our definition to characterize the graph but also directly relate it to the performance of graph filters (or GNNs).

Original:  T HEORETICAL A NALYSIS OF G RAPH I NFORMATION 3. N OTATION
Revised:  T HEORETICAL A NALYSIS OF G RAPH I NFORMATION 3. N OTATION

Original: Let G n =
Revised: Let G n =

Original: ( V n , E n ) be an undirected graph with additional self-connection, where V n = { v 0 , . .
Revised: ( V n , E n ) be an undirected graph with additional self-connection, where V n = { v 0 , . .

Original: . , v n − 1 } is the set of nodes and E n ⊂ V n × V n is the set of edges.
Revised: . , v n − 1 } is the set of nodes and E n ⊂ V n × V n is the set of edges.

Original: Let A ∈ R n × n be the adjacency matrix and L = D − A be the Laplacian matrix, where D is a diagonal degree matrix with D ii =
Revised: Let A ∈ R n × n be the adjacency matrix and L = D − A be the Laplacian matrix, where D is a diagonal degree matrix with D ii =

Original: (cid:80)
Revised: (cid:80)

Original: j
Revised: j

Original: A ij .
Revised: A ij .

Original: we denote ˜
Revised: we denote ˜

Original: A
Revised: A

Original: = D − 12 AD − 12 , then ˜ L
Revised: = D − 12 AD − 12 , then ˜ L

Original: = D − 12 LD − 12 =
Revised: = D − 12 LD − 12 =

Original: I − ˜
Revised: I − ˜

Original: A is the symmetric normalized Laplacian.
Revised: A is the symmetric normalized Laplacian.

Original: Let ( λ i , u i ) be a pair of eigenvalue and unit eigenvector of ˜ L , where
Revised: Let ( λ i , u i ) be a pair of eigenvalue and unit eigenvector of ˜ L , where

Original: 0 = λ 0 ≤ · · · ≤ λ n − 1 ≤ 2 .
Revised: 0 = λ 0 ≤ · · · ≤ λ n − 1 ≤ 2 .

Original: 3. P ROBLEM S ETTING
Revised: 3. P ROBLEM S ETTING

Original: In this paper, we are mainly interested in node classification problems on undirected graphs.
Revised: In this paper, we are mainly interested in node classification problems on undirected graphs.

Original: Given
Revised: Given

Original: G n =
Revised: G n =

Original: ( V n , E n ) , we consider T = { 0 , . .
Revised: ( V n , E n ) , we consider T = { 0 , . .

Original: , K − 1 } as the set of all node labels.
Revised: , K − 1 } as the set of all node labels.

Original: For ∀ k ∈ T , we denote
Revised: For ∀ k ∈ T , we denote

Original: C k as the set of nodes with label k and R ∈
Revised: C k as the set of nodes with label k and R ∈

Original: R K × K as a size matrix which is a diagonal matrix with
Revised: R K × K as a size matrix which is a diagonal matrix with

Original: R k =
Revised: R k =

Original: |C k | .
Revised: |C k | .

Original: Considering single-label problems in which classes are mutually exclusive, we use onehot encoding to indicate the class label and introduce a label matrix Y ∈ R n × K =
Revised: Considering single-label problems in which classes are mutually exclusive, we use onehot encoding to indicate the class label and introduce a label matrix Y ∈ R n × K =

Original: ( y 0 , . .
Revised: ( y 0 , . .

Original: , y K − 1 ) to represent the labels of V n , where y k is the indicator vector of C k .
Revised: , y K − 1 ) to represent the labels of V n , where y k is the indicator vector of C k .

Original: Obviously, R =
Revised: Obviously, R =

Original: Y ⊤ Y ,
Revised: Y ⊤ Y ,

Original: Y ⊤ 1 = diag
Revised: Y ⊤ 1 = diag

Original: ( R ) and Y 1 = 1 .
Revised: ( R ) and Y 1 = 1 .

Original: A signal x on G n can be arranged the signal values in a vector form x = ( x 0 , . . .
Revised: A signal x on G n can be arranged the signal values in a vector form x = ( x 0 , . . .

Original: , x n − 1 ) ⊤ .
Revised: , x n − 1 ) ⊤ .

Original: Particularly, labels { y k | k ∈ T } are also graph signals.
Revised: Particularly, labels { y k | k ∈ T } are also graph signals.

Original: 3. A S TRUCTURE I NDICATOR - I NTERACTION P ROBABILITY
Revised: 3. A S TRUCTURE I NDICATOR - I NTERACTION P ROBABILITY

Original: Homophily of graphs is an implicit assumption widely leveraged in graph learning methods including
Revised: Homophily of graphs is an implicit assumption widely leveraged in graph learning methods including

Original: GNNs.
Revised: GNNs.

Original: It is considered an indisputable common property of most graphs, despite its descriptive and unquantifiable definition, which introduces a variety of uncertainties.
Revised: It is considered an indisputable common property of most graphs, despite its descriptive and unquantifiable definition, which introduces a variety of uncertainties.

Original: In this section, starting with the random walk, we introduce interaction probability to overcome this challenge.
Revised: In this section, starting with the random walk, we introduce interaction probability to overcome this challenge.

Original: For a random walk on G n , we denote P =
Revised: For a random walk on G n , we denote P =

Original: D − 1 A as its transition matrix which is also a row Markov matrix.
Revised: D − 1 A as its transition matrix which is also a row Markov matrix.

Original: From the random walk theory, P k is the k -step transition matrix, and P kij is the probability that a random walker starting from node v
Revised: From the random walk theory, P k is the k -step transition matrix, and P kij is the probability that a random walker starting from node v

Original: i arrives at v j after k steps.
Revised: i arrives at v j after k steps.

Original: For a node v and a class C l , we denote π ki ( C l ) as the probability that a random walker starting from v i stays in C l at the k -th step.
Revised: For a node v and a class C l , we denote π ki ( C l ) as the probability that a random walker starting from v i stays in C l at the k -th step.

Original: It is trivial that π ki ( C l ) =
Revised: It is trivial that π ki ( C l ) =

Original: (cid:80) j ∈C l P kij with (cid:80) l ∈T π ki ( C l ) = 1 .
Revised: (cid:80) j ∈C l P kij with (cid:80) l ∈T π ki ( C l ) = 1 .

Original: π ki ( C l ) demonstrates the relative preference/closeness of node v i for C l with k -scale.
Revised: π ki ( C l ) demonstrates the relative preference/closeness of node v i for C l with k -scale.

Original: To meet the homophily assumption, for v i in C l , π k i ( C l ) is expected to gap away from others.
Revised: To meet the homophily assumption, for v i in C l , π k i ( C l ) is expected to gap away from others.

Original: Since π k i ( C l ) − (cid:80) m ̸
Revised: Since π k i ( C l ) − (cid:80) m ̸

Original: = l π k i ( C m ) =
Revised: = l π k i ( C m ) =

Original: 2 π k i ( C l ) − 1 , π k i ( C l ) can be regarded as a measure of the k -scale homophily degree of node v i .
Revised: 2 π k i ( C l ) − 1 , π k i ( C l ) can be regarded as a measure of the k -scale homophily degree of node v i .

Original: Particularly, for ∀ k ∈ N and v i ∈ C l , π ki ( C l ) = 1 means that C l is a community and will never communicate with other classes.
Revised: Particularly, for ∀ k ∈ N and v i ∈ C l , π ki ( C l ) = 1 means that C l is a community and will never communicate with other classes.

Original: However, this case is rare in real graphs.
Revised: However, this case is rare in real graphs.

Original: Below, we investigate the homophily of a class and propose a method to measure the communication strength between two classes.
Revised: Below, we investigate the homophily of a class and propose a method to measure the communication strength between two classes.

Original: Definition 3.1 ( k -step interaction probability) .
Revised: Definition 3.1 ( k -step interaction probability) .

Original: For l, m ∈ T , we define Π k as the k -step interaction probability matrix formulated as follows:
Revised: For l, m ∈ T , we define Π k as the k -step interaction probability matrix formulated as follows:

Original: [Equation]
Revised: [Equation]

Original: Π klm is the probability that a random walker from C l arrives at C m after k steps.
Revised: Π klm is the probability that a random walker from C l arrives at C m after k steps.

Original: Remark 1.
Revised: Remark 1.

Original: Obviously, Π k 1 = 1 .
Revised: Obviously, Π k 1 = 1 .

Original: Π klm is the mean proportion of C m in the k -hop neighbors of nodes from C l .
Revised: Π klm is the mean proportion of C m in the k -hop neighbors of nodes from C l .

Original: Noting that rank ( Y )
Revised: Noting that rank ( Y )

Original: =
Revised: =

Original: K , when K ̸ = n , Y R − 1 Y ⊤ ̸
Revised: K , when K ̸ = n , Y R − 1 Y ⊤ ̸

Original: = I , thus ( R − 1 Y ⊤ PY )
Revised: = I , thus ( R − 1 Y ⊤ PY )

Original: k
Revised: k

Original: ̸
Revised: ̸

Original: = R − 1 Y ⊤ P k Y , i.e. (Π) m ̸
Revised: = R − 1 Y ⊤ P k Y , i.e. (Π) m ̸

Original: = Π m .
Revised: = Π m .

Original: More generally, for an arbitrary polynomial function g , R − 1 Y ⊤ g ( P ) Y is likely not equal to g ( R − 1 Y ⊤ P k Y ) .
Revised: More generally, for an arbitrary polynomial function g , R − 1 Y ⊤ g ( P ) Y is likely not equal to g ( R − 1 Y ⊤ P k Y ) .

Original: In the rest of paper, we write ˜ g
Revised: In the rest of paper, we write ˜ g

Original: (Π) =
Revised: (Π) =

Original: R − 1 Y ⊤ g ( P ) Y and g (Π)
Revised: R − 1 Y ⊤ g ( P ) Y and g (Π)

Original: =
Revised: =

Original: g ( R − 1 Y ⊤ P k Y ) .
Revised: g ( R − 1 Y ⊤ P k Y ) .

Original: For instance, if g ( · ) = ( · ) m , then ˜ g (Π) =
Revised: For instance, if g ( · ) = ( · ) m , then ˜ g (Π) =

Original: Π m and g (Π)
Revised: Π m and g (Π)

Original: =
Revised: =

Original: (Π) m .
Revised: (Π) m .

Original: Also, we denote Π kll , the self-interaction probability , as π kl for short.
Revised: Also, we denote Π kll , the self-interaction probability , as π kl for short.

Original: 1 -step interaction probability intuitively reflects the degree of clustering of two classes and (cid:80)
Revised: 1 -step interaction probability intuitively reflects the degree of clustering of two classes and (cid:80)

Original: ki
Revised: ki

Original: =1 Π i measures the strength of interaction between classes in the scale of k steps.
Revised: =1 Π i measures the strength of interaction between classes in the scale of k steps.

Original: Since P is not symmetric, Π klm ̸ = Π kml .
Revised: Since P is not symmetric, Π klm ̸ = Π kml .

Original: To facilitate analysis, here we propose a symmetric variant of interaction probability to identify the interactions between two classes.
Revised: To facilitate analysis, here we propose a symmetric variant of interaction probability to identify the interactions between two classes.

Original: We denote this symmetric k -step interaction probability matrix as ˜Π k , by replacing P with ˜
Revised: We denote this symmetric k -step interaction probability matrix as ˜Π k , by replacing P with ˜

Original: A
Revised: A

Original: = D − 12 AD − 12 , we obtain ˜Π k = R − 12 Y ⊤ ˜
Revised: = D − 12 AD − 12 , we obtain ˜Π k = R − 12 Y ⊤ ˜

Original: A k Y R − 12 .
Revised: A k Y R − 12 .

Original: Below, we investigate the important properties of Π k and ˜Π k .
Revised: Below, we investigate the important properties of Π k and ˜Π k .

Original: Proposition 3.1.
Revised: Proposition 3.1.

Original: For l, m ∈ T and an arbitrary polynomial function g ( · ) , we have:
Revised: For l, m ∈ T and an arbitrary polynomial function g ( · ) , we have:

Original: a. R l Π klm + R m
Revised: a. R l Π klm + R m

Original: Π kml ≥ 2 √ R l R m ˜Π klm , where R l is the l -th diagonal element of R ; b. (˜ g 2 (˜Π)) ll ≥ (˜ g (˜Π) ll ) 2 , where ˜ g k (˜Π)
Revised: Π kml ≥ 2 √ R l R m ˜Π klm , where R l is the l -th diagonal element of R ; b. (˜ g 2 (˜Π)) ll ≥ (˜ g (˜Π) ll ) 2 , where ˜ g k (˜Π)

Original: =
Revised: =

Original: R − 12 Y ⊤ g k ( ˜ A ) Y R − 12 .
Revised: R − 12 Y ⊤ g k ( ˜ A ) Y R − 12 .

Original: The proof can be found in Appendix B. Since ˜Π k 1 ̸ = 1 , that is, the measure is no longer a probability measure.
Revised: The proof can be found in Appendix B. Since ˜Π k 1 ̸ = 1 , that is, the measure is no longer a probability measure.

Original: However, according to Prop.3.1.a ( let m = l ), ˜ π kl is the lower bound of π kl , and ˜ π kl =
Revised: However, according to Prop.3.1.a ( let m = l ), ˜ π kl is the lower bound of π kl , and ˜ π kl =

Original: π kl when G n is a regular graph.
Revised: π kl when G n is a regular graph.

Original: In the rest of theoretical analysis, we use ˜ π kl to measure the degree of
Revised: In the rest of theoretical analysis, we use ˜ π kl to measure the degree of

Original: C l ’s clustering.
Revised: C l ’s clustering.

Original: Let g ( · ) =
Revised: Let g ( · ) =

Original: ( · ) k , from Prop.3.1.b, we have ˜ π 2 k l ≥ (˜ π k l ) 2 .
Revised: ( · ) k , from Prop.3.1.b, we have ˜ π 2 k l ≥ (˜ π k l ) 2 .

Original: In Section.4.2, we leverage this inequality to derive a lower bound of our prediction error and further illustrate how structure influences the performance of a given filter.
Revised: In Section.4.2, we leverage this inequality to derive a lower bound of our prediction error and further illustrate how structure influences the performance of a given filter.

Original: 3. A F EATURE I NDICATOR - F REQUENCY D ISTRIBUTION
Revised: 3. A F EATURE I NDICATOR - F REQUENCY D ISTRIBUTION

Original: Following the graph signal processing (GSP) concepts, λ 0 , . . .
Revised: Following the graph signal processing (GSP) concepts, λ 0 , . . .

Original: , λ n − 1 are graph frequencies and u 0 . .
Revised: , λ n − 1 are graph frequencies and u 0 . .

Original: , u n − 1 are the corresponding frequency components which are invariant of graph filters.
Revised: , u n − 1 are the corresponding frequency components which are invariant of graph filters.

Original: Through Fourier transform, we obtain { α i = ⟨ u i , x ⟩| i = 0 , . . . , n − 1 } the spectral representation of a graph signal x , called graph signal spectrum.
Revised: Through Fourier transform, we obtain { α i = ⟨ u i , x ⟩| i = 0 , . . . , n − 1 } the spectral representation of a graph signal x , called graph signal spectrum.

Original: Moreover, a graph signal can be represented as a linear combination of frequency components, i.e., x = (cid:80) α
Revised: Moreover, a graph signal can be represented as a linear combination of frequency components, i.e., x = (cid:80) α

Original: i u i .
Revised: i u i .

Original: For a label vector y l which is also a graph signal, we denote { γ 0 , . . .
Revised: For a label vector y l which is also a graph signal, we denote { γ 0 , . . .

Original: , γ n − 1 } as its spectrum.
Revised: , γ n − 1 } as its spectrum.

Original: There is an intuitive assumption: information of label vectors is all we need for classification - we will validate this assumption in Section 4.1.
Revised: There is an intuitive assumption: information of label vectors is all we need for classification - we will validate this assumption in Section 4.1.

Original: Under this context, γ 2 i / (cid:80) i γ 2 i reflects how much the frequency component u k contributes to the distinctiveness of C l , without considering the positivity and negativity of effects.
Revised: Under this context, γ 2 i / (cid:80) i γ 2 i reflects how much the frequency component u k contributes to the distinctiveness of C l , without considering the positivity and negativity of effects.

Original: Interestingly, we find that the normalized signal spectrum is a histogram/discrete distribution defined below.
Revised: Interestingly, we find that the normalized signal spectrum is a histogram/discrete distribution defined below.

Original: Definition 3.2 (Frequency distribution) .
Revised: Definition 3.2 (Frequency distribution) .

Original: We define f , the frequency of signal x , as a random variable taking values in the set of graph frequencies with probability Pr( f = λ k )
Revised: We define f , the frequency of signal x , as a random variable taking values in the set of graph frequencies with probability Pr( f = λ k )

Original: =
Revised: =

Original: α 2 k (cid:14) (cid:80) i α 2 i .
Revised: α 2 k (cid:14) (cid:80) i α 2 i .

Original: The probability describes the frequency distribution of signal x .
Revised: The probability describes the frequency distribution of signal x .

Original: With this definition, we derive distributional representations of signals from their spectral representations/spectra.
Revised: With this definition, we derive distributional representations of signals from their spectral representations/spectra.

Original: One can evaluate the signal effect by comparing frequency distributions of signals and label vectors under a specified distribution metric, such as Wasserstein distance.
Revised: One can evaluate the signal effect by comparing frequency distributions of signals and label vectors under a specified distribution metric, such as Wasserstein distance.

Original: Below, we consider the moment of frequency distribution to show how graph structure influences signal frequency.
Revised: Below, we consider the moment of frequency distribution to show how graph structure influences signal frequency.

Original: Proposition 3.2.
Revised: Proposition 3.2.

Original: For G = {V , E} , let f be the frequency of signal x , then E [ f n ] = x ⊤ ( I − ˜ A ) n x x ⊤ x .
Revised: For G = {V , E} , let f be the frequency of signal x , then E [ f n ] = x ⊤ ( I − ˜ A ) n x x ⊤ x .

Original: The proof of this proposition can be found in Appendix B. With the definition of interaction probability, we further represent the moment of the label vector’s frequency.
Revised: The proof of this proposition can be found in Appendix B. With the definition of interaction probability, we further represent the moment of the label vector’s frequency.

Original: Corollary 3.3.
Revised: Corollary 3.3.

Original: For label frequency f l of y l , we have E [ f nl ] =
Revised: For label frequency f l of y l , we have E [ f nl ] =

Original: (cid:0) ˜ g
Revised: (cid:0) ˜ g

Original: ( I − ˜Π) (cid:1) ll with g = ( · ) n .
Revised: ( I − ˜Π) (cid:1) ll with g = ( · ) n .

Original: Recall that ˜ g
Revised: Recall that ˜ g

Original: ( I − ˜Π)
Revised: ( I − ˜Π)

Original: =
Revised: =

Original: R − 12 Y ⊤ ( I − ˜ A ) n Y R − 12 , we have E [ f l ] = 1 − ˜ π l , E
Revised: R − 12 Y ⊤ ( I − ˜ A ) n Y R − 12 , we have E [ f l ] = 1 − ˜ π l , E

Original: [ f 2 l ] = 1 − 2˜
Revised: [ f 2 l ] = 1 − 2˜

Original: π l + ˜ π 2 l and the variance of f l : Var ( f l ) =
Revised: π l + ˜ π 2 l and the variance of f l : Var ( f l ) =

Original: ˜ π
Revised: ˜ π

Original: 2 l − (˜ π l ) 2 .
Revised: 2 l − (˜ π l ) 2 .

Original: It can be seen that both the mean and variance of label frequency are close to 0 when ˜ π l approaches 1, which reflects a high homophily degree (as ˜ π l ≤ π l ≤ 1 ).
Revised: It can be seen that both the mean and variance of label frequency are close to 0 when ˜ π l approaches 1, which reflects a high homophily degree (as ˜ π l ≤ π l ≤ 1 ).

Original: In Section 4.1, we conduct a more detailed analysis of feature information of spectral space with frequency distribution.
Revised: In Section 4.1, we conduct a more detailed analysis of feature information of spectral space with frequency distribution.

Original:  A NALYSIS OF G RAPH F ILTERS
Revised:  A NALYSIS OF G RAPH F ILTERS

Original: A graph filter is defined as a function g with applied Laplacian matrix or adjacency matrix.
Revised: A graph filter is defined as a function g with applied Laplacian matrix or adjacency matrix.

Original: Denote
Revised: Denote

Original: R [ ˜ A ] as a polynomial ring in ˜ A over R , here we are mainly interested in g ∈ R [ ˜ A ] .
Revised: R [ ˜ A ] as a polynomial ring in ˜ A over R , here we are mainly interested in g ∈ R [ ˜ A ] .

Original: In this section, we provide a deep understanding of the performance of graph filters concerning label prediction based on the above theoretical analysis of graph information.
Revised: In this section, we provide a deep understanding of the performance of graph filters concerning label prediction based on the above theoretical analysis of graph information.

Original: In general, there are two major concerns: with fixed graph structure, how does the input impact the performance of a given filter?
Revised: In general, there are two major concerns: with fixed graph structure, how does the input impact the performance of a given filter?

Original: and with fixed input, how does graph structure impact the performance of a given filter? .
Revised: and with fixed input, how does graph structure impact the performance of a given filter? .

Original: In this section, we provide the theoretical analysis of these two questions in Sections 4.1 and 4.2, respectively.
Revised: In this section, we provide the theoretical analysis of these two questions in Sections 4.1 and 4.2, respectively.

Original: The general formulation of the l + 1 -th layer of spectral GNNs is X ( l +1)
Revised: The general formulation of the l + 1 -th layer of spectral GNNs is X ( l +1)

Original: =
Revised: =

Original: σ ( g ( ˜ A )
Revised: σ ( g ( ˜ A )

Original: X
Revised: X

Original: ( l ) W ( l +1) ) , here σ is an activation function, X ( l ) is the output of the l -th layer, X (0) is a feature matrix and
Revised: ( l ) W ( l +1) ) , here σ is an activation function, X ( l ) is the output of the l -th layer, X (0) is a feature matrix and

Original: W ( l +1) is a learnable transformation matrix.
Revised: W ( l +1) is a learnable transformation matrix.

Original: We call X ( l ) W ( l +1) the input of g ( ˜ A ) in l + 1 -th layer and denote X as the input of g ( ˜ A ) in the last layer.
Revised: We call X ( l ) W ( l +1) the input of g ( ˜ A ) in l + 1 -th layer and denote X as the input of g ( ˜ A ) in the last layer.

Original: In the following sections, we discuss the prediction error of spectral GNNs with a given graph filter without activation function before prediction.
Revised: In the following sections, we discuss the prediction error of spectral GNNs with a given graph filter without activation function before prediction.

Original: That is, in the last layer with X as input, g ( ˜ A ) X is directly used for prediction.
Revised: That is, in the last layer with X as input, g ( ˜ A ) X is directly used for prediction.

Original: Definition 4.1 (Prediction error) .
Revised: Definition 4.1 (Prediction error) .

Original: Let X ∈ R n × K be the input of graph filter g ( ˜ A ) , Y ∈ R n × K is the label matrix, the prediction error is formulated by:
Revised: Let X ∈ R n × K be the input of graph filter g ( ˜ A ) , Y ∈ R n × K is the label matrix, the prediction error is formulated by:

Original: [Equation]
Revised: [Equation]

Original: Remark 2.
Revised: Remark 2.

Original: For a label vector y l , we denote Er ( g, x l ) =
Revised: For a label vector y l , we denote Er ( g, x l ) =

Original: ∥ g ( ˜ A ) x l − y l ∥ 2 F as the error of g ( ˜ A ) predicting class l .
Revised: ∥ g ( ˜ A ) x l − y l ∥ 2 F as the error of g ( ˜ A ) predicting class l .

Original: Obviously, Er ( g, X ) =
Revised: Obviously, Er ( g, X ) =

Original: (cid:80)
Revised: (cid:80)

Original: l ∈T
Revised: l ∈T

Original: Er ( g, x l ) , where x l is the l -th column of X .
Revised: Er ( g, x l ) , where x l is the l -th column of X .

Original: In particular, we will apply our conclusion to specified filters and make concrete analysis.
Revised: In particular, we will apply our conclusion to specified filters and make concrete analysis.

Original: Definition 4.2.
Revised: Definition 4.2.

Original: With ϵ ∈ [0 , ϵ 0 ] and ϵ ′ ∈ [ − 1 , 1] , ϵ 0 is a small constant, we define low-pass filters g l ( ϵ ) ( ˜ A ) , high-pass filters g h ( ϵ ) ( ˜ A ) and band-pass filters g b ( ϵ ′ )
Revised: With ϵ ∈ [0 , ϵ 0 ] and ϵ ′ ∈ [ − 1 , 1] , ϵ 0 is a small constant, we define low-pass filters g l ( ϵ ) ( ˜ A ) , high-pass filters g h ( ϵ ) ( ˜ A ) and band-pass filters g b ( ϵ ′ )

Original: ( ˜ A ) as:
Revised: ( ˜ A ) as:

Original: [Equation]
Revised: [Equation]

Original: For λ , an eigenvalue of ˜ L , we have g l ( ϵ ) ( λ ) ∈
Revised: For λ , an eigenvalue of ˜ L , we have g l ( ϵ ) ( λ ) ∈

Original: [ ϵ − 1 , 1 + ϵ ] , g h ( ϵ ) ( λ ) ∈
Revised: [ ϵ − 1 , 1 + ϵ ] , g h ( ϵ ) ( λ ) ∈

Original: [ ϵ − 1 , 1 + ϵ ] and g b ( ϵ ′ )
Revised: [ ϵ − 1 , 1 + ϵ ] and g b ( ϵ ′ )

Original: ( λ ) ∈ [0 , 1] since λ ∈ [0 , 2] .
Revised: ( λ ) ∈ [0 , 1] since λ ∈ [0 , 2] .

Original: Particularly, g l (0) is the GCN filter.
Revised: Particularly, g l (0) is the GCN filter.

Original: 4. H OW I NPUT M ATTERS
Revised: 4. H OW I NPUT M ATTERS

Original: Denote ˜ X
Revised: Denote ˜ X

Original: = U ⊤ X
Revised: = U ⊤ X

Original: =
Revised: =

Original: (˜ x 0 , . . .
Revised: (˜ x 0 , . . .

Original: , ˜ x K − 1 ) and ˜ Y =
Revised: , ˜ x K − 1 ) and ˜ Y =

Original: U ⊤ Y =
Revised: U ⊤ Y =

Original: (˜ y 0 , . .
Revised: (˜ y 0 , . .

Original: ˜ y K − 1 ) , where U is a matrix with unit eigenvectors of ˜ L (recall that eigenvectors of ˜ A are consistent with that of ˜ L ), revisiting
Revised: ˜ y K − 1 ) , where U is a matrix with unit eigenvectors of ˜ L (recall that eigenvectors of ˜ A are consistent with that of ˜ L ), revisiting

Original: Er ( g, x l ) and Er ( g, y l ) in spectral domain, we have:
Revised: Er ( g, x l ) and Er ( g, y l ) in spectral domain, we have:

Original: [Equation]
Revised: [Equation]

Original: where Λ is the eigenvalue matrix of ˜ L , α i and γ i are the spectra of x l and y l respectively, p i =
Revised: where Λ is the eigenvalue matrix of ˜ L , α i and γ i are the spectra of x l and y l respectively, p i =

Original: Pr( f l = λ i ) , f l is the frequency of y l .
Revised: Pr( f l = λ i ) , f l is the frequency of y l .

Original: For better comparison, we normalize the input x l : ∥ x l ∥ 2 F = ∥
Revised: For better comparison, we normalize the input x l : ∥ x l ∥ 2 F = ∥

Original: y l ∥ 2 F , i.e., (cid:80)
Revised: y l ∥ 2 F , i.e., (cid:80)

Original: α 2 i = (cid:80) γ 2 i .
Revised: α 2 i = (cid:80) γ 2 i .

Original: g is re-scaled function with g ([0 , 2]) concentrating in [ − 1 , 1] .
Revised: g is re-scaled function with g ([0 , 2]) concentrating in [ − 1 , 1] .

Original: How input information matter?
Revised: How input information matter?

Original: With normalized feature and graph filters, it indicates that the performance of graph filters greatly depends on label spectra.
Revised: With normalized feature and graph filters, it indicates that the performance of graph filters greatly depends on label spectra.

Original: Particularly, when the frequency response of a graph filter does not fit the label frequency, it might be inferior to all-pass filters, such as MLP.
Revised: Particularly, when the frequency response of a graph filter does not fit the label frequency, it might be inferior to all-pass filters, such as MLP.

Original: On the other hand, it poses a principle of filter design: make feature response of filters be consistent with the main frequency band of label frequency as much as possible .
Revised: On the other hand, it poses a principle of filter design: make feature response of filters be consistent with the main frequency band of label frequency as much as possible .

Original: In terms of input information, it determines the performance of a filter - if the frequency distribution of input vector is far from that of label vector, even an ideal filter would fail .
Revised: In terms of input information, it determines the performance of a filter - if the frequency distribution of input vector is far from that of label vector, even an ideal filter would fail .

Original: This observation is identical to our assumption in Section 3.4 - information of label vector is all we need and the distance between frequency distribution of input and label vectors reflects its usefulness.
Revised: This observation is identical to our assumption in Section 3.4 - information of label vector is all we need and the distance between frequency distribution of input and label vectors reflects its usefulness.

Original: Therefore, Er ( g, y l ) is the lower bound of Er ( g, x l ) when g ( ˜ A ) are given.
Revised: Therefore, Er ( g, y l ) is the lower bound of Er ( g, x l ) when g ( ˜ A ) are given.

Original: While an input vector may be useful for distinguishing one class, it may be helpless for another.
Revised: While an input vector may be useful for distinguishing one class, it may be helpless for another.

Original: In most GNNs, they tune the frequency distribution of features with a learnable linear transformation to generate a more informative input.
Revised: In most GNNs, they tune the frequency distribution of features with a learnable linear transformation to generate a more informative input.

Original: Here, we discuss the Er ( g, y l ) of three types of filters:
Revised: Here, we discuss the Er ( g, y l ) of three types of filters:

Original: [Equation]
Revised: [Equation]

Original: Er ( g b ( ϵ ′ ) , y l ) /R
Revised: Er ( g b ( ϵ ′ ) , y l ) /R

Original: l ≈ ( E [ f l ] + ϵ ′ − 1) 4 + 6 Var ( f l )( E [ f l ] + ϵ ′ − 1) 2 + 8(
Revised: l ≈ ( E [ f l ] + ϵ ′ − 1) 4 + 6 Var ( f l )( E [ f l ] + ϵ ′ − 1) 2 + 8(

Original: [Equation]
Revised: [Equation]

Original: − ϵ ′ )
Revised: − ϵ ′ )

Original: Var ( f l
Revised: Var ( f l

Original: [Equation]
Revised: [Equation]

Original: E [ f l ] (1 + |
Revised: E [ f l ] (1 + |

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: (8) where we use Var ( f 2 l ) ≈ 4 E [ f l ] 2 Var ( f l ) derived from the delta method.
Revised: (8) where we use Var ( f 2 l ) ≈ 4 E [ f l ] 2 Var ( f l ) derived from the delta method.

Original: Discussion.
Revised: Discussion.

Original: An interesting observation is that for a class with high dispersive spectrum, efforts of any single filters are to no avail .
Revised: An interesting observation is that for a class with high dispersive spectrum, efforts of any single filters are to no avail .

Original: From Corollary 3.3, we know that E [ f l ] = 1 − ˜ π l and Var ( f l ) =
Revised: From Corollary 3.3, we know that E [ f l ] = 1 − ˜ π l and Var ( f l ) =

Original: ˜ π
Revised: ˜ π

Original: 2 l − (˜ π l ) 2 .
Revised: 2 l − (˜ π l ) 2 .

Original: It demonstrates that higher homophily means lower E
Revised: It demonstrates that higher homophily means lower E

Original: [ f l ] , lower Var ( f l ) , and also lower prediction error for low-pass filters.
Revised: [ f l ] , lower Var ( f l ) , and also lower prediction error for low-pass filters.

Original: On the other hand, we indicate that, in most cases, band-pass filters are more powerful than low-pass filters, let alone high-pass filters.
Revised: On the other hand, we indicate that, in most cases, band-pass filters are more powerful than low-pass filters, let alone high-pass filters.

Original: However, the prediction capacity of a signal filter is very limited when the means of spectra vary widely .
Revised: However, the prediction capacity of a signal filter is very limited when the means of spectra vary widely .

Original: 4. H OW S TRUCTURE M ATTERS
Revised: 4. H OW S TRUCTURE M ATTERS

Original: Above, we catch a glimpse of spectral explanation of the behavior of graph filters.
Revised: Above, we catch a glimpse of spectral explanation of the behavior of graph filters.

Original: Below, we expand more understanding of graph filters.
Revised: Below, we expand more understanding of graph filters.

Original: Assume that with learnable transformation, GNNs enable to generate an informative input.
Revised: Assume that with learnable transformation, GNNs enable to generate an informative input.

Original: Here we discuss the prediction error of different graph filters under the optimal input Y .
Revised: Here we discuss the prediction error of different graph filters under the optimal input Y .

Original: We revisit
Revised: We revisit

Original: Er ( g, y l ) using symmetric interaction matrix and propose a lower bound er ( g, y l ) leveraging Proposition 3.1:
Revised: Er ( g, y l ) using symmetric interaction matrix and propose a lower bound er ( g, y l ) leveraging Proposition 3.1:

Original: [Equation]
Revised: [Equation]

Original: How structural information matters?
Revised: How structural information matters?

Original: We indicate that, in the spatial point of view, graph filters can be interpreted as weight-tuning mechanisms on edges.
Revised: We indicate that, in the spatial point of view, graph filters can be interpreted as weight-tuning mechanisms on edges.

Original: The lower bound clearly demonstrates that a graph filter would have unsatisfactory prediction accuracy if it fails to make the homophily degree of the tuned graph large enough ( g [˜Π] ll are far from 1).
Revised: The lower bound clearly demonstrates that a graph filter would have unsatisfactory prediction accuracy if it fails to make the homophily degree of the tuned graph large enough ( g [˜Π] ll are far from 1).

Original: Applying the prediction error lower bound to aforementioned specified filters, we have:
Revised: Applying the prediction error lower bound to aforementioned specified filters, we have:

Original: [Equation]
Revised: [Equation]

Original: Discussion.
Revised: Discussion.

Original: These error bounds indicate that: 1.
Revised: These error bounds indicate that: 1.

Original: a low-pass filter would fail on classes with low homophily degree - in turn, it confirms that the importance of homophily assumption for low-pass filters like GCN - it is identical with our spectral point of view; 2.
Revised: a low-pass filter would fail on classes with low homophily degree - in turn, it confirms that the importance of homophily assumption for low-pass filters like GCN - it is identical with our spectral point of view; 2.

Original: high-pass filters have poor performances particularly on the high homophily graphs; 3.
Revised: high-pass filters have poor performances particularly on the high homophily graphs; 3.

Original: for a graph whose classes have consistent homophily degree (their self-interaction probabilities concentrate around a constant ¯ ϵ ), g b (¯ ϵ ) would work better than others.
Revised: for a graph whose classes have consistent homophily degree (their self-interaction probabilities concentrate around a constant ¯ ϵ ), g b (¯ ϵ ) would work better than others.

Original: However, it is predictable that any single filters would fail on graphs with diverse self-interaction probabilities .
Revised: However, it is predictable that any single filters would fail on graphs with diverse self-interaction probabilities .

Original:  M ODEL AND E MPIRICAL S TUDY
Revised:  M ODEL AND E MPIRICAL S TUDY

Original: Our theoretical analysis of graph information demonstrates that: 1. when node classes have inconsistent homophily degree or their label frequency distribution are far from each other, a single graph filter is prone to fail; 2.
Revised: Our theoretical analysis of graph information demonstrates that: 1. when node classes have inconsistent homophily degree or their label frequency distribution are far from each other, a single graph filter is prone to fail; 2.

Original: in most cases, band-pass filters would perform better than low-pass and high-pass filters; 3.
Revised: in most cases, band-pass filters would perform better than low-pass and high-pass filters; 3.

Original: a feature may contribute to the classification of one class but hinder the discrimination of another.
Revised: a feature may contribute to the classification of one class but hinder the discrimination of another.

Original: Inspired by these, we propose a disentangled multi band-pass filter framework (DEMUF) which can be applied to any type of graphs no matter what kinds of graph information they have.
Revised: Inspired by these, we propose a disentangled multi band-pass filter framework (DEMUF) which can be applied to any type of graphs no matter what kinds of graph information they have.

Original: The key point of our model is to learn multi band-pass filters which are used to capture different disentangled feature information respectively.
Revised: The key point of our model is to learn multi band-pass filters which are used to capture different disentangled feature information respectively.

Original: 5. A RCHITECTURE OF T WO F RAMEWORKS OF DEMUF
Revised: 5. A RCHITECTURE OF T WO F RAMEWORKS OF DEMUF

Original: Our framework includes feature disentanglement and frequency filtering .
Revised: Our framework includes feature disentanglement and frequency filtering .

Original: As we have emphasized the limitations of single filters, it is natural to leverage multi graph filters.
Revised: As we have emphasized the limitations of single filters, it is natural to leverage multi graph filters.

Original: Theoretically, piling up sufficient numbers of graph filters to capture all the frequency components can improve prediction performance.
Revised: Theoretically, piling up sufficient numbers of graph filters to capture all the frequency components can improve prediction performance.

Original: However, it is very expensive.
Revised: However, it is very expensive.

Original: To avoid this problem, we consider feature disentanglement - essentially, it is to disentangle frequency distributions of features into different families.
Revised: To avoid this problem, we consider feature disentanglement - essentially, it is to disentangle frequency distributions of features into different families.

Original: Features in the same family are expected to have similar spectral properties, that is, they have similar frequency distributions or have overlap on their main frequency bands.
Revised: Features in the same family are expected to have similar spectral properties, that is, they have similar frequency distributions or have overlap on their main frequency bands.

Original: Then for each family, we apply
Revised: Then for each family, we apply

Original: 𝐻 𝑀𝐿𝑃 𝑋 𝐹 (cid:2869) 𝜔 (cid:2869) 𝐻 (cid:2869) 𝜔 (cid:3015) 𝐻 (cid:3015) …
Revised: 𝐻 𝑀𝐿𝑃 𝑋 𝐹 (cid:2869) 𝜔 (cid:2869) 𝐻 (cid:2869) 𝜔 (cid:3015) 𝐻 (cid:3015) …

Original: 𝐹 (cid:2870) 𝐹 (cid:3015)(cid:2879)(cid:2869) 𝐹 (cid:3015)
Revised: 𝐹 (cid:2870) 𝐹 (cid:3015)(cid:2879)(cid:2869) 𝐹 (cid:3015)

Original: 𝐻 (cid:2869) 𝐻 (cid:2870)
Revised: 𝐻 (cid:2869) 𝐻 (cid:2870)

Original: 𝐻 (cid:3015)(cid:2879)(cid:2869) 𝐻 (cid:3015) …
Revised: 𝐻 (cid:3015)(cid:2879)(cid:2869) 𝐻 (cid:3015) …

Original: 𝐷𝐸 𝜔 (cid:2869) 𝐻 (cid:2869) 𝜔 (cid:3015) 𝐻 (cid:3015) …
Revised: 𝐷𝐸 𝜔 (cid:2869) 𝐻 (cid:2869) 𝜔 (cid:3015) 𝐻 (cid:3015) …

Original: 𝑋 𝐻 (cid:2870) 𝐻 (cid:2869) 𝐹 (cid:2869) … 𝐷𝐸 (cid:2871) |𝐹
Revised: 𝑋 𝐻 (cid:2870) 𝐻 (cid:2869) 𝐹 (cid:2869) … 𝐷𝐸 (cid:2871) |𝐹

Original: (cid:2871) 𝐷𝐸 (cid:2870)
Revised: (cid:2871) 𝐷𝐸 (cid:2870)

Original: |𝐹 (cid:2870) 𝑀𝐿𝑃 𝐻 𝐷𝐸 (cid:2869) =
Revised: |𝐹 (cid:2870) 𝑀𝐿𝑃 𝐻 𝐷𝐸 (cid:2869) =

Original: 𝑋 (cid:3036)
Revised: 𝑋 (cid:3036)

Original: [Figure]
Revised: [Figure]

Original: (cid:3284) (cid:4667)
Revised: (cid:3284) (cid:4667)

Original: [Figure]
Revised: [Figure]

Original: a band-pass graph filter to capture their main frequency components.
Revised: a band-pass graph filter to capture their main frequency components.

Original: We propose two frameworks with different structures of filters: Plain-DEMUF and Tree-DEMUF (depicted in Fig. 1).
Revised: We propose two frameworks with different structures of filters: Plain-DEMUF and Tree-DEMUF (depicted in Fig. 1).

Original: The D ISENTANGLE block and F ILTER block are formulated as follows:
Revised: The D ISENTANGLE block and F ILTER block are formulated as follows:

Original: [Equation]
Revised: [Equation]

Original: In our implementation, we provide two samples of D ISENTANGLE functions Φ k : one is linear transformations, the other is GUMBEL _ SOFTMAX (Jang et al., 2017) used to generate learnable masks for feature selection.
Revised: In our implementation, we provide two samples of D ISENTANGLE functions Φ k : one is linear transformations, the other is GUMBEL _ SOFTMAX (Jang et al., 2017) used to generate learnable masks for feature selection.

Original: In terms of the F ILTER block, we use the band-pass filter defined in Definition 4.2, i.e., g b ( ϵ ) =
Revised: In terms of the F ILTER block, we use the band-pass filter defined in Definition 4.2, i.e., g b ( ϵ ) =

Original: I − (1 + | ϵ | ) − 2 ( ˜ A − ϵI ) 2 as the identical filter form.
Revised: I − (1 + | ϵ | ) − 2 ( ˜ A − ϵI ) 2 as the identical filter form.

Original: Here, ϵ is the parameter of filter constrained in [ − 1 , 1] noting that 1 − ϵ is the center of frequency response g b ( ϵ ) .
Revised: Here, ϵ is the parameter of filter constrained in [ − 1 , 1] noting that 1 − ϵ is the center of frequency response g b ( ϵ ) .

Original: In each F ILTER block, h is the number of layers.
Revised: In each F ILTER block, h is the number of layers.

Original: The framework of Plain-DEMUF with N graph filters is:
Revised: The framework of Plain-DEMUF with N graph filters is:

Original: [Equation]
Revised: [Equation]

Original: Based on this, we implement a simple model called P-DEMUF.
Revised: Based on this, we implement a simple model called P-DEMUF.

Original: Precisely, we leverage a GUMBEL _ SOFTMAX to generate N learnable masks { M 1 , . . .
Revised: Precisely, we leverage a GUMBEL _ SOFTMAX to generate N learnable masks { M 1 , . . .

Original: , M N } for feature sampling at once followed by different MLP.
Revised: , M N } for feature sampling at once followed by different MLP.

Original: That is, Φ k ( X ) =
Revised: That is, Φ k ( X ) =

Original: MLP k
Revised: MLP k

Original: ( X ⊙ M k ) .
Revised: ( X ⊙ M k ) .

Original: Similarly, we develop a model, T-DEMUF, under the framework of Tree-DEMUF formulated by:
Revised: Similarly, we develop a model, T-DEMUF, under the framework of Tree-DEMUF formulated by:

Original: [Equation]
Revised: [Equation]

Original: In each T-DEMUF layer, we use GUMBEL _ SOFTMAX with different parameters to generate two masks M k and M ′ k and Φ k
Revised: In each T-DEMUF layer, we use GUMBEL _ SOFTMAX with different parameters to generate two masks M k and M ′ k and Φ k

Original: ( X k ) =
Revised: ( X k ) =

Original: X k ⊙ M k and Ψ k ( X k ) =
Revised: X k ⊙ M k and Ψ k ( X k ) =

Original: X
Revised: X

Original: k ⊙ M ′ k .
Revised: k ⊙ M ′ k .

Original: In each layer, we stop further disentangling of the branch of H k by utilizing an additional constraint L ( X k − 1 , H k ) =
Revised: In each layer, we stop further disentangling of the branch of H k by utilizing an additional constraint L ( X k − 1 , H k ) =

Original: ∥
Revised: ∥

Original: X k − 1 ⊙ M ′ k − H k ∥ 22 .
Revised: X k − 1 ⊙ M ′ k − H k ∥ 22 .

Original: Noting that
Revised: Noting that

Original: H k = ( g b ( ϵ k ) )
Revised: H k = ( g b ( ϵ k ) )

Original: h k X k −
Revised: h k X k −

Original: 1 ⊙ M ′ k , this constraint is to make the main frequency bands of H
Revised: 1 ⊙ M ′ k , this constraint is to make the main frequency bands of H

Original: k be consistent with frequency response of ( g b ( ϵ k ) )
Revised: k be consistent with frequency response of ( g b ( ϵ k ) )

Original: h k .
Revised: h k .

Original: Model discussion.
Revised: Model discussion.

Original: Compared with filter-bank learning methods which directly apply an array of filters to features, our models use subsets of features.
Revised: Compared with filter-bank learning methods which directly apply an array of filters to features, our models use subsets of features.

Original: It can greatly reduce the amount of computation and parameters and help learning filters more efficiently and effectively.
Revised: It can greatly reduce the amount of computation and parameters and help learning filters more efficiently and effectively.

Original: In addition, T-DEMUF uses an additional constraint to guide the filter learning process while P-DEMUF is a combination of multi graph neural networks which would not interfere with each other.
Revised: In addition, T-DEMUF uses an additional constraint to guide the filter learning process while P-DEMUF is a combination of multi graph neural networks which would not interfere with each other.

Original: Therefore, P-DEMUF is likely to obtain similar filters and require more filters to improve performance than T-DEMUF.
Revised: Therefore, P-DEMUF is likely to obtain similar filters and require more filters to improve performance than T-DEMUF.

Original: The model visualization results in Fig.
Revised: The model visualization results in Fig.

Original: 2 validate this statement.
Revised: 2 validate this statement.

Original: 5. E XPERIMENTS
Revised: 5. E XPERIMENTS

Original: To validate DEMUF, we compare the performances of P-DEMUF and T-DEMUF with that of spectral
Revised: To validate DEMUF, we compare the performances of P-DEMUF and T-DEMUF with that of spectral

Original: GNNs, spatial GNNs and MLP on extensive datasets.
Revised: GNNs, spatial GNNs and MLP on extensive datasets.

Original: 5.2. E XPERIMENT S ETTINGS
Revised: 5.2. E XPERIMENT S ETTINGS

Original: Datasets.
Revised: Datasets.

Original: We use four types of real datasets - Citation network , WebKB , Actor co-occurrence network and Wikipedia network , to validate our proposed models.
Revised: We use four types of real datasets - Citation network , WebKB , Actor co-occurrence network and Wikipedia network , to validate our proposed models.

Original: Cora and Citeseer (Sen et al., 2008) are widely used citation benchmarks which represent paper as nodes and citation between two papers as edges.
Revised: Cora and Citeseer (Sen et al., 2008) are widely used citation benchmarks which represent paper as nodes and citation between two papers as edges.

Original: Cornell, Texas, and Wisconsin (Pei et al., 2020) are three subgraphs of WebKB which is a webpage network with web pages as nodes and hyperlinks between them as edges.
Revised: Cornell, Texas, and Wisconsin (Pei et al., 2020) are three subgraphs of WebKB which is a webpage network with web pages as nodes and hyperlinks between them as edges.

Original: Chameleon and Squirrel (Rozemberczki et al., 2021) are two Wikipedia networks with web pages as nodes and links between pages as edges.
Revised: Chameleon and Squirrel (Rozemberczki et al., 2021) are two Wikipedia networks with web pages as nodes and links between pages as edges.

Original: The nodes originally have five classes while Bo et al.
Revised: The nodes originally have five classes while Bo et al.

Original: (2021) proposed a new classification criteria which divides nodes into three main categories.
Revised: (2021) proposed a new classification criteria which divides nodes into three main categories.

Original: In this paper, the relabeled networks are called Chameleon2 and Squirrel2.
Revised: In this paper, the relabeled networks are called Chameleon2 and Squirrel2.

Original: Actor (Tang et al., 2009) is a subgraph of the fillm-director-actor-writer network whose nodes only represent actors and edges represent their collaborations.
Revised: Actor (Tang et al., 2009) is a subgraph of the fillm-director-actor-writer network whose nodes only represent actors and edges represent their collaborations.

Original: For all data, we use 60% nodes for training, 20% for validation and 20% for testing.
Revised: For all data, we use 60% nodes for training, 20% for validation and 20% for testing.

Original: To intuitively show the homophily degree of a dataset, we calculate the mean of self-interaction probability (diagonal of interaction probability matrix) and show it in Table 1.
Revised: To intuitively show the homophily degree of a dataset, we calculate the mean of self-interaction probability (diagonal of interaction probability matrix) and show it in Table 1.

Original: This metric is similar to the node homophily in (Pei et al., 2020).
Revised: This metric is similar to the node homophily in (Pei et al., 2020).

Original: More statistics of datasets can be found in Appendix A.
Revised: More statistics of datasets can be found in Appendix A.

Original: Baselines.
Revised: Baselines.

Original: We compare our models with four spectral GNNs: GCN (Kipf & Welling, 2017),
Revised: We compare our models with four spectral GNNs: GCN (Kipf & Welling, 2017),

Original: ChebNet (Defferrard et al., 2016), GIN (Xu et al., 2019)
Revised: ChebNet (Defferrard et al., 2016), GIN (Xu et al., 2019)

Original: (despite a spatial GNN, we can easily get its spectral form), ARMA (Bianchi et al., 2021).
Revised: (despite a spatial GNN, we can easily get its spectral form), ARMA (Bianchi et al., 2021).

Original: We list their spectral filter forms in Appendix A.
Revised: We list their spectral filter forms in Appendix A.

Original: In short, GCN is a well-known low-pass filter.
Revised: In short, GCN is a well-known low-pass filter.

Original: The filter shape of GIN depends on its parameter ϵ .
Revised: The filter shape of GIN depends on its parameter ϵ .

Original: In this paper, we fix ϵ = 0 .
Revised: In this paper, we fix ϵ = 0 .

Original: and thus it is also a low-pass filter.
Revised: and thus it is also a low-pass filter.

Original: ChebNet and ARMA are high-order polynomial filters.
Revised: ChebNet and ARMA are high-order polynomial filters.

Original: In addition, we also add three spatial GNNs (whose spectral forms are hardly analyzed): GAT (Veliˇckovi´c et al., 2018), FAGCN (Bo et al., 2021), Geom_GCN
Revised: In addition, we also add three spatial GNNs (whose spectral forms are hardly analyzed): GAT (Veliˇckovi´c et al., 2018), FAGCN (Bo et al., 2021), Geom_GCN

Original: (Pei et al., 2020).
Revised: (Pei et al., 2020).

Original: Both GAT and FAGCN utilize attention mechanism and FAGCN takes high frequency information into account.
Revised: Both GAT and FAGCN utilize attention mechanism and FAGCN takes high frequency information into account.

Original: Geom_GCN is a novel aggregation method based on the geometry of graph (it is related because it was also empirically studied on graphs with different levels of homophily (Pei et al., 2020)).
Revised: Geom_GCN is a novel aggregation method based on the geometry of graph (it is related because it was also empirically studied on graphs with different levels of homophily (Pei et al., 2020)).

Original: Finally, we also compare with MLP, a baseline without using any graph information.
Revised: Finally, we also compare with MLP, a baseline without using any graph information.

Original: Experimental Setup.
Revised: Experimental Setup.

Original: For all experiments, we report the mean prediction accuracy on the testing data for 10 runs.
Revised: For all experiments, we report the mean prediction accuracy on the testing data for 10 runs.

Original: We search learning rate, hidden unit, weight decay and dropout for all models in the same search space.
Revised: We search learning rate, hidden unit, weight decay and dropout for all models in the same search space.

Original: Finally, we choose learning rate of 0 .
Revised: Finally, we choose learning rate of 0 .

Original: 01 , dropout rate of 0 . 5 , and hidden unit of 32 over all datasets.
Revised: 01 , dropout rate of 0 . 5 , and hidden unit of 32 over all datasets.

Original: The number of filters are searched between 2 to 10, and the final setting is: for T-DEMUF, we use 4 filters with 7 layers for Citation networks, 2 filters with 15 layers for all
Revised: The number of filters are searched between 2 to 10, and the final setting is: for T-DEMUF, we use 4 filters with 7 layers for Citation networks, 2 filters with 15 layers for all

Original: WebKB and Wikipedia networks, 5 filters with 1 layer for Actor.
Revised: WebKB and Wikipedia networks, 5 filters with 1 layer for Actor.

Original: The numbers of MLP layers are 2, 2, 3 and 4, respectively.
Revised: The numbers of MLP layers are 2, 2, 3 and 4, respectively.

Original: P-DEMUF uses: 3 filters with 8 layers for Citation networks; 5 filters for
Revised: P-DEMUF uses: 3 filters with 8 layers for Citation networks; 5 filters for

Original: Cornell, 4 filters for Wisconsin and 3 filters for Texas - all of them are 1 layer; 7 filters with 9 layers for WebKB; 5 filters with 2 layers for Actor.
Revised: Cornell, 4 filters for Wisconsin and 3 filters for Texas - all of them are 1 layer; 7 filters with 9 layers for WebKB; 5 filters with 2 layers for Actor.

Original: P-DEMUF applies 2-layer MLP to all benchmarks.
Revised: P-DEMUF applies 2-layer MLP to all benchmarks.

Original: In addition, as the setting of benchmarks are the same as that in Geom_GCN, we refer to the results reported in Pei et al.
Revised: In addition, as the setting of benchmarks are the same as that in Geom_GCN, we refer to the results reported in Pei et al.

Original: 5. R ESULT AND A NALYSIS
Revised: 5. R ESULT AND A NALYSIS

Original: The experimental results are summarized in Table 1.
Revised: The experimental results are summarized in Table 1.

Original: Our models consistently outperform baselines over most benchmarks with significant improvement.
Revised: Our models consistently outperform baselines over most benchmarks with significant improvement.

Original: On Cora and Citeseer, the datasets with a high level of homophily, our models are only comparable to GCN and other baselines.
Revised: On Cora and Citeseer, the datasets with a high level of homophily, our models are only comparable to GCN and other baselines.

Original: However, on all other datasets with a lower level of homophily, our models both obtained great performance gain.
Revised: However, on all other datasets with a lower level of homophily, our models both obtained great performance gain.

Original: To understand the impact of graph homophily on different types of graph filters, let us analyze the performance of all spectral GNNs.
Revised: To understand the impact of graph homophily on different types of graph filters, let us analyze the performance of all spectral GNNs.

Original: On high-homophily datasets, all GNNs perform similarly and the accuracy is much higher than MLP.
Revised: On high-homophily datasets, all GNNs perform similarly and the accuracy is much higher than MLP.

Original: That means the graph structure information is extremely useful in this case.
Revised: That means the graph structure information is extremely useful in this case.

Original: However, on low-homophily datasets, many of them are even worse than MLP.
Revised: However, on low-homophily datasets, many of them are even worse than MLP.

Original: GCN and GIN, the two low-pass filter based models, perform worst.
Revised: GCN and GIN, the two low-pass filter based models, perform worst.

Original: The two GNNs with high-order graph filters, ChebNet and ARMA, are clearly superior to other models due to their higher spectrum coverage.
Revised: The two GNNs with high-order graph filters, ChebNet and ARMA, are clearly superior to other models due to their higher spectrum coverage.

Original: However, they cannot beat our models with specially designed multiple filters.
Revised: However, they cannot beat our models with specially designed multiple filters.

Original: The
Revised: The

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Table]

Original: [Table] 
Revised: [Table] [Table] [Figure] [Table]

Original: [Figure] [Table] 
Revised: [Figure] [Table] [Figure]

Original: [Table][Table] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Actor 0.
Revised: Actor 0.

Original: here and filters g hb (1 − ϵ ) learnt by P-DEMUF and T-DEMUF, where the thickness of curve reflects its weight ω .
Revised: here and filters g hb (1 − ϵ ) learnt by P-DEMUF and T-DEMUF, where the thickness of curve reflects its weight ω .

Original: It shows that frequency responses of our filters are consistent with labels frequency.
Revised: It shows that frequency responses of our filters are consistent with labels frequency.

Original: reason might be that the high complexity of their filters makes it more difficult to learn one optimal single filter.
Revised: reason might be that the high complexity of their filters makes it more difficult to learn one optimal single filter.

Original: Finally, our model T-DEMUF yields over 18% higher accuracy than the best baselines (Geom_GCN) on Squirrel; and P-DEMUF yields almost 10% higher accuracy than MLP on Texas
Revised: Finally, our model T-DEMUF yields over 18% higher accuracy than the best baselines (Geom_GCN) on Squirrel; and P-DEMUF yields almost 10% higher accuracy than MLP on Texas

Original: In addition, we select some typical datasets and show the frequency distribution on these graphs in
Revised: In addition, we select some typical datasets and show the frequency distribution on these graphs in

Original: Fig.
Revised: Fig.

Original: We can obviously see that on Cora the spectrum is focused on low frequency components.
Revised: We can obviously see that on Cora the spectrum is focused on low frequency components.

Original: This can explain why the low-pass filter based models can also perform well on it.
Revised: This can explain why the low-pass filter based models can also perform well on it.

Original: On other datasets, the frequency distribution is more diverse, so the low-pass filters can not match with the important frequency components anymore.
Revised: On other datasets, the frequency distribution is more diverse, so the low-pass filters can not match with the important frequency components anymore.

Original: In contrast, both of our models, T-DEMUF and P-DEMUF, learn graph filters corresponding well to those components (as shown in the last two rows of Fig. 2).
Revised: In contrast, both of our models, T-DEMUF and P-DEMUF, learn graph filters corresponding well to those components (as shown in the last two rows of Fig. 2).

Original: TDEMUF uses fewer number of (more dispersed) filters but achieves comparable or better performance.
Revised: TDEMUF uses fewer number of (more dispersed) filters but achieves comparable or better performance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose a theoretical analysis of graph information with the introduction of interaction probability and frequency distribution.
Revised: In this paper, we propose a theoretical analysis of graph information with the introduction of interaction probability and frequency distribution.

Original: We develop a deep understanding of how different structures and input influence the performance of graph filters.
Revised: We develop a deep understanding of how different structures and input influence the performance of graph filters.

Original: We also design a simple framework to learn a filter bank.
Revised: We also design a simple framework to learn a filter bank.

Original: Empirical results on extensive datasets validate the power of our model.
Revised: Empirical results on extensive datasets validate the power of our model.

Original: A B ENCHMARKS AND M ODEL D ISCUSSION A. S TATISTICS INFORMATION OF BENCHMARKS .
Revised: A B ENCHMARKS AND M ODEL D ISCUSSION A. S TATISTICS INFORMATION OF BENCHMARKS .

Original: We provide statistics information of our benchmarks in Table.
Revised: We provide statistics information of our benchmarks in Table.

Original: A.1.
Revised: A.

Revised: [Table]

Original: [Table]
Revised: [Table]

Revised: A. S PECTRAL FILTERS .

Original: In our paper
Revised: In our paper

Original: [Table]
Revised: [Table]

Original: we use four spectral GNN as baselines whose spectral filters are listed as Table.
Revised: we use four spectral GNN as baselines whose spectral filters are listed as Table.

Original: A.2 and define a band-pass filter g b ( ϵ ) as a quadratic function with respect to the adjacency matrix
Revised: A.2 and define a band-pass filter g b ( ϵ ) as a quadratic function with respect to the adjacency matrix.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation]{+k L k ) − 1 ( (cid:80) K − 1 k =
 p k L k )+}
Revised: {+k L k ) − 1 ( (cid:80) K − 1 k =
 p k L k )+}

Original: Ours
Revised: Ours

Original: [Equation]
Revised: I − (1 + | ϵ | ) − 2 ( ˜ A − ϵI )

Original: Since g b ( ϵ ) ( ˜ A )
Revised: Since g b ( ϵ ) ( ˜ A )

Original: =
Revised: =

Original: I − (1+ | ϵ ′ | ) − 2 ( ϵI − ˜ A ) 2 =
Revised: I − (1+ | ϵ ′ | ) − 2 ( ϵI − ˜ A ) 2 =

Original: (1+ | ϵ | ) − 2 ((1+ | ϵ |− ϵ ) I + A )((1+ | ϵ | + ϵ )
Revised: (1+ | ϵ | ) − 2 ((1+ | ϵ |− ϵ ) I + A )((1+ | ϵ | + ϵ )

Original: I − A ) , it is exactly an overlap between a low-pass filter (1+
Revised: I − A ) , it is exactly an overlap between a low-pass filter (1+

Original: | ϵ |− ϵ )
Revised: | ϵ |− ϵ )

Original: I + A ) and a high-pass filter ((1+ | ϵ |
Revised: I + A ) and a high-pass filter ((1+ | ϵ |

Original: + ϵ )
Revised: + ϵ )

Original: I − A ) .
Revised: I − A ) .

Original: That is why g b ( ϵ ) is a band-pass filter.
Revised: That is why g b ( ϵ ) is a band-pass filter.

Original: A. M ODEL D ISCUSSION . A.3. M OTIVATION OF DISENTANGLEMENT .
Revised: A. M ODEL D ISCUSSION . A.3. M OTIVATION OF DISENTANGLEMENT .

Original: Overlap if not disentangled.
Revised: Overlap if not disentangled.

Original: Without disentanglement, it is highly possible that the learned filters have large overlaps if we do not induce any constraints on them.
Revised: Without disentanglement, it is highly possible that the learned filters have large overlaps if we do not induce any constraints on them.

Original: In our algorithm, we aim to train filters to capture the main frequency information of their input and assign different weights to that captured information depending on how much they contribute to label prediction.
Revised: In our algorithm, we aim to train filters to capture the main frequency information of their input and assign different weights to that captured information depending on how much they contribute to label prediction.

Original: Therefore, it is natural to assume that if the input of filters is different, filters are less likely to overlap.
Revised: Therefore, it is natural to assume that if the input of filters is different, filters are less likely to overlap.

Original: Disentanglement can make the “input” different and more adaptable to each filter.
Revised: Disentanglement can make the “input” different and more adaptable to each filter.

Original: Disentanglement reduces the model complexity.
Revised: Disentanglement reduces the model complexity.

Original: With disentanglement, we divide node features into several subsets by learnable masking or map them into several low-dimensional spaces through linear transformations which lowers the dimension of corresponding features for each filter and meanwhile makes the input feature fits better to each filter.
Revised: With disentanglement, we divide node features into several subsets by learnable masking or map them into several low-dimensional spaces through linear transformations which lowers the dimension of corresponding features for each filter and meanwhile makes the input feature fits better to each filter

Original:  A.3. M OTIVATION OF T-DEMUF.
Revised: . A.3. M OTIVATION OF T-DEMUF.

Original: As we clarified in Section 5.1, our implementation of disentanglement is not random masking but learnable masking leveraging GUMBEL-SOFTMAX.
Revised: As we clarified in Section 5.1, our implementation of disentanglement is not random masking but learnable masking leveraging GUMBEL-SOFTMAX.

Original: These learnable maskings disentangle node features into several subsets of features.
Revised: These learnable maskings disentangle node features into several subsets of features.

Original: With a constraint L ( X k − 1 , H k ) =
Revised: With a constraint L ( X k − 1 , H k ) =

Original: ∥
Revised: ∥

Original: X k − 1 ⊙ M ′ k − H k ∥ 22 , for each subset of features we train a band-pass filter to through their main frequency.
Revised: X k − 1 ⊙ M ′ k − H k ∥ 22 , for each subset of features we train a band-pass filter to through their main frequency.

Original: As shown in Figure.2, this constraint guides the process of filter learning which can help reduce the overlap of filters’ frequency responses such that can reduce the number of graph filters.
Revised: As shown in Figure.2, this constraint guides the process of filter learning which can help reduce the overlap of filters’ frequency responses such that can reduce the number of graph filters.

Original: At the same time, to minimize the supervised loss, maskings are trained to disentangle features whose frequency distribution are similar to those of labels and assign a higher weight; those useless captured feature information will be assigned a lower weight.
Revised: At the same time, to minimize the supervised loss, maskings are trained to disentangle features whose frequency distribution are similar to those of labels and assign a higher weight; those useless captured feature information will be assigned a lower weight.

Original: Weights are also learnable.
Revised: Weights are also learnable.

Original: A.3. H OW CAN OUR FILTER BANK SELECTION BE DATA - DRIVEN .
Revised: A.3. H OW CAN OUR FILTER BANK SELECTION BE DATA - DRIVEN .

Original: In our algorithm, although the form of our band-pass filter g b ( ϵ ) is predefined, its parameters including ϵ and weight ω are learned from specified graphs.
Revised: In our algorithm, although the form of our band-pass filter g b ( ϵ ) is predefined, its parameters including ϵ and weight ω are learned from specified graphs.

Original: Moreover, the parameters of our feature disentanglement blocks (the linear transformations and learnable masking) are also learned from data which will affect the learning of the filter bank.
Revised: Moreover, the parameters of our feature disentanglement blocks (the linear transformations and learnable masking) are also learned from data which will affect the learning of the filter bank.

Original: Therefore, our filter bank selection is data-driven.
Revised: Therefore, our filter bank selection is data-driven

Original:  A.  A BLATION S TUDY
Revised: . A. M ORE E XPERIMENTAL R ESULTS . A.4. A DDITIONAL BASELINE - GPRGNN.

Revised: Here we compare our models with a related baseline GPRGNN(Chien et al., 2021). It is worth noting that the splitting in our paper is different from that in GPRGNN. In GPRGNN, its training set consists of the same number of nodes from each class while we just randomly choose our training data.

Revised: We find that WebKB datasets are sensitive to the way of splitting due to their uneven distribution of labels ( the numbers of classes are: Cornell and Texas: 33/1/18/101/30, Wisconsin: 10/70/118/32/21).

Revised: Although GPRGNN’s splitting is likely better for model training, our model still outperforms it in the Wikipedia datasets, i.e., Chameleon (our 69.52% v.s. GPRGNN 67.48%) and Squirrel (our 56.47% v.s. GPRGNN 49.93%).

Revised: In GPRGNN’s setting, the performances of MLP on WebKB are comparable to GPRGNNs’ while in our setting, our proposed models’ performances are much better than MLPs’.

Revised: We also test T-DEMUF on Actor, Cornell, and Texas following the splitting of

Revised: GPRGNN.

Revised: On Actor, our accuracy is 41.11% while GPRGNN’s is 39.30%; on Cornell, our accuracy is 91.26% and GPRGNN’s is 91.36%; on Texas, our accuracy is 92.79% while GPRGNN’s is 92.92%.

Revised: Therefore, in most of the benchmarks, our model performs better than GPRGNN.

Revised: A.4. A BLATION STUDY .

Original: To show the advantage of using disentanglement, we provide an ablation study on five benchmarks.
Revised: To show the advantage of using disentanglement, we provide an ablation study on five benchmarks.

Original: Here, we propose two ablation models based on P-DEMUF.
Revised: Here, we propose two ablation models based on P-DEMUF.

Original: Recall that the disentanglement block of P-DEMUF consists of learnable masking and linear transformations, we design our ablation models by taking off the component of masking and linear transformation.
Revised: Recall that the disentanglement block of P-DEMUF consists of learnable masking and linear transformations, we design our ablation models by taking off the component of masking and linear transformation.

Original: Also, for fair and intuitive comparison, we simply fix the number of filters as 2.
Revised: Also, for fair and intuitive comparison, we simply fix the number of filters as 2.

Original: The results shown as Table.
Revised: The results shown as Table.

Original: A.4 validate that if we take off the disentanglement blocks of P-DEMUF, the results become worse in most of benchmarks.filters as 2.
Revised: A.4.2 validate that if we take off the disentanglement blocks of P-DEMUF, the results become worse in most of benchmarks.

Original: [Table]
Revised: [Table]

Revised: P-DEMUF (w/o masking ) 80.

Revised: P-DEMUF (w/o masking & linear) 77.

Original: B P ROOF OF P ROPOSITION
Revised: B P ROOF OF P ROPOSITION

Original: Here, we provide the proof of Proposition 3.2.
Revised: Here, we provide the proof of Proposition 3.2.

Original: Proof.
Revised: Proof.

Original: Since x = n − 1 (cid:80) i =

Revised: Since x = n − 1 (cid:80) i =

Original: α i u
Revised: α i u

Original: i , u i is the i -th unit eigenvector of ˜ L and λ n = u ⊤ i ˜
Revised: i , u i is the i -th unit eigenvector of ˜ L and λ n = u ⊤ i ˜

Original: L n u
Revised: L n u

Original: i then we have
Revised: i then we have

Original: [Equation]
Revised: [Equation]

Original: Below is the proof of Proposition 3.1.
Revised: Below is the proof of Proposition 3.1.

Original: Proof.
Revised: Proof.

Original: For P = D − 1 A and ˜
Revised: For P = D − 1 A and ˜

Original: A
Revised: A

Original: = D − 12 AD − 12 , and Π , ˜Π defined by Definition 3.1, the inequality can be represented as ( R Π k + (Π k ) ⊤ R ) lm ≥ 2( R 12 ˜Π k R 12 ) lm which is equivalent to prove y ⊤ m ( P k + ( P k ) ⊤ ) y l ≥ 2 y ⊤ m ˜
Revised: = D − 12 AD − 12 , and Π , ˜Π defined by Definition 3.1, the inequality can be represented as ( R Π k + (Π k ) ⊤ R ) lm ≥ 2( R 12 ˜Π k R 12 ) lm which is equivalent to prove y ⊤ m ( P k + ( P k ) ⊤ ) y l ≥ 2 y ⊤ m ˜

Original: A k y l .
Revised: A k y l .

Original: Noting that, ( P k ) ⊤ =
Revised: Noting that, ( P k ) ⊤ =

Original: DP k D − 1 and ˜
Revised: DP k D − 1 and ˜

Original: A k
Revised: A k

Original: = D 12 P k D − 12 , with B =
Revised: = D 12 P k D − 12 , with B =

Original: P k + ( P k ) ⊤ , we have B ij =
Revised: P k + ( P k ) ⊤ , we have B ij =

Original: P kij + d i d j P kij ≥ 2 (cid:113) d i d j P kij = 2 ˜
Revised: P kij + d i d j P kij ≥ 2 (cid:113) d i d j P kij = 2 ˜

Original: A kij .
Revised: A kij .

Original: Therefore, y ⊤ m ( B − 2 ˜
Revised: Therefore, y ⊤ m ( B − 2 ˜

Original: A k ) y l ≥ 0 .
Revised: A k ) y l ≥ 0 .

Original: Let m =
Revised: Let m =

Original: l , then we get π k l ≥ ˜ π k l .
Revised: l , then we get π kl ≥ ˜ π kl .

Original: To prove proposition b, we utilize lemma B.1.
Revised: To prove proposition b, we utilize lemma B.1.

Original: Since g ( ˜ A ) is symmetric, then we have
Revised: Since g ( ˜ A ) is symmetric, then we have

Original: [Equation]
Revised: [Equation]

Original: Lemma B.1.
Revised: Lemma B.1.

Original: Let B ∈ R n × n is a symmetric matrix, ∀ ij , y ∈ R n , we have
Revised: Let B ∈ R n × n is a symmetric matrix, ∀ ij , y ∈ R n , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Since B is symmetric, then we have B =
Revised: Since B is symmetric, then we have B =

Original: U Λ U ⊤ , here U is matrix of unit eigenvectors of B .
Revised: U Λ U ⊤ , here U is matrix of unit eigenvectors of B .

Original: From the proof of Proposition 3.2, we obtain that y ⊤ B 2 y y ⊤ y =
Revised: From the proof of Proposition 3.2, we obtain that y ⊤ B 2 y y ⊤ y =

Original: [Equation]
Revised: [Equation]

Original: (cid:80)

Original: [Equation]
Revised: [Equation]

Original:  and (cid:16) y ⊤ B y y ⊤ y (cid:17) 2{+
+} =
Revised: i and (cid:16) y ⊤ B y y ⊤ y (cid:17) {+
+} =

Original: ( (cid:80)
Revised: ( (cid:80)

Original: α 2 i λ i ) 2 ( (cid:80)
Revised: α 2 i λ i ) 2 ( (cid:80)

Original: α 2 i ) 2 .
Revised: α 2 i ) 2 .

Original: From Hölder’s inequality, we have ( (cid:80) ( α i λ i ) 2 )( (cid:80) α 2 i ) ≥ ( (cid:80)
Revised: From Hölder’s inequality, we have ( (cid:80) ( α i λ i ) 2 )( (cid:80) α 2 i ) ≥ ( (cid:80)

Original: α 2 i λ i ) 2 .
Revised: α 2 i λ i ) 2 .

Original: Therefore, we have(cid:80) ( α i λ i [Equation] 2 (cid:80) α 2 i ≥
Revised: Therefore, we have

Original: [Equation]
Revised: [Equation]