Original: R E D-GCN: R EVISIT THE D EPTH OF G RAPH C ONVOLUTIONAL N ETWORK
Revised: R E D-GCN: R EVISIT THE D EPTH OF G RAPH C ONVOLUTIONAL N ETWORK

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Finding the proper depth d of a GNN that provides strong representation power has drawn significant attention, yet nonetheless largely remains an open problem for the graph learning community.
Revised: Finding the proper depth d of a GNN that provides strong representation power has drawn significant attention, yet nonetheless largely remains an open problem for the graph learning community.

Original: Although noteworthy progress has been made, the depth or the number of layers of a corresponding GCN is realized by a series of graph convolution operations, which naturally makes d a positive integer ( d ∈ N + ).
Revised: Although noteworthy progress has been made, the depth or the number of layers of a corresponding GCN is realized by a series of graph convolution operations, which naturally makes d a positive integer ( d ∈ N + ).

Original: An interesting question is whether breaking the constraint of N + by making d a real number ( d ∈ R ) can bring new insights into graph learning mechanisms.
Revised: An interesting question is whether breaking the constraint of N + by making d a real number ( d ∈ R ) can bring new insights into graph learning mechanisms.

Original: In this work, by redefining GCN’s depth d as a trainable parameter continuously adjustable within ( −∞ , + ∞ ) , we open a new door of controlling its expressiveness on graph signal processing to model graph homophily/heterophily (nodes with similar/dissimilar labels/attributes tend to inter-connect).
Revised: In this work, by redefining GCN’s depth d as a trainable parameter continuously adjustable within ( −∞ , + ∞ ) , we open a new door of controlling its expressiveness on graph signal processing to model graph homophily/heterophily (nodes with similar/dissimilar labels/attributes tend to inter-connect).

Original: A simple and powerful
Revised: A simple and powerful

Original: GCN model R E D-GCN, is proposed to retain the simplicity of GCN and meanwhile automatically search for the optimal d without the prior knowledge regarding whether the input graph is homophilic or heterophilic.
Revised: GCN model R E D-GCN, is proposed to retain the simplicity of GCN and meanwhile automatically search for the optimal d without the prior knowledge regarding whether the input graph is homophilic or heterophilic.

Original: Negative-valued d intrinsically enables high-pass frequency filtering functionality for graph heterophily.
Revised: Negative-valued d intrinsically enables high-pass frequency filtering functionality for graph heterophily.

Original: Variants extending the model flexibility/scalability are also developed.
Revised: Variants extending the model flexibility/scalability are also developed.

Original: The theoretical feasibility of having a real-valued depth with explainable physical meanings is ensured via eigen-decomposition of the graph Laplacian and a properly designed transformation function from the perspective of functional calculus.
Revised: The theoretical feasibility of having a real-valued depth with explainable physical meanings is ensured via eigen-decomposition of the graph Laplacian and a properly designed transformation function from the perspective of functional calculus.

Original: Extensive experiments demonstrate the superiority of R E D-GCN on node classification tasks for a variety of graphs.
Revised: Extensive experiments demonstrate the superiority of R E D-GCN on node classification tasks for a variety of graphs.

Original: Furthermore, by introducing the concept of eigengraph, a novel graph augmentation method is obtained: the optimal d effectively generates a new topology through a properly weighted combination of eigengraphs, which dramatically boosts the performance even for a vanilla GCN.
Revised: Furthermore, by introducing the concept of eigengraph, a novel graph augmentation method is obtained: the optimal d effectively generates a new topology through a properly weighted combination of eigengraphs, which dramatically boosts the performance even for a vanilla GCN.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph convolutional network (GCN) (Kipf & Welling, 2016; Veliˇckovi´c et al., 2017; Hamilton et al., 2017) has exhibited great power in a variety of graph learning tasks, such as node classification (Kipf & Welling, 2016), link prediction (Zhang & Chen, 2018), community detection (Chen et al., 2020), and many more.
Revised: Graph convolutional network (GCN) (Kipf & Welling, 2016; Veliˇckovi´c et al., 2017; Hamilton et al., 2017) has exhibited great power in a variety of graph learning tasks, such as node classification (Kipf & Welling, 2016; Luan et al., 2019; 2022a), link prediction (Zhang & Chen, 2018), community detection (Chen et al., 2020), and many more.

Original: Since the representation power of GCN is largely determined by its depth, i.e., the number of graph convolution layers, tremendous research efforts have been made on finding the optimal depth that strengthens the model’s ability for downstream tasks.
Revised: Since the representation power of GCN is largely determined by its depth, i.e., the number of graph convolution layers, tremendous research efforts have been made on finding the optimal depth that strengthens the model’s ability for downstream tasks.

Original: Upon increasing the depth, the over-smoothing issue arises: a GCN’s performance is deteriorated if its depth exceeds a uncertain threshold (Kipf & Welling, 2016).
Revised: Upon increasing the depth, the over-smoothing issue arises: a GCN’s performance is deteriorated if its depth exceeds a uncertain threshold (Kipf & Welling, 2016).

Original: It is unveiled in (Li et al., 2018) that a graph convolution operation is a special form of Laplacian smoothing (Taubin, 1995).
Revised: It is unveiled in (Li et al., 2018) that a graph convolution operation is a special form of Laplacian smoothing (Taubin, 1995).

Original: Thus, the similarity between the graph node embeddings grows with the depth so that these embeddings eventually become indistinguishable.
Revised: Thus, the similarity between the graph node embeddings grows with the depth so that these embeddings eventually become indistinguishable.

Original: Various techniques are developed to alleviate this issue, e.g., applying pairwise normalization can make distant nodes dissimilar (Zhao & Akoglu, 2019), and dropping sampled edges during training slows down the growth of embedding smoothness as depth increases (Rong et al., 2019).
Revised: Various techniques are developed to alleviate this issue, e.g., applying pairwise normalization can make distant nodes dissimilar (Zhao & Akoglu, 2019), and dropping sampled edges during training slows down the growth of embedding smoothness as depth increases (Rong et al., 2019).

Original: Other than the over-smoothing issue due to large GCN depth, another fundamental phenomenon widely existing in real-world graphs is homophily and heterophily.
Revised: Other than the over-smoothing issue due to large GCN depth, another fundamental phenomenon widely existing in real-world graphs is homophily and heterophily.

Original: In a homophilic graph, nodes with similar labels or attributes tend to inter-connect, while in a heterophily graph, connected nodes usually have distinct labels or dissimilar attributes.
Revised: In a homophilic graph, nodes with similar labels or attributes tend to inter-connect, while in a heterophily graph, connected nodes usually have distinct labels or dissimilar attributes.

Original: Most graph neural networks (GNNs) are developed based on homophilic assumption (Yang et al., 2016), while models able to perform well on heterophilic graphs often need special treatment and complex designs (Bianchi et al., 2021; Zhu et al., 2020).
Revised: Most graph neural networks (GNNs) are developed based on homophilic assumption (Yang et al., 2016), while models able to perform well on heterophilic graphs often need special treatment and complex designs (Bianchi et al., 2021; Zhu et al., 2020).

Original: Despite the achievements made by these methodologies, little correlation has been found between the adopted GNN model’s depth and its capability of characterizing graph heterophily.
Revised: Despite the achievements made by these methodologies, little correlation has been found between the adopted GNN model’s depth and its capability of characterizing graph heterophily.

Original: For most GNNs, if not all, the depth needs to be manually set as a hyper-parameter before training, and finding the proper depth usually requires a considerable amount of trials or good prior knowledge of the graph dataset.
Revised: For most GNNs, if not all, the depth needs to be manually set as a hyper-parameter before training, and finding the proper depth usually requires a considerable amount of trials or good prior knowledge of the graph dataset.

Original: Since the depth represents the number of graph convolution operations and naturally takes only positive integer values, little attention has been paid to the question whether a non-integer depth is realizable, and if yes, whether it is practically meaningful, and whether it can bring unique advantages to current graph learning mechanisms.
Revised: Since the depth represents the number of graph convolution operations and naturally takes only positive integer values, little attention has been paid to the question whether a non-integer depth is realizable, and if yes, whether it is practically meaningful, and whether it can bring unique advantages to current graph learning mechanisms.

Original: This work revisits the GCN depth from spectral and spatial perspectives and explains the interdependencies between the following key ingredients in graph learning: the depth of a GCN, the spectrum of the graph signal, and the homophily/heterophily of the underlying graph.
Revised: This work revisits the GCN depth from spectral and spatial perspectives and explains the interdependencies between the following key ingredients in graph learning: the depth of a GCN, the spectrum of the graph signal, and the homophily/heterophily of the underlying graph.

Original: Firstly, through eigen-decomposition of the symmetrically normalized graph Laplacian, we present the correlation between graph homophily/heterophily and the eigenvector frequencies.
Revised: Firstly, through eigen-decomposition of the symmetrically normalized graph Laplacian, we present the correlation between graph homophily/heterophily and the eigenvector frequencies.

Original: Secondly, by introducing the concept of eigengraph, we show the graph topology is equivalent to a weighted linear combination of eigengraphs, and the weight values determine the GCN’s capability of capturing homophilic/heterophilic graph signals.
Revised: Secondly, by introducing the concept of eigengraph, we show the graph topology is equivalent to a weighted linear combination of eigengraphs, and the weight values determine the GCN’s capability of capturing homophilic/heterophilic graph signals.

Original: Thirdly, we reveal that the eigengraph weights can be controlled by GCN’s depth, so that an automatically tunable depth parameter is needed to adjust the eigengraph weights into the designated distribution in match of the underlying graph homophily/heterophily.
Revised: Thirdly, we reveal that the eigengraph weights can be controlled by GCN’s depth, so that an automatically tunable depth parameter is needed to adjust the eigengraph weights into the designated distribution in match of the underlying graph homophily/heterophily.

Original: To realize the adaptive GCN depth, we extend its definition from a positive integer to an arbitrary real number with theoretical feasibility guarantees from functional calculus (Shah & Okutmu¸stur, 2020).
Revised: To realize the adaptive GCN depth, we extend its definition from a positive integer to an arbitrary real number with theoretical feasibility guarantees from functional calculus (Shah & Okutmu¸stur, 2020).

Original: With a trainable depth parameter, we propose a simple and powerful model, Redefined Depth-GCN (ReD-GCN), with two variants.
Revised: With a trainable depth parameter, we propose a simple and powerful model, Redefined Depth-GCN (ReD-GCN), with two variants.

Original: Extensive experiments demonstrate the automatically optimal depth searching ability, and it is found that negative-valued depth plays the key role in handling heterophilic graphs.
Revised: Extensive experiments demonstrate the automatically optimal depth searching ability, and it is found that negative-valued depth plays the key role in handling heterophilic graphs.

Original: Systematical investigation on the optimal depth is conducted in both spectral and spatial domains.
Revised: Systematical investigation on the optimal depth is conducted in both spectral and spatial domains.

Original: It in turn inspires the development of a novel graph augmentation methodology.
Revised: It in turn inspires the development of a novel graph augmentation methodology.

Original: With clear geometric explanability, the augmented graph structure possesses supreme advantages over the raw input topology, especially for graphs with heterophily.
Revised: With clear geometric explanability, the augmented graph structure possesses supreme advantages over the raw input topology, especially for graphs with heterophily.

Original: The main contributions of this paper are summarized as following:
Revised: The main contributions of this paper are summarized as following:

Original: • The interdependence between negative GCN depth and graph heterophily is discovered; In-depth geometric and spectral explanations are presented.
Revised: • The interdependence between negative GCN depth and graph heterophily is discovered; In-depth geometric and spectral explanations are presented.

Original: • A novel problem of automatic GCN depth tuning for graph homophily/heterophily detection is formulated.
Revised: • A novel problem of automatic GCN depth tuning for graph homophily/heterophily detection is formulated.

Original: To our best knowledge, this work presents the first trial to make GCN’s depth trainable by redefining it on the real number domain.
Revised: To our best knowledge, this work presents the first trial to make GCN’s depth trainable by redefining it on the real number domain.

Original: • A simple and powerful model R E D-GCN with two variants (R E D-GCN-S and R E D-GCND) is proposed.
Revised: • A simple and powerful model R E D-GCN with two variants (R E D-GCN-S and R E D-GCND) is proposed.

Original: A novel graph augmentation method is discussed.
Revised: A novel graph augmentation method is discussed.

Original: • Our model achieves superior performance on semi-supervised node classification tasks on 11 graph datasets.
Revised: • Our model achieves superior performance on semi-supervised node classification tasks on 11 graph datasets.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Notations.
Revised: Notations.

Original: We utilize bold uppercase letters for matrices (e.g., A ), bold lowercase letters for column vectors (e.g., u ) and lowercase letters for scalars (e.g., α ).
Revised: We utilize bold uppercase letters for matrices (e.g., A ), bold lowercase letters for column vectors (e.g., u ) and lowercase letters for scalars (e.g., α ).

Original: We use the superscript ⊤ for transpose of matrices and vectors (e.g., A ⊤ and u ⊤ ).
Revised: We use the superscript ⊤ for transpose of matrices and vectors (e.g., A ⊤ and u ⊤ ).

Original: An attributed undirected graph G =
Revised: An attributed undirected graph G =

Original: { A , X } contains an adjacency matrix A ∈ R n × n and an attribute matrix X ∈ R n × q with the number of nodes n and the dimension of node attributes q .
Revised: { A , X } contains an adjacency matrix A ∈ R n × n and an attribute matrix X ∈ R n × q with the number of nodes n and the dimension of node attributes q .

Original: D denotes the diagonal degree matrix of A .
Revised: D denotes the diagonal degree matrix of A .

Original: The adjacency matrix with self-loops is given by ˜
Revised: The adjacency matrix with self-loops is given by ˜

Original: A =
Revised: A =

Original: A
Revised: A

Original: + I
Revised: + I

Original: ( I is the identity matrix), and all variables derived from ˜ A are decorated with symbol ˜ , e.g., ˜ D represents the diagonal degree matrix of ˜ A .
Revised: ( I is the identity matrix), and all variables derived from ˜ A are decorated with symbol ˜ , e.g., ˜ D represents the diagonal degree matrix of ˜ A .

Original: M d stands for the d -th power of matrix M , while the parameter and node embedding matrices in the d -th layer of a GCN are denoted by W ( d ) and H ( d ) .
Revised: M d stands for the d -th power of matrix M , while the parameter and node embedding matrices in the d -th layer of a GCN are denoted by W ( d ) and H ( d ) .

Original: Graph convolutional network (GCN) and simplified graph convolutional network (SGC).
Revised: Graph convolutional network (GCN) and simplified graph convolutional network (SGC).

Original: The layer-wise message-passing and aggregation of GCN (Kipf & Welling, 2016) is given by
Revised: The layer-wise message-passing and aggregation of GCN (Kipf & Welling, 2016) is given by

Original: [Equation]
Revised: [Equation]

Original: where H ( d ) / H ( d +1) stands for the embedding matrix ( H (0) =

Original: X ) in the d -th/ ( d + 1) -th layer; W ( d ) is the trainable parameter matrix; and σ ( · ) is the non-linear activation function.

Original: With σ ( · ) removed in

Original: 𝑣 ! 𝑣 "
Revised: 𝑣 ! 𝑣 "

Original: 𝑣 # 𝑣 ! 𝑣 " 𝑣 # normalization 11 2 = 𝑣 ! 𝑣 " 𝑣 # 1 − 𝜆 !
Revised: 𝑣 # 𝑣 ! 𝑣 " 𝑣 # normalization 11 2 = 𝑣 ! 𝑣 " 𝑣 # 1−𝜆 !

Original: = 1,(𝜆 !
Revised: = 1,(𝜆 !

Original: = 0) 111111𝑣 ’  𝑣 ! 𝑣 "  − 1− 211𝑣 ! 
Revised: = 0) 113  𝑣 ! 𝑣 " 𝑣 # − 1−13  𝑣 ! 𝑣 " 𝑣 # 1−𝜆 " =

Original: 𝑣 " 𝑣 # 1 − 𝜆 " = −12,(𝜆 " = 3 2) −12 11 2 + + 𝐀 𝐒 = 𝐃 !"
Revised: −12,(𝜆 " = 3 2) −12 112 + + 𝐀 𝐒 = 𝐃 !"

Original: # 𝐀𝐃 !"
Revised: # 𝐀𝐃 !"

Revised: # eigengrapheigengrapheigengraph1−𝜆 # = −12,(𝜆 # = 3 2) where H ( d ) / H ( d +1) stands for the embedding matrix ( H (0) =

Revised: X ) in the d -th/ ( d + 1) -th layer; W ( d ) is the trainable parameter matrix; and σ ( · ) is the non-linear activation function.

Original: # eigengrapheigengrapheigengraph1 − 𝜆 # = −12,(𝜆 # = 3 2) each layer, SGC (Wu et al., 2019) is obtained as below:
Revised: With σ ( · ) removed in each layer, SGC (Wu et al., 2019) is obtained as below:

Original: [Equation]
Revised: [Equation]

Original: where ˜ S =
Revised: where ˜ S =

Original: ˜ D − 12 ˜
Revised: ˜ D − 12 ˜

Original: A ˜ D − 12 , and the parameter of each layer W
Revised: A ˜ D − 12 , and the parameter of each layer W

Original: ( i ) are compressed into one trainable
Revised: ( i ) are compressed into one trainable

Original: W = (cid:81)
Revised: W = (cid:81)

Original: d − 1 i =0 W ( i ) .
Revised: [Equation]

Original: Graph Laplacian and spectrum.
Revised: Graph Laplacian and spectrum.

Original: In graph theory, graph Laplacian L = D − A and its symmetrically normalized correspondence L sym
Revised: In graph theory, graph Laplacian L = D − A and its symmetrically normalized correspondence L sym

Original: =
Revised: =

Original: I − D − 12 AD − 12 possess critical properties of the underlying graph G .
Revised: I − D − 12 AD − 12 possess critical properties of the underlying graph G .

Original: L sym has eigenvalues [ λ 1 , λ 2 , . . .
Revised: L sym has eigenvalues [ λ 1 , λ 2 , . . .

Original: , λ n ] , where λ i ∈ [0 , 2) , ∀ i ∈ { 1 , 2 , . .
Revised: , λ n ] , where λ i ∈ [0 , 2) , ∀ i ∈ { 1 , 2 , . .

Original: . , n } (Chung &
Revised: . , n } (Chung &

Original: Graham, 1997).
Revised: Graham, 1997).

Original: Here they are put in ascending order: 0 = λ 1 ≤ λ 2 ≤ · · · ≤ λ n < 2 .
Revised: Here they are put in ascending order: 0 = λ 1 ≤ λ 2 ≤ · · · ≤ λ n < 2 .

Original: It can be eigen-decomposed as: L sym =
Revised: It can be eigen-decomposed as: L sym =

Original: UΛU ⊤ , where U =
Revised: UΛU ⊤ , where U =

Original: [ u 1 , u 2 , . . .
Revised: [ u 1 , u 2 , . . .

Original: , u n ] is the eigenvector matrix ( u i ⊥ u j , ∀ i ̸
Revised: , u n ] is the eigenvector matrix ( u i ⊥ u j , ∀ i ̸

Original: = j ), and Λ is the diagonal eigenvalue matrix:Λ =  
Revised: = j ), and Λ is the diagonal eigenvalue matrix:

Original: λ· · · ... · · · λ n   .
Revised: [Equation]

Original: For each eigenvector u i , we have u i u ⊤ i ∈ R n × n .
Revised: For each eigenvector u i , we have u i u ⊤ i ∈ R n × n .

Original: As we will show in Section 3, this n × n matrix can be viewed as a weighted adjacency matrix of a graph with possible negative edges, which we name u i u
Revised: As we will show in Section 3, this n × n matrix can be viewed as a weighted adjacency matrix of a graph with possible negative edges, which we name u i u

Original: ⊤ i as the i -th eigengraph of G .
Revised: ⊤ i as the i -th eigengraph of G .

Original: Accordingly, L sym can be written as the linear combination of all eigengraphs weighted by the corresponding eigenvalues (Chung & Graham, 1997):
Revised: Accordingly, L sym can be written as the linear combination of all eigengraphs weighted by the corresponding eigenvalues (Chung & Graham, 1997):

Original: [Equation]
Revised: [Equation]

Original: where the first eigenvalue λ 1 = 0 , and the corresponding eigengraph u 1 u ⊤ 1 has an identical value 1 n for all entries (Shuman et al., 2013).
Revised: where the first eigenvalue λ 1 = 0 , and the corresponding eigengraph u 1 u ⊤ 1 has an identical value 1 n for all entries (Shuman et al., 2013).

Original: Thus, for SGC, we have
Revised: Thus, for SGC, we have

Original: [Equation]
Revised: [Equation]

Original: A SGC with d layers requires d consecutive graph convolution operations, which involves the multiplication of ˜ S by d times.
Revised: A SGC with d layers requires d consecutive graph convolution operations, which involves the multiplication of ˜ S by d times.

Original: Due to the orthogonality of ˜ U , namely, ˜ U ⊤ ˜
Revised: Due to the orthogonality of ˜ U , namely, ˜ U ⊤ ˜

Original: U
Revised: U

Original: =
Revised: =

Original: I , we obtain ˜ Sd
Revised: I , we obtain ˜ S

Original: [Equation]
Revised: [Equation]

Original: 5) where 1 − ˜ λ i ∈ ( − 1 , 1] , and the depth d of SGC serves as the power of ˜ S ’s eigenvalues.
Revised: where 1 − ˜ λ i ∈ ( − 1 , 1] , and the depth d of SGC serves as the power of ˜ S ’s eigenvalues.

Original: ˜
Revised: ˜

Original: S
Revised: S

Original: d can be viewed as the sum of eigengraphs ˜ u i ˜
Revised: d can be viewed as the sum of eigengraphs ˜ u i ˜

Original: u ⊤ i weighted by coefficients (1 − ˜ λ i ) d .
Revised: u ⊤ i weighted by coefficients (1 − ˜ λ i ) d .

Original: Graph homophily and heterophily.
Revised: Graph homophily and heterophily.

Original: Graph homophily describes to what extent edges tend to link nodes with the same labels and similar features.
Revised: Graph homophily describes to what extent edges tend to link nodes with the same labels and similar features.

Original: In this work, we focus on edge homophily (Zhu et al., 2020): h ( G ) =
Revised: In this work, we focus on edge homophily (Zhu et al., 2020): h ( G ) =

Original: (cid:80) i,j, A
Revised: (cid:80) i,j, A

Original: [ i,j ]=1 ⟨ y
Revised: [ i,j ]=1 ⟨ y

Original: [ i ]= y
Revised: [ i ]= y

Original: [ j ] ⟩ (cid:80) i,j A
Revised: [ j ] ⟩ (cid:80) i,j A

Original: [ i,j ] ∈ [0 , 1] , where ⟨ x ⟩ = 

Revised: [ i,j ] ∈ [0 , 1] , where ⟨ x ⟩ =

Original: if x is true and 0 otherwise.
Revised: if x is true and 0 otherwise.

Original: A graph is more homophilic for h ( G ) closer to 1 or more heterophilic for h ( G ) closer to 0 .
Revised: A graph is more homophilic for h ( G ) closer to 1 or more heterophilic for h ( G ) closer to 0 .

Original:  M ODEL
Revised:  M ODEL

Original: Firstly, we establish the intrinsic connections between eigengraphs with small/large weights, graph signals with high/low frequencies, and graphs with homophilic/heterophilic properties.
Revised: Firstly, we establish the intrinsic connections between eigengraphs with small/large weights, graph signals with high/low frequencies, and graphs with homophilic/heterophilic properties.

Original: Secondly, we show how the positive/negative depth d of a GCN affects the eigengraph weights and in turn determines the algorithm’s capability to handle homophilic/heterophilic graph signals.
Revised: Secondly, we show how the positive/negative depth d of a GCN affects the eigengraph weights and in turn determines the algorithm’s expressive power to process homophilic/heterophilic graph signals.

Original: Thirdly, with the help of functional calculus (Shah & Okutmu¸stur, 2020), we present the theoretical feasibility of extending the domain of d from N + to R .
Revised: Thirdly, with the help of functional calculus (Shah & Okutmu¸stur, 2020), we present the theoretical feasibility of extending the domain of d from N + to R .

Original: Finally, by making d a trainable parameter, we present our model R E D-GCN and its variants, which are capable of automatically detecting the homophily/heterophily of the input graph and finding the corresponding optimal depth.
Revised: Finally, by making d a trainable parameter, we present our model R E D-GCN and its variants, which are capable of automatically detecting the homophily/heterophily of the input graph and finding the corresponding optimal depth.

Original: The eigenvectors of a graph Laplacian form a complete set of basis vectors in the n -dimensional space capable of expressing the original node attribute X as a linear combination.
Revised: The eigenvectors of a graph Laplacian form a complete set of basis vectors in the n -dimensional space capable of expressing the original node attribute X as a linear combination.

Original: From the perspective of graph spectrum analysis (Shuman et al., 2013), the frequency of eigenvector u i reflects how much the j -th entry u i [ j ] deviates from the k -th entry
Revised: From the perspective of graph spectrum analysis (Shuman et al., 2013), the frequency of eigenvector u i reflects how much the j -th entry u i [ j ] deviates from the k -th entry

Original: u i [ k ] for each connected node pair v j and v k in G .
Revised: u i [ k ] for each connected node pair v j and v k in G .

Original: This deviation is measured by the set of zero crossings of u i : Z ( u i ) :
Revised: This deviation is measured by the set of zero crossings of u i : Z ( u i ) :

Original: = { e = ( v j , v k ) ∈ E : u i [ j ] u i [ k ] < 0 } , where E is the set of edges in graph G .
Revised: = { e = ( v j , v k ) ∈ E : u i [ j ] u i [ k ] < 0 } , where E is the set of edges in graph G .

Original: Larger/smaller |Z ( u i ) | indicates higher/lower eigenvector frequency.
Revised: Larger/smaller |Z ( u i ) | indicates higher/lower eigenvector frequency.

Original: A zero-crossing also corresponds a negative weighted edge in an eigengraph.
Revised: A zero-crossing also corresponds a negative weighted edge in an eigengraph.

Original: Due to the widely existing positive correlation between λ i and |Z ( u i ) | (Shuman et al., 2013), large/small eigenvalues mostly correspond to the high/low frequencies of the related eigenvectors.
Revised: Due to the widely existing positive correlation between λ i and |Z ( u i ) | (Shuman et al., 2013), large/small eigenvalues mostly correspond to the high/low frequencies of the related eigenvectors.

Original: As illustrated by the toy example of n = 3 in Figure 1, for λ 1 = 0 , we have |Z ( u 1 ) | = 0 , and eigengraph u 1 u ⊤is well-connected with identical edge weight 1 n ; negative edge weights exist in the 2nd and 3rd eigengraphs, indicating more zero crossings ( |Z ( u 2 )
Revised: As illustrated by the toy example of n = 3 in Figure 1, for λ 1 = 0 , we have |Z ( u 1 ) | = 0 , and eigengraph u 1 u ⊤is well-connected with identical edge weight 1 n ; negative edge weights exist in the 2nd and 3rd eigengraphs, indicating more zero crossings ( |Z ( u 2 )

Original: |
Revised: |

Original: = 1 and |Z ( u 3 ) | = 2 ) and higher eigenvector frequencies.
Revised: = 1 and |Z ( u 3 ) | = 2 ) and higher eigenvector frequencies.

Original: Since node labels correlate with their attributes (Zheng et al., 2022a), and node  attributes can be expressed by eigenvectors, the deviation between eigenvector entry pairs naturally implies the extent of heterophily.
Revised: Since node labels correlate with their attributes (Zheng et al., 2022a), and node attribute similarities indicate the extent of smoothness/homophily (Luan et al., 2020; 2021), plus node attributes can be expressed by eigenvectors, the deviation between eigenvector entry pairs naturally implies the extent of heterophily.

Original: Apparently, high frequency eigenvectors and their corresponding eigengraphs have advantage on capturing graph heterophily.
Revised: Apparently, high frequency eigenvectors and their corresponding eigengraphs have advantage on capturing graph heterophily.

Original: High frequency eigengraphs should accordingly take larger weights when modeling heterophilic graphs, while low frequency ones should carry larger weights when dealing with homophilic graphs.
Revised: High frequency eigengraphs should accordingly take larger weights when modeling heterophilic graphs, while low frequency ones should carry larger weights when dealing with homophilic graphs.

Original: In turn, eigengraph weights are controlled by GCN/SGC’s depth d , e.g., for a SGC of depth d , the weight of the i -th eigengraph is (1 − ˜ λ i ) d , and changing the layer d of SGC adjusts the weights of different eigengraphs.
Revised: In turn, eigengraph weights are controlled by GCN/SGC’s depth d , e.g., for a SGC of depth d , the weight of the i -th eigengraph is (1 − ˜ λ i ) d , and changing the layer d of SGC adjusts the weights of different eigengraphs.

Revised: Therefore, depth d controls the model’s expressive power to effectively filter low/high-frequency signals for graph homophily/heterophily.

Original: A question is naturally raised: instead of manually setting the depth d , can d be built into the model as a trainable parameter so that a proper set of the eigengraph weights matching the graph homophily/heterophily can be automatically reached by finding the optimal d in an end-to-end fashion during training?
Revised: A question is naturally raised: instead of manually setting the depth d , can d be built into the model as a trainable parameter so that a proper set of the eigengraph weights matching the graph homophily/heterophily can be automatically reached by finding the optimal d in an end-to-end fashion during training?

Original: Differentiable variables need continuity, which requires the extension of depth d from the discrete positive integer domain ( N + ) to the continuous real number domain R .
Revised: Differentiable variables need continuity, which requires the extension of depth d from the discrete positive integer domain ( N + ) to the continuous real number domain R .

Original: According to functional calculus
Revised: According to functional calculus

Original: (Shah & Okutmu¸stur, 2020), applying an arbitrary function f on a graph
Revised: (Shah & Okutmu¸stur, 2020), applying an arbitrary function f on a graph

Original: Laplacian L sym is equivalent to applying the same function only on the eigenvalue matrix Λ :f
Revised: Laplacian L sym is equivalent to applying the same function only on the eigenvalue matrix Λ :

Original: [Equation]
Revised: [Equation]

Original:  sym [Equation] Λ [Equation]
Revised: which also applies to ˜ L sym and ˜ S .

Original: U ⊤

Original: [Equation]

Original: U   f [Equation] [Equation] f (
Revised: Armed with this, we seek to realize an arbitrary depth SGC via a power function as f (˜

Original: [Equation]
Revised: [Equation]

Original: n [Equation]   U ⊤
Revised: =

Original: [Equation]
Revised: [Equation]

Original: (6) which also applies to ˜ L sym and ˜ S .

Original: Armed with this, we seek to realize an arbitrary depth SGC via a power function as f (˜ S ) =

Original: ˜ S d = ˜

Original: U ( I − ˜ Λ ) d ˜ 
Revised: Λ ) d ˜ U

Original: U ⊤ =
Revised: ⊤ =

Original: (cid:80) n i =1 (1 − ˜ λ i )
Revised: (cid:80) ni =1 (1 − ˜ λ i )

Original: d ˜ u i ˜
Revised: d ˜ u i ˜

Original: u ⊤ i ( d ∈ R ) .
Revised: u ⊤ i ( d ∈ R ) .

Original: 1.53113() d d =2 d =1 d =0 d =d =(a) (1 − ˜ λ ) d .
Revised: 1.53113() d d =2 d =1 d =0 d =d =(a) (1 − ˜ λ ) d .

Original: (0 . 5 ) d d =2 d =1 d[-=
-]
Revised: (0 . 5 ) d d =2 d =1 d[-=
-]

Original: d =d =(b) (1 − 12 λ ) d
Revised: [Figure] [Figure]

Original: However, since ˜ λ i ∈ [0 , 2) , we have (1 − ˜ λ i ) ≤ 0 when 1 ≤ ˜ λ
Revised: However, since ˜ λ i ∈ [0 , 2) , we have (1 − ˜ λ i ) ≤ 0 when 1 ≤ ˜ λ

Original: i < 2 , and for (1 − ˜ λ i ) taking zero or negative values, (1 − ˜ λ i )
Revised: i < 2 , and for (1 − ˜ λ i ) taking zero or negative values, (1 − ˜ λ i )

Original: d is not well-defined or involving complex-number-based calculations for a real-valued d (e.g., ( − 0 . 5) 38 ) (Shah &
Revised: d is not well-defined or involving complex-number-based calculations for a real-valued d (e.g., ( − 0 . 5) 38 ) (Shah &

Original: Okutmu¸stur, 2020).
Revised: Okutmu¸stur, 2020).

Original: Moreover, even for integer-valued d s under which (1 − ˜ λ i )
Revised: Moreover, even for integer-valued d s under which (1 − ˜ λ i )

Original: d is easy to compute, the behavior of (1 − ˜ λ i ) d is complicated versus ˜ λ
Revised: d is easy to compute, the behavior of (1 − ˜ λ i ) d is complicated versus ˜ λ

Original: i and diverges when ˜ λ
Revised: i and diverges when ˜ λ

Original: i = 1 for negative d s, as shown in Figure 2a.
Revised: i = 1 for negative d s, as shown in Figure 2a.

Original: Thus, the favored weight distribution may be hard to obtain by tuning d .
Revised: Thus, the favored weight distribution may be hard to obtain by tuning d .

Original: To avoid such complications and alleviate the difficulties for manipulating the eigengraph weights, a transformation function g ( · ) operating on the graph Laplacian L sym or ˜ L sym is in need to shift g ( λ i ) or g (˜ λ i ) into a proper value range so that its power of a real-valued d is easy to obtain and well-behaved versus λ i or ˜ λ i .
Revised: To avoid such complications and alleviate the difficulties for manipulating the eigengraph weights, a transformation function g ( · ) operating on the graph Laplacian L sym or ˜ L sym is in need to shift g ( λ i ) or g (˜ λ i ) into a proper value range so that its power of a real-valued d is easy to obtain and well-behaved versus λ i or ˜ λ i .

Original: Without the loss of generality, our following analysis focuses on L sym and λ i .
Revised: Without the loss of generality, our following analysis focuses on L sym and λ i .

Original: There may exist multiple choices for g ( · ) satisfying the requirements.
Revised: There may exist multiple choices for g ( · ) satisfying the requirements.

Original: In this work, we focus on the following form:
Revised: In this work, we focus on the following form:

Original: [Equation]
Revised: [Equation]

Original: This choice of g ( · ) holds three properties: (1) Positive eigenvalues.
Revised: This choice of g ( · ) holds three properties: (1) Positive eigenvalues.

Original: Since we have L sym ’s i -th eigenvalue λ i ∈ [0 , 2) , the corresponding eigenvalue of ˆ S is g ( λ i ) =
Revised: Since we have L sym ’s i -th eigenvalue λ i ∈ [0 , 2) , the corresponding eigenvalue of ˆ S is g ( λ i ) =

Original: 12 (2 − λ i ) ∈ (0 , 1] .
Revised: 12 (2 − λ i ) ∈ (0 , 1] .

Original: Thus, the d -th power of g ( λ i ) is computable for any d ∈ R .
Revised: Thus, the d -th power of g ( λ i ) is computable for any d ∈ R .

Original: (2) Monotonicity versus eigenvalues λ .
Revised: (2) Monotonicity versus eigenvalues λ .

Original: As shown in Figure 2b, g ( λ i )
Revised: As shown in Figure 2b, g ( λ i )

Original: d = (1 − 12 λ )
Revised: d = (1 − 12 λ )

Original: d is monotonically increasing/decreasing when λ varies betweenand 2 under negative/positive depth.
Revised: d is monotonically increasing/decreasing when λ varies betweenand 2 under negative/positive depth.

Original: (3) Geometric interpretability.
Revised: (3) Geometric interpretability.

Original: Filter ˆ S can be expressed as:
Revised: Filter ˆ S can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: As shown in Figure 3, in spatial domain, ˆ S is obtained via three operations on adjacency matrix A : normalization, adding self-loops, and scaling all edge weights by 12 , while ˜ S in vanilla GCN/SGC contains only two operations: adding self-loops and normalization.
Revised: As shown in Figure 3, in spatial domain, ˆ S is obtained via 3 operations on adjacency matrix A : normalization, adding self-loops, and scaling all edge weights by 12 (a type of lazy random walk (Luan et al., 2020)), while ˜ S in vanilla GCN/SGC contains 2 operations: adding self-loops and normalization.

Original: With the help of transformation g , the depth d is redefined on real number domain, and the message propagation process of depth d can be realized via the following steps: (1) Eigen-decompose L sym ; (2) Calculate ˆ S d via weight g ( λ i ) d and the weighted sum of all eigengraphs
Revised: With the help of transformation g , the depth d is redefined on real number domain, and the message propagation process of depth d can be realized via the steps sym 𝑣 !

Original: : ˆ S d = U ˆ Λ d U ⊤ =

Original: (cid:80) ni

Original: =1 g ( λ i )

Original: d u i u ⊤

Original: i

Original: (3) Multiply ˆ S d with original node attributes X .

Original: 𝑣! 𝑣 " 𝑣 # 𝑣 ! 𝑣
Revised: 𝑣 " 𝑣 # 𝑣 ! 𝑣

Original: " 𝑣 # 14 12 𝑣 !
Revised: " 𝑣 # 14 12 𝑣 !

Original: 𝑣 " 𝑣 #  1 2𝐃 !"
Revised: 𝑣 " 𝑣 #  1 2𝐃 !"

Original: # 𝐀𝐃 !"
Revised: # 𝐀𝐃 !"

Original: # 1 2𝐈 !
Revised: # 1 2𝐈 !

Original: 𝐒 𝑣 !
Revised: 𝐒 𝑣 !

Original: 𝑣 " 𝑣 # 13 13 #𝐒
Revised: 𝑣 " 𝑣 # 13 13 #𝐒

Original: Remark.
Revised: Negative depth explained.

Original: An intuitive explanation of negative d can be obtained from the perspective of matrix inverse and message diffusion process when d takes integer values.
Revised: An intuitive explanation of negative d can be obtained from the perspective of matrix inverse and message diffusion process when d takes integer values.

Original: Since ˆ S − 1 ˆ S =
Revised: Since ˆ S − 1 ˆ S =

Original: U ˆ Λ − 1 U
Revised: U ˆ Λ − 1 U

Original: ⊤ U ˆ Λ
Revised: ⊤ U ˆ Λ

Original: 1 U ⊤
Revised: 1 U ⊤

Original: =
Revised: =

Original: I , ˆ S − 1 is the inverse matrix of ˆ S .
Revised: I , ˆ S − 1 is the inverse matrix of ˆ S .

Original: In diffusion dynamics, X can be viewed as an intermediate state generated in a series of message propagation steps.
Revised: In diffusion dynamics, X can be viewed as an intermediate state generated in a series of message propagation steps.

Original: ˆ SX effectively propagates the message one-step forward, while ˆ S − 1 can cancel the effect of ˆ S on X and recover the original message by moving backward: ˆ S − 1 ˆ SX =
Revised: ˆ SX effectively propagates the message one-step forward, while ˆ S − 1 can cancel the effect of ˆ S on X and recover the original message by moving backward: ˆ S − 1 ˆ SX =

Original: X .
Revised: X .

Original: Accordingly, ˆ S − 1 X traces back to the message’s previous state in the series.
Revised: Accordingly, ˆ S − 1 X traces back to the message’s previous state in the series.

Original: However, neither A or L has inverse due to their non-positive eigenvalues.
Revised: However, neither A or L has inverse due to their non-positive eigenvalues.

Original: More discussions on the impact of negative depth in spatial domain are presented in Section 4.4.
Revised: More discussions on the impact of negative depth in spatial domain are presented in Section 4.4.

Original: Non-integer d indicates the back- or forward propagation can be a continuous process.
Revised: Non-integer d indicates the back- or forward propagation can be a continuous process.

Original: R E D-GCN-S.
Revised: R E D-GCN-S.

Original: By further making d a trainable parameter, we present our model,
Revised: By further making d a trainable parameter, we present our model,

Original: Redefined DepthGCN-Single
Revised: Redefined DepthGCN-Single

Original: (R E D-GCN-S), whose final node embedding matrix is given by
Revised: (R E D-GCN-S), whose final node embedding matrix is given by

Original: [Equation]
Revised: [Equation]

Original:  (9) where σ ( · ) is the nonlinear activation function; W is a trainable parameter matrix; and d is the trainable depth parameter.
Revised: ) , (9) where σ ( · ) is the nonlinear activation function; W is a trainable parameter matrix; and d is the trainable depth parameter.

Original: As observed from Figure 2b, weight distribution of different frequencies/eigengraphs is tuned via d : (1) for d = 0 , the weight is uniformed distributed among all frequency components ( g ( λ i ) d = 1 ), which implies that no particular frequency is preferred by the graph signal; (2) for d > 0 , weight g ( λ i )
Revised: As observed from Figure 2b, weight distribution of different frequencies/eigengraphs is tuned via d : (1) for d = 0 , the weight is uniformed distributed among all frequency components ( g ( λ i ) d = 1 ), which implies that no particular frequency is preferred by the graph signal; (2) for d > 0 , weight g ( λ i )

Original: d decreases with the corresponding frequency, which indicates the low frequency components are favored so that R E D-GCN-S effectively functions as a low-pass filter and therefore captures graph homophily; (3) for d < 0 , high frequency components gains amplified weights so that R E D-GCN-S serves as a high-pass filter capturing graph heterophily.
Revised: d decreases with the corresponding frequency, which indicates the low frequency components are favored so that R E D-GCN-S effectively functions as a low-pass filter and therefore captures graph homophily; (3) for d < 0 , high frequency components gains amplified weights so that R E D-GCN-S serves as a high-pass filter capturing graph heterophily.

Original: During training, R E D-GCN-S tunes its frequency filtering functionality to suit the underlying graph signal by automatically finding the optimal d .
Revised: During training, R E D-GCN-S tunes its frequency filtering functionality to suit the underlying graph signal by automatically finding the optimal d .

Original: R E D-GCN-D. During optimization, R E D-GCN-S embraces a single depth d unified for all eigengraphs and selects its preferences for either homophily or heterophily.
Revised: R E D-GCN-D. During optimization, R E D-GCN-S embraces a single depth d unified for all eigengraphs and selects its preferences for either homophily or heterophily.

Original: However, R E D-GCN-S requires a full eigen-decomposition of L sym , which can be expensive for large graphs.
Revised: However, R E D-GCN-S requires a full eigen-decomposition of L sym , which can be expensive for large graphs.

Original: Additionally, the high and low frequency components in a graph signal may not be mutually exclusive, namely, there exists the possibility for a graph to simultaneously possess homophilic and heterophilic counterparts.
Revised: Additionally, the high and low frequency components in a graph signal may not be mutually exclusive, namely, there exists the possibility for a graph to simultaneously possess homophilic and heterophilic counterparts.

Original: Therefore, we propose the second variant of R E D-GCN: R E D-GCN-D (Dual), which introduces two separate trainable depths, d h and d l , to gain more flexible weighting of the high and low frequency related eigengraphs respectively.
Revised: Therefore, we propose the second variant of R E D-GCN: R E D-GCN-D (Dual), which introduces two separate trainable depths, d h and d l , to gain more flexible weighting of the high and low frequency related eigengraphs respectively.

Original: Arnoldi method (Lehoucq et al., 1998) is adopted to conduct EVD on L sym and obtain the top- K largest and smallest eigen-pairs ( λ i , u i ) s. By denoting
Revised: Arnoldi method (Lehoucq et al., 1998) is adopted to conduct EVD on L sym and obtain the top- K largest and smallest eigen-pairs ( λ i , u i ) s. By denoting

Original: U l = U [: , 0 : K ] and U h
Revised: U l = U [: , 0 : K ] and U h

Original: =
Revised: =

Original: U
Revised: U

Original: [: , n − K : n ] ( U l and U h ∈ R n × K ), we define a new diffusion matrix ˆ S dual ( d l , d h , K ) as
Revised: [: , n − K : n ] ( U l and U h ∈ R n × K ), we define a new diffusion matrix ˆ S dual ( d l , d h , K ) as

Original: [Equation]
Revised: [Equation]

Original: where ˆ Λ l ∈ R K × K and ˆ Λ h ∈ R K × K are diagonal matrices of the top- K smallest and largest eigenvalues.
Revised: where ˆ Λ l ∈ R K × K and ˆ Λ h ∈ R K × K are diagonal matrices of the top- K smallest and largest eigenvalues.

Original: The final node embedding of R E D-GCN-D is presented as
Revised: The final node embedding of R E D-GCN-D is presented as

Original: [Equation]
Revised: [Equation]

Original: where depths d l and d h are trainable; and W is a trainable parameter matrix.
Revised: where depths d l and d h are trainable; and W is a trainable parameter matrix.

Original: We make R E D-GCN-D scalable on large graphs by choosing K ≪ n , so that ˆ S dual ( d l , d h , K ) approximates the full diffusion matrix by covering only a small subset of all eigengraphs.
Revised: We make R E D-GCN-D scalable on large graphs by choosing K ≪ n , so that ˆ S dual ( d l , d h , K ) approximates the full diffusion matrix by covering only a small subset of all eigengraphs.

Original: For small graphs, we use K =
Revised: For small graphs, we use K =

Original: ⌊ n 2 ⌋ to include all eigengraphs, and ˆ S dual ( d l , d h , K ) thus gains higher flexibility than ˆ S with the help of the two separate depth parameters instead of a unified one.
Revised: ⌊ n 2 ⌋ to include all eigengraphs, and ˆ S dual ( d l , d h , K ) thus gains higher flexibility than ˆ S with the help of the two separate depth parameters instead of a unified one.

Original: Differences with ODE-based GNNs .
Revised: Differences with ODE-based GNNs .

Original: Previous attempts on GNNs with continuous diffusion are mostly inspired by graph diffusion equation, an Ordinary Difference Equation (ODE) characterizing the dynamical message propagation process versus time.
Revised: Previous attempts on GNNs with continuous diffusion are mostly inspired by graph diffusion equation, an Ordinary Difference Equation (ODE) characterizing the dynamical message propagation process versus time.

Original: In contrast, our framework starts from discrete graph convolution operations without explicitly involving ODE.
Revised: In contrast, our framework starts from discrete graph convolution operations without explicitly involving ODE.

Original: CGNN (Xhonneux et al., 2020) aims to build a deep GNN immune to over-smoothing by adopting the neural ODE framework (Chen et al., 2018).
Revised: CGNN (Xhonneux et al., 2020) aims to build a deep GNN immune to over-smoothing by adopting the neural ODE framework (Chen et al., 2018).

Original: But its time parameter t is a non-trainable hyper-parameter predefined within the positive domain, which is the key difference with R E D-GCN.
Revised: But its time parameter t is a non-trainable hyper-parameter predefined within the positive domain, which is the key difference with R E D-GCN.

Original: A critical CGNN component for preventing over-smoothing, restart distribution (the skip connection from the first layer), is not needed in our framework.
Revised: A critical CGNN component for preventing over-smoothing, restart distribution (the skip connection from the first layer), is not needed in our framework.

Original: Moreover, CGNN applies the same depth to all frequency components, while
Revised: Moreover, CGNN applies the same depth to all frequency components, while

Original: R E D-GCN-D has the flexibility to adopt two different depths respectively to be adaptive to high and low frequency components.
Revised: R E D-GCN-D has the flexibility to adopt two different depths respectively to be adaptive to high and low frequency components.

Revised: GRAND (Chamberlain et al., 2021) introduces non-Eular multi-step schemes with adaptive step size to obtain more precise solutions of the diffusion equation.

Revised: Its depth (total integration time) is continuous but still predefined/non-trainable and takes only positive values.

Original: DGC (Wang et al., 2021) decouples the SGC depth into two prefinded non-trainable hyper-parameters: a positive real-valued T controlling the total time and a positive integer-valued K dgc corresponding to the number of diffusion steps.
Revised: DGC (Wang et al., 2021) decouples the SGC depth into two prefinded non-trainable hyper-parameters: a positive real-valued T controlling the total time and a positive integer-valued K dgc corresponding to the number of diffusion steps.

Original: However, realizing negative depth in DGC is non-applicable since the implementation is through propagation by K dgc times, rather than through an arbitrary real-valued exponent d on eigengraph weights in R E D-GCN.
Revised: However, realizing negative depth in DGC is non-applicable since the implementation is through propagation by K dgc times, rather than through an arbitrary real-valued exponent d on eigengraph weights in R E D-GCN.

Original:  E XPERIMENT
Revised:  E XPERIMENT

Original: In this section, we evaluate the proposed R E D-GCN on the semi-supervised node classification task on both homophilic graphs and heterophilic graphs.
Revised: In this section, we evaluate the proposed R E D-GCN on the semi-supervised node classification task on both homophilic graphs and heterophilic graphs.

Original: 4. E XPERIMENT S ETUP
Revised: 4. E XPERIMENT S ETUP

Original: Datasets.
Revised: Datasets.

Original: We use 11 datasets for evaluation, including 4 homophilic graphs: Cora (Kipf & Welling, 2016), Citeseer (Kipf & Welling, 2016), Pubmed (Kipf & Welling, 2016) and DBLP (Bojchevski &
Revised: We use 11 datasets for evaluation, including 4 homophilic graphs: Cora (Kipf & Welling, 2016), Citeseer (Kipf & Welling, 2016), Pubmed (Kipf & Welling, 2016) and DBLP (Bojchevski &

Original: Günnemann, 2017), and 7 heterophilic graphs: Cornell (Pei et al., 2020), Texas (Pei et al., 2020),
Revised: Günnemann, 2017), and 7 heterophilic graphs: Cornell (Pei et al., 2020), Texas (Pei et al., 2020),

Original: Wisconsin (Pei et al., 2020), Actor (Pei et al., 2020), Chameleon (Rozemberczki et al., 2021), Squirrel (Rozemberczki et al., 2021), and cornell5 (Fey & Lenssen, 2019).
Revised: Wisconsin (Pei et al., 2020), Actor (Pei et al., 2020), Chameleon (Rozemberczki et al., 2021), Squirrel (Rozemberczki et al., 2021), and cornell5 (Fey & Lenssen, 2019).

Original: We collect all datasets from the public GCN platform Pytorch-Geometric (Fey & Lenssen, 2019).
Revised: We collect all datasets from the public GCN platform Pytorch-Geometric (Fey & Lenssen, 2019).

Original: For Cora, Citeseer, Pubmed with data splits in Pytorch-Geometric, we keep the training/validation/testing set split as in GCN (Kipf &
Revised: For Cora, Citeseer, Pubmed with data splits in Pytorch-Geometric, we keep the training/validation/testing set split as in GCN (Kipf &

Original: Welling, 2016).
Revised: Welling, 2016).

Original: For the remaining 8 datasets, we randomly split every dataset into 20/20/60% for training, validation, and testing.
Revised: For the remaining 8 datasets, we randomly split every dataset into 20/20/60% for training, validation, and testing.

Original: The statistics of all datasets are presented in Appendix.
Revised: The statistics of all datasets are presented in Appendix.

Original: Baselines and Metrics.
Revised: Baselines and Metrics.

Original: We compare our model with 7 baseline methods,including 4 classic GNNs:
Revised: We compare our model with 7 baseline methods,including 4 classic GNNs:

Original: GCN (Kipf & Welling, 2016), SGC (Wu et al., 2019), APPNP (Klicpera et al., 2018) and ChebNet (Defferrard et al., 2016), and 3 GNNs tailored for heterophilic graphs: FAGCN (Bo et al., 2021),
Revised: GCN (Kipf & Welling, 2016), SGC (Wu et al., 2019), APPNP (Klicpera et al., 2018) and ChebNet (Defferrard et al., 2016), and 3 GNNs tailored for heterophilic graphs: FAGCN (Bo et al., 2021),

Original: GPRGNN (Chien et al., 2020) and H2GCN (Zhu et al., 2020).
Revised: GPRGNN (Chien et al., 2020) and H2GCN (Zhu et al., 2020).

Original: Accuracy (ACC) is used as the evaluation metric.
Revised: Accuracy (ACC) is used as the evaluation metric.

Original: We report the average ACCs with the standard deviation (std) for all methods, each obtained by 5 runs with different initializations.
Revised: We report the average ACCs with the standard deviation (std) for all methods, each obtained by 5 runs with different initializations.

Original: Implementation Details.
Revised: Implementation Details.

Original: See Appendix due to the page limit.
Revised: See Appendix due to the page limit.

Original: 4. N ODE C LASSIFICATION
Revised: 4. N ODE C LASSIFICATION

Original: The semi-supervised node classification performances on homophilic graphs and heterophilic graphs are shown in Table 1 and Table 2 respectively.
Revised: The semi-supervised node classification performances on homophilic graphs and heterophilic graphs are shown in Table 1 and Table 2 respectively.

Original: Homophilic graphs .
Revised: Homophilic graphs .

Original: From Table 1, it is observed that different methods have similar performance on homophilic graphs.
Revised: From Table 1, it is observed that different methods have similar performance on homophilic graphs.

Original: R E D-GCN-S achieves the best accuracies on two datasets: Cora and DBLP.
Revised: R E D-GCN-S achieves the best accuracies on two datasets: Cora and DBLP.

Original: On the remaining two datasets, R E D-GCN-S is only 1 . 1% and 0 .
Revised: On the remaining two datasets, R E D-GCN-S is only 1 . 1% and 0 .

Original: 4% below the best baselines (APPNP on Citeseer and SGC on Pubmed).
Revised: 4% below the best baselines (APPNP on Citeseer and SGC on Pubmed).

Original: For R E D-GCN-D, it obtains similar performance as the other methods, even though it only uses the top- K largest/smallest eigen-pairs
Revised: For R E D-GCN-D, it obtains similar performance as the other methods, even though it only uses the top- K largest/smallest eigen-pairs.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Heterophilic graphs .
Revised: Heterophilic graphs .

Original: R E D-GCN-S/R E D-GCN-D outperforms every baseline on all heterophilic graphs, as shown in Table 2.
Revised: R E D-GCN-S/R E D-GCN-D outperforms every baseline on all heterophilic graphs, as shown in Table 2.

Original: These results demonstrate that without manually setting the model depth and without the prior knowledge of the input graph, R E D-GCN has the capability of automatically detecting the underlying graph heterophily.
Revised: These results demonstrate that without manually setting the model depth and without the prior knowledge of the input graph, R E D-GCN has the capability of automatically detecting the underlying graph heterophily.

Original: We have an interesting observation: on 3 large datasets,
Revised: We have an interesting observation: on 3 large datasets,

Original: Squirrel, Chameleon, and cornell5, even with only a small portion of the eigengraphs, R E D-GCN-D is able to achieve better performance than R E D-GCN-S with the complete set of eigengraphs.
Revised: Squirrel, Chameleon, and cornell5, even with only a small portion of the eigengraphs, R E D-GCN-D is able to achieve better performance than R E D-GCN-S with the complete set of eigengraphs.

Original: This suggests that the graph signal in some real-world graphs might be dominated by a few low and high frequency components, and allowing two independent depth parameters in R E D-GCN-D brings the flexibility to capture the low and high frequencies at the same time.
Revised: This suggests that the graph signal in some real-world graphs might be dominated by a few low and high frequency components, and allowing two independent depth parameters in R E D-GCN-D brings the flexibility to capture the low and high frequencies at the same time.

Original: 4. TRAINABLE DEPTH
Revised: 4. TRAINABLE DEPTH

Original: A systematic study is conducted on the node classification performance w.r.t the trainable depth d .
Revised: A systematic study is conducted on the node classification performance w.r.t the trainable depth d .

Original: Optimal depth .
Revised: Optimal depth .

Original: In Figure 4, the optimal depths and their corresponding classification accuracies are annotated.
Revised: In Figure 4, the optimal depths and their corresponding classification accuracies are annotated.

Original: For two homophilic graphs, Cora and Citeseer, the optimal depths are positive ( 5 . 029 and 3 . 735 ) in terms of the best ACCs, while for two heterophilic graphs, Actor and Squirrel, the optimal depths are negative ( − 0 .
Revised: For two homophilic graphs, Cora and Citeseer, the optimal depths are positive ( 5 . 029 and 3 . 735 ) in terms of the best ACCs, while for two heterophilic graphs, Actor and Squirrel, the optimal depths are negative ( − 0 .

Original: 027 and − 3 .
Revised: 027 and − 3 .

Original: These results demonstrate our model indeed automatically capture graph heterophily/homophily by finding the suitable depth to suppress or amplify the relative weights of the corresponding frequency components.
Revised: These results demonstrate our model indeed automatically capture graph heterophily/homophily by finding the suitable depth to suppress or amplify the relative weights of the corresponding frequency components.

Original: Namely, high/low frequency components are suppressed for homophilic/heterophilic graphs respectively. 
Revised: Namely, high/low frequency components are suppressed for homophilic/heterophilic graphs respectively. [Figure]

Original: Close to zero depth.
Revised: Close to zero depth.

Original: For the two homophilic graphs in Figures 4a and 4b, sharp performance drop is observed when depth d approaches 0, since the eigengraphs gain close-to-uniform weights.
Revised: For the two homophilic graphs in Figures 4a and 4b, sharp performance drop is observed when depth d approaches 0, since the eigengraphs gain close-to-uniform weights.

Original: For the[Figure] heterophilic Actor dataset, its optimal depth − 0 .
Revised: For the heterophilic Actor dataset, its optimal depth − 0 .

Original: 027 is close to 0 , as shown in Figure 4c.
Revised: 027 is close to 0 , as shown in Figure 4c.

Original: In addition, the performance of R E D-GCN-D ( 27 . 6% ) is similar to that of GCN ( 27 . 5% ), both of which are much worse than R E D-GCN-S ( 35 . 3% ).
Revised: In addition, the performance of R E D-GCN-D ( 27 . 6% ) is similar to that of GCN ( 27 . 5% ), both of which are much worse than R E D-GCN-S ( 35 . 3% ).

Original: This result indicates that Actor is a special graph where all frequency components have similar importance.
Revised: This result indicates that Actor is a special graph where all frequency components have similar importance.

Original: Due to the absence of the intermediate frequency components between the high- and low-end ones, the performance of R E D-GCN-D is severely impacted.
Revised: Due to the absence of the intermediate frequency components between the high- and low-end ones, the performance of R E D-GCN-D is severely impacted.

Original: For vanilla GCN, the suppressed weights of the high frequency components deviate from the near-uniform spectrum and thus lead to low ACC on this dataset.
Revised: For vanilla GCN, the suppressed weights of the high frequency components deviate from the near-uniform spectrum and thus lead to low ACC on this dataset.

Original: 4. G RAPH AUGMENTATION AND G EOMETRIC INSIGHTS
Revised: 4. G RAPH AUGMENTATION AND G EOMETRIC INSIGHTS

Original: [Table]
Revised: [Table]

Original: It is especially interesting to analyze what change a negative depth brings to the spatial domain and how such change impacts the subsequent model performance.
Revised: It is especially interesting to analyze what change a negative depth brings to the spatial domain and how such change impacts the subsequent model performance.

Original: Graph augmentation.
Revised: Graph augmentation.

Original: By picking the optimal depth d according to the best performance on the validation set, a new diffusion matrix ˆ S d is obtained.
Revised: By picking the optimal depth d according to the best performance on the validation set, a new diffusion matrix ˆ S d is obtained.

Original: With the optimal d fixed, substituting the normalized adjacency matrix ˜ D − 12 ˜
Revised: With the optimal d fixed, substituting the normalized adjacency matrix ˜ D − 12 ˜

Original: A ˜ D −in Eq.
Revised: A ˜ D −in Eq.

Original: 1 by ˆ S d is equivalent to applying the vanilla GCN to a new topology.
Revised: 1 by ˆ S d is equivalent to applying the vanilla GCN to a new topology.

Original: This topology effectively plays the role of a structural augmentation for the original graph.
Revised: This topology effectively plays the role of a structural augmentation for the original graph.

Original: The impact of such augmentation on performance is tested on 3 heterophilic graphs: Texas, Cornell and Wisconsin, as shown in Table 3.
Revised: The impact of such augmentation on performance is tested on 3 heterophilic graphs: Texas, Cornell and Wisconsin, as shown in Table 3.

Original: Apparently, for the vanilla GCN, the performance obtained with this new topology is superior over that with the raw input graph: it dramatically brings 20% - 30% lifts in ACC.
Revised: Apparently, for the vanilla GCN, the performance obtained with this new topology is superior over that with the raw input graph: it dramatically brings 20% - 30% lifts in ACC.

Original: Moreover, the augmented topologies also make vanilla GCN outperform R E D-GCN-S and R E D-GCN-D on 2 out of the 3 datasets.
Revised: Moreover, the augmented topologies also make vanilla GCN outperform R E D-GCN-S and R E D-GCN-D on 2 out of the 3 datasets.

Original: By nature, the augmented graph is a re-weighted linear combination of the eigengraphs, and its topological structure intrinsically assigns higher weights to eigengraphs corresponding to higher frequencies, as shown in Figures 5.
Revised: By nature, the augmented graph is a re-weighted linear combination of the eigengraphs, and its topological structure intrinsically assigns higher weights to eigengraphs corresponding to higher frequencies, as shown in Figures 5.

Original: [Figure]
Revised: [Table]

Original: matrix ˆ S dand original ˆ Sfor Cornell ( d = − 0 . 362 ).
Revised: W e i g h t s S d S

Original: Geometric properties.
Revised: Geometric properties.

Original: To further understand how the topology of ˆ
Revised: To further understand how the topology of ˆ

Original: S d with a negative optimal d differs from that of ˆ S and why the performance is significantly boosted, a heat map of (ˆ S d − ˆ S ) is presented in Figure 6 for Cornell.
Revised: S d with a negative optimal d differs from that of ˆ S and why the performance is significantly boosted, a heat map of (ˆ S d − ˆ S ) is presented in Figure 6 for Cornell.

Original: 3 First, the dark red diagonal line in the heat map indicates the weights of self-loops are significantly strengthened in the augmented graph, and as a result, in consistency with the previous findings (Zheng et al., 2022a), the raw node attributes make more contributions in determining their labels.
Revised: 3 First, the dark red diagonal line in the heat map indicates the weights of self-loops are significantly strengthened in the augmented graph, and as a result, in consistency with the previous findings (Zheng et al., 2022a), the raw node attributes make more contributions in determining their labels.

Original: These strengthened self-weights also play the similar role as restart distribution or skip connections (Xhonneux et al., 2020) preventing the node embeddings becoming over-smoothed.
Revised: These strengthened self-weights also play the similar role as restart distribution or skip connections (Xhonneux et al., 2020) preventing the node embeddings becoming over-smoothed.

Original: In addition, there is a horizontal line and a vertical line (light yellow line marked by dashed ovals) in the heat map in Figure 6, correspond to the hub node in the graph, namely the node with the largest degree.
Revised: In addition, there is a horizontal line and a vertical line (light yellow line marked by dashed ovals) in the heat map in Figure 6, correspond to the hub node in the graph, namely the node with the largest degree.

Original: Interestingly, the connections between this node and most other nodes in the graph experience a negative weight change.
Revised: Interestingly, the connections between this node and most other nodes in the graph experience a negative weight change.

Original: Therefore, the influence of the hub node on most other nodes are systematically reduced.
Revised: Therefore, the influence of the hub node on most other nodes are systematically reduced.

Original: Consequently, the augmentation amplifies the deviations between node embeddings and facilitates the characterization of graph heterophily.
Revised: Consequently, the augmentation amplifies the deviations between node embeddings and facilitates the characterization of graph heterophily.

Original:  R 
Revised:  R ELATED W ORKS

Original: [Figure]

Original: Graph Convolutional Network (GCN).
Revised: Graph Convolutional Network (GCN).

Original: GCN models can be mainly divided into two categories: (1) spectral graph convolutional networks and (2) spatial convolutional networks.
Revised: GCN models can be mainly divided into two categories: (1) spectral graph convolutional networks and (2) spatial convolutional networks.

Original: In (1), Spectral CNN (Bruna et al., 2013) borrows the idea from convolutional neural network (Goodfellow et al., 2016) to construct a diagonal matrix as the convolution kernel.
Revised: In (1), Spectral CNN (Bruna et al., 2013) borrows the idea from convolutional neural network (Goodfellow et al., 2016) to construct a diagonal matrix as the convolution kernel.

Original: ChebNet
Revised: ChebNet

Original: (Defferrard et al., 2016) adopts a polynomial approximation of the convolution kernel.
Revised: (Defferrard et al., 2016) adopts a polynomial approximation of the convolution kernel.

Original: GCN (Kipf & Welling, 2016) further simplifies the ChebNet via the first order approximation.
Revised: GCN (Kipf & Welling, 2016) further simplifies the ChebNet via the first order approximation.

Original: Recently, (He et al., 2021; Bianchi et al., 2021; Wang & Zhang, 2022) propose more advanced filters as the convolution kernel.
Revised: Recently, (He et al., 2021; Bianchi et al., 2021; Wang & Zhang, 2022) propose more advanced filters as the convolution kernel.

Original: Most works in (2) follow the message-passing mechanism.
Revised: Most works in (2) follow the message-passing mechanism.

Original: GraphSAGE (Hamilton et al., 2017) iteratively aggregates features from local neighborhood.
Revised: GraphSAGE (Hamilton et al., 2017) iteratively aggregates features from local neighborhood.

Original: GAT (Veliˇckovi´c et al., 2017) applies self-attention to the neighbors.
Revised: GAT (Veli ˇ ckovi ´ c et al., 2017) applies self-attention to the neighbors.

Original: APPNP (Klicpera et al., 2018)
Revised: APPNP (Klicpera et al., 2018)

Original: deploys personalized pagerank (Tong et al., 2006) to sample nodes for aggregration.
Revised: deploys personalized pagerank (Tong et al., 2006) to sample nodes for aggregration.

Original: MoNet (Monti et al., 2017) unifies GCNs in the spatial domain.
Revised: MoNet (Monti et al., 2017) unifies GCNs in the spatial domain.

Original: The Depth of GCN and Over-smoothing.
Revised: The Depth of GCN and Over-smoothing.

Original: A large amount of works focus on the over-smoothing issue.
Revised: A large amount of works focus on the over-smoothing issue.

Original: Its intrinsic cause is demystified: a linear GCN layer is a Laplacian smoothing operator (Li et al., 2018; Wu et al., 2019).
Revised: Its intrinsic cause is demystified: a linear GCN layer is a Laplacian smoothing operator (Li et al., 2018; Wu et al., 2019).

Original: PairNorm (Zhao & Akoglu, 2019) forces distant nodes to be distinctive by adding an intermediate normalization layer.
Revised: PairNorm (Zhao & Akoglu, 2019) forces distant nodes to be distinctive by adding an intermediate normalization layer.

Original: Dropedge (Rong et al., 2019), DeepGCN (Li et al., 2019), AS-GCN (Huang et al., 2018), and JK-net (Xu et al., 2018) borrow the idea of ResNet (He et al., 2016) to dis-intensify smoothing.
Revised: Dropedge (Rong et al., 2019), DeepGCN (Li et al., 2019), AS-GCN (Huang et al., 2018), and JK-net (Xu et al., 2018) borrow the idea of ResNet (He et al., 2016) to dis-intensify smoothing.

Original: DeeperGXX (Zheng et al., 2021) adopts a topology-guided graph contrastive loss for connected node pairs to obtain discriminative representations.
Revised: DeeperGXX (Zheng et al., 2021) adopts a topology-guided graph contrastive loss for connected node pairs to obtain discriminative representations.

Original: Most works aim to build deep GCNs (i.e., d is a large positive integer) by reducing over-smoothing, while
Revised: Most works aim to build deep GCNs (i.e., d is a large positive integer) by reducing over-smoothing, while

Original: R E D-GCN extends the depth from N + to R and explores the negative depth.
Revised: R E D-GCN extends the depth from N + to R and explores the negative depth.

Original: Node Classification on Homophilic and Heterophilic Graphs.
Revised: Node Classification on Homophilic and Heterophilic Graphs.

Original: GCN/GNN models mostly follow the homophily assumption that connected nodes tend to share similar labels (Kipf & Welling, 2016;
Revised: GCN/GNN models mostly follow the homophily assumption that connected nodes tend to share similar labels (Kipf & Welling, 2016;

Original: Veli ˇ ckovi ´ c et al., 2017; Hamilton et al., 2017).
Revised: Veli ˇ ckovi ´ c et al., 2017; Hamilton et al., 2017).

Original: Recently, heterophilic graphs, in which neighbors often have disparate labels, attract lots of attention.
Revised: Recently, heterophilic graphs, in which neighbors often have disparate labels, attract lots of attention.

Original: Geom-GCN (Pei et al., 2020) and H2GCN (Zhu et al., 2020) extend the neighborhood for aggregation.
Revised: Geom-GCN (Pei et al., 2020) and H2GCN (Zhu et al., 2020) extend the neighborhood for aggregation.

Original: FAGCN (Bo et al., 2021) and GPRGNN (Chien et al., 2020) adaptively integrate the high/low frequency signals with trainable parameters.
Revised: FAGCN (Bo et al., 2021) and GPRGNN (Chien et al., 2020) adaptively integrate the high/low frequency signals with trainable parameters.

Original: Alternative message-passing mechanisms have been proposed in HOG-GCN (Wang & Zhang, 2022) and CPGNN (Zhu et al., 2021).
Revised: Alternative message-passing mechanisms have been proposed in HOG-GCN (Wang & Zhang, 2022) and CPGNN (Zhu et al., 2021).

Revised: The latest related works include ACM-GCN (Luan et al., 2021; 2022b), LINKX (Lim et al., 2021), BernNet (He et al., 2021), GloGNN (Li et al., 2022) and GBKGNN (Du et al., 2022).

Original: More related works can be found in a recent survey (Zheng et al., 2022b).
Revised: Other works can be found in a recent survey (Zheng et al., 2022b).

Original:  C ONCLUSION AND F UTURE W ORK
Revised:  C ONCLUSION AND F UTURE W ORK

Original: To our best knowledge, this work presents the first effort to make GCN’s depth trainable by redefining it on the real number domain.
Revised: To our best knowledge, this work presents the first effort to make GCN’s depth trainable by redefining it on the real number domain.

Original: We unveil the interdependence between negative GCN depth and graph heterophily.
Revised: We unveil the interdependence between negative GCN depth and graph heterophily.

Original: A novel problem of automatic GCN depth tuning for graph homophily/heterophily detection is formulated, and we propose a simple and powerful solution named R E D-GCN with two variants (R E D-GCN-S and R E D-GCN-D).
Revised: A novel problem of automatic GCN depth tuning for graph homophily/heterophily detection is formulated, and we propose a simple and powerful solution named R E D-GCN with two variants (R E D-GCN-S and R E D-GCN-D).

Original: An effective graph augmentation method is also discussed via the new understanding on the message propagation mechanism generated by the negative depth.
Revised: An effective graph augmentation method is also discussed via the new understanding on the message propagation mechanism generated by the negative depth.

Original: Superior performance of our method is demonstrated via extensive experiments with semi-supervised node classification on 11 graph datasets.
Revised: Superior performance of our method is demonstrated via extensive experiments with semi-supervised node classification on 11 graph datasets.

Original: The new insights on GCN’s depth obtained by our work may open a new direction for future research on spectral and spatial GNNs.
Revised: The new insights on GCN’s depth obtained by our work may open a new direction for future research on spectral and spatial GNNs.

Original: Since R E D-GCN requires to conduct eigen-decomposition of the graph Laplacian, it is not directly applicable to inductive and dynamic graph learning problems, which we leave for future exploration.
Revised: Since R E D-GCN requires to conduct eigen-decomposition of the graph Laplacian, it is not directly applicable to inductive and dynamic graph learning problems, which we leave for future exploration