SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\sigma (\cdot)$  is the nonlinear activation function;  $\mathbf{W}$  is a trainable parameter matrix; and  $d$  is the trainable depth parameter. As observed from Figure 2b, weight distribution of different frequencies/eigengraphs is tuned via  $d$ : (1) for  $d = 0$ , the weight is uniformly distributed among all frequency components  $(g(\lambda_i)^d = 1)$ , which implies that no particular frequency is preferred by the graph signal; (2) for  $d > 0$ , weight  $g(\lambda_i)^d$  decreases with the corresponding frequency, which indicates the low frequency components are favored so that TEDGCN-S effectively functions as a low-pass filter and therefore captures graph homophily; (3) for  $d < 0$ , high frequency components gains amplified weights so that TEDGCN-S serves as a high-pass filter capturing graph heterophily. During training, TEDGCN-S tunes its frequency filtering functionality to suit the underlying graph signal by automatically finding the optimal  $d$ .
TEDGCN-D algorithm. During optimization, TEDGCN-S embraces a single depth  $d$  unified for all eigengraphs and selects its preferences for either homophily or heterophily. However, TEDGCN-S requires a full eigen-decomposition of  $\mathbf{L}_{sym}$ , which can be expensive for large graphs. Additionally, the high and low frequency components in a graph signal may not be mutually exclusive, namely, there exists the possibility for a graph to simultaneously possess homophilic and heterophilic counterparts. Therefore, we propose the second variant of TEDGCN: TEDGCN-D (Dual), which introduces two separate trainable depths,  $d_h$  and  $d_l$ , to gain more flexible weighting of the high and low frequency related eigengraphs respectively. Arnoldi method [30] is adopted to conduct EVD on  $\mathbf{L}_{sym}$  and obtain the top- $K$  largest and smallest eigen-pairs  $(\lambda_i, \mathbf{u}_i)\mathrm{s}$ . By denoting  $\mathbf{U}_l = \mathbf{U}[:, 0:K]$  and  $\mathbf{U}_h = \mathbf{U}[:, n - K:n]$  ( $\mathbf{U}_l$  and  $\mathbf{U}_h \in \mathbb{R}^{n \times K}$ ), we define a new diffusion matrix  $\hat{\mathbf{S}}_{\text{dual}}(d_l, d_h, K)$  as
$$
\hat {\mathbf {S}} _ {d u a l} \left(d _ {l}, d _ {h}, K\right) = \mathbf {U} _ {l} \hat {\boldsymbol {\Lambda}} _ {l} ^ {d _ {l}} \mathbf {U} _ {l} ^ {\top} + \mathbf {U} _ {h} \hat {\boldsymbol {\Lambda}} _ {h} ^ {d _ {h}} \mathbf {U} _ {h} ^ {\top}, \tag {10}
$$
where  $\hat{\Lambda}_l\in \mathbb{R}^{K\times K}$  and  $\hat{\Lambda}_h\in \mathbb{R}^{K\times K}$  are diagonal matrices of the top-  $K$  smallest and largest eigenvalues. The final node embedding of TEDGCN-D is presented as
$$
\mathbf {H} = \sigma \left(\hat {\mathbf {S}} _ {d u a l} \left(d _ {l}, d _ {h}, K\right) \mathbf {X} \mathbf {W}\right), \tag {11}
$$
where depths  $d_{l}$  and  $d_{h}$  are trainable; and  $\mathbf{W}$  is a trainable parameter matrix. We make TEDGCN-D scalable on large graphs by choosing  $K \ll n$ , so that  $\hat{\mathbf{S}}_{\text{dual}}(d_{l}, d_{h}, K)$  approximates the full diffusion matrix by covering only a small subset of all eigengraphs. For small graphs, we use  $K = \left\lfloor \frac{n}{2} \right\rfloor$  to include all eigengraphs, and  $\hat{\mathbf{S}}_{\text{dual}}(d_{l}, d_{h}, K)$  thus gains higher flexibility than  $\hat{\mathbf{S}}$  with the help of the two separate depth parameters instead of a unified one.
Complexity Analysis. TEDGCN has two major steps: (1) conduct eigen-decomposition of  $\hat{\mathbf{A}}$ ; (2) train a one-layer GCN on the augmented graph  $\hat{\mathbf{S}}^d$ . For Step (1), naive eigen-decomposition has a time complexity of  $O(n^{3})$ . However, by adopting the Lanczos method [30] and only keeping the top- $K$  largest/smallest eigenvalues in TEDGCN-D, the time complexity can be reduced to  $O(nK^2 + mK)$  [24], where  $m$  is the number of edges and  $K << n$ . For Step (2), via dense matrix multiplications, computing the approximated  $\hat{\mathbf{S}}^d$  takes  $O(nK^2 + n^2 K)$ , and conducting message propagation takes  $O(n^2 q + nqc)$ , where  $q$  and  $c$  denote the dimension of node feature vectors and the number of node classes. Potential improvements of efficiency: since the diffusion matrix in Subsection 4.4 of the augmented graph is sparse for known real-world graphs, the complexity of Step (2) can also be further simplified: the  $O(n^2 K)$  term can be reduced to  $O(|E_{aug}|K)$ , where  $E_{aug}$  is the edge set of the augmented graph.
Differences with ODE-based GNNs. Previous attempts on GNNs with continuous diffusion are mostly inspired by graph diffusion equation, an Ordinary Difference Equation (ODE) characterizing the dynamical message propagation process versus time. In contrast, our framework starts from discrete graph convolution operations without explicitly involving ODE. CGNN [60] aims to build a deep GNN immune to over-smoothing by adopting the neural ODE framework [9]. But its time parameter  $t$  is a non-trainable hyper-parameter predefined within the positive domain, which is the key difference with TEDGCN. A critical CGNN component for preventing over-smoothing, restart distribution (the skip connection from the first layer), is not needed in our framework. Moreover, CGNN applies the same depth to all frequency components, while TEDGCN-D has the flexibility to adopt two different depths respectively to be adaptive to high and low frequency components. GRAND [8] introduces non-Eular multi-step schemes with adaptive step size to obtain more precise solutions of the diffusion equation. Its depth (i.e., the total integration time) is continuous but still
Case  $\lambda_{i} = 2$ , namely  $g(\lambda_i) = 0$ , is excluded since it corresponds to the existence of bipartite components.
predefined/non-trainable and only admits positive values. DGC [58] decouples the SGC depth into two predefined non-trainable hyper-parameters: a positive real-valued  $T$  controlling the total time and a positive integer-valued  $K_{dgc}$  corresponding to the number of diffusion steps. However, realizing negative depth in DGC is non-applicable since the implementation is through propagation by  $K_{dgc}$  times, rather than through an arbitrary real-valued exponent  $d$  on eigraph weights in TEDGCN.
In this section, we evaluate the proposed TEDGCN on the semi-supervised node classification task on both homophilic and heterophilic graphs.
Datasets. We use 11 datasets for evaluation, including 4 homophilic graphs: Cora [28], CiteSeer [28], Pubmed [28] and DBLP [6], and 7 heterophilic graphs: Cornell [41], Texas [41], Wisconsin [41], Actor [41], Chameleon [44], Squirrel [44], and cornell5 [15]. We collect all datasets from the public GCN platform Pytorch-Geometric [15]. For Cora, CiteSeer, Pubmed with data splits in Pytorch-Geometric, we keep the same training/validation/testing set split as in GCN [28]. For the remaining 8 datasets, we randomly split every dataset into  $20/20/60\%$  for training, validation, and testing.
Baselines and metrics. Here, we compare our model with 7 baseline methods, including 4 classic GNNs: GCN [28], SGC [59], APPNP [29] and ChebNet [12], and 3 GNNs tailored for heterophilic graphs: FAGCN [5], GPRGNN [10] and H2GCN [76]. Accuracy (ACC) is used as the evaluation metric. We report the average ACCs with the standard deviation (std) for all methods, each obtained by 5 runs with different initializations.
Additional contents. In Appendix, we provide more contents related to experiments including (1) the statistics of all datasets (Appendix A.1); (2) experimental results on synthetic graphs with controllable edge homophily/heterophily levels to demonstrate the effectiveness of negative depth (Appendix A.2); (3) comparison with additional methods including ACM-GCN [37, 38], LINKX [34], BernNet [22], GBK-GNN [14], HOG-GCN [57], Geom-GCN [41] and GloGNN [33] under the commonly used fully-supervised setting that training/validation/testing set split is  $48/32/20\%$  for training, validation, and testing based on the results reported in the original papers (Appendix A.3); (4) implementation details (Appendix A.4); and (5) visualization of node embeddings w.r.t.  $d$  (Appendix A.5).
4.2 Node Classification
The semi-supervised node classification performances on homophilic graphs and heterophilic graphs are shown in Table 1 and Table 2 respectively.
Homophilic graphs. From Table 1, it is observed that different methods have similar performance on homophilic graphs. TEDGCN-S achieves the best accuracies on two datasets: Cora and DBLP. On the remaining two datasets, TEDGCN-S is only  $1.1\%$  and  $0.4\%$  below the best baselines (APPNP on Citeseer and SGC on Pubmed). For TEDGCN-D, it obtains similar performance as the other methods, even though it only uses the top- $K$  largest/smallest eigen-pairs.
Table 1: Performance comparison (mean±std accuracy  $(\%)$ ) on homophilic graphs.
Datasets	Cora	Citeseer	Pubmed	DBLP
GCN	80.8±0.8	70.5±0.6	78.8±0.6	84.1±0.2
SGC	80.9±0.4	70.8±0.8	79.6±0.4	84.1±0.2
APPNP	81.0±1.0	71.9±0.4	79.3±0.2	83.0±0.5
GPRGNN	82.0±0.7	69.3±0.9	78.6±0.7	84.5±0.3
FAGCN	80.3±0.4	71.7±0.8	78.5±0.9	82.4±0.7
H2GCN	78.8±1.0	70.5±1.0	77.9±0.3	82.4±0.3
ChebNet	78.8±0.5	71.1±0.4	78.1±0.8	83.1±0.1
TEDGCN-S	82.5±1.1	70.8±0.7	79.2±0.2	84.7±0.3
TEDGCN-D	82.4±0.7	70.6±0.6	77.9±0.3	84.2±0.2
Heterophilic graphs. Our TEDGCN-S or TEDGCN-D outperforms every baseline on all heterophilic graphs, as shown in Table 2. These results demonstrate that without manually setting the
Table 2: Performance comparison (mean±std accuracy  $(\%)$ ) on heterophilic graphs.
Datasets	Texas	Cornell	Wisconsin	Actor	Squirrel	Chameleon	cornell5
GCN	55.9±3.4	44.3±4.4	51.4±2.2	27.5±0.5	35.8±1.3	55.2±1.8	67.9±0.2
SGC	58.7±3.1	43.8±4.4	47.3±2.1	28.0±0.8	37.2±1.8	55.3±1.0	67.4±0.5
APPNP	55.1±3.7	51.5±2.4	58.0±3.1	32.8±0.8	29.5±0.9	46.7±0.8	68.3±0.5
GPRGNN	61.3 ±5.8	53.3±4.6	71.0±4.8	33.6±0.4	34.1±1.0	55.0±3.9	67.3±0.3
FAGCN	60.2±7.8	54.8±7.4	60.1±5.2	32.3±0.5	31.2±1.6	50.4±1.9	68.3±0.7
H2GCN	68.8±6.5	61.4±4.4	69.9±5.3	33.9±0.3	30.4±0.9	48.8±1.9	68.4±0.2
ChebNet	76.2±2.9	66.7±3.9	75.4±3.5	34.3±0.5	31.8±0.5	49.6±1.8	OOM
TEDGCN-S	77.6±5.9	72.0±5.8	82.0±2.6	35.3±0.7	38.2±1.2	55.7±1.3	68.5±0.4
TEDGCN-D	77.1 ±2.5	72.0 ±2.8	81.5 ±2.4	27.6 ±0.8	44.2±0.9	56.9±0.9	70.0±0.2
(b)CiteSeer
Figure 4: Node classification accuracy w.r.t. the trainable depth  $d$  on four datasets: Cora, Citeseer, Actor and Squirrel. (the optimal  $d$ , accuracy) is annotated (e.g.,  $(-0.027, 36.9\%)$  for Actor).
(c) Actor
model depth and without the prior knowledge of the input graph (i.e., whether the input graph is homophilic or heterophilic), TEDGCN has the capability of automatically detecting the underlying graph heterophily. We have an interesting observation: on 3 large datasets, Squirrel, Chameleon, and cornell5, even with only a small portion of the eigengraphs, TEDGCN-D is able to achieve better performance than TEDGCN-S with the complete set of eigengraphs. This suggests that the graph signal in some real-world graphs might be dominated by a few low and high frequency components, and allowing two independent depth parameters in TEDGCN-D brings the flexibility to capture the low and high frequencies at the same time.
4.3 Trainable Depth
A systematic study is conducted on the node classification performance w.r.t. the trainable depth  $d$ . Optimal depth. In Figure 4, the optimal depths and their corresponding classification accuracies are annotated. For two homophilic graphs, Cora and Citeseer, the optimal depths are positive (5.029 and 3.735) in terms of the best ACCs, while for two heterophilic graphs, Actor and Squirrel, the optimal depths are negative (-0.027 and -3.751). These results demonstrate that our model can indeed automatically capture graph heterophily/homophily by finding the suitable depth to suppress or amplify the relative weights of the corresponding frequency components. Namely, high/low frequency components are suppressed for homophilic/heterophilic graphs respectively.
Close to zero depth. For the two homophilic graphs in Figures 4a and 4b, sharp performance drop is observed when depth  $d$  approaches 0, since the eigengraphs gain close-to-uniform weights. For the heterophilic Actor dataset, its optimal depth  $-0.027$  is close to 0, as shown in Figure 4c. In addition, the performance of TEDGCN-D  $(27.6\%)$  is similar to that of GCN  $(27.5\%)$ , both of which are much worse than TEDGCN-S  $(35.3\%)$ . This result indicates that Actor is a special graph where all frequency compo
nents have similar importance. Due to the absence of the intermediate frequency components between the high- and low-end ones, the performance of TEDGCN-D is severely impacted. For vanilla GCN, the suppressed weights of the high frequency components deviate from the near-uniform spectrum and thus lead to low ACC on this dataset.
Table 3: The performance of one-layer vanilla GCN over the augmented  $\hat{\mathbf{S}}^d$ .
Datasets	Texas	Cornell	Wisconsin
GCN	55.9	44.3	51.4
TEDGCN-S	77.6	72.0	82.0
TEDGCN-D	77.1	72.0	81.5
GCN (Sd)	75.9	72.7	83.4
4.4 Graph Augmentation and Geometric Insights
It is especially interesting to analyze what change a negative depth brings to the spatial domain and how such change impacts the subsequent model performance.
Graph augmentation via negative depth. By picking the optimal depth  $d$  according to the best performance on the validation set, a new diffusion matrix  $\hat{\mathbf{S}}^d$  is obtained. With the optimal  $d$  fixed,
(a) Texas  $(d = -0.328)$
(b) Cornell  $(d = -0.362)$
Figure 5: The weights of eigengraphs w.r.t. eigenvalues on the augmented diffusion matrix  $\hat{\mathbf{S}}^d$  and original  $\hat{\mathbf{S}}$ .
(c) Wisconsin  $(d = -0.280)$
(a) Texas
Figure 6: The difference between the augmented diffusion matrix and the original one  $\hat{\mathbf{S}}^d -\hat{\mathbf{S}}$
(b) Cornell
(c) Wisconsin
substituting the normalized adjacency matrix  $\tilde{\mathbf{D}}^{-\frac{1}{2}}\tilde{\mathbf{A}}\tilde{\mathbf{D}}^{-\frac{1}{2}}$  in Eq. (1) by  $\hat{\mathbf{S}}^d$  is equivalent to applying the vanilla GCN to a new topology. This topology effectively plays the role of a structural augmentation for the original graph. The impact of such augmentation on performance is tested on 3 heterophilic graphs: Texas, Cornell and Wisconsin, as shown in Table 3. Apparently, for the vanilla GCN, the performance obtained with this new topology is superior over that with the raw input graph: it dramatically brings  $20\% -30\%$  lifts in ACC. Moreover, the augmented topologies also make vanilla GCN outperform TEDGCN-S and TEDGCN-D on 2 out of the 3 datasets. By nature, the augmented graph is a re-weighted linear combination of the eigengraphs, and its topological structure intrinsically assigns higher weights to eigengraphs corresponding to higher frequencies, as shown in Figure 5.
Geometric properties. To further understand how the topology of  $\hat{\mathbf{S}}^d$  with a negative optimal  $d$  differs from that of  $\hat{\mathbf{S}}$  and why the performance is significantly boosted, heat maps of  $(\hat{\mathbf{S}}^d - \hat{\mathbf{S}}) \mathrm{s}$  are presented in Figure 6. First, the dark red diagonal line in the heat map indicates the weights of self-loops are significantly strengthened in the augmented graph, and as a result, in consistency with the previous findings [71], the raw node attributes make more contributions in predicting their labels. These strengthened self-weights also play the similar role as restart distribution or skip connections [60] preventing the node embeddings becoming over-smoothed. In addition, there is a horizontal line and a vertical line (light yellow line marked by dashed ovals) in each heat map in Figure 6, correspond to the hub node in the graph, namely the node with the largest degree. Interestingly, the connections between this node and most other nodes in the graph experience a negative weight change. Therefore, the influence of the hub node on most other nodes are systematically reduced. Consequently, the augmentation amplifies the deviations between node embeddings and facilitates the characterization of graph heterophily.
Graph convolutional network (GCN). GCN models can be mainly divided into two categories: (1) spectral graph convolutional networks and (2) spatial convolutional networks. In (1), Spectral CNN [7] borrows the idea from convolutional neural network [18] to construct a diagonal matrix as the convolution kernel. ChebNet [12] adopts a polynomial approximation of the convolution kernel. GCN [28] further simplifies the ChebNet via the first order approximation. Recently, [22, 4, 57] propose more advanced filters as the convolution kernel. Most works in (2) follow the message-passing mechanism. GraphSAGE [19] iteratively aggregates features from local neighborhood. GAT [50] applies self-attention to the neighbors. APPNP [29] deploys personalized pagerank [49] to sample nodes for aggregation. MoNet [40] unifies GCNs in the spatial domain.
The depth of GCN and over-smoothing. A large amount of works focus on the over-smoothing issue. Its intrinsic cause is demystified: a linear GCN layer is a Laplacian smoothing operator [32, 59]. PairNorm [69] forces distant nodes to be distinctive by adding an intermediate normalization layer. Droppedge [43], DeepGCN [31], AS-GCN [23], and JK-net [61] borrow the idea of ResNet [21] to dis-intensify smoothing. DeeperGXX [70] adopts a topology-guided graph contrastive loss for connected node pairs to obtain discriminative representations. Most works aim to build deep GCNs (i.e.,  $d$  is a large positive integer) by reducing over-smoothing, while TEDGCN extends the depth from  $\mathbb{N}+$  to  $\mathbb{R}$  and explores the negative depth.
Node classification on homophilic and heterophilic graphs. GCN/GNN models mostly follow the homophily assumption that connected nodes tend to share similar labels [28, 50, 19]. Recently, heterophilic graphs, in which neighbors often have disparate labels, attract lots of attention. GeomGCN [41] and H2GCN [76] extend the neighborhood for aggregation. FAGCN [5] and GPRGNN [10] adaptively integrate the high/low frequency signals with trainable parameters. Alternative message-passing mechanisms have been proposed in HOG-GCN [57] and CPGNN [75]. The latest related works include ACM-GCN [37, 38], LINKX [34], BernNet [22], GloGNN [33] and GBK-GNN [14]. Other works can be found in a recent survey [72].
To our best knowledge, this work presents the first effort to make GCN's depth become negative and trainable by redefining it on the real number domain. We unveil the intrinsic connection between negative GCN depth and graph heterophily. A novel problem of automatic GCN depth tuning for graph homophily/heterophily detection is formulated, and we propose a simple and powerful solution named TEDGCN with two variants (TEDGCN-S and TEDGCN-D). An effective graph augmentation method is also discussed via the new understanding on the message propagation mechanism generated by the negative depth. Superior performance of our method is demonstrated via extensive experiments with semi-supervised node classification on 11 graph datasets. The new insights on GCN's depth obtained by our work may open a new direction for future research on spectral and spatial GNNs. Our paper studies the depth of graph convolutional network and graph spectrum, which has no negative ethical impacts on society. Since TEDGCN requires to conduct eigen-decomposition of the graph Laplacian, it is not directly applicable to inductive and dynamic graph learning problems. In addition, there may exist other options for the transformation functions such as the pseudoinverse of the Laplacian. We leave these for future exploration.
[1] Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In Proceedings of the 36 th International Conference on Machine Learning, 2019.
[2] Yikun Ban, Yuchen Yan, Arindam Banerjee, and Jingrui He. Ee-net: Exploitation-exploration neural networks in contextual bandits. arXiv preprint arXiv:2110.03177, 2021.
[3] Yikun Ban, Yuchen Yan, Arindam Banerjee, and Jingrui He. Neural exploitation and exploration of contextual bandits. arXiv preprint arXiv:2305.03784, 2023.
[64] Yuchen Yan, Si Zhang, and Hanghang Tong. Bright: A bridging algorithm for network alignment. In Proceedings of the Web Conference 2021, pages 3907-3917, 2021.
[65] Yuchen Yan, Qinghai Zhou, Jinning Li, Tarek Abdelzaher, and Hanghang Tong. Dissecting cross-layer dependency inference on multi-layered inter-dependent networks. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 2341-2351, 2022.
[66] Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, pages 40-48. PMLR, 2016.
[67] Zhichen Zeng, Boxin Du, Si Zhang, Yinglong Xia, Zhining Liu, and Hanghang Tong. Hierarchical multi-marginal optimal transport for network alignment. arXiv preprint arXiv:2310.04470, 2023.
[68] Zhichen Zeng, Si Zhang, Yinglong Xia, and Hanghang Tong. Parrot: Position-aware regularized optimal transport for network alignment. In Proceedings of the ACM Web Conference 2023, pages 372-382, 2023.
[69] Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. In International Conference on Learning Representations, 2019.
[70] Lecheng Zheng, Dongqi Fu, and Jingrui He. Tackling oversmoothing of gnns with contrastive learning. arXiv preprint arXiv:2110.13798, 2021.
[71] Wenqing Zheng, W Edward Huang, Nikhil Rao, Sumeet Katariya, Zhangyang Wang, and Karthik Subbian. Cold brew: Distilling graph node representations with incomplete or missing neighbor- hoods. In International Conference on Learning Representations, 2022.
[72] Xin Zheng, Yixin Liu, Shirui Pan, Miao Zhang, Di Jin, and Philip S Yu. Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv:2202.07082, 2022.
[73] Qinghai Zhou, Liangyue Li, Nan Cao, Lei Ying, and Hanghang Tong. Admiring: Adversarial multi-network mining. In 2019 IEEE International Conference on Data Mining (ICDM), pages 1522-1527. IEEE, 2019.
[74] Qinghai Zhou, Liangyue Li, Xintao Wu, Nan Cao, Lei Ying, and Hanghang Tong. Attent: Active attributed network alignment. In Proceedings of the Web Conference 2021, pages 3896-3906, 2021.
[75] Jiong Zhu, Ryan A Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K Ahmed, and Danai Koutra. Graph neural networks with heterophily. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11168-11176, 2021.
[76] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33:7793-7804, 2020.
- Subsection A.1 presents the statistics of datasets used in the Experiment Section (Section 4) in the main contents.
- Subsection A.2 presents the experimental results on synthetic datasets with controllable homophily/heterophily level.
- Subsection A.3 compares TEDGCN with some latest methods on a different split (48%, 32%, 20%) for training, validation and testing of benchmark datasets. In this subsection, we also compare all methods (15 baselines and TEDGCN) under a unified setting on Texas, Cornell, Wisconsin and Actor datasets.
- Subsection A.4 introduces the details of the implementation of TEDGCN.
- Subsection A.5 is a visualization of node embeddings w.r.t. different depths ( $d$  values) on both homophilic (Cora) and heterophilic (Texas) graphs.
A.1 The Statistics of Datasets
Table 4: The statistics of datasets.
Datasets	Cora	Citeseer	Pubmed	DBLP	Texas	Wisconsin	Cornell	Actor	Chameleon	Squirrel	cornell5
#Nodes	2,708	3,327	19,717	17,716	183	251	183	7,600	2,277	5,201	18,660
#Edges	10,556	9,104	88,648	105,734	325	515	298	30,019	62,792	396,846	158,1554
#Features	1,433	3,703	500	1,639	1,703	1,703	1,703	932	2,325	2,089	4,735
#Classes	7	6	3	4	5	5	5	5	5	5	5
A.2 Experiments on Synthetic Datasets