SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Intuitively, the first two channels perform distinct topology-based propagations on node feature  $X$ , and employ inverse or constant summation to aggregate multi-hop information, in contrast to the local decaying summation ( $\theta_l \to 0$  when  $l \to \infty$ ) commonly adopted in homophilous GNNs. Hence, such summations are suitable for retrieving long-range information under heterophily. The raw matrix  $X$  is also directly used as one input channel to depict node identity, which is a ubiquitous practice known as the skip connection or all-pass filter in heterophilous GNNs [12, 45, 20, 22].
Illustrated in Figure 2, the inverse embedding  $P_{X,H}$  is based on the intuition that, as neighbors tend to be different from the ego node, their features are also dissimilar. Hence in propagation, the embedding of the ego node should contain the previous embedding of itself, as well as the inverse of adjacent embeddings, which is exactly the interpretation of propagating node features by graph Laplacian matrix  $\tilde{L} = I - \tilde{A}$ , while an additional identity matrix is applied to balance the embedding distribution. The second embedding  $P_{X,L2}$  performs a 2-hop propagation through the graph and aggregates the results of multi-scale neighbors. It echoes the earlier statement on the importance of 2-hop adjacency from the feature aspect. Note that for  $P_{X,L2}$ , the employed adjacency matrix is  $\bar{A}$  which escapes self-loops, since it is shown to be relatively favorable for capturing non-local homophily in multi-hop propagations compared with  $\tilde{A}$  [13, 23].
Spectral Analysis. Assume that  $\| X(u)\| = \| P(u)\| = 1$ . We first examine the following regularization problem optimizing the embedding  $P$  based on input  $X$  for homophilous graphs [43]:
$$
\boldsymbol {P} _ {X, L} = \underset {\| \boldsymbol {P} (u) \| = 1, \forall u \in V} {\arg \min } \sum_ {u, v \in V} A (u, v) \| \boldsymbol {P} (u) / d ^ {a} (u) - \boldsymbol {P} (v) / d ^ {b} (v) \| ^ {2} + \| \boldsymbol {P} - \boldsymbol {X} \| _ {F} ^ {2}. \tag {4}
$$
Differentiating the objective function with respect to  $\pmb{P}$  leads to  $(\pmb{I} - \tilde{\pmb{A}})\pmb{P} - \pmb{X} = 0$ . Therefore the closed-form solution is  $P_{X,L}^{*} = (I - \tilde{A})^{-1}\pmb{X} = \sum_{l=0}^{\infty}\tilde{A}^{l}\pmb{X}$ . In practical implementation, a limited  $L_{P,L}$ -hop summation is used instead due to the over-smoothing issue that the infinite form converges to identical node-wise embeddings. This Markov diffusion kernel  $P_{X,L} = \frac{1}{L_{P,L}}\sum_{l=0}^{L_{P,L}}\tilde{A}^{l}\pmb{X}$  is investigated in  $S^2GC$  [43] as an approach for balancing locality and multi-hop propagation, functioning as a low-pass filter to the signal  $\pmb{X}$  but also preserves high frequency. Its interpretation can be observed from Eq. [4], that it simultaneously minimizes the embedding difference of neighboring nodes as well as the approximation closeness to the input feature  $\pmb{X}$ .
To obtain the channel  $P_{X,L2}$  used in  $\mathrm{LD}^2$ , we introduce the low-frequency regularization preferably to 2-hop adjacency in the graph, as 1-hop neighbors exhibit heterophily. Therefore, replacing  $\tilde{A}(u,v)$  in Eq. (4) with  $\tilde{A}_2(u,v)$  yields our constant 2-hop embedding  $P_{X,L2}$ . It shares similar spectral properties with  $S^2GC$  for acting as a low-pass filter in 2-hop neighborhoods, while maintaining certain long-distance knowledge thanks to the multi-scale aggregation. The other channel in feature embedding, i.e. the Laplacian propagation, can be derived as  $P_{X,H} = (I + \tilde{L})(I - \tilde{A})^{-1}X = (I + \tilde{L})P_{X,L}$ . Based on the above analysis, the embedding  $P_{X,L}$  contains multi-hop neighborhood information, while  $(I + \tilde{L})$  can be seen as the improved Laplacian operator extracting the high-frequency components. The embedding  $P_{X,H}$  thus serves as a high-pass filter focusing on discriminative structures in a non-local manner. In terms of spatial domain interpretation, such high-frequency information corresponds to the fine-grained embedding differences between two nodes [18, 46]. It is noticeable that these three channels  $P_{X,L2}, P_{X,H}, P_{X,0}$  respectively represent low-pass, high-pass, and all-pass propagations through the graph while addressing heterophily. Combining them as inputs to the neural network benefits the model performance with expressive information at various distances including identity, local, and global perspectives.
3.4 Approximate Adjacency Propagation Precomputation
Conventionally, calculating the graph propagation  $\tilde{A} \cdot P$  for an arbitrary feature matrix  $P$  is conducted by the sparse-dense matrix multiplication. However, such an approach does not recognize the property of the adjacency matrix  $\tilde{A}$ , that it can be represented by the adjacency list of nodes, and non-zero values in its data are solely determined by node degrees. Furthermore, since the propagation result is subsequently processed by the neural network, it is not necessary to be precise as the model is robust to handle noisy data [46, 47]. We first define the precision bound for approximate embedding:
Definition 3.1 (Approximate Vector Embedding). Given a relative error bound  $0 < \epsilon < 1$ , a norm threshold  $\delta > 0$ , and a failure probability  $0 < \phi < 1$ , the estimation  $\hat{P}(u)$  for an arbitrary embedding vector  $P(u)$  should satisfy that, for each  $u \in V$  with  $\|P(u)\| > \delta$ ,  $\|P(u) - \hat{P}(u)\| \leq \epsilon \cdot \|P(u)\|$  with probability at least  $1 - \phi$ .
Graph power iteration algorithm is the variant of power iteration particularly applied for calculating powers of adjacency matrix  $\mathbf{A}$  [48]. In essence, the algorithm can be derived by maintaining a residue  $\mathbf{R}^{(l)}(u)$  that holds the current  $l$ -hop propagation results for each node, and iteratively updating the next-hop residues of neighboring nodes  $\mathbf{R}^{(l+1)}(v), v \in \mathcal{N}(u)$  for all nodes  $u$ . For each iteration, the reserve  $\hat{\mathbf{P}}^{(l)}$  is also added up and converges to an underestimation of  $\mathbf{P}$ .
We propose Algorithm [1] for our specific scenario, namely Approximate Adjacency Propagation (A²Prop). Based on power iteration, our algorithm is greatly generalized to accommodate normalized adjacency, feature vectors for nodes, and a limited number of hops. We show that the algorithmic output can be bounded by Definition [3.1]. For  $L_{P}$  iterations, denote the acceptable error per entry for push as  $\delta_{P}$ , the matrix-wise absolute error is  $\| P - \hat{P} \|_{1,1} \leq \sum_{f=1}^{L_{P}} \sum_{f=1}^{F} \sum_{u \in V} d(u) \delta_{P} = L_{P} m F \delta_{P}$ . By setting  $\delta_{P} = \epsilon \delta / L_{P} m$ , the estimation  $\hat{P}$  satisfies Definition [3.1].
Approximate Feature Embedding. The feature embedding formed as  $P_{X} = \sum_{l=0}^{L_{P}} \theta_{l} T^{l} X$  can be computed by iteratively applying graph power iterations to the initial residue  $R^{(0)} = X$ . The implicit propagation behavior is described by matrix  $T$ . For example, for Laplacian propagation  $T = I + \tilde{L}$  to node  $u$ , the embeddings from the previous iteration are aggregated as  $R^{(l+1)}(u) = 2R^{(l)}(u) - \sum_{v \in \mathcal{N}(u)} R^{(l)}(v) / d^{a}(u) d^{b}(v) = \sum_{v \in \mathcal{N}(u) \cup \{u\}} \frac{\alpha_{L}(u,v)}{d^{a}(u) d^{b}(v)} \cdot R^{(l)}(v)$ . Here  $\alpha_{T}(u,v)$  is a propagation factor for unifying the aggregation by  $T$ , that  $\alpha_{L}(u,u) = 2d^{a+b}(u), \alpha_{L}(u,v) = -1, v \in \mathcal{N}(u)$ . For propagation  $\tilde{A}$  and  $\bar{A}$ , the factor is  $\alpha_{A}(u,v) = 1$  and  $\alpha_{A}(u,u) = 1, 0$ , respectively.
In each iteration  $l$ , the reserve is updated after propagation according to the coefficient  $\theta_{l}$  to sum up corresponding embeddings. Intuitively, one multiplication of  $\bar{A}^2$  is equivalent to two iterations of  $\bar{A}$  propagation. Hence for  $P_{X,L2}$  there is  $\theta_{l} = l \mod 2 = 0,1,0,1,\dots$  under the summation scheme
Algorithm 1 A²Prop: Approximate Adjacency Propagation
Input: graph  $G$ , feature matrix  $\mathbf{X}$ , max hop  $L_{P}$ , normalization factor  $a, b$ , propagation factor  $\alpha_{T}$ , summation factor  $\theta_{l}$ , push threshold  $\delta_{P}$
Output: adjacency embedding  $P_A$ , feature embedding  $P_X$
1  $R_A^{(0)}\gets N(0,1),R_X^{(0)}\gets X$
2 for  $l$  from 0 to  $L_{P} - \bar{1}$  do
3  $R_{A}^{(l + 1)}\gets 0,R_{X}^{(l + 1)}\gets 0$
4 for all  $u\in V$  such that  $\| R^{(l)}(u)\| >\delta_P$  do
5 for all  $v\in \mathcal{N}(u)\cup \{u\}$  do 6  $\begin{array}{rl} & {\pmb{R}_{A}^{(l + 1)}(v)\gets \pmb{R}_{A}^{(l + 1)}(v) + \alpha_{A}(u,v)\cdot \pmb{R}_{A}^{(l)}(u)}\\ & {\pmb{R}_{X}^{(l + 1)}(v)\gets \pmb{R}_{X}^{(l + 1)}(v) + \frac{\alpha_{T}(u,v)}{d^{a}(v)d^{b}(u)}\cdot \pmb{R}_{X}^{(l)}(u)} \end{array}$  7
8 if  $l$  mod  $2 = 1$  and  $l <   L_{P} - 1$  then
9  $\begin{array}{rl}\big{\downarrow} & P_A\gets \mathrm{orthonormalize}(R_A^{(l)}) \end{array}$
10  $P_{X}\gets P_{X} + \theta_{l}\cdot R_{X}^{(l)}$
11 empty  $\pmb{R}_A^{(l)},\pmb{R}_X^{(l)}$
12  $P_A\gets P_A\cdot |(R_A^{(L_P)})^\top \cdot P_A|^{1 / 2}$
13  $P_{X}\leftarrow P_{X} + \theta_{L_{P}}\cdot R_{X}^{(L_{P})}$
14 return  $P_A, P_X$
(a) genius
(b)pokec
Figure 3: Effect of embedding channels and propagation hops on accuracy.
in Algorithm 1. Since all embeddings we consider are constant, that is,  $\theta_l \in \{0, 1\}$ , the reserve can be simply increased without the rescaling terms in more general cases such as [32].
Approximate Adjacency Embedding. The adjacency embedding is represented by leading eigenvectors  $P_A = U|\Lambda|^{1/2}$ . This eigendecomposition of  $A^2$  can be solved by the truncated power iteration [49]: Initialize the  $n \times F$  residue by i.i.d. Gaussian noise  $R^{(0)} = N(0,1)$ . For each iteration  $l$ , firstly multiply the residue by  $A^2$  as  $R^{(l+1)} = A^2 R^{(l)}$ ; then, perform column-wise normalization to the residue orthonormalize  $(R^{(l+1)})$  so that its columns are orthogonal to each other and of L2 norm 1. After convergence, the matrix satisfies  $A^2 R^{(L_P)} = R^{(L_P)}\Lambda$  within the error bound, which leads to the estimated output  $\hat{U} = R^{(L_P)}, \hat{P}_A = \hat{U}|\hat{\Lambda}|^{1/2}$ .
Similarly, the 2-hop power iteration of  $P_A$  can be merged with those for  $P_X$  with a shared maximal iteration  $L_P$ , and orthonormalization is conducted every two  $A$  iterations. When the algorithm converges with error bound  $\delta$ , the number of iterations follows  $L_P = O(\log (F / \delta) / (1 - |\lambda_{F + 1} / \lambda_F|))$ . By selecting proper values for  $F$  and  $\delta$ , the algorithm produces satisfying results within  $L_P$  iterations.
Precomputation Complexity. Since  $\mathrm{A}^2\mathrm{Prop}$  serves as a general approximation for various adjacency-based propagations, the computation of all feature channels can be performed simultaneously in a single run. The memory overhead of the algorithm is mainly the residue and reserve matrices for  $C$  embedding channels, which is  $O(CnF)$  in total. Note that  $\mathrm{A}^2\mathrm{Prop}$  precomputation is performed in the main memory, and benefits from a less-constrained budget compared to GPU memory.
For each  $\mathrm{A}^2\mathrm{Prop}$  iteration, neighboring connections are accessed for at most  $m$  times. The time complexity of Algorithm 1 can thus be bounded by  $O(L_{P}mF)$ . In addition, its loops over nodes and features can be parallelized and vectorized to reduce execution time. Moreover, the graph power iteration design is also amendable for further enhancements, such as reduction to sub-linear complexity [50, 25], better memory utilization [51, 52], and precision-efficiency trade-offs [53, 54]. We leave these potential improvements on  $\mathrm{A}^2\mathrm{Prop}$  for future work.
We implement the  $\mathrm{LD}^2$  model and evaluate its performance from the perspectives of both efficacy and scalability. In this section we highlight key empirical results compared to minibatch GNNs on large-scale heterophilous graphs, while parameter settings, further experiments, and subsequent discussions can be found in the Appendix.
Datasets. We mainly perform experiments on million-scale and above heterophilous datasets [26, 55] for the transductive node classification task, with the largest available graph wiki ( $m = 243\mathrm{M}$ ) included. Evaluations on more homophilous and heterophilous graphs can be found in Appendices D and E. We leverage settings as per [26] such as the random train/test splits and the induced subgraph testing for GSAINT-sampling models.
Baselines. We focus on GNN models applicable to minibatch training in our evaluation regarding scalability, and hence most full-batch networks mentioned in Section 2 are excluded in the main experiments, while more comprehensive results for both minibatch and full-batch models are in Appendices D and E. Conventional baselines in the main experiments include MLP which only processes node attributes without considering graph topology, as well as PPRGo [29] and SGC [30] representing decoupled schemes for traditional graph propagation. For GNNs under non-homophily, we investigate GCNJK-GS [21] and MixHop-GS [12], where GSAINT random walk sampling [27] is utilized to empower the original backbone models for minibatching. LINKX is the decoupled heterophilous GNN proposed by [26]. Simple i.i.d. node batching is adopted for decoupled networks. Explorations on the model settings are displayed in Appendix C.
Evaluation Metrics. We uniformly use classification accuracy on the test set to measure network effectiveness. Note that since the datasets are updated and the minibatch scheme is employed, results may be different from their original works. In order to evaluate scalability performance, we conduct repeated experiments and record the network training/inference time and peak memory footprint as efficiency metrics. For precomputed methods, we consider the learning process combining both
Table 2: Average test accuracy  $(\%)$  of minibatch  $\mathrm{LD}^2$  and baselines on heterophilous datasets.  $>12h$  means the model requires more than 12h clock time to produce proper results. Respective results of the first and second best performances on each dataset are marked in bold and underlined fonts.
Dataset	genius	tolokers	arxiv-year	penn94	twitch-gamers	pokec	snap-patents	wiki
Nodes n	421,858	11,758	169,343	41,536	168,114	1,632,803	2,738,035	1,770,981
Edges m	922,864	1,038,000	1,157,799	1,362,220	6,797,557	22,301,964	13,967,949	242,507,069
F / Nc	12 / 2	10 / 2	128 / 5	4,814 / 2	7 / 2	65 / 2	269 / 5	600 / 5
MLP	82.47 ±0.06	73.38 ±0.25	37.23 ±0.31	74.41 ±0.48	61.26 ±0.19	61.81 ±0.07	23.03 ±1.48	35.64 ±0.10
PPRGo	79.81 ±0.00	78.16 ±0.00	39.35 ±0.12	58.75 ±0.31	47.19 ±2.26	50.61 ±0.04	(>12h)	(>12h)
SGC	79.85 ±0.01	71.16 ±0.06	43.40 ±0.16	68.31 ±0.27	57.05 ±0.21	56.58 ±0.06	37.70 ±0.06	28.12 ±0.08
GCNJK-GS	80.65 ±0.07	74.41 ±0.73	48.26 ±0.64	65.91 ±0.16	59.91 ±0.42	59.38 ±0.21	33.64 ±0.05	42.95 ±0.39
MixHop-GS	80.63 ±0.04	77.47 ±0.40	49.26 ±0.16	75.00 ±0.37	61.80 ±0.00	64.02 ±0.02	34.73 ±0.15	45.52 ±0.11
LINKX	82.51 ±0.10	77.74 ±0.13	50.44 ±0.30	78.63 ±0.25	64.15 ±0.18	68.64 ±0.65	52.69 ±0.05	50.59 ±0.12
LD2 (ours)	85.31 ±0.06	79.76 ±0.26	50.29 ±0.11	75.52 ±0.10	64.33 ±0.19	74.93 ±0.10	58.58 ±0.34	52.91 ±0.16
Table 3: Time and memory overhead of  $\mathrm{LD}^2$  and baselines on large-scale datasets. "Learn", "Infer", and "Mem." respectively refer to minibatch learning and inference time (s) and peak GPU memory (GB). Precomputation time is appended when applicable.  $>12\mathrm{h}$  means the model requires more than  $12\mathrm{h}$  clock time to produce proper results. Respective results of the first and second best performances among heterophilous models per metric are marked in bold and underlined fonts.
Dataset	twitch-gamers	pokec	snap-patents	wiki
Learn	Infer	Mem.	Learn	Infer	Mem.	Learn	Infer	Mem.	Learn	Infer	Mem.
MLP	6.36	0.02	0.61	47.86	0.11	13.77	27.39	0.28	9.33	133.55	0.62	18.15
PPRGo	10.46+15.88	0.41	9.64	121.95+56.11	2.69	3.82	(>12h)	(>12h)
SGC	0.09+0.74	0.01	0.28	1.05+8.08	0.01	0.28	4.94+23.54	0.01	0.42	12.66+7.98	0.01	0.52
GCNJK-GS	71.48	0.02*	7.33	27.33	0.09*	9.03	19.02	0.23*	9.21	95.52	0.69*	16.36
MixHop-GS	52.12	0.01*	1.49	71.35	0.03*	12.91	45.24	0.16*	19.58	84.22	0.23*	16.28
LINKX	10.99	0.19	2.35	28.77	0.33	9.03	39.80	0.22	21.53	180.71	1.14	14.53
LD2(ours)	0.85+1.96	0.01	1.44	17.95+6.18	0.01	3.82	31.32+6.96	0.02	3.96	28.12+6.50	0.01	4.47
* Inference time of GSAINT sampling is less precise since they are conducted on induced subgraphs smaller than the raw graph.
precomputation and training. Evaluations are conducted on a machine with 192GB RAM, two 28-core Intel Xeon CPUs (2.2GHz), and an NVIDIA RTX A5000 GPU (24GB memory).
The main evaluations of  $\mathrm{LD}^2$  and baselines on 8 large heterophilous datasets are presented in Tables $^2$  and  $\boxed{3}$  for effectiveness and efficiency metrics, respectively. As an overview, our model demonstrates its scalability in completing training and inference with fast running speed and efficient memory utilization, especially on large graphs. At the same time, it achieves comparable or superior prediction accuracy against the state-of-the-art minibatch heterophilous GNNs on most datasets.
Time Efficiency. We first highlight the scalability performance of our  $\mathrm{LD}^2$  model. Specifically, compared to heterophilous benchmarks on the four largest graphs with million-scale data,  $\mathrm{LD}^2$  speeds up the minibatch training process by 3-15 times, with an acceptable precomputation cost. Its inference time is also consistently below 0.1 seconds. The outstanding efficiency of  $\mathrm{LD}^2$  is mainly attributed to the simple model architecture that removes graph-scale operations while ensuring rapid convergence. In contrast, the execution speeds of MixHop and LINKX are highly susceptible to node and edge sizes, given their design dependency on the entire input graph. The extensive parameter space also causes them to converge slower, necessitating relatively longer training times. PPRGo shows limited scalability due to the costly post-transformation propagation. The superiority of  $\mathrm{LD}^2$  efficiency even holds when compared to simple methods such as MLP and SGC, indicating that the model is favorable for incorporating extra heterophilous information with no significant additional overhead. The empirical results affirm that  $\mathrm{LD}^2$  exhibits optimized training and inference complexity at the same level as simple models.
Memory Footprint.  $\mathrm{LD}^2$  remarkably reduces run-time GPU memory consumption. As the primary overhead only comprises the model parameters and batch representations, it enables flexible configuration of the model size and batch size to facilitate powerful training. Even for the largest graph
wiki with  $n = 1.77\mathrm{M}$  and  $F = 600$ , the footprint remains below 5GB under our hyperparameter settings. Other heterophilous GNNs, though adopting the minibatch scheme, experience high memory requirements and even occasionally encounter out-of-memory errors during experiments, as their space-intensive graph propagations are executed on the GPU. Consequently, when the graph scales up, they can only be applied with highly constrained model capacities to conserve space, potentially resulting in compromised performance.
Test Accuracy. With regard to efficacy,  $\mathrm{LD}^2$  achieves top testing accuracy on 6 out of 8 heterophilous graphs and comparable performance on the remaining ones. It also consistently outperforms the sampling-based GCNJK and MixHop, as well as conventional GNNs. Particularly, by extracting embeddings from not only node features but pure graph topology as well,  $\mathrm{LD}^2$  obtains significant improvements over feature-based networks on datasets such as genius, snap-patents, and wiki, demonstrating the importance of pure graph information in heterophilous learning. We deduce that the relatively suboptimal accuracy on penn94 may be correlated with the difficulty of fitting one-hot encoding features into informative embeddings, as explored in Appendix G. Consistent with the previous studies [26], regular GNN baselines suffer from performance loss on most heterophilous graphs, while MLP achieves comparably high accuracy when node attributes are discriminative enough. For non-homophilous models GCNJK and MixHop, the minibatch scheme hinders them from reaching higher results because of the neglect of their full-graph relationships.
4.3 Effect of Parameters
To gain deeper insights into the multi-channel embeddings of  $\mathrm{LD}^2$ , in Figure 3 we explore the effect of embeddings channels and propagation hops which are critical to our model design, while more discussions on other parameters and factors are displayed in Appendix F.
Embedding Channels. Lines in Figure represent the results of learning on separate inputs on two representative datasets genius and poking. It can be observed that different graphs imply varying patterns when embedding channels and propagation hops are changed. For the genius dataset where raw node attributes already achieve an accuracy above  $82\%$ , applying the other two feature embeddings further improves the result. While the adjacency embedding alone shows secondary performance, integrating it with other channels proves beneficial. In comparison, on poking, it is the inverse embedding  $\pmb{P}_{X,H}$  that becomes the key contributor. The empirical evaluation supports our design that by adopting multi-channel and heterophily-oriented embeddings,  $\mathrm{LD}^2$  benefits from learning both topology and feature for a more comprehensive understanding of the graph data.
Propagation Hops. As elaborated in Section 3.4 propagation hops  $L_{P}$  determines the number of iterations in Algorithm  $\square$ . Particularly, for the approximate adjacency embedding  $P_A$ , it also affects the convergence of decomposition. As shown by the brown dashed lines in Figure  $\boxed{3}$  the accuracy typically becomes stable when  $L_{P} > 8$ , indicating the utility of the low-dimensional approximation in producing effective topology embedding within limited iterations. For the multi-channel scheme in general, as the graph scale increases, employing more propagation hops becomes advantageous in capturing distant information. Aligned with our analysis, above observation validates that  $\mathrm{LD}^2$  is powerful in capturing implicit information of various frequencies and scales that is important in the presence of heterophily.
In this work, we propose  $\mathrm{LD}^2$ , a scalable GNN design for heterophilous graphs, that leverages long-distance propagation to capture non-local relationships among nodes, and incorporates low-dimensional yet expressive embeddings for effective learning. The model decouples full-graph dependency from the iterative training, and adopts an efficient precomputation algorithm for approximating multi-channel embeddings. Theoretical and empirical evidence demonstrates its optimized training characteristics, including time efficiency with a complexity linear to  $O(n)$ , and GPU memory independence from the graph size  $n$  and  $m$ . As a noteworthy result,  $\mathrm{LD}^2$  successfully applies to million-scale datasets under heterophily, with learning times as short as 1 minute and GPU memory expense below 5GB. We also recognize the current limitations of our work including potential accelerations for precomputation and adaptability to diverse feature patterns. Detailed limitations and broader impacts are addressed in the Appendix.