SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1: while  $\eta < T_0$  do
2: randomly sample a pretrained training trajectory in  $P_{\Theta_T}$  and calculate the  $\mathcal{L}_{\mathrm{meta - tt}}$  according to Eq. (5);
3: for  $t < T_{1}$  do
4: train  $\mathrm{GNN}_S$  through gradient descent by  $\widetilde{\pmb{\theta}}_S^{t + 1}\gets \widetilde{\pmb{\theta}}_S^t -\zeta \nabla_{\widetilde{\pmb{\theta}}}L_{\mathrm{cls}}\left[\mathrm{GNN}_S\left(\widetilde{\mathbf{X}},\mathbf{I}\right),\widetilde{\mathbf{Y}}\right]$  where  $\zeta$  is the step size;
5: end for
6: update the condensed graph-free data  $S_{\eta}$  according to Eq. (4);
7: calculate current  $\eta$ -th step  $\gamma_{\mathrm{gnf}}(S_{\eta})$  according to Eq. (7);
8: end while
9: select the optimal condensed graph-free data  $S_{\eta}^{*}$  with the smallest score  $\gamma_{\mathrm{gnf}}^{*}$  as the final output.
distribution  $P_{\Theta_{\mathcal{T}}}$ , which can be taken as a 'meta' way to make the distilled dataset  $S$  adapt different parameter initialization. That is why we call it 'meta-matching'. In this way, when initializing  $\mathrm{GNN}_{\mathcal{T}}$  and  $\mathrm{GNN}_S$  with the same model parameters, Eq. (5) contributes to aligning the learning behaviors of  $\mathrm{GNN}_{\mathcal{T}}$  that experiences  $p$ -steps optimization, to  $\mathrm{GNN}_S$  that experiences  $q$ -steps optimization. In this way, the proposed training trajectory meta-matching schema could comprehensively imitate the long-term learning behavior of GNN training. As a result, the informative knowledge of the large-scale graph  $\mathcal{T}$  can be effectively transferred to the small-scale condensed graph-free data  $S = (\widetilde{\mathbf{X}},\widetilde{\mathbf{Y}})$  in the above outer-loop optimization objective of Eq. (4).
For the inner loop, we train  $\mathrm{GNN}_S$  on the synthesized small-scale condensed graph-free data for optimizing its model parameter until the optimal  $\widetilde{\theta}_{\mathcal{S}}^{*}$ . Therefore, the final optimization objective of the proposed SFGC is
$$
\min  _ {\mathcal {S}} \operatorname {E} _ {\boldsymbol {\theta} _ {t} ^ {*}, i} \sim P _ {\boldsymbol {\Theta} _ {\mathcal {T}}} \left[ \mathcal {L} _ {\text {m e t a - t t}} \left(\boldsymbol {\theta} _ {t} ^ {*} | _ {t = t _ {0}} ^ {p}, \widetilde {\boldsymbol {\theta}} _ {t} | _ {t = t _ {0}} ^ {q}\right) \right], \tag {6}
$$
$$
s. t. \quad \widetilde {\boldsymbol {\theta}} _ {\mathcal {S}} ^ {*} = \underset {\widetilde {\boldsymbol {\theta}}} {\arg \min } \mathcal {L} _ {\mathrm {c l s}} \left[ \operatorname {G N N} _ {\widetilde {\boldsymbol {\theta}}} (\mathcal {S}) \right],
$$
where  $\mathcal{L}_{\mathrm{cls}}$  is the node classification loss calculated with the cross-entropy on graphs. Compared with the triple-level optimization in Eq. (1), the proposed SFGC directly replaces the learnable  $\mathbf{A}'$  in Eq. (1) with a fixed identity matrix  $\mathbf{I}$ , resulting in the condensed structure-free graph data  $\mathcal{S} = (\widetilde{\mathbf{X}},\mathbf{I},\widetilde{\mathbf{Y}})$ . Without synthesizing condensed graph structures with  $\mathrm{GSL}_{\psi}$ , the proposed SFGC refines the complex triple-level optimization to the bi-level one, ensuring effectiveness of the condensed graph-free data.
Hence, the advances of the training trajectory meta-matching schema in the proposed SFGC can be summarized as follows: (1) compared with the online gradient calculation, SFGC's offline parameter sampling avoids dynamically computing and storing gradients of both the large and condensed small graphs, reducing computation and memory costs during the condensation process; (2) compared with short-range matching, SFGC's long-term meta-matching avoids condensed data to short-sightedly fit certain optimization steps, contributing to a more holistic and comprehensive way to imitate GNN's learning behaviors.
2.4 Graph Neural Feature Score
For each update of the outer loop in Eq. (6), we would synthesize the brand-new condensed graph-free data. However, evaluating the quality of the underlying condensed graph-free data in the dynamical meta-matching condensation process is quite challenging. That is because we cannot quantify a graph dataset's performance without blending it in a GNN model. And the condensed graph-free data itself cannot be measured by convergence or decision boundary. Generally, to evaluate the condensed graph-free data, we use it to train a GNN model. If the condensed data at a certain meta-matching step achieves better GNN test performance on node classification, it indicates the higher quality of
the current condensed data. That means, evaluating condensed graph-free data needs an extra process of training a GNN model from scratch, leading to much more time and computation costs.
In light of this, we aim to derive a metric to dynamically evaluate the condensed graph-free data in the meta-matching process, and utilize it to select the optimal small-scale graph-free data. Meanwhile, the evaluation progress would not introduce extra GNN iterative training for saving computation and time. To achieve this goal, we first identify what characteristics such a metric should have: (1) closed-form solutions of GNNs to avoid iterative training in evaluation; (2) the ability to build strong connections between the large-scale graph and the small-scale synthesized graph-free data. In this case, the graph neural tangent kernel (GNTK) stands out, as a typical class of graph kernels, and has the full expressive power of GNNs with provable closed-form solutions. Moreover, as shown in Eq. (2), GNTK naturally builds connections between arbitrary two graphs even with different sizes.
Based on the graph kernel method with GNTK, we proposed a graph neural feature score metric  $\gamma_{\mathrm{gnf}}$  to dynamically evaluate and select the optimal condensed graph-free data as follows:
$$
\gamma_ {\mathrm {g n f}} (\mathcal {S}) = \frac {1}{2} \left\| \mathbf {Y} _ {\text {v a l}} - \mathcal {K} \langle \mathcal {T} _ {\text {v a l}}, \mathcal {S} \rangle \left(\mathcal {K} \langle \mathcal {S}, \mathcal {S} \rangle + \lambda \mathbf {I}\right) ^ {- 1} \tilde {\mathbf {Y}} \right\| ^ {2}, \tag {7}
$$
where  $\mathcal{K}\langle \mathcal{T}_{\mathrm{val}},\mathcal{S}\rangle \in \mathbb{R}^{N_{\mathrm{val}}\times N^{\prime}}$  and  $\mathcal{K}\langle \mathcal{S},\mathcal{S}\rangle \in \mathbb{R}^{N^{\prime}\times N^{\prime}}$  denote the node-level GNTK matrices derived according to Eq. (2). And  $\mathcal{T}_{\mathrm{val}}$  is the validation sub-graph of the large-scale graph with  $N_{\mathrm{val}}$  numbers of nodes. Concretely,  $\gamma_{\mathrm{gnf}}(S)$  calculates the graph neural tangent kernel based ridge regression error. It measures that, given an infinitely-wide GNN trained on the condensed graph  $S$  with ridge regression, how close such GNN's prediction on  $\mathcal{T}_{\mathrm{val}}$  to its ground truth labels  $\mathbf{Y}_{\mathrm{val}}$ . Note that Eq. (7) can be regarded as the Kernel Inducing Point (KIP) algorithm [39, 40] adapted to the GNTK kernel on GNN models.
Hence, the proposed graph neural feature score meets the above-mentioned characteristics as: (1) it calculates a closed-form GNTK-based ridge regression error for evaluation without iteratively training GNN models; (2) it strongly connects the condensed graph-free data with the large-scale validation graph; In summary, the overall algorithm of the proposed SFGC is presented in Algo. 1.
Datasets. Following [27], we evaluate the node classification performance of the proposed SFGC method on Cora, Citeseer [61], and Ogbn-arxiv [17] under the transductive setting, on Flickr [66] and Reddit [16] under the inductive setting. For all datasets under two settings, we use the public splits and setups for fair comparisons. We consider three condensation ratios  $(r)$  for each dataset. Concretely,  $r$  is the ratio of condensed node numbers  $rN(0 < r < 1)$  to large-scale node numbers  $N$ . In the transductive setting,  $N$  represents the number of nodes in the entire large-scale graph, while in the inductive setting,  $N$  indicates the number of nodes in the training sub-graph of the whole large-scale graph. The dataset statistic details are shown in Appendix C.
Baselines & Implementations. We adopt the following baselines for comprehensive comparisons [27]: graph coarsening method [20], graph coreset methods, i.e., Random, Herding [60], and K-Center [51], the graph-based variant DC-Graph of general dataset condensation method DC [77], which is introduced by [27], graph dataset condensation method GCOND [27] and its variant GCONDX without utilizing the graph structure. The whole pipeline of our experimental evaluation can be divided into two stages: (1) the condensation stage: synthesizing condensed graph-free data, where we use the classical and commonly-used GCN model [61]; (2) the condensed graph-free data test stage: training a certain GNN model (default with GCN) by the obtained condensed graph-free data from the first stage and testing the GNN on the test set of the large-scale graph with repeated 10 times. We report the average transductive and inductive node classification accuracy  $(\mathrm{ACC}\%)$  with standard deviation (std). Following [27], we use the two-layer GNN with 256 hidden units as the defaulted setting. Besides, we adopt the K-center [51] features to initialize our condensed node attributes for stabilizing the training process. Additional hyper-parameter setting details are listed in Appendix E.
Table 1: Node classification performance (ACC%±std) comparison between condensation methods and other graph size reduction methods with different condensation ratios. (Best results are in bold, and the second-best are underlined.)
Datasets	Ratio (r)	Other Graph Size Reduction Baselines	Condensation Methods	Whole Dataset
Coarsening [20]	Random [60]	Herding [60]	K-Center [51]	DC-Graph [77]	GCOND-X [27]	GCOND [27]	SFGC (ours)
Citeseer	0.9%	52.2±0.4	54.4±4.4	57.1±1.5	52.4±2.8	66.8±1.5	71.4±0.8	70.5±1.2	71.4±0.5	71.7±0.1
1.8%	59.0±0.5	64.2±1.7	66.7±1.0	64.3±1.0	66.9±0.9	69.8±1.1	70.6±0.9	72.4±0.4
3.6%	65.3±0.5	69.1±0.1	69.0±0.1	69.1±0.1	66.3±1.5	69.4±1.4	69.8±1.4	70.6±0.7
Cora	1.3%	31.2±0.2	63.6±3.7	67.0±1.3	64.0±2.3	67.3±1.9	75.9±1.2	79.8±1.3	80.1±0.4	81.2±0.2
2.6%	65.2±0.6	72.8±1.1	73.4±1.0	73.2±1.2	67.6±3.5	75.7±0.9	80.1±0.6	81.7±0.5
5.2%	70.6±0.1	76.8±0.1	76.8±0.1	76.7±0.1	67.7±2.2	76.0±0.9	79.3±0.3	81.6±0.8
Ognb-arxiv	0.05%	35.4±0.3	47.1±3.9	52.4±1.8	47.2±3.0	58.6±0.4	61.3±0.5	59.2±1.1	65.5±0.7	71.4±0.1
0.25%	43.5±0.2	57.3±1.1	58.6±1.2	56.8±0.8	59.9±0.3	64.2±0.4	63.2±0.3	66.1±0.4
0.5%	50.4±0.1	60.0±0.9	60.4±0.8	60.3±0.4	59.5±0.3	63.1±0.5	64.0±0.4	66.8±0.4
Flickr	0.1%	41.9±0.2	41.8±2.0	42.5±1.8	42.0±0.7	46.3±0.2	45.9±0.1	46.5±0.4	46.6±0.2	47.2±0.1
0.5%	44.5±0.1	44.0±0.4	43.9±0.9	43.2±0.1	45.9±0.1	45.0±0.2	47.1±0.1	47.0±0.1
1%	44.6±0.1	44.6±0.2	44.4±0.6	44.1±0.4	45.8±0.1	45.0±0.1	47.1±0.1	47.1±0.1
Reddit	0.05%	40.9±0.5	46.1±4.4	53.1±2.5	46.6±2.3	88.2±0.2	88.4±0.4	88.0±1.8	89.7±0.2	93.9±0.0
0.1%	42.8±0.8	58.0±2.2	62.7±1.0	53.0±3.3	89.5±0.1	89.3±0.1	89.6±0.7	90.0±0.3
0.2%	47.4±0.9	66.3±1.9	71.0±1.6	58.5±2.1	90.5±1.2	88.8±0.4	90.1±0.5	89.9±0.4
Figure 3: Comparisons between five variants of synthesizing graph structures vs. our structure-free SFGC (discrete  $k$ -nearest neighbor ( $k\mathbf{NN}$ ) structure variants: SFGC-d1 ( $k = 1$ ), SFGC-d2 ( $k = 2$ ), and SFGC-d5 ( $k = 5$ ), continuous graph structure variant: SFGC-c, parameterized graph structure variant: SFGC-p).
Performance of SFGC on Node Classification. We compare the node classification performance between SFGC and other graph size reduction methods, especially the graph condensation methods. The overall performance comparison is listed in Table 1. Generally, SFGC achieves the best performance on the node classification task with 13 of 15 cases (five datasets and three condensation ratios for each of them), compared with all other baseline methods, illustrating the high quality and expressiveness of the condensed graph-free data synthesized by our SFGC. More specifically, the better performance of SFGC than GCOND and its structure-free variant GCOND-X experimentally verifies the superiority of the proposed method. We attribute such superiority to the following two aspects regarding the condensation stage. First, the long-term parameter distribution matching of our SFGC works better than the short-term gradient matching in GCOND and GCOND-X. That means capturing the long-range GNN learning behaviors facilitates to holistically imitate GNN's training process, leading to comprehensive knowledge transfer from the large-scale graph to the small-scale condensed graph-free data. Second, the structure-free paradigm of our SFGC enables more compact small-scale graph-free data. For one thing, it liberates the optimization process from triple-level objectives, alleviating the complexity and difficulty of condensation. For another thing, the obtained optimal condensed graph-free data compactly integrates node attribute contexts and topology structure information. Furthermore, on Cora and Citeseer, SFGC synthesizes better condensed graph-free data that even exceeds the whole large-scale graph dataset. These results confirm that SFGC is able to break the information limitation under the large-scale graph and effectively synthesize new, small-scale graph-free data as an optimal representation of the large-scale graph.
Effectiveness of Structure-free Paradigm in SFGC. The proposed SFGC introduces the structure-free paradigm without condensing graph structures in graph condensation. To verify the effectiveness of the structure-free paradigm, we compare the proposed SFGC with its variants, which synthesize graph structures in the condensation process. Specifically, we evaluate the following three different methods of synthesizing graph structures with five variants of SFGC: (1) discrete  $k$ -nearest neighbor ( $k$ NN) structures calculated by condensed node features under  $k = (1,2,5)$ , corresponding to the variants SFGC-d1, SFGC-d2, and SFGC-d5; (2) cosine similarity based continuous graph structures
Table 2: Performance across different GNN architectures.
Datasets (Ratio)	Methods	MLP	GAT [56]	APPNP [14]	Cheby [7]	GCN [61]	SAGE [16]	SGC [63]	Avg.
Citeseer (r=1.8%)	DC-Graph [77]	66.2	-	66.4	64.9	66.2	65.9	69.6	66.6
GCOND-X [27]	69.6	-	69.7	70.6	69.7	69.2	71.6	70.2
GCOND [27]	63.9	55.4	69.6	68.3	70.5	66.2	70.3	69.0
SFGC (ours)	71.3	72.1	70.5	71.8	71.6	71.7	71.8	71.5
Cora (r=2.6%)	DC-Graph [77]	67.2	-	67.1	67.7	67.9	66.2	72.8	68.3
GCOND-X [27]	76.0	-	77.0	74.1	75.3	76.0	76.1	75.7
GCOND [27]	73.1	66.2	78.5	76.0	80.1	78.2	79.3	78.4
SFGC (ours)	81.1	80.8	78.8	79.0	81.1	81.9	79.1	80.3
Ogbn-arxiv (r=0.25%)	DC-Graph [77]	59.9	-	60.0	55.7	59.8	60.0	60.4	59.2
GCOND-X [27]	64.1	-	61.5	59.5	64.2	64.4	64.7	62.9
GCOND [27]	62.2	60.0	63.4	54.9	63.2	62.6	63.7	61.6
SFGC (ours)	65.1	65.7	63.9	60.7	65.1	64.8	64.8	64.3
Flickr (r=0.5%)	DC-Graph [77]	43.1	-	45.7	43.8	45.9	45.8	45.6	45.4
GCOND-X [27]	42.1	-	44.6	42.3	45.0	44.7	44.4	44.2
GCOND [27]	44.8	40.1	45.9	42.8	47.1	46.2	46.1	45.6
SFGC (ours)	47.1	45.3	40.7	45.4	47.1	47.0	42.5	45.0
Reddit (r=0.1%)	DC-Graph [77]	50.3	-	81.2	77.5	89.5	89.7	90.5	85.7
GCOND-X [27]	40.1	-	78.7	74.0	89.3	89.3	91.0	84.5
GCOND [27]	42.5	60.2	87.8	75.5	89.4	89.1	89.6	86.3
SFGC (ours)	89.5	87.1	88.3	82.8	89.7	90.3	89.5	88.2
calculated by condensed node features, corresponding to the variant SFGC-c; (3) parameterized graph structure learning module with condensed node features adapted by [27], corresponding to the variant SFGC-p. We conduct experiments on three transductive datasets under nine condensation ratios for each graph structure synthesis variant and the proposed SFGC. The results are presented in Fig. 3. In general, the proposed SFGC achieves the best performance over various graph structure synthesis methods, and these results empirically verify the effectiveness of the proposed structure-free condensation paradigm. More specifically, for discrete  $k$ -nearest neighbor ( $k$ NN) structure variants, different datasets adapt different numbers of  $k$ -nearest neighbors under different condensation ratios, which means predefining the value of  $k$  can be very challenging. For example, CiteSeer dataset has better performance with  $k = 1$  in SFGC-d1 under  $r = 0.9\%$  than SFGC-d2 and SFGC-d5, but under  $r = 1.8\%$ ,  $k = 2$  in SFGC-d2 performs better than others two. Besides, for continuous graph structure variant SFGC-c, it generally cannot exceed the discrete graph structure variants, except for Ogbn-arxiv dataset under  $r = 0.05\%$ . And the parameterized variant SFGC-p almost fails to synthesize satisfied condensed graphs under the training trajectory meta-matching scheme. The superior performance of SFGC to all structure-based methods demonstrates the effectiveness of its structure-free paradigm.
Effectiveness of Graph Neural Feature Score in SFGC. We compare the learning time between GNN iterative training vs. our proposed GNTK-based closed-form solutions of  $\gamma_{\mathrm{gnf}}$ . Note that the iterative training evaluation strategy mandates the complete training of a GNN model from scratch at each meta-matching step, hence, we calculate its time that covers all training epochs under the best test performance for fair comparisons. Typically, for Flickr dataset ( $r = 0.1\%$ ), our proposed  $\gamma_{\mathrm{gnf}}$  based GNTK closed-form solutions takes only 0.015s for dynamic evaluation, which significantly outperforms the iterative training evaluation with 0.845s. The superior performance can also be observed in Ogbn-arxiv dataset ( $r = 0.05\%$ ) with 0.042s of our  $\gamma_{\mathrm{gnf}}$ , compared with 4.264s of iterative training, illustrating our SFGC's high dynamic evaluation efficiency. More results and analysis of our proposed  $\gamma_{\mathrm{gnf}}$  in GNTK-based closed-form solutions can be found in Appendix.
Generalization Ability of SFGC across Different GNNs. We evaluate and compare the generalization ability of the proposed SFGC and other graph condensation methods. Concretely, we test the node classification performance of our synthesized graph-free data (condensed on GCN) with seven different GNN architectures: MLP, GAT [56], APPNP [14], Cheby [7], GCN [61], SAGE [16], and SGC [63]. It can be generally observed that the proposed SFGC achieves outstanding performance over all tested GNN architectures, reflecting its excellent generalization ability. This is because our method reduces the graph structure to the identity matrix, so that the condensed graph node set can no longer be influenced by different convolution operations of GNNs along graph structures, enabling it consistent and good performance with various GNNs.
More experimental analysis and discussions, including the effects of different ranges of long-term meta-matching, the performance on downstream unsupervised graph clustering task, visualization of our condensed structure-free node set, as well as time complexity analysis, are detailed in Appendix E.
This work proposes a novel Structure-Free Graph Condensation paradigm, named SFGC, to distill the large-scale graph into the small-scale graph-free node set without graph structures. Under the structure-free learning paradigm, the training trajectory meta-matching scheme and the graph neural feature score measured dynamic evaluation work collaboratively to synthesize small-scale graph-free data with superior effectiveness and good generalization ability. Extensive experimental results and analysis under large condensation ratios confirm the superiority of the proposed SFGC method in synthesizing excellent small-scale graph-free data. It can be anticipated that our work would bridge the gap between academic GNNs and industrial MLPs by synthesizing small-scale, graph-free data to address graph data scalability, while retaining the expressive performance of graph learning. Our method works on condensing the number of nodes in a single graph at the node level, and we will explore extending it to condense the number of graphs in a graph set at the graph level in the future. We will also explore the potential of unifying graphs and large language models [44] for the graph condensation task.
In this work, S. Pan was supported by an Australian Research Council (ARC) Future Fellowship (FT210100097), and M. Zhang was supported by National Natural Science Foundation of China (NSFC) grant (62306084). This research is partially sponsored by the U.S. National Science Foundation through Grant No IIS-2302786.
[1] Joshua Batson, Daniel A Spielman, Nikhil Srivastava, and Shang-Hua Teng. 2013. Spectral Sparsification of Graphs: Theory and Algorithms. Commun. ACM 56, 8 (2013), 87-94.
[2] Davide Buffelli, Pietro Lio, and Fabio Vandin. 2022. SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS).
[3] Chen Cai, Dingkang Wang, and Yusu Wang. 2020. Graph Coarsening with Neural Networks. In International Conference on Learning Representations(ICLR).
[4] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. 2022. Dataset Distillation by Matching Training Trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4750-4759.
[5] Tianlong Chen, Yongduo Sui, Xuxi Chen, Aston Zhang, and Zhangyang Wang. 2021. A Unified Lottery Ticket Hypothesis For Graph Neural Networks. In International Conference on Machine Learning (ICML). PMLR, 1695-1706.
[6] Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-GCN: An Efficient Algorithm For Training Deep And Large Graph Convolutional Networks. In Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD). 257-266.
[7] Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. Advances in Neural Information Processing Systems (NeurIPS) 29 (2016).
[8] Zhiwei Deng and Olga Russakovsky. 2022. Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS).
[9] Mucong Ding, Xiaoyu Liu, Tahseen Rabbani, and Furong Huang. 2022. Faster Hyperparameter Search on Graphs via Calibrated Dataset Condensation. In NeurIPS 2022 Workshop: New Frontiers in Graph Learning.
[77] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. Dataset Condensation with Gradient Matching. In International Conference on Learning Representations (ICLR).
[78] Xin Zheng, Yixin Liu, Zhifeng Bao, Meng Fang, Xia Hu, Alan Wee-Chung Liew, and Shirui Pan. 2023. Towards Data-centric Graph Machine Learning: Review and Outlook. arXiv preprint arXiv:2309.10979 (2023).
[79] Xin Zheng, Yixin Liu, Shirui Pan, Miao Zhang, Di Jin, and Philip S Yu. 2022. Graph Neural Networks for Graphs with Heterophily: A Survey. arXiv preprint arXiv:2202.07082 (2022).
[80] Xin Zheng, Miao Zhang, Chunyang Chen, Chaojie Li, Chuan Zhou, and Shirui Pan. 2022. Multi-relational Graph Neural Architecture Search with Fine-grained Message Passing. In IEEE International Conference on Data Mining (ICDM).
[81] Xin Zheng, Miao Zhang, Chunyang Chen, Qin Zhang, Chuan Zhou, and Shirui Pan. 2023. Autoheg: Automated graph neural network on heterophilic graphs. arXiv preprint arXiv:2302.12357 (2023).
[82] Yizhen Zheng, Shirui Pan, Vincent Cs Lee, Yu Zheng, and Philip S Yu. 2022. Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination. In Advances in Neural Information Processing Systems (NeurIPS).
[83] Fan Zhou and Chengtai Cao. 2021. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Proceedings of the Association for the Advanced of Artificial Intelligence (AAAI), Vol. 35. 4714-4722.
[84] Daniel Zügner and Stephan Gunnemann. 2019. Adversarial Attacks on Graph Neural Networks via Meta Learning. In International Conference on Learning Representations (ICLR).