Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models
Authors: Wenzhuo Tang
Haitao Mao
Danial Dervovic
Ivan Brugere
Saumitra Mishra
Yuying Xie
Jiliang Tang
Venue: NeurIPS 2025 poster ()
Keywords: Graph Foundation Model
Graph Data Augmentation
Diffusion Models
Primary area: deep_learning
TL;DR: We introduce a universal graph structure augmentor for cross-domain data scaling.

Abstract: Models for natural language and images benefit from data scaling behavior: the more data fed into the model, the better they perform. This 'better with more' phenomenon enables the effectiveness of large-scale pre-training on vast amounts of data. However, current graph pre-training methods struggle to scale up data due to heterogeneity across graphs. To achieve effective data scaling, we aim to develop a general model that is able to capture diverse data patterns of graphs and can be utilized to adaptively help the downstream tasks. To this end, we propose UniAug, a universal graph structure augmentor built on a diffusion model. We first pre-train a discrete diffusion model on thousands of graphs across domains to learn the graph structural patterns. In the downstream phase, we provide adaptive enhancement by conducting graph structure augmentation with the help of the pre-trained diffusion model via guided generation. By leveraging the pre-trained diffusion model for structure augmentation, we consistently achieve performance improvements across various downstream tasks in a plug-and-play manner. To the best of our knowledge, this study represents the first demonstration of a data-scaling graph structure augmentor on graphs across domains.

Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models
Wenzhuo Tang', Haitao Mao?; Danial Dervovic', Ivan Brugere', Saumitra Mishra',
Yuying Xie Jiliang Tang'
Michigan State University 2 AAmaoon 3 J.P. Morgan Al Research [tangwen2, xyy, tangjili] @msu edu, macht@amazon. com [danial. dervovic, ivan. brugere, saumitra saumitra.mishra)d)
com
Abstract
Models for natural language and images benefit from data scaling behavior: the data fed into the model, the better they perform. This 'better with more' phenomenon enables the effectiveness of large-scale pre-training on vast amounts data. However, current graph pre-training methods struggle scale up data due to heterogeneity across graphs. To achieve effective data scaling, we aim to develop general model that is able to capture diverse data patterns of graphs and can be utilized to adaptively help the downstream tasks. To this end, we propose UniAug, universal graph structure augmentor built on diffusion model. We first pre-train discrete diffusion model on thousands of graphs across domains learn the graph structural patterns. the downstream phase, we provide adaptive enhancement by conducting graph structure augmentation with the help of the pre-trained diffusion model via guided generation. By leveraging the pre- pre-trained diffusion model for structure augmentation, we consistently achieve performance improvements across various downstream tasks plug-and-play manner. To the best of our knowledge, this study represents the first demonstration of a data-scaling graph structure augmentor on graphs across domains.
Introduction
The effectiveness of existing foundation models [1, 2, 3] heavily relies on the availability of substantial amounts of data, where the relationship manifests as scaling behavior between model performance and data scale [4]. Consistent performance gain has been observed with the increasing scale of
pre-training data in both Natural Language Processing [4, 5] and Computer Vision [6. 7] domains. This data scaling phenomenon facilitates the development of general models endowed with extensive knowledge and effective data pattern recognition capabilities. In downstream applications, these models are capable of adaptively achieving performance gains across tasks.
In the context of graphs, the availability of large-scale graph databases [8, 9, 10] enables possible data scaling across datasets and domains. Existing works have demonstrated graph data scaling following two limited settings: in-domain pre- pre-training [11, 12] and task-specific selection for pre-training data [13]. During the pre-training process, each graph in the pre-training pool must be validated in-domain or relevant to the downstream dataset. Given a specific domain or task, the crucial discriminative data patterns are likely confined to a fixed set [14], leaving other potential patterns in diverse graph data distribution noisy input. In terms of structure, graphs from different domains particularly composed of varied patterns [15], making it hard to transfer across domains. For example, considering the building blocks of the graphs, the motifs shared by the World Wide Web hyperlinks
"Tis work conducted while H.M. was Ph.D. student at Michigan State University and is independent of H.M.'s current position at Amazon.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

only partially align with those shared by genetic networks [15]. Therefore, closely aligning the characteristics of the pre-training graphs and the downstream data both in feature and structure is essential for facilitating positive transfer [13]. As consequence, the necessity of such meticulous data filtering restricts these methods from sealing up graphs effectively, as they can only utilize a small part of the available data. Given the limitation of the graph pre-training methods, a pertinent question emerges: How can we effectively leverage the increasing scale of graph data across domains?
Rather than focusing solely on data patterns specific to particular domains, we aim to develop a model that has comprehensive understanding of data patterns inherent across various types of graphs. In line with the principles of data scaling, we hypothesize that incorporating broader range of training datasets can help the model build an effective and universal graph pattern library, avoiding an overemphasis on major data patterns specific to any single dataset [16]. To construct such a model, we propose to utilize a diffusion model operating only on the structure as the backbone, for the following key reasons. (1) Unlike features, graph structures follow uniform construction principle, namely, the connections between nodes. This allows for positive transfer across domains when the upstream and downstream data exhibit similar topological patterss [13]. In particular, while the graph representations of neurons and forward electronic circuits are derived from distinct domains, they still share common motifs [15]. (2) Current supervised and self-supervised methods tend to capture only specific patterns of graph data, with models designed for particular inductive biases [16, 14, 17]. For instance, graph convolutional networks (GCNs) excel in nove- representation learning by emphasizing homophily, whereas graph-level representation learning benefits from expressive GNNs capable of distinguishing complex graph structures. (3) We opt for a structure-only model due to the heterogeneous feature spaces across graphs, which often include missing features or mismatched semantics [18]. For instance, node features yield completely different interpretations in citation networks, where they represent keywords of documents, compared to molecular networks, where they denote properties of atoms. To this end, we pre-train a structure-only diffusion model on thousands of graphs, which serves as the upstream component of our framework.
In the downstream stage, we employ the pre-trained diffusion model as Universal graph structure Augmentor (UniAug) to enhance the dataset, where diffusion guidance [19, 20, 21] is employed to generate synthetic
align the generated structure with the downstream requirements. Specifically, structures with various guidance objectives, and the resulting graphs consist of generated structures and original node features. This data augmentation paradigm strategically circumvents feature het- erogeneity and fully utilizes downstream inductive biases by applying carefully designed downstream models to the augmented graphs in plug-and-play manner. Empirically, we apply UniAug to graphs from diverse domains and consistently observe performance improvement in node classification, link prediction, and graph property prediction. To the best of our knowledge, this study represents the first demonstration of a cross-domain data-scaling graph structure augmentor.
2 Preliminary and Related Work
Learning from unlabeled graphs. Graph self-supervised learning (SSL.) methods provide exam- ples of pre-training and fine-tuning paradigm [22, 23, 24, 25, 26]. However, these methods benefit from limited data scaling due to feature heterogeneity, structural pattern differences across domains, and varying downstream inductive biases. It is worth mentioning that DCT [27] presents a pre-training and then data augmentation pipeline on molecules. Despite its impressive performance improvement on graph-level tasks, DCT is bounded with molecules and thus the use cases are limited.
Graph data augmentation. There have been many published works exploring graph data aug- mentation (GDA) since the introduction of graph neural networks (GNNs), with a focus on node- level [28, 29, 30], link- [31, 32], and graph-level [33, 34, 35, 36, 37]. These GDA methods have been generally designed for specific tasks or particular aspects of graph data. In addition, they are often tailored for a single dataset and struggle to transfer to unseen patterns, which limits their generalizability to a broader class of applications.
Diffusion models on graphs. Diffusion models [38, 39, 40] are latent variable models that learn data distribution by gradually adding noise into the data and then recovering the clean input. Existing diffusion models on graphs can be classified into two main categories depending on the type of noise injected, i.. Gaussian discrete. Previous works employed Gaussian diffusion models
2

---

Pretrain
Structure Augmentation
Downstream GNN
Diffusion Model
Guidance Head
+
Augmented Graphs
Diffuslon Model
Downstream GNN
Frozen
Accive
Figure The pipeline of UniAug. We pre-train a diffusion model across domains and perform structure augmentation the downstream graphs. The augmented graphs consist of generated structures and original node features and are then processed by downstream GNN.
both on general graphs [41, 42] and molecules [43, 44). However, adding Gaussian noise into the adjacency matrix will destroy the sparsity of the graph, which hinders the scalability of the diffusion models [45]. Recent works adapted discrete diffusion models to graphs with categorical transition kernels [46, 47, 48]. We denote the adjacency matrix of graph A E {0, _|"X= with nodes. With details Appendix A, we write the forward process to corrupt the adjacency matrix into a sequence of latent variables Bernoulli distribution
A $
A 1 Bernoulli (A a A (1-aa)m))
A A 9 (A*
A' 1) (A'-1 (A))
A
q (A' A")
where is the converging non-zero probability, a' is the noise scale, and a' I a'. Under predict-A" parameterization, the reverse process denoise the adjacency matrix with Markov chain
A 1- A') x Σ (A
A A0) Do
(A AA).
pe
An
where Po (Ag A²) represents the denoising network that predicts the original adjacency matrix from the noisy adjacency matrix. The parameters are estimated by optimizing the variational lower bound on the negative aigliiklihood [49]
Lub
E (A(AAA))) [KKL. (g (A A At,A")I po (A'- AA))
(3)
A lo (A A')] EA(A [KL (q (A' A") Il (A'))]
3 Method
In this section, our goal is build UniAug understand the diverse structure patterns of graphs and perform data augmentation with range of objectives. As illustrated in Fig. UniAug consists two main components: pre-trained diffusion model and the downstream adaptation through structure augmentation. We first collect thousands of graphs from varied domains with diverse patterns. To construct general model free of downstream inductive biases, we train self-conditioned discrete diffusion model on graph structures. In the downstream stage, we train an MLP guidance head on top of the diffusion model with objectives across different levels of granularity. We then augment the downstream dataset by generating synthetic structures through guided generation, where the augmented graph composed generated structures and original node features. Subsequently, we apply the augmented data to train a task apec-specific model for performing downstream tasks. Below, we elaborate on the data collection process, the architecture of the discrete diffusion model, and the guidance objectives.
3

---

3.1 Pre-training data collection
In light of the data scaling spirit, we expect our pre-training data to contain diverse data patterns with sufficient volume. As graphs from differ- ent domains exhibit different patterns [15], we wish to build a collection of graphs from nu- merous domains to enable universal graph pattern library with pre-training. Within the publicly available graph databases, Network Repository [8] provides a comprehensive collec- tion of graphs with varied scales from different domains, such as biological networks, chemi- cal networks, social networks, and many more. Among the thousands of graphs in the Network Repository, some of them contain irregular patterns, including multiple levels of edges, extremely high density, et cetera. To ensure the quality of the graphs, we analyze the graph properties following Xu al. [50] and filter the outliers. In addition, we observe that the coverage of graphs in the Network Repository is incomplete according to the network entropy and scale-free exponent, as we observe a relatively scattered space in the middle of Fig. 2. To fill in the gap, we include subset of the GutHub Star dataset [51] by random sampling 1000 graphs into our graph collection. The selected graphs are utilized to train a discrete diffusion model.
Propertins
entropp
Nersack Repaskay Oati & Say
Netwer
Scale-free exponent
Figure 2: Normalized structural properties Network Repository and Github Star. We enlarge the distribution coverage of our collection by combining both datasets.
3.2 Pre-training through diffusion model
Diffusion models have demonstrated the ability to facilitate transferability from a data augmentation perspective on the images [52, 53, 54]. Unlike the traditional hand-crafted data augmentation methods, diffusion models can produce more diverse patterns with high quality [52]. With the aid of diffusion guidance [19, 20], these methods can achieve domain customization tailored specific semantic spaces [53, 54]. Despite the success of data augmentation through diffusion models on images, the Euclidden nature of graph structures poses challenges for data-centric learning on graphs. In addition, the fact that most graphs in the Network Repository are unlabeled exacerbates the challenges, as the absence of labeled data results in substantially lower generation quality for diffusion models [20, 55].
To address the aforementioned challenges, we propose to construct self-conditioned discrete diffusion model on graph structures. Unlike Gaussian- diffusion models, discrete diffusion models [56, 49, 57, 46] operate with discrete transition kernels between latent variables, shown in Section 2. The key reason we opt for the discrete diffusion models lies in the sparse nature of graphs, where adding Gaussian noise into the adjacency matrix will result in dense graph [45]. On the contrary, discrete diffusion models effectively preserve the sparse structure of graphs during the diffusion process, thus maintaining the efficiency of the models on graphs.
To accommodate for unlabeled graphs, we adopt a self-supervised labeling strategy as an auxiliary conditioning procedure [58, 59]. By leveraging the self-labeling technique, we able to upscale the diffusion model to data with more diverse patterns [58]. The self-labeling technique requires two components: feature extractor and self-supervised annotator.
Feature extractor. We extract graph-level features by calculating graph propertics, including the number of nodes, density, network entropy, average degree, degree variance, and scale-free exponent following [50]. The first two represent the scale of the graph corresponding to nodes and edges, and the rest indicate the amount of information contained within a graph [50]. We compute the properties of one graph and concatenate them to get a graph-level representation.
Self-supervised annotator. To assign labels to graphs in self-supervised manner, we employ clustering algorithms on the graph-level representations. The number of clusters determined jointly by the silhouette score [60] and the separation of the graphs. The candidates of the number of clusters are chosen to ensure different clusters are well separated. Among the candidates, we select the final number of clusters by maximizing the mean Silhouette Coefficient of all samples.

---

link prediction Table I: Comparison between GDA methods, pre-training methods, and UniAug. By cross-domain transfer, we emphasize the ability of the method to train on vastly different domains and benefit all them.
GDA methods
Pre-training methods
UniAug
GraphAug
CFLP
Half-Hop
FLAG
AttrMask
D-SLA
GraphMAE
Effective graph-level task Ellective on edge level lask
Effective node-level task In-domain transfer
Cross-domain transfer
Next we detail the parameterization of the denoising model Po A graph-level labels k. The denoising model recovers the edges of the original adjacency matrix by predicting the connectivity of the upper triangle, which can be formulated problem [6], 62]. Following the link prediction setup, the denoising model composed of graph transformer (GT) [63] and an MLP link predictor. Denote the hidden dimension as d, we treat the node degrees as node features and utilize a linear mapping fa R -> Rd match the dimension. Similarly, we utilize another linear mapping f. R + R for timestep & and learnable embeddings fu : .O....,.................... K} + H for labels k, where K is the number clusters. The outputs are summed together and then fed into the GT. Mathematically, we have
A") with the self-assigned
h' GT (Ja (degree (A')) + f((()) (((k),A), (A 3 = (Ag h') MLP (Ih), hj])
With the above denoising network, our diffusion model is trained on the collected graphs by optimizing the variational lower bound in (3). After the pre-training process, we perform adaptive downstream enhancement through graph structure augmentation.
3.3 Downstream adaptation through data augmentation
The downstream phase of UniAug is to augment the graph topology through guided generation. This guidance process serves to provide downstream semantics for the diffusion model, thus bridging the gap between the pre-training distribution and the downstream datasets. Among the techniques for diffusion guidance, gradient-based methods [20, 21] offer versatile approaches by incorporating external conditions that are not present during training. For the discrete diffusion process, opt for the gradient-based NOS method [21] due to its flexibility and efficiency. Specifically, we build an MLP regression head go : Rd + R' that takes the hidden representations h' as the input and outputs the guidance objective of dimension r. Denote т as the temperature, as the step-size, as the regularization strength, and & drawn from (0, I), we sample from (AAAA h²) x (A h²) exp (go (h')) via Langevin dynamics
0
(5) One key question to answer is how to choose the proper guidance objectives. Our goal is to find numerical characteristics that can best describe the structural properties of graph. This includes supervision signal and self- supervisdd information on the level of node, edge, and graph.
h - h Van
AKL (D' (Ā* I (
(X ((1n")) go (b**) + 2yre.
Node level. Node labels provide the supervision signal for node classification tasks. Beyond node labels, node degrees are a fundamental factor in the evolutionary process of a graph [64]. From the perspective of network analysis, centrality measures indicate the importance of nodes from various viewpoints [65]. Empirically, we observe that utilizing different node-level heuristics as guidance targets tends to yield similar outcomes. Therefore, we focus on node labels and node degrees.
Edge level. Edge-level heuristics can be broadly classified into two categories: local structural heuristics, such as Common Neighbor and Adamic Adar [66], and global structural heuristics, such as Katz [67] and SimRank [68]. Similar node-level heuristics, empirical observations suggest that different edge-level heuristics tend to yield comparable guidance effects. In this work, we focus on the Common Neighbors (CN) heuristic due to its efficiency. Another edge-level guidance objective to recover the adjacency matrix from the node representations in a link prediction way, similar to how we parameterize the denoising network. We anticipate that such link prediction objective helps to align the generated graph with the downstream data the granularity of edges.
5

---

Table 2: Mean and standard deviation of accuracy (%) with 10-fold cross-validation on graph classification. The best result is bold. The highlighted results indicate negative transfer for pre-training methods compared to GIN. The last column is the average rank.
DD
Engyyee
Procents
NCII
IMD0
DMDB M
Reddit D
Reddit 12K
Collab
A.R.
GIN
75.81 6.11
06.00
73,32
2.200
2
49.07
1.30
48.63
74.54
AurMask
7 2.55
48.00
87.50
38.00
75.64
8.00
Chathed
15. 2:67
21.67
72.21 4.00
1.22
N000 1.55
48:20 2.25
90.33 2.31
47.62
25. 73.00 1.89
7.67
EdgePred
15.64 2.77
2200 +
71.22 3.53
N0.20
47:00 2.42
90.80 1.89
48.35
24.64 +
8.56
MnfoMax
75.23 4:443
22:50 2
2 1.48
71,00 2.06
46.70:2.44 2.46
89.15 2.84
48.98 1.53
2 1.12
8.00
10N0
75.96 2.86
22.17 3.67
71.51&5.331 25.31
2.227
71,02
48,25 2.06
90.17 2.13
49.01 =
74.77 4 1.73
7.11
D-SLA
74.66.33.30
22.67 4.31
71.97 2:4.1
77.05 2.311
71.92 2:75
47.28 1.58
80.77 1.87
48.50 =: 1:33
7.00
GropeMABl
16.07 4:2.5
2300 3,64
70.65 4,419
19.08
2.72
71,50 01.00:8:00 2.01
47.93 3.63
85,
3,63
47.67 e 1.16
24.84 1.35
7.67
2.222
66.85
74.61
5.33
48.30 2.50
75.
3.26
6.67
GrajhAu
2..23
68.14
3.221
74.00
1.85
49.00
76.02
3.67
FLAG
74.31
3,45
75.90
Unitug
78.13+2..1
9.88
75.47
30.54
2..0 2:45
92.28 a 1.59
=.74
7 2.02
1.11
Graph level. Graph labels offer the supervision signal for graph classes or regression targets. In addition, we incorporate graph-level properties [50] as quantitative measures to bridge the gap between the pre-training distribution and the downstream dataset. We empirically observe that graph label guidance offers significantly higher performance boosts compared to properties graph-level tasks. Therefore, we focus on graph labels in our experiments.
We provide our choice of objectives for each task in Appendix B. We note that all the above objectives are natural choices inspired by heuristics and downstream tasks. There exist many other self-supervised objectives to be explored, such as community-levy spectral change [69] and motif occurrence prediction [70]. We leave the study of objectives as one future work. With the diffusion guidance, we assemble the augmented graphs with generaled structures and original node features. The augmented graphs are then fed into cownstream-specifee GNNs.
3.4 Comparison to existing methods
The data augmentation paradigm of UniAug allows us to disentangle the upstream and downstream. We construct a diffusion model as the upstream component to comprehend the structural patterns of graphs across various domains. In addition, we leverage downstream inductive biases with wownstream-specinee models in plug-and-play manner. This allows UniAug to facilitate cross- domain transfer, offering a unified method that benefits graphs across different domains for various downstream tasks. On the contrary, existing GDA methods are typically designed for specific tasks and hard to transfer to unseen patterns. In the meantime, existing pre-training methods fail to transfer across domains due to heterogeneity in features and structures. This comparison highlights the success of UniAug as data-scaling graph structure augmentor across domains. We summarize the comparison between methods in Table 1.
4 Experiment
In this section, conduct experiments validate the effectiveness of UniAug. We first pre-train our discrete diffusion model on thousands of graphs collected from diverse domains. For each downstream task, we train MLP guidance head with corresponding objectives on top of the diffusion model. We then perform structure augmentation using UniAng and subsequently train task-specific GNN on augmented data for prediction. Through the experiments, aim to answer the following research questions:
RQ1: Can UniAug benefit graphs from various domains across different downstream tasks?
RQ2: What is the scaling behavior of UniAug corresponding to data scale and amount of compute?
RQ3:
Which components of UniAug are effective in preventing negative transfer?
4.1
Main results
To get a comprehensive understanding of UniAug, we evaluate it on 25 downstream datasets from 7 domains for graph property prediction, link prediction, and node classification. The statistics of the datasets can be found in Appendix C, and technical details of the experiments are in Appendix D.

---

Table 3: Mean and standard deviation across 10 runs link prediction. Results scaled x100. The last two methods are based on NCN, while the rest are based. The best result bold for two backbones, respectively. The last column is the average rank of each GCN based method.
Com MRR
Creseer MRR
Pubmeed MRR
Power Hiud 10
Yeast Faav 10
kidde Hise 10
Flickr Hinde 10
R.R.
30.26 4: 4,80
50.37 7.91
16.38 1.30
30.61 a 4.07
24.71 a 4.92
35.71 2.65
2.58
MVCIRL
3..03+
41.32+4..1
15.21 + 2.35
31.71 + 3.78
23.74 +
36.21 = 2.81
8.42 + 2,18
5.29
Sclf-superiisdd
GRACE
31.27
49.13
16.88 1.74
28.21 = 5.04
23.96
3
9.87
BGRL
2.14
51.914
33.71 a
25.91
37.95 =
a
GrsphMAR
$2.98 5,01
52.71 5.39
1883 + 1,30
32.81 2.12
26.51 - 2,92
35.63
7.01 -
GDA
UniAng GCN
CFLP
35.36 7,88
33.62 6:44
55.20 4.16
54.66
17.01 2,75 a
34.36 a
16.02 e
27.52
24.23 + 5.23
39.67
28.74 2.38
0:00 0.00
4.57
NCN-bused
UniAug NCN
NCN
35.92 7.85
31.72 4.48
61.09 3.21
50,03 3,45
38.26 6. 2.56
40.30 2.53
27.36 5.00
30.20 à 1.46
42.11 5.74
39.85 e: 5.07
39.26 2.88
36.81 3.29
9.53 a 0.92
8,85 a 0.90
Table 4: Mean and standard deviation of accuracy (%) across 10 splits on node classification of heterophilic graphs. The hest result is bold. The last column is the average rank of each method.
Comell
Wisconsin
Texas
Actor
Chameleon"
Squirel
A.R.
GCN
59.41 &: 6.03
51.68 1 4.34
63.78 4,80
30.58 à 1.29
40.94 & 3.91
39.11 a 1.74
4,50
MVGRL
50.64 5.89
61.70 3.94
31.37 a 0.83
32.34 1: 2.11
35.32 1 1.32
6.83
Sclf-suprriied
GRACE
$6.39 2.111
53.83 + 3.56
63.54
28.14 0.81
35.71 1.95
33.65 + 2.51
7,00
BGRI.
2.16
59.80 4.08
65.78 2.66
29,80 0.31
37.01 2.89
34.77 + 2.01
5.33
GrathME
37.31
58.27 2.91
58.34 3.57
28.97 a 0.27
36.75 & 1.78
39.13 1.133.2000
5.50
Half- Hop
6..46:27.58
76.47
72.35 4.27
3
38.59 2.89
2.13
GDA
UniAAg
68.11=6.773
69:02 + 4.96
73.51 5.06
33.11 1.57
43.84 + 3.39
41.90 + 1.90
2.00
UniAug Holl-Hop
72,43
79.61 5.56
77,03 4.27
34.97 0.55
41,94 2,77
38.79 1 2.61
1.67
Baselines. We evaluate our model against three main groups of baselines. (1) Task-specific GNNs:
For graph property prediction, we use GIN [17]; for link prediction, we use GCN [71] and NCN [72]; and for node classification, we use GCN [71]. (2) Graph pre-training methods: These include AttrMask, CtxtPred, EdgePred, and InfoMax [22], JOAO [25], D-SLA [24]. and GraphMAE [23]. For each of these methods, we pre-train it on the same pre-training set as UniAug. While most of the pre-training graphs lack node features, we calculate the node degrees as the input. Each method consists of three pre-trained variants with different backbone GNNs, including GIN, GCN, and GAT. (3) Self-supervised methods: These include MVGRL [73], GRACE [74], BGRL [75] and GraphMAE [23]. For the self-supervised methods, we extract the node embeddings and feed them into downstream specific heads. (4) Graph data augmentation (GDA) methods: For graph property
prediction, include S-Mixup [34], GraphAug [35], FLAG [37], GREA [36], and DCT [27]; for link prediction, we include CFLP [3 I]: and for node classification on heterophilic graphs, we include Half-Hop [30]. The GDA methods are implemented based on chosen task- GNNs.
Graph property prediction. We employ graph label guidance for UniAng through- out the graph-level tasks by training a 2- layer MLP as the guidance head on the graph labels in the training set. In the augmentation stage, we generate multiple graphs per training sample, and the gener- ated graphs then fed into the baseline GIN. We present the results of molecule re- gression in Table and graph classification in Table 2. Three key observations emerge from the analysis: (1) Existing pre-training methods show negative transfer compared to GIN. Some special cases are the En- zymes and molecule regression datasets, where all pre-training methods fail to yield satisfactory results. In these datasets, the features are one of the driving components for graph property prediction, while the pre-training methods fail to encode such information due to
Table 5: Mean and standard deviation of MAE J. across 10 runs on molecule regression. The last column is the average rank of each method. Among the methods, all pre-training methods discard atom and bond features due to dimension mismatch and we include the best best-performing method JOAO into comparison; GIN and UniAug remove the bond features; others incorporate both.
oghg-TLipo
oghg-ESOI. ESOL.
oghg- Fregg-rreSool
A.R.
GINE
0.545 0.019
0,766 = 0.016
1.639 = 0.146
5.00
GIN
0.543 0.021
0.729 0.018
0.
3.67
JOAO
0.859 0.007
1.458 0.040
3.292 = 0.11
7,00
FLAG* GREA*
0.528 0.012
0.755 + 0.039
1.565 + 0.098
3.00
0.586
0.036
0.805 0.135
1,829 + 0.368
6.00
DCT
0.516 + 0.071
0.717 0.020
1,339 + 0.075
1,33
UniAng
0.528 0.006
0.677 0.026
1.448 0.049
1.67
*Results are taken from DCT [27].
7

---

incompatibility with the feature dimension. This reveals one critical drawback of the pre-training methods: their inability handle feature heterogeneity. (2) GDA methods yield inconsistent results across different datasets. While these methods enhance performance in some datasets, they cause performance declines in others. This variability is directly reflected in the average rank, where some of them even fall behind the GIN. (3) Unlike the pre-training methods and GDA methods, UniAug shows consistent performance improvements against GIN with a large margin. In the molecule regression tasks, UniAug effectively compensates for the absence of bond features and achieves performance comparable to DCT, which a data augmentation method pre-trained on in-domain molecule graphs. We also adapt the remi-supervised [76] and self-supervised [77] setting for the baselines for a comprehensive benchmark in Appendix D.1 Table 13, where we observe that Uni presents consistently satisfactory performance according to the average rank, matching outper- forming the best baseline. These findings affirm that the pre-training and structure augmentation paradigm of UniAug effectively benefits the downstream datasets at the graph level.
Link prediction. We choose three guidance objectives for UniAug, including node degree, CN, and link prediction objective, as described in Section 3.3. For each objective, we train an MLP to provide guidance information. We then augment the graph structure by generating a synthetic graph and preserving the original training edges, ensuring that the augmented graph does not remove any existing edges. The augmented graph is then fed into a GCN for link prediction. We summarize the results in Table 3, where we observe that (1) GDA method CFLP leads performance drops on the datasets without features and also suffers from high computation complexity during preprocessing, (2) UniAug enhances performance across all tested datasets. In addition, we employ UniAug to NCN [72], one of the ofate-of-thh-art methods for link prediction. The results demonstrate consistent performance boosts from UniAug when we apply NCN as the backbone. The structure augmentation paradigm of UniAug allows plug-and-play and-play applications to any downstream- specific models, showcasing its adaptability and effectiveness. Additional results, including the performance of pre-training baselines and the effects of three guidance objectives, can be found in Appendix D.2.
Node classification. To demonstrate the ef- fectiveness of UniAug in node-level tasks, we transform the node classification into subgraph classification. Specifically, we extract the aggre- gation tree of each node, i.e.. 2-hop subgraph for 2-layer GCN, and label the subgraph with the center node. We then adopt a strategy similar to graph classification and train 2-layer classifier as a guidance head. Inspired by the success of structure augmentation on heterophilic graphs [78, 30], we evaluate UniAug on 6 heterophilic datasets. We observe phenomena similar to those seen in graph- and link-level tasks in Table 4, with results of pre-training baselines in Appendix D.3. One thing mention is the combination of UniAug and Half-Hop. Half-Hop offers performance improvements in four out of six datasets via data augmentation, and combining it with UniAug yields even higher results. This highlights the flexibility of UniAug and opens up possibilities for further exploration of its use cases. Given the impressive results of UniAug on heterophilic graphs, we anticipate will also help balance the performance disparities among nodes with different homophily ratios on homophilic graphs [16]. We split the nodes into five groups according to their homophily ratios and calculate the standard deviation (SD) across groups. As shown in Table 6, UniAug matches the performance of vanilla GCN and also reduces the performance discrepancies corresponding to SD.
Table 6: Results of node classification on homophily graphs. Results are scaled x x100.
Cora
Citeseer
Pubmed
GCN UniAng
81.75 a 0.73
70.71 a 0.76
79.53 a 0.25 79.54 : 0.35
ACC
81.78 a: 0.60
71.17 0.58
SD
GCN
24.51 1.06
22.57 + 0.80
27.02 0.56
UniAug
23.45 = 0.99
19.90 = 0.81
26.50 = 0.55
4.2 Scaling behavior of UniAug
In light of the neural scaling law [4, 5, 6, 7, 79], we expect UniAug to benefit from an increased coverage of data and more compute budget. In this subsection, we investigate the scaling behavior of UniAug in terms of data scale and amount of compute for pre-training.
Data coverage During the data collection process, we prepare three versions of the training data with increasing magnitude and growing coverage on the graph distribution. We first sample 10 graphs per category from the Network Repository [8] to build a SMALL collection. Next, we gather all the graphs from the Network Repository and filter out large-scale graphs and outliers for FULL
8

---

80
45
Metrics Values
40
75-
Accuracy
135
70-
100
65
25
60
20
Enzymes
IMDB-B
Proteins
Yeast
Power
Coza
Erdos
Figure 3: Effects of pre-training data scale graph classification (left) and link prediction (right). The groups
SMA, FUT,, and EXT represent SMALL, FUTLL, and EXTRA data collection.
Enzymes IMDB-B
Proteins
Yeast
Power
Erdos
Citeseer
Accurasy
74
Metrics Values
3.0
72
40
70
30
68
15
20
25
10
15
20
25
Compute (10-) PF-days)
Compute (10-3 PF-days)
Figure 4: Effects pre-training amount of compute on graph classification (left) link prediction (right), where one PF-days 10th x 24 3600 8.64 x 10 floating point operations.
collection. In addition, we add 1000 graphs subset of the GitHub Star dataset from TUDataset [80] to enlarge the coverage of diverse patterns and form EXTRA collection. We pre-train three versions of UniAug respectively on the three collections and evaluate them on graph classification and link prediction. As shown in Fig. 3, we observe clear trend of increase in performance we enlarge the coverage of pre-training data. This paves the way to scale up UniAug to even more pre-training datasets with expanding distribution of graphs. Additional results on scaling effects with respect to the proportion of the full data can be found in Appendix D.4.
Amount of compute We sought to understand how effectively our diffusion model can learn data patterns as we continue to train it. To this end, we checkpointed UniAng every 2,000 epochs (5 x 10-3 PF-days) while training on the EXTRA collection, and then applied it to graph classification and link prediction tasks. The results are illustrated in Fig. 4. We observe that downstream performance generally improves with prolonged training. while the trend slows down for some datasets when we reach 8,000 epochs. We take the checkpoint the 10,000th epoch for evaluations. Given the scaling behavior observed, we anticipate UniAug to become even more effective with additional resources.
4.3 Preventing negative transfer
In the previous parts of the experi- ments, showcase the positive trans- fer of UniAug across different tasks. We now investigate which aspects of the design prevent negative trans- fer. UniAug consists of two main components: pre-trained diffusion model and the structure augmentation through guided generation. In the pre-training process, we inject self- supervised graph labels into the dif- fusion model and we wonder about the performance of its unconditioned counterpart. Regarding the augmenta- tion process, we examine the impact
Table 7: Demonstration of negative transfer on graph classification (up) and link prediction (down).
Enzymes
Proteins
IMDB-M
73.32
UniAug
71.50 5.85
75.47 2.50
73.50 2.48
50.13 2.05
wlo self-cond
71.11 7.50
73.31 4.63
71.50
2.27
49.00 2.74
wio guidance w/ cross-guide
62.17 3.93
71.15
$3,80 = 3.29
35.33
3.17
51.50
72.46 4.35
71.10
2.38
49.20
2.59
Cora MRR
Clesser MRR
Power
Yeast
Endos Hivek 10
Hivd 10
Hiudh 1o
GCN
30.26
50.57 7.91
30.61 4.00
24.21
Liniing
35.36 7.88
34.36
27:52 a 4.50
39.67
wio self-cond
27.97 16.11
37.65 6.00
28.95 7.73
23.54 2
34.35
wio guidunce
29,00 6.06
51.41 7.10
25.57 6.00
25.36 6.06
37.11
wf coos-guide
12.37 4.50
50 19.59.55 5.67
92.99 - 2.54
26.76 3.58
36 3.67

---

of diffusion guidance by exploring outcomes when the guidance is either removed or applied using another dataset from a different domain (cross-guide). We summarize the results Table 7 for graph classification and link prediction. All modifications investigated lead to performance declines in both tasks. We observe that removing guidance results in significant negative transfers for graph classification, while the effects of seff-conditionin are more pronounced for link prediction. We conclude that both the self-condiiioning strategy and diffusion guidance are crucial in preventing negative transfer, underscoring their importance in the design of UniAug.
Conclusion and Discussion
In this work, we propose a graph structure augmentor UniAug to leverage the increasing scale of graph data. We collect thousands of graphs from various domains and pre-train self-conditioned discrete diffusion model on them. In the downstream stage, we augment the graphs by preserving the original node features and generating synthetic structures. We apply UniAug to node-, link-, and graph-level tasks and achieve consistent performance gain. We have successfully developed a showcase that benefits from cross-domain graph data scaling using diffusion models.
One limitation of the current analysis is the absence of an investigation into the effects of model parameters due to limited resources. Given the scaling behavior of UniAug in terms of data scale and amount of compute, we anticipate that large-scale model will provide significant performance improvements. One future direction is to investigate the adaptation of fast sampling methods to the discrete diffusion models on graphs. This will lead to lower time complexity and enable broader application scenarios.
Impact Statements
In this work, we build a universal graph structure augmentor that benefits from data scaling across domains. Given the consistent performance improvements for different tasks, we expect this work to contribute significantly towards the goal of building graph foundation model. In the meantime, we showcase the power of the deep generative models graphs by introducing new application scenarios. We anticipate such success will contribute to the community of generative models and graph learning.
important to mention that the model backbones of our method and baselines heavily rely on neighboring node information as an inductive bias. However, this characteristic can result in biased predictions, especially when patterns in neighborhood majorities dominate, leading to potential ethical issues in model predictions.
7 Acknowledgment
W.T. and J.T. are supported by the National Science Foundation (NSF) under grant numbers CNS2321416, IIS2212032, IIS2212144, IIS 2504089, DUE2234015, CNS2246050, DRL2405483 and IOS2035472, the Michigan Department of Agriculture and Rural Development, US Dept of Com- merce, Amazon Faculty Award, Meta, NVIDIA, Microsoft and SNAP. W.T. and Y.X. are supported by the National Institutes of Health (NIH) under grant numbers U01DE033330, R01HL.16650S, R01DE026728 and the NSF under grant numbers OISE2434687 and 1OS2107215.
8 Disclaimer
This paper was prepared for informational purposes by the Artificial Intelligence Research group of JPMorgan Chase & Co. and its affiliates ("JP Morgan"), and is not a product of the Research Department of JP Morgan. JP Morgan makes no representation and warranty whatsoever and disclaims all liability, for the completeness, accuracy or reliability of the information contained herein. This document is not intended as investment research or investment advice, or recommendation offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction, and shall not constitute a solicitation under any jurisdiction or to any person, if such solicitation under such jurisdiction or to such person would be unlawful.
10

---

References
[1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 9748-8763. PMLR, 2021.
[2] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Ilama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
[3] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015-4026, 2023.
[4] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint 2rXiv:2007.08361, 2020.
[5] Jordan Hoffmmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203. /5556, 2022.
[6] Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, and Hanie Sedghi. Exploring the limits of large scale pre-training. In International Conference on Learning Representations, 2022.
[7] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2204-12113, 2022.
[8] Ryan Rossi and Nesreen Ahmed. The network data repository with interactive graph analyties and visualization. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015.
[9] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 13:22118-2213 2020.
[10] Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http: //sapp. stanf ord. edu/data, June 2014.
[11] Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z. Li. Mole-BERT: Rethinking pre-training graph neural networks for molecules. In The Eleventh International Conference on Learning Representations, 2023.
[12] Zhiyuan Liu, Yaorui Shi, An Zhang, Enzhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. Advances in Neural Information Processing Systems, 36, 2024.
[13] Yuxuan Cao, Jiarong Xu, Carl Yang. Jiaan Wang, Yunchao Zhang, Chunping Wang. Lei Chen, and Yang Yang. When to pre-train graph neural networks? an answer from data generation perspective! arXiv preprint arXiv:2303.166588 2023.
[14] Haitao Mao, Juanhui Li, Harry Shomer, Bingheng Li, Wenqi Fan, Yao Ma, Tong Zhao, Neil Shah, and Jiliang Tang. Revisiting link prediction: a data perspective. In The Twelfth International Conference on Learning Representations, 2024.
[15] Ron Milo, Shai Shen-Orr, Shalev Itzkovitz, Nadav Kashtan, Dmitri Chklovskii, and Uri Alon. Network motifs: simple building blocks of complex networks. Science, 298(5594);824-827,, 2002.
[16] Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, and Jiliang Tang. Demystifying structural disparity in graph neural networks: Can one size fit all? Advances in Neural Information Processing Systems, 36, 2024.
11

---

[17] Keyulu Xu, Weihua Hu. Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
[18] Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Michael Galkin, and Jiliang Tang, Graph foundation models. arXiv preprini arXiv:2402.02216. 2024.
[19] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
[20] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[21] Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances in Neural Information Processing Systems, 36, 2024.
[22] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-traue pre-training graph neural networks. arXiv preprint 1226655 2019.
[23] Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 594-604, 2022.
[24] Dongki Kim, Jinheon Back, and Sung Ju Hwang. Graph self-supervised learning with accurate discrepancy learning. Advances in Neural Information Processing Systems, 35: 14085- 2022.
[25] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In International Conference on Machine Learning, pages 12121-12132. PMLR, 2021.
[26] Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. Self-supervised graph- level representation learning with local and global structure. In International Conference on Machine Learning, pages 11548-15558 PMLR, 2021.
[27] Gang Liu, Eric Inae, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Data-centric learning from unlabeled graphs with diffusion model. Advances in neural information processing systems, 36, 2024.
[28] Hyeonjin Park, Seunghun Lee, Sihyeon Kim, Jinyoung Park, Jisu Jeong, Kyung-Min Kim, Jung-Woo Ha, and Hyunwoo J Kim. Metropolis-hasting data augmentation for graph neural networks. Advances in Neural Information Processing Systems, 34:19010-19020, 2021.
[29] Songtao Liu, Rex Ying, Hanze Dong, Lanqing Li, Tingyang Xu, Yu Rong, Peilin Zhao, Junzhou Huang, and Dinghao Wu. Local augmentation for graph neural networks. In International conference on machine learning, pages 14054-14072. PMLR, 2022.
[30] Mehdi Azabou, Venkataramana Ganesh, Shantanu Thakoor, Chi-Heng Lin, Lakshmi Sathidevi, Ran Liu, Michal Valko, Petar Veličković, and Eva L Dyer. Half-hop: A graph upsampling ap- proach for slowing down message passing. In International Conference on Machine Learning, pages 1 1341-1360. PMLR, 2023.
[31] Tong Zhao, Gang Liu, Daheng Wang, Wenhao Yu, and Meng Jiang. Learning from counter- factual links for link prediction. In International Conference on Machine Learning, pages 26911-26928 PMLR, 2022.
[32] Trung-Kien Nguyen and Yuan Fang. Diffusion- negative sampling on graphs for link prediction. arXiv preprint arXiv:2403. 172599 2024.
[33] Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. G-mixup: Graph data augmentation for graph classification. International Conference on Machine Learning, pages -248. PMLR, 2022.
12

---

[34] Hongyi Ling, Zhimeng Jiang, Meng Liu, Shuiwang Ji, and Na Zou. Graph mixup with soft alignments. International Conference on Machine Learning. pages 1333-211449 PMLR, 2023.
[35] Youzhi Luo, Michael McThrow, Wing Yee Au, Tao Komikado, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Automated data augmentations for graph classification. arXiv preprint arXiv:2202.1 13248, 2022.
[36] Gang Liu, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph rationalization with environment-based augmentations. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1069-1078, 2022.
[37] Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard Ghanem, Gavin Taylor, and Tom Goldstein. Robust optimization data augmentation for large-scale graphs In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 60-69, 2022.
[38] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[39] Yang Song, Jascha Sohl-Dickstein, Diederík P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[40] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[41] Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. In International Conference on Artificial Intelligence and Statistics, pages 4474 4484. PMLR, 2020.
[42] Jaehyeong Jo, Seul Lee, and Sung Ju Hwang, Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning, pages
PMLR, 2022.
[43] Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International conference on machine learning, pages 9558-9568. PMLR, 2021.
[44] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations, 2022.
[45] Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, and Roger Wattenhofer. Diffusion models for graphs benefit from discrete state spaces. In NeurlPS 2022 Workshop: New Frontiers in Graph Learning, 2022.
[46] Clement Vignac, Igor Krawezuk, Antoine Siraudin, Bohan Wang. Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. In The Eleventh International Conference on Learning Representations, 2023.
[47] Xiaohui Chen, Jiaxing He, Xu Han, and Liping Liu. Efficient and degree-guided graph generation via discrete diffusion modeling. In International Conference on Machine Learning, pages 4585-4610 PMLR, 2023.
[48] Jialin Chen, Shirley Wu, Abhijit Gupta, and Zhitao Ying. D4explainer: In-distribution explanations of graph neural network via discrete denoising diffusion. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[49] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Struetured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 14:179911-1 17993, 2021.
13

---

[50] Jiarong Xu, Renhong Huang, Xin Jiang, Yuxuan Cao, Carl Yang, Chunping Wang, and Yang Yang. Better with less: data-active perspective on pre-training graph neural networks. Advances in Neural Information Processing Systems, (6:56946-56978 2023.
[51] Benedek Rozemberezki, Oliver Kiss, and Rik Sarkar. Karate club: an api oriented open-source python framework for unsupervised learning on graphs. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 3125-3132, 2020.
[52] Brandon Trabucco, Kyle Doherty, Max A Gurimas, and Ruslan Salakhutdinov. Effective data augmentation with diffusion models. In The Twelfth International Conference on Learning Representations, 2024.
[53] Zebin You, Yong Zhong, Fan Bao, Jiacheng Sun, Chongxuan Li, and Jun Zhu. Diffusion models and semi-supervised learners benefit mutually with few labels. Advances in Neural Information Processing Systems, 36, 2024.
[54] Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and XIAOJUAN QL Is synthetic data from generative models ready for image recognition? In The Eleventh International Conference on Learning Representations, 2023.
[55] Fan Bao, Chongxuan Li, Jiacheng Sun, and Jun Zhu. Why are conditional generative models better than unconditional ones? arXiv preprint arXiv:22/2.00362 2022.
[56] Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454-12463 2021.
[57] Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems, 35:28266-28279, 2022.
[58] Shanghua Gao, Zhong-Yu Li, Ming-Hsuan Yang, Ming-Ming Cheng, Junwei Han, and Philip Torr. Large-scale unsupervised semantic segmentation. IEEE transactions on pattern analysis and machine intelligence, 2022.
[59] Vincent Tao Hu, David W Zhang, Yuki M Asano, Gertjan J Burghouts, and Cees GM Snoek. Self-guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 18413-18422, 2023.
[60] Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53-65, 1987.
[61] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018
[62] Ajay Kumar, Shashank Sheshar Singh, Kuldeep Singh, and Bhaskar Biswas. Link prediction techniques, applications, and performance: A survey. Physica A: Statistical Mechanics and its Applications, 553: 144899, 2020.
[63] Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi supervisd classification. arXiv preprint arx1v:2009.03300, 2020.
[64] Yang-Yu Bung-Yuloe Liu, Jean-Jacques Slotine, and Albert-LászIó Barabási. Controllability of complex networks. nature, 177-7336):167-173, 2011.
[65] Stephen P Borgatti. Centrality and network flow. Social networks, 27(1)555-71, 2005.
[66] Lada A Adamic and Eytan Adar. Friends and neighbors on the web. Social networks, 25(3):21 1--200, 2003.
[67] Leo Katz, new status index derived from sociometric analysis. Psychometrika, 18(1):39 43, 1953,
14

---

[68] Glen Jeh and Jennifer Widom. Simrank: measure of structural-contet similarity. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 538-543. 2002.
[69] Shiyin Tan, Dongyuan Li, Renhe Jiang, Ying Zhang, and Manabu Okumura. Community- invariant graph contrastive learning. arXiv preprint arXiv:2405.01350, 2024.
[70] Yu Rong. Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. Advances in neural information processing systems, 33:12559-12571, 2020.
[71] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
[72] Xiyuan Wang, Haotong Yang, and Muhan Zhang. Neural common neighbor with completion for link prediction. In The Twelfth International Conference Learning Representations, 2024.
[73] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning graphs. In International conference on machine learning, pages 41 16 4126. PMLR, 2020.
[74] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. arXiv preprint vrXi:22006.04131, 2020.
[75] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L Dyer, Remi Munos, Petar Veličković, and Michal Valko. Large-scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations, 2022.
[76] Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. L22gen: Layer-wise and learned efficient training of graph convolutional networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2127-2135, 2020.
[77] Fan-Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020.
[78] Wendong Bi, Lun Du, Qiang Fu, Yanlin Wang, Shi Han, and Dongmei Zhang. Make het- erophily graphs better fit gnn: A graph rewiring approach. arXiv preprint 2:X1:2200.082644, 2022.
[79] Jingzhe Liu, Haitao Mao, Zhikai Chen, Tong Zhao, Neil Shah, and Jiliang Tang. Neural scaling laws on graphs. arXiv preprint M.Xiv:2402.020555 2024.
[80] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.0866. 2020.
[81] Ze Wang, Jiang Wang, Zicheng Liu, and Qiang Qiu. Binary latent diffusion. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 22576-22585, 2023.
[82] Paul D Dobson and Andrew J Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology, 330(4))771-783, 2003.
[83] Ida Schomburg, Antje Chang, Christian Ebeling, Marion Gremse, Christian Heldt, Gregor Huhn, and Dietmar Schomburg. Brenda, the enzyme database: updates and major new developments. Nucleic acids research, ([ssupp__)::43333
2004.
[84] Nikil Wale, Ian A Watson, and George Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems, 14:347-375, 2008.
[85] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1365-1374, 2015.
15

---

[86] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad, Collective classification in network data, Al magazine, 29(3):93-93, 2008.
[87] Duncan J Watts and Steven H Strogatz. Collective dynamics of 'small-world networks. nature, 393(6684):440- 442, 1998.
[88] Dongbo Bu, Yi Zhao, Lun Cai, Hong Xue, Xiaopeng Zhu, Hongchao Lu, Jingfen Zhang, Shiwei Sun, Lunjiang Ling. Nan Zhang, al. Topological structure analysis of the protein- protein interaction network in budding yeast. Nucleic acids research, 1109):2443-2450 2003.
[89] Vladimir Batagelj and Andrej Mrvar. Pajek datasets. http://vlado fmf uni 1j sssssi/pppp ppbb networks/datal, 2006.
[90] Oleksandr Shehur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05866 2018
[91] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen Geometric graph convolutional networks. In International Conference on Learning Represen- lations, 2020.
[92] Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social influence analysis in large-scale networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 807-816, 2009.
[93] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we re- ally making progress? In The Eleventh International Conference on Learning Representations, 2023.
[94] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zhigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826, 2016.
[95] Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. A fair comparison of graph neural networks for graph classification. In International Conference on Learning Representations, 2020.
[96] Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv preprint arXiv:2406. 10727, 2024.
[97] Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: A simple yet effective method towards cross-domann graph pretraining. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4443-4455 2024.
[98] Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre- training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pages 417-428, 2023.
[99] Quang Truong and Peter Chin. Weisfeiler and lehman go paths: Learning topological fea- tures via path complexes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15382-15391, 2024.
[100] Juanhui Li, Harry Shomer, Haitao Mao, Shenglai Zeng, Yao Ma, Neil Shah, Jiliang Tang, and Dawei Yin. Evaluating graph neural networks for link prediction: Current pitfalls and new benchmarking. Advances in Neural Information Processing Systems, 36, 2024.
[101] WANG Botao, Jia Li, Yang Liu, Jiashun Cheng, Yu Rong, Wenjia Wang, and Fugee Tsung. Deep insights into noisy pseudo labeling on graph data, In shirth- Conference on Neural Information Processing Systems, 2023.
16

---

[102] George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, 20(1):359-392 1998.
[103] Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference machine learning, pages 40-48. PMLR, 2016.
[104] Jingyu Chen, Runlin Lei, and Zhewei Wei. PolyGCL: GRAPH CONTRASTIVE LEARNING via learnalle spectral polynomial filters. In The Twelfth International Conference Learning Representations, 2024.
17

---

A Derivation of Diffusion Process
In the following, will formulate the existing diserete diffusion models into binary diffusion on the adjacency matrix. We denote the adjacency matrix of a graph as A E {0, 1] 1 with n nodes. Following D3PM [49], we corrupt the adjacency matrix into sequence of latent variables A by independently injecting noise into each element with Markov process
A 1:7 A A²,
q (A' A²-1)=
Cat (Aljip= = Q²)) {Jaс}
II
(6)
where Q' E [0, 1]2x2 is the transition probability of timestep f. The above Markov process is called forward process. Existing works provide different designs for the transition matrix Q', including
g
3
Uniform [48]
3
8r
Absorbing [47] :
8
8
i
1 3r π (1 7t
B P x
Predefined [46]
where is the converging non-zero probability and 3 is the noise scale. All three transition matrices can be written as binary diffusion with Bernoulli distribution
(A | = Bernoulli (A": A (1-)))))))
9 (A' A Bernouli (A): A" + (1)
(8)
(A" 1 A A)) 9 (A' (A A
(AA' A9)
where a' 3r and a' I a². The prior A determined by with p (AT)
Bernoulli(x). i.c. the existence of each edge follows a Bernoulli distribution with probability r. The main difference of the forward process among the existing works is the choice of æ, where = 0 for EDGE [47], 0.5 for D4Explainer [48], and pre-computed average density for DiGress [46].
In our early experiments, observe that the absorbing kernel of efficiency and effectiveness for graph generation. The forward process with non-zero π will add non-existing edges, which brings in additional computations, When sampling from prior, non-zero will introduce additional uncertainty because we will first sample every edge from Bernoulli TT). in this work and leave the exploration of other
surpasses the other two in terms
Therefore, we choose the absorbing prior transition kernels as future work.
We note that in our implementation, early experiments and some existing works [81, 47]. We leave the study of the effects diffusion timesteps on downstream tasks future work.
choose the number of timesteps T 128 according to
B Guidance Objective for Downstream Tasks
We mention various guidance objectives in Section 3.3 with different granularity. Here, we specify the objectives use for each downstream task. Our empirical results suggest that supervision signals will lead to better performance. Thus, we use node labels for node classification and graph labels for graph property prediction in Section 4. Regarding link prediction, we anticipate that both node-level and edge-level objectives may help the downstream adaptation. Therefore, we choose three objectives including node degree, CN heuristic, and link prediction objective.
C Datasets
The license of the datasets use this work Table 8.
Graph property prediction datasets include DD and Proteins [82], Enzymes [83], NCI1 [84], IMDB-Binary, IMDB-Multi, Reddit-Binary and Reddit-Multi-12 [85], ogbg-Lipo. ogbg-ESSOL and Frbe-FreeSolv [9]. The statistics are summarized in 9.
18

---

Table 8: List of datasets and corresponding License
Dataset
License
Network Repository
CCBY-SA
Github Star
CC 4.0
Cora
NLM license
Citeseer
NLMM license
Pubmed
NLM license
WebKB
MIT license
Wikipedia Network
license
Actor
MIT license
Power
BSD License
Yeast
BSD License
Fidos
RSD License
Amazon Photo
MIT license
Flickr
MIT license
DD
CC
Eazymes
CC BY 4.0
Proceins
CC 4.0
NCTI
CC BY 4.0
TMDR
CC BY 4.0
Reddit
CC BY 4.0
Table Statistics of graph property prediction datasets.
Domain
Dataset
Task type
Graphs
Tasks
Nodes
Edges
DD
Classification
1.178
284
716
Biology
Enzymes
Classifization
600
33
Classification
1,113
Academic
Collab
Classification
5,000
2458
IMDB
Classification
1,000
20
97
IMDB-M
Classification
1,500
66
Social
Reddit-5k
Classification
4,999
509
595
Classification
11,929
NCT
Classification
4,110
30
32
oghg Lipo
Regression
4200
27
59
Chemical
ogbg-ESOL
Regression
1128
ogbg-FreeSolv
Regression
642
Link prediction datasets include Cora, Citeseer, and Pubmed [86], Power [87], Yeast [88], Er- dos [89], Amazon Photo [90], and Flickr [10]. The statistics are summarized in 10.
Table 10: Statisties of link prediction datasets.
Domain
Citation
Transport
Biology
Academic
Social
#Nodes
2,708
3,327
18.717
4,941
2,284
6,927
334,863
#Edges
5,278
4,676
44,327
6,646
899,756
Mean Degree
3.9
2.81
4.74
2.67
5.82
3.42
5.69
Node classification datasets include Cora, Citeseer, and Pubmed [86], WebKB (Texas, Cornell, and Wisconsin) [91], Wikipedia Network (Chameleon and Squirrel) [91], and Actor [92]. The first three are homophilic graphs, and the others are heterophilic. The statisties are summarized in 11.
D Experiment
In this section, we introduce the implementation details and additional results for the experiments. Throughout all the experiments, we train all the methods with Adam optimizer an A A100 GPU. We train the guidance head of UniAug with cross-entropy loss for class labels and mean squared error loss for all other objectives. For multi-class objectives, apply the label smoothing [94] technique
19

---

Table 11: Statistics of node classification datasets.
Cora
Citeseer
Pubmed
Comell
Wisconsin
Texas
Chamelson
Squirel
Actio
Domain
Citation
Web
Social
#Nodes #Edges #Classes
2,708 5,278
3,327 4,676
19.717 44,324
183
251
183
890
2,223 46,998
7,600 33,544
295
499
309
8,854
3
5
5
*Chameleon and Squirrel are filtered to remove duplicated nodes [93].
following NOS [21]. Denote y as the one-hot label and C as the number of classes, we have
y. āi + y (1 -1l-;)) ā.) +C
(9)
D.1 Graph property prediction
For graph classification, we follow [95] for the setting with 10-fold cross-validation. We utilize a 5-layer GIN with latent dimensions of 64 throughout the datasets. For molecule regression, we implement& a 5-layer GIN with a virtual node, and the latent dimensions are 300. We have mainly four hyperparameters for UniAug: step-size and regularization strength À in (5), number of repeats per training graph, and whether augment validation and test graphs with the trained guidance head. For each training graph, we repeatedly generate structures and plug in the original node features for multi-repeat augmentation. We perform the update (5) for times per each sampling step. The hyperparameters are tuned from the choices in Table 12.
Table 12: Hyperparameter choices for graph property prediction.
0.01 [0.1. 1.0]
repeals
s, 10. 641 [True, False]
Aug val and test
In Section 4.1, we aim to benchmark the capability of cross-domain pre-training of different methods on the same set of pre-training graphs. While the pre-training graphs contain vastly different features, we have to align the feature space to allow pre-training for the baseline methods. There are two ways to tackle the feature heterogeneity issues in the existing literature. One line of them utilizes LLMs to align text-space graphs [96], which is not applicable to broader classes graphs. Other works, like GCOPE [97]. perform dimension reduction to align the feature dimension of different graphs. We emphasize that dimension reduction methods fail to deal with extreme cases like missing features. This phenomenon is pretty common in real life, large proportion of the graphs in the Network Repository do not have corresponding features. Therefore, we simply use the node degrees as the features in Section 4.1.
We understand that removing the node features may result in a performance drop for the baseline methods. Note that most of the baselines follow the pre-training paradigm of [22] with domain- specific model designs for chemistry and biology datasets, and thus cannot be directly applied to the chosen graph classification datasets. Therefore, supervised [77] setting for the baselines for a comprehensive benchmark. The semi-supervised setting involves pre-train pre-training with all data of that specific dataset and finetuning the training set of each split. Meanwhile, baselines of the self-supervised setting pre-train on the whole dataset and then classify the learned graph embeddings with downstream SVM classifier. The results summarized in Table 13, where the best and second-best results are highlighted bold and italic, respectively. We observe that UniAug presents consistently satisfactory performance according the average rank, matching or outperforming the best bascline.
adapt the semi-supervised [76] and self-
To demonstrate the effectiveness of UniAug in scenarios with limited labeled data, we perform 5-shot graph classification following [98]. The results are summarized in Table 14 These results show that UniAug achieves significant performance improvements over the self-supervised baselines, underscoring its robustness and adaptability in sho- settings.
In addition, to showcase the flexibility of UniAug on the downstream backbone, we pick one of the SOTA method PIN [99] for graph classification and evaluate UniAug on the basis of it. The results summarized in Table 15, where see UniAug offers constant improvements over PIN.
20

---

Table 13: Mean and standard deviation of accuracy (%) with 10-fold cross-validation on graph classification.
The best and second-best results are highlighted in bold and italic. The last column is the average rank.
DD
Proteins
NC11
IMDB-8
IMDB-M
Reddit-B
Collab
A.R.
ChiPPrdd
74 51
70 23-1.33
75.00++030
66.66-9955
73.69-0037
6,80
Semi-sugervisd
GraphCL
InfoMax
76.1741.37
75.7810.34
72.27.0.00
74.17=0.34
74.63#0.25
74.8610.26
89.1 10.11
88, 03.66100.95
73.7610.29
74,23=0.21
5.60
4,60
I0N0
75.8120.73
73.3120.18
74,8610.39
88.7910.65
75.5320.18
4.60
InfoGraph
74.44e0.31
76.20el D6
73.03=0877
49 09e11.53
82, 22.Stell.k
70.68=1.13
5.17
Self-xupesvised
GraphCL.
14.39:11.45
71,14s0
48
89.5910.84
11.36:1.15
10AO
74.55.0.41
78.0720:17
70.2113.05
19:20s0.77
85 15.2911.35
69,5010.36
5.17
GrathMAE
75.3010,39
80 10.4000.30 30
75.5220.66
51.63,0,522
88.0110.19
80.32:00.66
2.17
UniAug
78.13:2.1
75.47=2.50
90.54:1.77
73.50=2.48
50.1322.05
92.2821.59
77.00=2.02
1.43
Table 14: Accuracy of 5-shot graph classification.
Proteins
Enzymes
GIN
58.17 + 8.58
20.34 + 5.01
InfoGraph
54,12 8.20
20.90 + 3.32
GraphCL.
56.38 7.24
28.11 + 4.00
JOAO
57.21
35.31 3.79
GraphMAE
60.03
33.91 # 6.58
UniAng
66.85 4.7.1
48.37 4.77
Table 15: Accuracy of graph classification with PIN.
Proteins
NCII
IMDB-B
PIN
78.8 =
85.1 1,5
76.6
UniAng PIN
80.2 = 2.8
86.5 1.4
77.9 1.8
D.2 Link prediction
For link prediction, we follow the model designs and evaluation protocols of [100]. For results based on GCN and NCN, with prediction head. The prediction head of GCN is a 3-layer MLP. The number of layers and the latent dimension of the GCN encoder taken from [100]. We have mainly three hyperparameters for UniAug: step-size y and regularization strength X, and the number of updates in (5) per each sampling step. In addition, inspired by the pseudo labeling strategy [101], we provide an option threshold for the sampling process of the diffusion model. Specifically, we only keep the edges with the probability of existence higher than g for each sampling step. After the sampling process, we recover the training edges of the original graph structure. The hyperparameters are tuned from the choices in Table 16. One thing to mention is that we handle the large graphs by graph partitioning with METIS [102]. Specifically, we augment the partitions of large graph and then assemble the partitions back into single graph. The edges between different partitions are recovered after the assembling process.
GCN encoder to produce node embeddings and perform link prediction
Table 16: Hyperparameter choices for link prediction.
[0.01, 100]
[0.1. 1.0, 10.0]
[None, 0.9, 0.99. 0.999]
# updates
[5, 10, 20]
In addition to the results shown in Table 3, we have the pre-training baselines mentioned in Section 4. The results are summarized in Table 17. We observe that existing pre-training methods provide negative transfer, especially datasets with node features. More explanation removing the node features can be found in Appendix D.I.
As mentioned in Section 1. we choose three guidance objectives for link prediction with different granularity. The effects of different objectives can be found Table 18. We observe that the outcomes of different objectives differ across datasets and there is consistently winning strategy.
21

---

Table 17: Mean and standard deviation across 10 rums link prediction. Results are scaled x x100. The last two methods are based on NCN, while the rest are GCN- based. The best result bold for two backbones, respectively. The highlighted results indicate negative transfer for pre-training methods compared to GCN. The last column is the average rank of each GCN-based method.
Dom MRRI
Cloaser MIR
Pulwnod MRR
Power Hisek
Yrast Hostt 10
Fabo Have to
Flickr Hisee 10
A.K.
GCN
90.26 x 4,80
50.57 x 7.91
16.38 1.30
50.61 407
24 71 4.92
35.71 i 2.65
8.10 2.58
4.14
Aurbaask
20:25
=
25.10 a
5.77
CiatPred
15:08 2.91
3:00
29:30
22:40
7.86
Edpeited
25.75
6.66
545
InfoMax
16.35 2.57
22.90
2.7.
35.82 4.12
3.23
6.00
23.6
23.10
15.33 4.770
33.77 #3.05
6:01 a 1.57
SLA
25.55
23.05
3.396
29.37
26 3.32
36,02
6,70 2.03
5.29
GraphMAL
13.9
13,00
27,00 1.99
2.92
35,80 a
5.41 2.44
CRLP
6..4
17.01 2.755
16,02 a 8.31
24.23
28.74 2.38
0OM
6.43
Uniksg-GCN
2.836
$4.66 4.55
17.28 1.59
34.36 L68
27.52 4.90
39.67 a 4.51
9:46 a 1.18
1.29
NCN
4.472
59,03
+.2.6
5000
5,07
Uniong NCN
61.69
1,46
42.11
Table 18: Effects of different guidance objectives.
Cora MRR
Citeseer MRR
Puhmed MRR
Power Hits 10
Yeast @
Erdos
Flickr Hits@l
@
Link guide
30.45 2.90
54.66 4.55
16.97
31.41
25.8 4.10
9,46 =
Degree guide
32.73 6.71
51.13 a 5.51
16.37 0.58
32.88 2.02
27.52 4.80
39.67 4.51
9.11 0.88
CN guide
35.36 + 7,88
50.86 + 5.73
17.28 + 1.89
34.36 1.68
26.67 = 4.02
36.18 + 4.32
9.28 = 1.18
D.3 Node classification
For node classification on heterophilic graphs, we use the fixed splits from Geom-GCN [91] for Cornell, Wisconsin, Texas, and Actor. For Chameleon and Squirrel, we remove duplicated nodes following [93] and take their fixed splits. Regarding node classification on homophilic graphs, we employ the superviper setting [103]. The GCN backbone is implemented as a 2-layer classifier. Similar to graph property prediction, we have mainly four hyperparameters for UniAug: step-size y and regularization strength À in (5), number of repeats per training graph, and whether augment validation and test graphs with the trained guidance head. The hyperparameters are tuned from the choices in Table 19.
Table 19: Hyperparameter choices for node classification.
A
0.01
10.1.0.5, 0.5, 1.0]
repeats
3. 10]
Aug val and test
[True, False]
Table 20: Mean and standard deviation of accuracy (%) across 10 splits on node classification heterophilic graphs. The best result bold. The highlighted results indicate negative transfer for pre-training methods compared GCN. The last column the average rank of each method.
Camell
Wiscnesin
Texas
Actor
Chancleon*
Squirre
A.R.
GCN
59.41 6.03
51.68 4.34
63.78 4,80
30 à 1.29
10.913.99
39.11 1.74
3.83
AttMask
44,86
4,31
15.82
a 1.03
= 1.68
ChxPPied
4081 7.78
36.67
58.92
4.32
23
2.63
26.26
9.50
EdgePPee
=..10
48.04
+ 6.63
50.37
22.99
6.22
21.02
5.06
27.94 8.41
8.83
InfoMax
39.19 4 12:75
39.80 16.38
58.87
4.06
23.30 4.37
22.59
27.5
10.17
JOAO
8.00
44.70 7.45
$7.06 3.43
24.17 5.02
25,81 3.79
31.72-
8.33
D-SLA
41.05 6.38
59.93 9.553
23.74 4.06
2..29
=8.50=66:0
8.00
GraphMAE
47.05
57.06 4.559
63.10
24.69 a 0.68
37.18
31.94 1.65
Half-Hop
2..56
76.47 2.66
72.35
33.95 2 0.68
38.59
27.34:2.18
3.00
UniAng
68.11 6.772
69.02 + 4.96
75.51
5:06
$3.11 + 1.57
43.84
3,39
41.90 = 1.90
2.00
Unitug Half-Hop
72.43 5.81
79.61 5.56
4.273
34,97 + 0.55
41.94 1.744+2.777 2.77
38.79 = 2.61
1.50
*Chameleon and Squirrel are filtered remove duplicated nodes [93].
In addition to the results shown Table 4, we have the pre-training baselines as mentioned in Section 4. The results are summarized in Table 17, with similar scenarios graph- and link-level
22

---

tasks. The existing pre-training methods provide negative transfer when pre- pre-trained on the same data collection as UniAug. More explanation on removing the node features can be found in Appendix D.1.
To showcase the flexibility of UniAug on the downstream backhone, we pick one of the SOTA methods PolyGCI. [104] for node classification heterophilic datasets and evaluate UniAug on the basis of it. The results are summarized in Table 21, where we see UniAug produces improvements over PolyGCL in three out of four datasets.
Table 21: Accuracy of node classification with PolyGCL
Cornell
Wisconsin
Texas
Actor
PolyGCL
82,62 a 3.11 84.31 # 2.88
85,50 1.88
88.03 1,80
41.15 0.88
UniAug PolyCCL
88.35 + 2.58
86.70 2.77
43.01 1.27
D.4 Investigation on scaling
In Section 4.2, we investigate the scaling behavior of UniAug regarding data scale and pre-training time. We omit some of the results for better visualization. Here we present the numerical results in Table 22 and Table 23.
80
45
10
75
Metrics Valuees
Accuracy
35
70
30
65-
25
60
20
100
50 100
75 100
Endymes mee
[MDB-B
Proteins
Yeast
Power
Cora
Erdos
Figure Effects pre-training data scale (ratio) on graph classification (left) link prediction (right).
We recognize that these three sets vary in both scale and diversity. To analyze the scaling effect of UniAug based solely data quantity, clustered the pre-training into 10 clusters based graph-level representations (Section 3.2) and performed stratified sampling within these clusters. From this, we created three subsets containing 25%, 50%, and 75% of the total graphs, and pre- trained UniAng on each subset. The results, summarized in Fig.5, show a clear trend of performance improvement as the size of the pre-training set increases. Combined with experiments on the SMALL, FULL, and EXTRA sets, these findings suggest that UniAug benefits from both increasing the scale and enhancing the diversity of the pre-training data.
Table 22: Effects of pre-training data scale on graph classification (up) and link prediction (down).
Engymes
Proteins
IMDB B
IMDB M
GIN
66.00 7.52
73.32 4.03
71.10 2.90
49.07 2.81
UniAug SMALL
3.20
UniAug- FULL
71.33 + 6.51 71.17 + 7.10
74.05 4.82
73.11 2.35
49.67 + 2,41
UniAug- EXTRA
73.50
50.13 2.05
Corn MRR
Citeseer MRR
Power
Yeast
Einos Hits@10
Hits@
Hits@
GCN
30.26
50.57 = 7.91
30.61 + 4.07
24.71 4.92
35.71 2.65
UniAugg SMALL
32.25 8.71
47.91 a 3.87
32.25 a 3.72
25.81 a 4.89
36.28 3.56
UniAug.
32.97 3.75
UnlAng- EXTRA
35.36 7,88
54.66 4,55
34,36 + 1,68
27.52 4,80
39.67 4,51
23

---

Table 23: Effects of pre-training amount of compute on graph classification (up) and link prediction (down).
10
PF-days
Enzyyms
Proteins
IMDB-E B
IMDB M
6.21
3.63
2.90
48.28 2.75
10
69.00 e 5.10
74.30 + 5.33
72.80 3.85
48.60 2.23
68.83 $ 5.88
75.11 3.18
71.77
2.38
48,60 2.48
20
70.79 5.73
74.87
5.30
73.04
2.82
49.47
25
71.50 5,85
75.47 + 2.50
73.50 + 2.48
50.13
2.05
10 PF-days
Cora
Ciieseer
Power
Yeast Hit
Erdos
5
27.56 +: 4.36
49.45 4: 9.20
22.81 a 9,47
23.62 a: 9.77
35.33 =: 3.16
31,02 6.53
50.72 6.22
32.49 2.52
26.70 + 4.85
36.10 4.66
15
33,24 a 7.97
49.02 5.92
32.88 a 1.31
27.80 4.55
39.70 3.67
25
35.36 * 7.88
54.66 4.55
34.36 1.68
27.52 + 4.80
39.67 4.51
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
It's in the Conclusion and Discussion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.

---

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The experimental details can be found in the main text and the appendix. We also provide the code in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
25

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The data publicly available and the code is in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c ec/
public/ mublic/guides. /tt.dde/Cooeeuummiiiii SeteeesumiissooPooo
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. wwtpcccwpppppcetcccccccccccccctt wtaiiic/ggiee//eeeee Saaeeee///ooSSommeeeeeee http://odiiiiiiiisoooooooi.ooo.o
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see the experiment section and the corresponding appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
26

---

Does the paper report enror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report the standard deviation across runs.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurip:: httphh/wwwpp.ppppppppeeeeppeeeee
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
27

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See the impact statement.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: See appendix.
Guidelines:
28

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
See method and experiment sections.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
29

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (https: pettps://neurre for what should or should not be described.
30


Reviews:
[{"review_id": "hQfEuwxQ3X", "reviewer": "Reviewer_NC11", "summary": "This paper addresses the challenge of leveraging large-scale, cross-domain graph data for effective learning. It introduces UniAug, a universal augmentation framework that pretrains a discrete diffusion model to capture structural patterns from diverse graphs. The pretrained model is then used to guide structure-based augmentations for downstream tasks. Experiments demonstrate that UniAug benefits from increased data scale and improves performance across a range of graph learning tasks.", "questions": "* While Figure 3 and Table 22 show a general trend of improved performance with more pretraining data, many of the gains (especially between FULL and EXTRA) appear small and fall within overlapping confidence intervals. Could the authors provide formal statistical tests to confirm whether these improvements are significant?  \n* The paper uses older models like GCN and GIN as downstream backbones. Have the authors evaluated UniAug with more recent GNN architectures, such as GraphSAGE for node classification or GPS for graph classification?  \n* Have the authors explored the effects of model scaling, such as increasing the size of the diffusion model (e.g., number of layers or hidden dimensions)? Given the observed data and compute scaling behavior, it would be interesting to know whether UniAug also benefits from larger model capacity.  \n* Could the authors include a discussion of recent graph foundation model work such as GraphAny \\[1\\], GraphFM \\[3\\], GraphProp \\[2\\], GFSE \\[4\\], etc? These methods also explore cross-domain generalization, and comparing or contrasting them with UniAug would help clarify the unique contributions of this work.\n\nSorry about the mixup again, and I would totally understand if the 2nd question is not feasible to answer in the remaining time. Just answering 1st, 3rd, and 4th should be enough.\n\n\\[1\\] Zhao, Jianan, et al. \"Graphany: A foundation model for node classification on any graph.\" *arXiv preprint arXiv:2405.20445* (2024).  \n\\[2\\] Sun, Ziheng, et al. \"GraphProp: Training the Graph Foundation Models using Graph Properties.\"  \n\\[3\\] Lachi, Divyansha, et al. \"Graphfm: A scalable framework for multi-graph pretraining.\" *arXiv preprint arXiv:2407.11907* (2024).  \n\\[4\\] Chen, Jialin, et al. \"GFSE: A Foundational Model For Graph Structural Encoding.\"", "limitations": "Yes, they discuss the data scaling limitation of the methods that could benefit from more diversity. To go further, introducing diversity on the decoding task, brain area studied, or species (e.g. human data for the BCI application) seems interesting directions. The pretraining task limitation is interesting but would benefit from more details such as information about staying in a generative and studying which masking scheme is the most appropriate (MtM or NEDS) or switching to a discriminative approach (Pop-t)? For the transfer of knowledge beyond cross modality taking for example the cross-brain region distillation. This direction goes towards specialized models specific for some brain region instead of a foundation model of the brain, are the authors more convinced by this direction?", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n* The paper presents a novel structural augmentation approach using a diffusion model that focuses only on graph structure. By avoiding the use of node features, it handles the challenge of feature heterogeneity and can be applied across domains where features are inconsistent or missing.  \n* UniAug is model-agnostic and can be easily integrated with any downstream GNN, making it a flexible and practical approach that does not require modifications to existing model architectures.  \n* The paper includes robust evaluation across a wide range of tasks and datasets, which helps support the general effectiveness of the proposed method.\n\nWeakness:\n\n* The paper mainly evaluates UniAug using older models like GCN and GIN. While this helps isolate the effect of the augmentation, these models are no longer considered state-of-the-art. It remains unclear whether UniAug would still provide strong improvements when used with more recent models, such as GraphSAGE or H2GCN for node classification. It’s possible that the benefits of UniAug could be reduced—or even disappear—when tested with stronger and more modern graph architectures.   \n* The paper does not discuss several recent works that also address cross-domain graph generalization, such as GraphAny \\[1\\], GraphProp \\[2\\], and GraphFM \\[3\\]. These models have already shown that cross-domain transfer is both feasible and beneficial. As a result, the claim that this is the *first* method to demonstrate cross-domain data scaling in graphs feels overstated. While this paper has its own clear and novel contributions, it is important to acknowledge recent progress in this area.  \n* While Figure 3 and Table 22 show that UniAug generally improves with more pretraining data (SMALL → FULL → EXTRA), the results often have high standard deviations and overlapping performance ranges. For example, on Enzymes, Proteins, and IMDB-M, the gains between FULL and EXTRA are small and likely not statistically significant. Similar issues appear in Table 2 (DD, IMDB-M), Table 4 (Pubmed, Power), and Table 5 (Actor, Chameleon).", "was_revised": true, "final_justification": "My major concerns were regarding statistical significance of some reported improvements, regarding model scaling behavior, and regarding positioning with existing literature. All of them have been appropriately discussed by the authors in their rebuttal response.", "rebuttal": "We thank the reviewer for their time and effort in the review process.\n\nAfter carefully reading the comments, we believe there may have been a misunderstanding, as the review appears to be for a different paper. The points discussed do not align with the content of our submission.\n\nTo resolve this, we have already contacted the Area Chair to bring this to their attention.\n\nWe appreciate your understanding and service to the conference."}, {"review_id": "RTWq6dU7V7", "reviewer": "Reviewer_6ARW", "summary": "The authors propose UniAug, a graph structure augmentor based on a discrete diffusion model. UniAug is first pre-trained on a large number of graphs across domains to learn generalizable structural patterns, and then used to augment the structure of graphs in downstream tasks. The paper claims that this approach enables effective scaling behavior and yields consistent improvements on downstream tasks.", "questions": "Refer to Strengths and weaknesses", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "S1. The paper shows that learning from cross domains can benefit different downstream tasks.\n\nS2. The proposed method demonstrates consistent improvements over existing graph learning baselines.\n\nS3. Using discrete diffusion models for structure augmentation\n\nW1. My major concern is about the node features. It seems that UniAug only learns to capture the adjacency matrices of graphs from a collection. During the data augmentation stage, just use the same node features. From my perspective, the lack of consideration of node features is a limitation. \n\nW2. The selection of the guidance head appears to rely on trial-and-error, which may be time-consuming and task-specific. This reliance reduces the practicality and ease of generalization of UniAug across diverse downstream tasks.\n\nW3. From Table 3, DCT still has better results than UniAug. Does this mean that domain-specific pretrained models are still better than cross-domain models? This observation suggests that domain-specific models may remain more effective than cross-domain approaches in some settings, thereby weakening the paper’s central claim regarding the universal applicability and superiority of cross-domain pretraining.\n\nW4. Pretraining on only thousands of graphs may be insufficient to support the claim of universality. It would be better to show UniAug scales to more graphs. \n\nW5. What is the time cost for pretraining the discrete diffusion model? It could be critical when scaling to even larger datasets.", "was_revised": false, "final_justification": "", "rebuttal": "> Q1. From my perspective, the lack of consideration of node features is a limitation.\n\nIt is important to note that **node features exhibit great heterogeneity across datasets and domains**. For instance, the atom characteristics of molecules are fundamentally different from the paper keywords in citation networks. Additionally, a large proportion of real-world networks lack corresponding node features altogether. This suggests that **pre-training a feature-centric model to achieve positive transfer is extremely challenging**.\n\nMoreover, **even for graphs that share the same feature space, feature-centric models can lead to negative transfer**. For example, graphs with textual node features demonstrate that feature-centric pre-trained models like **GraphMAE** show a performance gap compared to **GCN** training from scratch [1]. This indicates that pre-training a feature-centric model may not be an adequate approach for achieving data scaling.\n\nGraph structures adhere to a uniform construction principle, namely, the connections between nodes. We believe that our pre-training and augmentation paradigm can effectively utilize the diverse topological patterns across domains and provide positive transfer to various downstream tasks. During the downstream application, we assemble the augmented graphs with generated structures and original node features. This approach allows UniAug to **bypass the feature heterogeneity problem and achieve consistent positive transfer**.\n\n> Q2. The selection of the guidance head appears to rely on trial-and-error, which may be time-consuming and task-specific. This reliance reduces the practicality and ease of generalization of UniAug across diverse downstream tasks.\n\nWe thank the reviewer for raising this concern. We respectfully clarify that the selection of the guidance head is **not based on trial-and-error, but follows a consistent and logical principle**: to use the most direct supervisory signal available for the given downstream task.\n\n1. For **supervised tasks** (e.g., node or graph classification), the most direct signal is the ground-truth labels. Therefore, we systematically use a prediction head trained on these labels as the guidance.\n2. For **self-supervised tasks** where explicit labels are absent (e.g., link prediction), we leverage well-established, task-specific heuristics from the literature. For link prediction, we follow this principle by using a guide based on Common Neighbors (CN), a proven and effective heuristic for this task [2].\n\nAs shown in Table 18, using the CN-based guidance **still achieves state-of-the-art performance**, attaining the highest average rank when compared with the results in Table 4. \n\n> Q3. From Table 3, DCT still has better results than UniAug. Does this mean that domain-specific pretrained models are still better than cross-domain models?\n\nThe reviewer is correct that the domain-specific DCT model achieves a better result on this particular molecular regression task. However, this comparison highlights a fundamental difference in approach and, we argue, showcases the strength of our domain-agnostic framework.\n\n1. **Different Inputs, Different Goals:** It is critical to note that DCT is a specialized model **designed exclusively for molecular graphs** and leverages rich, domain-specific **bond/atom features** which carry significant chemical information. In contrast, UniAug is intentionally designed as a **domain-agnostic framework** that operates **only on the graph's topological structure**. This design choice is what allows UniAug to generalize across wildly different domains (e.g., social networks, citation networks, *and* molecular graphs) using a single, unified model.\n2. **Highlighting the Power of Structural Learning:** The fact that our purely structure-based UniAug achieves results that are **highly competitive and remarkably close** to a specialized model using extra, powerful domain features is a strong testament to our method's effectiveness. It demonstrates the significant extent to which meaningful patterns can be learned from topology alone.\n\nWe emphasize that the goal of UniAug is not to outperform every specialized model in its specific domain, but to provide a **single, powerful pre-training framework** that is broadly applicable.\n\n> Q4. Pretraining on only thousands of graphs may be insufficient to support the claim of universality. It would be better to show UniAug scales to more graphs.\n\nWe agree that scaling is an important dimension for pre-training, and we'd like to clarify our rationale. For a **cross-domain** model, we argue that the **diversity** of the pre-training data is also crucial compared to the raw quantity. Our primary goal was to build a corpus with the broadest possible coverage of different graph structures and domains.\n\n- We chose to combine a subset of GitHub Star with the **Network Repository** precisely because it is the largest and most diverse public collection of graphs from disparate domains (social, biological, economic, etc.).\n- Our own results support this strategy. As we demonstrate empirically in **Figure 3**, improving the **coverage of different graph distributions** directly correlates with better downstream performance. This finding suggests that at this scale, diversity is a primary driver of generalization.\n\nWhile the resulting dataset of several thousand graphs may seem modest compared to huge, domain-specific corpora (e.g., billions of molecules in ZINC), it represents the state-of-the-art for a **heterogeneous, multi-domain collection**.\n\nIn Figure 5, we observe a clear trend of performance improvement as the size of the pre-training set increases. As part of our future work, we plan to explore the integration of other large-scale collections, such as SNAP, to continue pushing the boundaries of both scale and diversity.\n\n> Q5. What is the time cost for pretraining the discrete diffusion model? It could be critical when scaling to even larger datasets.\n\nWe thank the reviewer for this practical question. The entire pre-training process reported in our paper took approximately **26 hours**, using four NVIDIA RTX A6000 GPUs. We consider this a very reasonable **one-time cost** to produce a powerful, universal model that can then be rapidly applied to numerous downstream tasks. In the context of large-scale pre-training, this is a computationally efficient result. Future work could also incorporate standard engineering optimizations (e.g., mixed-precision training) to further improve efficiency for larger pre-training corpus.\n\n[1] Chen, Zhikai, et al. \"Text-space graph foundation models: Comprehensive benchmarks and new insights.”\n\n[2] Wang, Xiuan, et al. “Neural Common Neighbor with Completion for Link Prediction.”\n\n---\n\nWe appreciate your feedback and believe that the innovative aspects of our approach and its practical effectiveness in diverse scenarios will bring significant insights into the field. We would be grateful if you could consider increasing your score to support our work."}, {"review_id": "aLvOA3iOjy", "reviewer": "Reviewer_LHbU", "summary": "Authors present UniAug, which uses a diffusion model to learn fundamental graph properties, and create a general-purpose model that understands graph structures across a wide variety of domains. By training on cross-domain graphs, and being agnostic to the node features, UniAug can move to downstream applications where they preserve original node features but have new neighborhood structures. Authors show performance improvements in node classification, link prediction, and graph property prediction.", "questions": "My main question is how much total time is spent on training UniAug and applying it to a specific graph task. I saw some data on the impact of pre-training on the tasks but was only able to gather the trend that more pre-training results in higher metrics. This is important because readers should be able to understand the computational cost of applying UniAug. While the diffusion model can be applied, we still have to train the MLP guidance head.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: Datasets are comprehensive, domains are diverse, and improvements are mostly consistent across datasets. Authors also tackled all the major graph tasks. Homophilic and heterophilic graph datasets are also covered.\n\nWeaknesses: Improvements are generally not large, which could be offset by the fact that there are so many domains and datasets. Understanding the sensitivity of hyperparameters is missing, though called out as due to limited resources. Paper is also largely empirical, and given that the consistent improvements across homophilic and heterophilic graphs, I think a theoretical foundation for why UniAug works across these graph structures would bring confidence to the reader.", "was_revised": true, "final_justification": "Authors have sufficiently answered most of my questions to bump up to borderline accept. Some issues remain with metrics and theoretical foundation, so I won't go higher than that.", "rebuttal": "> Q1. Improvements are generally not large, which could be offset by the fact that there are so many domains and datasets.\n\nWe thank the reviewer for raising this concern. We respectfully argue that the improvements from UniAug are both **significant** and a **direct result of our core methodology**.\n\n**First, the improvements are not marginal; they are dominant and consistent.** While individual percentage gains may vary, the **average rank** across multiple datasets provides the clearest picture of robust performance:\n\n- In Table 4, UniAug achieves an average rank of **1.43**, far superior to the second-best method's 3.00.\n- Similarly, in Table 5, our method's rank of **1.43** decisively beats the next-best self-supervised method (BGRL), which has a rank of 5.33.\n\nThis pattern of consistently securing the top rank (Tables 4, 5, 12) demonstrates a non-marginal, highly reliable performance gain.\n\n**Second, these significant gains are caused by our proposed mechanism, not just the data.** The most direct proof is our ablation study in **Table 7**. This experiment shows that when we **disable the diffusion guidance, performance drops significantly**, leading to negative transfer. This proves that the success of UniAug is causally linked to its ability to perform task-specific augmentation, rather than being a simple side effect of a large and diverse pre-training set.\n\n> Q2. Understanding the sensitivity of hyperparameters is missing\n\nWe thank the reviewer for this excellent suggestion. We agree that a hyperparameter sensitivity analysis is important for understanding our model's robustness, and we have performed the requested analysis. We analyzed the effect of the guidance step size, a key hyperparameter. Below are the results for link prediction (MRR) on the Cora dataset. As the table shows, our method achieves strong performance across a reasonably wide range of step sizes (from 0.1 to 10), with performance peaking at 1.0. This demonstrates that our method is **not overly sensitive** to this hyperparameter and does not require expensive, fine-grained tuning to achieve good results.\n\n| Step size | 0.01 | 0.1 | 1 | 10 | 100 |\n| --- | --- | --- | --- | --- | --- |\n| Cora MRR (/times 100) | 32.22 ± 8.23 | 34.22 ± 7.91 | 35.36 ± 7.88 | 34.22 ± 5.12 | 31.05 ± 10.21 |\n\n> Q3. A theoretical foundation for why UniAug works across these graph structures would bring confidence to the reader\n\nOne of the key motivations of this manuscript is that **GNN model architectures should be aligned with task-specific inductive biases** [2]. For instance, GCN performs well on node classification for homophilic graphs, while GIN demonstrates superior performance on graph classification. This highlights the inherent difficulty in designing a single GNN that performs well across all types of tasks.\n\nTo address this, graph data augmentation (GDA) methods aim to **mitigate the issue by directly modifying the data, effectively aligning data patterns with model designs**. For example, Half-Hop [3] adds nodes on each edge to improve GCN's performance on node classification tasks for heterophilic graphs, achieving substantial gains. CFLP [4] generates counterfactual links to enhance link prediction. However, while GDA methods offer performance improvements, they often rely on handcrafted augmentation strategies tailored for specific tasks or datasets, limiting their generalizability to broader applications.\n\nUniAug address this issue by leveraging a diffusion model. Intuitively, UniAug learns diverse data patterns by pre-training on graphs across domains, enabling it to adaptively augment downstream graphs with task-specific and data-specific guidance. This allows UniAug to \"automatically\" align data patterns with the downstream GNN architecture, resulting in empirical success across a wide range of tasks and datasets.\n\n> Q4. My main question is how much total time is spent on training UniAug and applying it to a specific graph task.\n\nWe thank the reviewer for raising the concern regarding the computation cost. Here, we provide a comparison with the best self-supervised baseline, **GraphMAE**. In our experiment, we have two settings for **GraphMAE**: (1) pre-training on the same set of graphs and fine-tuning on downstream graphs, and (2) self-supervised learning to obtain embeddings and train a classifier. \n\nWe first compare the pre-training time of **GraphMAE** and **UniAug** in the following table, where we see that the pre-training time of **UniAug** is longer than that of **GraphMAE**. As emphasized in **Tables 2, 16, and 19**, the pre-training version of **GraphMAE** leads to **negative transfer**, while **UniAug** consistently achieves **positive transfer**.\n\n|  | Total time (minutes) |\n| --- | --- |\n| GraphMAE | 125 |\n| UniAug | 378 |\n\nNext, we perform the comparison under a self-supervised setting. The downstream application pipeline of UniAug consists of: (1) training the guidance head; (2) performing guided generation; and (3) training the downstream GCN. The self-supervised pipeline for GraphMAE includes: (1) self-supervised training; and (2) training a classifier on embeddings. For GraphMAE, we use the tuned hyperparameters provided by the authors. We report the total time of the full pipeline on a single A6000 GPU in the following table.\n\n| Total time (seconds) | NCI1 | PROTEINS | IMDB-B | IMDB-M |\n| --- | --- | --- | --- | --- |\n| GraphMAE | 1273.61 | 79.24 | 51.21 | 110.11 |\n| UniAug | 378.44 | 90.15 | 45.56 | 85.80 |\n\nWe observe that in most datasets, the total time of **GraphMAE** and **UniAug** are comparable. Note that **GraphMAE** performs self-supervised training with **300 epochs** for **NCI1** and **100 epochs** for **Reddit-B**, which is relatively time-consuming. \n\nThere are several reasons why the pipeline of **UniAug** is actually efficient: \n1. When training the guidance head, we freeze all other model parameters and **only update the weights for the MLP-based head**.\n2. The number of diffusion steps for **UniAug** is set to **128**, which is much smaller than the standard **1000 steps** for continuous diffusion models. It only takes **21 seconds** to sample **1000 graphs** for the IMDB-B dataset.\n3. We choose the diffusion kernel to be the **absorbing kernel**, which significantly reduces memory consumption during sampling.\n\nIn addition, we argue that the improvements provided by UniAug are **not marginal**. In the downstream application, UniAug achieves **consistent positive transfer** across tasks and datasets. As mentioned in Tables 4, 5, and 12, UniAug consistently presents the **highest average rank**. It is worth noting that in Table 4, the average rank of UniAug is 1.43, which is **much higher than the second-best method**, BGRL, with an average rank of 3.00. Similarly, in Table 5, the average rank of UniAug + Half-Hop is 1.43, **significantly surpassing the best self-supervised method**, BGRL, which has an average rank of 5.33.\n\n[1] Cao, Yuxuan, et al. \"When to pre-train graph neural networks? from data generation perspective!.” KDD, 2023\n\n[2] Mao, Haitao, et al. \"Demystifying structural disparity in graph neural networks: Can one size fit all?.\" NeurIPS, 2024\n\n[3] Azabou, Mehdi, et al. \"Half-Hop: A graph upsampling approach for slowing down message passing.\" ICML, 2023.\n\n[4] Zhao, Tong, et al. \"Learning from counterfactual links for link prediction.\" ICML, 2022.\n\n---\n\nWe understand your concerns and would like to emphasize the significant contributions of our work, which address key challenges for cross-domain positive transfer. We kindly ask you to reconsider your score, as we believe our findings offer valuable insights."}, {"review_id": "ayVJKCI35h", "reviewer": "Reviewer_L4ew", "summary": "The authors propose UniAug—a universal graph structure augmenter based on a discrete diffusion model. It enables cross-domain pretraining on large-scale graph data and can be seamlessly plugged into downstream tasks to enhance performance on node-level, edge-level, and graph-level tasks through structural augmentation. The results demonstrate that graph data can also benefit from the \"big data + big model\" scaling effect, similar to what has been observed in text and image domains.", "questions": "1.What were the thresholds and rules for filtering out “abnormal graphs”? Are the dataset licenses/publications compatible with large-scale public pretraining use?\n\n2. Which graph statistics were used for clustering? How were hyperparameters (e.g., number of clusters K, distance metrics) chosen?Where are the experiments evaluating model sensitivity to pseudo-label noise? Can the authors report performance curves under varying noise levels?\n\n3. In domains heavily reliant on node/edge attributes (e.g., molecular bonds, knowledge graph entity types), how do the authors ensure that modeling only structure does not omit critical information?\n\n4. What are the number of layers, hidden dimensions, and total parameter count of the diffusion network? How does this compare to mainstream “foundation models for graphs”? Did the authors attempt to scale up the model or data size? If limited by compute, can they report scaling law results across small, medium, and large setups?\n\n5. On average, how many diffusion steps and how many seconds are required to generate one augmented graph? How does this compare to existing augmenters like GDA or EDGE in terms of speed? Has the feasibility of one-step or few-step sampling methods (e.g., consistency distillation) been explored?\n\n6. How much labeled data is required to train a Guidance Head per task? What is the performance under low-label (<10%) or zero-shot settings? Has using a shared Guidance Head across multiple tasks been tested?\n\n7. Why are recent generative models (e.g., GraphFM, GraphGPT) and attribute-aligned pretraining methods (e.g., GraphMVP, GraphMAE) not directly compared? For some tasks, only top-k accuracy is reported while ROC-AUC or MAE is omitted. Does this omission affect the generality of the conclusions?", "limitations": "Please see above", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. First cross-domain graph augmentation paradigm leveraging data scaling: Demonstrates that large-scale heterogeneous graphs can be unified for pretraining and effectively transferred across domains.\n\n2. Structure-feature decoupling: Focuses solely on modeling graph structure, addressing the challenges of inconsistent feature dimensions or semantics across different graphs.\n\n3. Self-conditioned discrete diffusion with guided generation: Ensures efficient and stable learning on sparse graphs; the guided head aligns generation with downstream tasks, mitigating noise.\n\n4. Plug-and-play compatibility: Requires no modification to downstream GNNs and can be seamlessly integrated with existing Graph Data Augmentation (GDA) methods.\n\nWeaknesses\n1. Due to computational constraints, the authors trained only a moderately sized diffusion model and did not systematically investigate how scaling model parameters or depth would affect performance.\n\n2. The discrete diffusion process involves many sampling steps, resulting in slow generation; the authors list “introducing fast sampling” as future work.\n\n3. UniAug does not model node/edge features at all, instead relying on “directly concatenating the original features back.” This may lead to information loss or require extra processing in tasks that depend on attributes like chemical bonds or textual properties.\n\n4. During pretraining, pseudo-labels are generated via clustering based on graph statistics. However, the authors did not conduct ablation studies on the quality of these pseudo-labels or the model’s robustness to label noise.\n\n5. The authors manually filtered out “abnormal graphs” using thresholding, yet regions of “sparse distribution” still remain (as shown in Fig. 2).\n\n6. Although described as “plug-and-play,” each task still requires training a Guidance MLP Head, and the experiments make use of supervision signals (e.g., node/graph labels, CN-based heuristics).", "was_revised": true, "final_justification": "I will keep my score", "rebuttal": "> Q1. What were the thresholds and rules for filtering out “abnormal graphs”? Are the dataset licenses/publications compatible with large-scale public pretraining use?\n\nWe thank the reviewer for these important questions. To ensure the quality of our pre-training corpus, we applied a set of rules to programmatically filter out graphs with pathological or outlier structures. The thresholds we used are:\n\n- **Degree Mean:** ≤3\n- **Degree Variance:** ≤3\n- **Graph Density (for graphs with ≥100 nodes):** ≤0.1\n- **Entropy:** ≤10\n\nThese values were chosen based on an empirical analysis of the full data collection to remove clear outliers (e.g., star graphs, overly dense or sparse graphs) while retaining a diverse set of typical real-world networks.\n\nWe have meticulously verified the license for each data source and confirm that **all datasets are compatible** with large-scale public pre-training use. \n\n> Q2. Which graph statistics were used for clustering? How were hyperparameters (e.g., number of clusters K, distance metrics) chosen?Where are the experiments evaluating model sensitivity to pseudo-label noise? Can the authors report performance curves under varying noise levels?\n\nWe thank the reviewer for these insightful questions about our methodology. We provide the requested details below\n\n- **Graph Statistics Used:** As detailed in Section 3.2, we generated graph-level representations using six standard graph properties: number of nodes, density, network entropy, average degree, degree variance, and scale-free exponent.\n- **Hyperparameter Selection:** We employ the **K-Means** clustering algorithm on the graph statistics. The number of clusters (K) was chosen via a principled, two-step process. First, we identified candidate values for K that yielded good cluster separation. Then, from these candidates, we selected the final value of **K=10** by maximizing the mean Silhouette Coefficient, ensuring the most stable and meaningful grouping in the data.\n\nRegarding the “label noise”, unlike in semi-supervised learning where pseudo-labels are noisy estimates of ground-truth classes, their purpose here is to provide a **high-level structural conditioning signal**. They group graphs with similar topological properties to encourage our diffusion model to learn a structured latent space. The goal is not to predict a \"correct\" cluster ID, but to leverage this grouping to **enhance the model's ability to capture diverse structural patterns**, a strategy also noted in [1].\n\n> Q3. In domains heavily reliant on node/edge attributes (e.g., molecular bonds, knowledge graph entity types), how do the authors ensure that modeling only structure does not omit critical information?\n\nWe thank you for this critical question, as it allows us to clarify a key detail of our method and highlight one of its surprising strengths. We **retains original node attributes** for downstream tasks but **does not use edge attributes**.\n\nWe recognize that edge attributes are vital in domains like molecular chemistry. However, our results in **Table 3** show something remarkable: **even on the edge-critical molecular regression task, UniAug outperforms other general-purpose baselines** and achieves performance highly competitive with DCT—a model specifically pre-trained on molecules *with* their rich edge features. This powerful result demonstrates that our structure-focused pre-training is effective that it can learn to compensate for the absence of explicit edge features, likely by capturing their influence through topology alone. \n\n> Q4. What are the number of layers, hidden dimensions, and total parameter count of the diffusion network? How does this compare to mainstream “foundation models for graphs”? Did the authors attempt to scale up the model or data size? If limited by compute, can they report scaling law results across small, medium, and large setups?\n\nWe thank the reviewer for these important questions about model scale and its comparison to other large graph models.\n\n- **Model Size:** Our diffusion network uses **4 layers**, a **hidden dimension of 128**, and has approximately **2 million parameters**.\n- **Comparison:** The field of \"foundation models for graphs\" is still emerging, with few established benchmarks for model size. Comparing to recent works, our model's parameter count is comparable to that of OFA [2], and significantly larger than Graphany [3].\n\nWhile a full scaling analysis for larger models was beyond our available computing resources, the layer-wise study above serves as a preliminary result. As shown in the table below, we evaluated generation quality for varying numbers of layers. Performance on key structural metrics peaked at **4 layers** and then began to degrade. This indicates a \"sweet spot\" for model capacity on this task, suggesting larger models may be prone to overfitting or optimization difficulties.\n\n| Num Layers | Degree | Spectral | Clustering |\n|-|-|-|-|\n|2| 0.3851 | 0.4865 |1.25|\n|3| 0.3344 | 0.4659 | 1.15|\n|4| **0.2648** | **0.4592** | **1.11** |\n|5|0.3378|0.4917|1.19|\n\n> Q5. On average, how many diffusion steps and how many seconds are required to generate one augmented graph? How does this compare to existing augmenters like GDA or EDGE in terms of speed? Has the feasibility of one-step or few-step sampling methods (e.g., consistency distillation) been explored?\n\nWe thank the reviewer for these excellent questions about the efficiency of our augmentation process. Our standard process to generate one augmented graph uses **128 diffusion steps**. On an NVIDIA A6000 GPU, this takes approximately **0.2 seconds per graph**. In terms of speed, this is **directly comparable to EDGE**. As expected, this is slower than simple, non-diffusion methods, but our method provides much richer and more tailored structural augmentations.\n\nAdapting complex techniques like consistency distillation, which are primarily designed for continuous data, to our discrete graph diffusion framework is a significant research challenge in its own right. However, we have explored a simpler approach to acceleration. Our preliminary experiments show that reducing the number of sampling steps to just **32 (a 4x speedup)** still yields downstream performance that is **highly competitive and comparable** to using the full 128 steps. \n\n> Q6. How much labeled data is required to train a Guidance Head per task? What is the performance under low-label (<10%) or zero-shot settings? Has using a shared Guidance Head across multiple tasks been tested?\n\nWe thank the reviewer for these excellent questions about the data requirements and functionality of the guidance Head. The Guidance Head is highly flexible and **does not strictly require labeled data**. Its implementation depends on the downstream setting:\n\n- **Supervised Setting:** When task labels are available, the Guidance Head can be a simple, lightweight predictor trained on that data.\n- **Low-Label / Zero-Shot Setting:** A key strength of our framework is its performance in label-scarce scenarios. In these cases, the Guidance Head can be implemented as a simple, **unsupervised heuristic**. As we demonstrate in our appendix (Table 18), using an established heuristic like Common Neighbors for link prediction—which requires **zero labels**—yields excellent results. This highlights our method's effectiveness in zero-shot contexts.\n\nRegarding sharing a Guidance Head across tasks, we clarify that this would contradict the core design of our framework. The fundamental purpose of the Guidance Head is to be **dataset-specific**. It's a specialized, lightweight module that steers the universal, pre-trained diffusion model to generate augmentations tailored to the distribution of the targeting downstream graphs.\n\nSharing one head across different tasks would force it to learn a single, compromised guidance signal for distinct distributions. This would prevent our model from producing the specialized, high-quality augmentations that are key to its performance. The power of our approach lies in combining the **generalization of the core diffusion model** with the **specialization of the guidance Head**.\n\n> Q7. Why are recent generative models (e.g., GraphFM, GraphGPT) and attribute-aligned pretraining methods (e.g., GraphMVP, GraphMAE) not directly compared? For some tasks, only top-k accuracy is reported while ROC-AUC or MAE is omitted. Does this omission affect the generality of the conclusions?\n\nWe thank the reviewer for these questions about our experimental setup. We aimed to provide a comprehensive comparison against relevant and established baselines.\n\n- **Recent Generative Models:**\n    - **GraphFM** was not included as it is a concurrent work that was not publicly available or formally published at the time of our experiments.\n    - **GraphGPT** represents a different research paradigm focused on instruction-tuning large language models (LLMs) for graph-related tasks. A direct comparison is not meaningful due to the fundamentally different architectures, training methods, and objectives.\n- **Attribute-Aligned Pre-training:** We want to clarify that we **do compare extensively** against state-of-the-art attribute-aligned methods. As shown throughout our results, our baselines include prominent models like **GraphMAE, BGRL, JOAO, and D-SLA** (Tables 2, 3, 4, 5).\n\nRegarding evaluation metrics, we strictly adhere to the established benchmark to ensure a fair and direct comparison with prior work. The settings are clearly described in the experiment section.\n\n[1] Gao, Shanghua, et al. \"Large-scale unsupervised semantic segmentation.\"\n\n[2] Liu, Hao, et al. \"One for all: Towards training one graph model for all classification tasks.”\n\n[3] Zhao, Jianan, et al. \"Graphany: A foundation model for node classification on any graph.”\n\n---\n\nWe appreciate your feedback and would like to clarify any misunderstandings. We would sincerely appreciate your reconsideration of the score based on these contributions."}]

Author rebuttal:

Meta-review: The paper proposes UniAug, a universal graph structure augmentation framework that pretrains a discrete diffusion model to capture structural patterns from diverse graphs. The motivation of the paper is reasonable. The datasets, in experiments, are comprehensive, span diverse domains, and the improvements are largely consistent across all data.

Overall, the research problem is important and the approaching is novel and interesting. I vote for acceptance.
Decision: Accept (poster)