SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Authors: Junyou Zhu
Langzhou He
Chao Gao
Dongpeng Hou
Zhen Su
Philip S. Yu
Juergen Kurths
Frank Hellmann
Venue: ICML 2025 poster (2025)
Keywords: Graph Representation Learning
Graph Diffusion Model
Diffusion Probabilistic Models
Graph Self-supervised Learning
Primary area: general_machine_learning->representation_learning
TL;DR: SDMG bridges the gap between generative and representation learning by aligning low-frequency reconstruction, enhancing performance across graph tasks.

Abstract: Diffusion probabilistic models (DPMs) have recently demonstrated impressive generative capabilities. There is emerging evidence that their sample reconstruction ability can yield meaningful representations for recognition tasks. In this paper, we demonstrate that the objectives underlying generation and representation learning are not perfectly aligned. Through a spectral analysis, we find that minimizing the mean squared error (MSE) between the original graph and its reconstructed counterpart does not necessarily optimize representations for downstream tasks. Instead, focusing on reconstructing a small subset of features, specifically those capturing global information, proves to be more effective for learning powerful representations. Motivated by these insights, we propose a novel framework, the Smooth Diffusion Model for Graphs (SDMG), which introduces a multi-scale smoothing loss and low-frequency information encoders to promote the recovery of global, low-frequency details, while suppressing irrelevant high-frequency noise. Extensive experiments validate the effectiveness of our method, suggesting a promising direction for advancing diffusion models in graph representation learning.

SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Junyou Zhu 12
Langzhou He²
Chao Gao
Dongpeng Hou
Zhen Su
Philip S. Yu
Jürgen Kurths'
Frank Hellmann'
Abstract
0.9
0.9
Aceuracy
Accuracy
0.7
0.7
0.9
5%
25%
10%
30%
0.5
0.5
13%
50%
Diffusion probabilistic models (DPMs) have cently demonstrated impressive generative capa- bilities. There is emerging evidence that their sample reconstruction ability can yield meaning- ful representations for recognition tasks. this paper, we demonstrate that the objectives under- lying generation and representation learning are not perfectly aligned. Through spectral anal- ysis, find that minimizing the mean squared error (MSE) between the original graph and its reconstructed counterpart does not necessarily op- timize representations for downstream tasks. In- stead, focusing on reconstructing small subset of features, specifically those capturing global infor- mation, proves be more effective for learning powerful representations. Motivated by these in- sights, we propose novel framework, the Smooth Diffusion Model for Graphs (SDMG), which in- troduces a multi-scale smoothing loss and low- frequency information encoders to promote the recovery global, cow-frequency details, while suppressing irrelevant high-frequency noise. Ex- tensive experiments validate the effectiveness of our method, suggesting promising direction for advancing diffusion models in graph representa- tion learning.
0.8
100%
0.3
0.3
1000
2000
3000
1000
2000
3000
Epochs
Epochs
(a) No Noise
(b) With Noise
Figure 1. Impact of reconstructing various proportions of low- frequency components on node classification for Photo. (a) sort the frequency components (derived from the graph Laplacian) in ascending order and reconstruct only the lowest fraction (e.g.. 5%) of the frequency components. This partial reconstruction already achieves high accuracy, while restoring the full spectrum (100%) overfits higu-erryeency noise. (b) Under Gaussian noise, focusing on narrow low-frequency band remains robust, whereas full-spectrum reconstruction collapses in performance.
1. Introduction
Self-supervised learning (SSL) is transforming representa- tion learning by reducing our reliance on massive labeled datasets (Yu et al., 2024b; Xiao et al., 2024; Kalantidis et al., 2020; Zhu et al., 2022). Among its many variants, gen- erative diffusion probabilistic models (DPMs) (Ho et al., 2020; Song et al., 2021) have emerged as particularly pow- erful: by reconstructing clean signals from corrupted inputs, these models capture rich semantic insights. Recent vision research shows that stronger diffusion models yield more expressive embeddings (Hudson al., 2024; Wang al., 2023; Wei al., 2023). This has also spurred adoption in more general graph domains. A pioneering example, the DDM (Yang et al., 2024), deploys directional perturbations to denoise node features, revealing how diffusion-basse techniques can elegantly handle complex, anisotropic graph structures.
'Department of Complexity Science, Potsdam Institute for Climate Impact Research, 14473 Potsdam, Germany Machine Learning Group, Technical University Berlin, 10587 Berlin, Germany 'Department of Computer Science, University of Illi- nois at Chicago, Chicago, IL 60607, USA "School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Poly- technical University, an 710072, China. Correspondence to: Chao Gao srgaoonwwu.cduu.n>..
Despite the growing popularity of diffusion-based graph representation methods, existing approaches (Yang et al., 2024) adopt off-the-shelf DPMs tailored for generation (Niu et al., 2020), leaving open the question of whether the gen- eration goal to fully reconstruct every detail of a graph truly enhances the discriminative power of representations. How- ever, our empirical findings (Figure 1(a)) suggest: focusing
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
on reconstructing only a narrow band of low-frequency information (e-g., 5%) already yield strong embed- dings, while attempting to fully restore all features (i.e., 100%) often proves unnecessary- or, even worse, detrimen- tal. We push this point further in Figure ((b) by adding task-irrelevant Gaussian noise to node features; finding that reconstructing only low-frequency components preserves competitive accuracy even if the data is considerably perturbed, whereas insisting on reconstructing everything causes performance to drop significantly. These observa- tions highlight misalignment between generation-oriented objectives and representation learning. This raises fun- damental tension: should we aim to faithfully recover all graph details, or strategically suppress unnecessary details to better emphasize semantically meaningful signals for downstream tasks?
focus on semantically meaningful structures aligned with downstream recognition tasks. Our key contributions in- clude:
We theoretically and empirically uncover that purely generation-oriented objectives can conflict with recog- nition goals, revealing how excessive high-frequent reconstruction reduces the representation quality.
We propose the Smooth Diffusion Model for Graphs (SDMG), an approach that explicitly leverages low- frequency filters and a novel multí-scale smoothing (MSS) loss to align pre-training reconstruction with downstream recognition. To the best of our knowledge, we are the first to address this misalignment between graph generation and representation.
Extensive node- and graph-level experiments show that SDMG achieves state-of-the-ar performance, pointing to a promising direction for diffusion- based graph SSL.
To investigate this, we begin with an empirical study (Sec- tion 3) by analyzing how different frequency components affect classification performance. Our analysis reveals two main insights: (1) aggressively minimizing the MSE- loss, particularly for high- frequency details, can undermine downstream accuracy, and (2) focusing the model on a narrow range of low-frequency signals results in stronger representations and faster con- vergence. In fact, reconstructing additional frgh-frey structures introduces distracting noise for downstream tasks, even though it is necessary and beneficial for generation. This aligns with established GNN research emphasizing the predictive power of low- wwwffreuencyy features (Hoang et al., 2021; Liu et al., 2022). Although one might consider in- corporating GNNs as low-pass filters into the denoising encoder, we find that high-frequency components are in- evitably re-learned and encoded as we continue to minimize the MSE-based reconstruction objective (demonstrated in Figure 4 of Section 3). This issue remains challenging, espe- cially since standard constraints designed for filtering noise frequency components, like information bottlenecks (Yu et al., 2024a), rely labeled supervision, which is not readily available in our SSL setting.
2. Preliminaries
based generation-reconstuucee
2.1. Notations
Consider graph G = (V,E) with D N and the adja- cency matrix A E RNNN where Aij E {0, indicates the presence of an edge between nodes and j. and N de- notes the number of nodes. Let D = )ddgg)) denote the degree matrix, where deg. = EEEE Aij. For convenience, we define the augmented adjacency matrix A = A + and its degree matrix D = D + I. With these. we construct the normalized adjacency matrix Anum The symmetric normalized Laplacian ma- trix is L. I A Since É is symmetric normal- ized, its cigen-decomposition is UAUT, where A = diag(S1..... AN) and U = (u, :.... uNll E RNN are the eigenvalues and eigenvectors of L, respectively. Each node Vi E V is associated with a feature vector x; E IRd and we stack all node features into X = [ [2..22............. R xd Then, the graph Fourier transform based on L is represented by D and the inverse transform is x= D I/PUX.
degn)
D-1/2 DD -1/2,
E
Motivated by these findings, we propose Smooth Diffusion Model for Graphs (SDMG), a novel self-supervised frame- work designed to learn recognition-oriented representa- tions without labels. SDMG employs two dedicated low- frequency encoders, one for node features and another for topology, to distill global, low-frequency signals. To avoid reintroducing unhelpful high-frequency details, we propose a new multi-scale smoothing objective. Rather than strictly enforcing a pointwise reconstruction, our objec- tive aligns the original and reconstructed graphs at multiple scales, theoretically encouraging the model to emphasize low-frequency signals while suppressing tigeunryy noise. This strategy effectively "filters out" distracting de- tails, allowing the diffusiion-aasee representation learner to
2.2. Diffusson Models
DPMs define a forward-noising process and a corresponding reverse-time denoising process to generate samples from complex distributions (Ho et al., 2020; Song et al., 2021; Cho et al., 2024). Formally, consider continuous-time stochastic process (xx)T Rdd. The forward process gradually perturbs an initial data point X by noise via:
dx: = f(t)x; dt + g(t) dw.,
(I)
where w, is a standard Wiener process, and f()), (t) dic- tate the drift and diffusion terms, respectively. As t + T.
2

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
XT converges to Gaussian distribution independent of the initial input.
3. An Investigation into the Effects of Smooth Global Signal Reconstruction
By reversing time, one obtains reverse SDE:
We begin by empirically and theoretically examining how the reconstruction of different frequency components in graph signals impacts the performance on downstream tasks (e.g.. node classification). Our study uncovers two findings:
- g²t px dt - g(t) dwo. (2) dx; = -
where W. is reverse-time Wiener process, and Vx Po is the score function. Approximating this score enables simulating the reverse dynamics to recover clean samples from noise.
Reconstructing Low-Frequeney Components Suffices for High-Qualis Representations. We investi- gate which frequency components most benefit representa- tion learning by selectively reconstructing only the lower end of the spectrum (see Figure 2). We first express the node-feature matrix X in the spectral domain based on the graph Fourier transform as
Finding I:
In practice, a neural network x (x,,1) () with parameters 1/ approximates the score or directly predicts Xu- com- mon objective is weighted mean-squared error between (x,, t) and Xo, known as the denoising score matching (DSM) loss:
X = DU/2X RRd
(5)
COSM = E. {A(1) Ex,1xo (x)
x (3)
where U and D are defined in Section 2.1. To retain only the q smallest eigenvalues (i.e., the lowest-frequency compo- nents), we conduct low-frequency truncation. Specifically, U(4) == [ui,..., Ug R denote the sub-matrix that collects the first g eigenvectors (i.e. the eigenmodes with the smallest eigenvalues). Projecting onto this sub- space yields the retained coefficients
While DPMs have achieved remarkable success in Euclidean domains, their application to graph representation learning is still emerging. DDM (Yang et al., 2024) is the first work to integrate diffusion into node feature spaces and employ a denoising decoder (a U-Net architecture) to recover Xo from X₁. The latent representations H are then extracted from intermediate layers of this decoder:
( = U D E Rexd
(6)
H = hww (xx),
(4)
while all then map X()) back to the spatial domain via
rows q - 1:N are discarded. We
where hn denotes the encoder portion of the denoising net- work chosen intermediate layer. However, we argue that strictly generative objectives in DDM may not align with discriminative goals, prompting us to rethink the training strategy for improved representation learning.
Now D-1/2 U (4) X (g),
thus obtaining a feature matrix X that captures only the globally smooth, low- low-frequency signal. Varying y smoothly controls the amount of high-frequency detail that is re- moved.
Fiequreyyrry
High-Frequercy
With X in hand, we feed these low-frequency node fea- tures into a vanilla diffusion- based denoising model without using any low-frequency encoders (e.g., GNNs) but using MLPs and extracting intermediate-aate layer representations for linear probing in node classification.
Xiow eg. 20%
MSE
Denoise Unot
Linear Probing Classification
Loss
Figure 3 shows the results on two real-world datasets. the accuracy rises sharply when the model reconstructs roughly the lowest 20 of the entire frequency spectrum (orange re- gion). Once components beyond this cowest-frequenc band are added (green region), performance levels off or declines. In other words, recovering only the bottom fifth of the spectrum already captures most of the discriminative signal, whereas reconstructing additional higher-frequeng components brings little benefit and can be detrimental.
Now
Roconstuction of Low www.uruuaycy Part
Figure 2. A case study model illustrates how only the lowest pro- portions of the firequecc components (green boxes) that are ex- tracted from node features or topology are reconstructed.
Finding 2: Overemphasizing MSE Minimization Rein- troduces Irrelevant Details. Simply pushing the MSE
3

---

Smoothing Your Diffusson Models for Powerful Graph Representation Learning
0.8
0.9
Y is bounded by
Accuracy
Accuracy
0.7
0.6
(Z.Y) -
0.5
0.4
0.3
Therefore, optimizing the encoding Z in order minimize mams( (Z) is partially misaligned with optimizing the encoding for (Z; Y). The former benefits from an increase in encoded superfluous information, the latter is penalized by it.
0%
25%
100%
0%
25%
50%
75%
100%
Fraction of X Reconstructed
Fraction X Reconstructed
(a) Cora
(b) Photo
Figure 3. Node-classitee claessssssiiitoonn accuracy when we only rebuild the low-freen fww-rreuency part of the feature matrix X. We keep the first g Laplacian eigenmodes, reconstruct X from these components, and plot the ratio q/N on the z-axis (N = total number modes, so q/N = 1 means full-spectrum reconstruction). Accuracy peaks for small (orange band) and falls as higher-frequeng components are progressively added (green band). "X" denotes node-feature reconstruction.
The proof is in Appendix A.I. Theorem 3.1 indicates that as the portion of the encoder's capacity allocated to irrelevant information increases, the model's ability to encode label- relevant information diminishes.
Empirically, Figure 4 compares three diffusion variants on the Photo dataset: (i) Vanilla, trained purely with MSE; (i) SDMG-MSE, which adds low-frequency encoders but still minimizes MSE; (iii) SDMG, our method which em- ploys low-frequency encoders with our multi-scale smooth- ing loss. Panel (a) reports classification accuracy, while panel (b) tracks low-frequency spectral energy. Three obser- vations stand out:
as low as possible can unintentionally force the model to encode high- high-frequency or noisy features that do not help, and can even harm, downstream classification. Below, we first provide theoretical evidence and then empirical results to support this point.
Observation 1: Vanilla overfits. Without low-frequency filter, the model struggles to capture only low-frequency components, resulting in a substantial performance gap. Moreover, the MSE objective eventually overfits high- frequency details, causing an accuracy drop after around 100 epochs.
0.9
0.9
Spectral Energy
Ageuracy
0.7
0.7
A.00
0.5
0.5
0.94
SONSO
SUASS-MSE
0.3
0.53
250
500
25h
0.3
Varlla
-5DMG
SDDG MSE
250
750
200
400
600
Observation 2: Low-frequency encoders boost early gains. By introducing low-frequency GNN encoders, SDMG-MSE and SDMG quickly captures global signals, achieving nearly 88% accuracy by epoch 0, far surpassing the baseline initialization.
Epochs
Epochs
(a) Classification Performance
(b) Spectral Energy of components
Figure 4. Performance and spectral energy for three model variants on Photo datasets. (a) Node classification accuracy for Vanilla diffusion, SDMG (with low-frequency encoders and MSS loss), and SDMG-MSE (with low-frequency encoders but MSE loss). (b) The spectral energy of the lowest 40% frequency components, where higher values indicate better low- wwwfrequency (I.F) capture.
Pure MSE reintroduces noise. Despite the initial gains, SDMG-MSE eventually exhibits the downward trend of Vanilla in later epochs. Because MSE penalizes errors in all frequency bands equally, the model eventu- ally reconstructs high-frequency details, causing drop in classification accuracy. This is further demonstrated by Fig- ure 4(b), which shows that SDMG-MSE converges toward the same (reduced) low-frequency energy level Vanilla, indicating an increased emphasis on reconstructing unhelp- ful high- frequency signals.
Observation 3:
Theorem 3.1. Consider an encoding Z of X with a bounded capacity Z( S C, where T(::-) denotes mutual infor- mation. Assume that X. takes values in the unit ball of an Euclidean space. Let Y be target variable to be pre- dicted from X, and assume X can be decomposed as X = (X,,X, X,) such that X, is relevant to Y and X, is super- fluous to predicting Y given X,, that is T(X.; Y = O.
3.1. Broader Implications and Insights
Then we have:
Although high-frequenc, details may occasionally help (Fig- ure 3(b),, results show that low-frequency, global signals are more crucial for graph tasks. Prioritizing these smooth components over complete reconstruction is more effective under limited capacity (Section 4). Future work could ex- plore frequency-adaptive objectives to focus on relevant spectral bands for downstream tasks.
The minimal mean square error (mmse) for predicting X
from Z is bounded
mmsee X Z) 2
(X)())
(12:X.)++1(2 X.)
The information that the encoding Z carries on the variable

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
X
C
%
NY
(x (X,)
Pointwise MSE Loss
Multi-scale Smooth Loss
Fleg Encoders
Skip Connection
N
Faward Process
MLP
Denoise U-Net
Denoise U-Net
NNY
NN
MLP
X
X
(a) Forward Process
(b) Vanilla
(c) SDMG
(d) Denoise U-Net Decoder
Figure 5. Overview of the SDMG framework. (a) Forward diffusion: Gaussian noise is incrementally added to the original sample Xo obtain the comupted version X.. (b) Standard diffuusinn model: X, is input into denoise U-Net decoder, which reconstructs using an MSE loss, with activations from specific U-Net layers serving as representations H. (c) SDMG enhancements: Low-frequence encoders (Eg and Eg) extract information from node features and graph topology as conditional imputs for the U-Net (Section 4.2), and a new multi-scale smoothing loss prioritizes the reconstruction of important low-frequency information (Section 4.3). (d) Customized conditional U-Net architecture designed for diffusion-based graph representation learning (Section 4.3 and Appendix D).
4. Method: Smooth Diffusion Model for Graph Representation Learning
4.1. Reconstruction Objective
Graph data consists of node features and topology, each providing distinet perspectives for reconstruction. In self- supervised learning, need decide which part of this signal to reconstruct. Note that in most real-world graphs the adjacency matrix is extremely sparse. The number of present edges grows linearly with |V], whereas the number of absent edges grows quadratically. This produces a severe class imbalance if every entry of is treated as a regres- sion target. Node features, on the other hand, are typically dense and directly related downstream prediction tasks. We therefore formulate the denoising objective solely with respect to the clean feature matrix X:
Current diffusion diffusion model-based graph representation learning faces the challenge of misalignment between generative and recognition objectives. As shown in Section 3, focusing wwwffreuencyy components of node features and adjacency matrices significantly alleviates this misalignment, thereby improving downstream performance. However, computing the low-frequency embeddings via graph Fourier transforms (Equations (6) and (7)) is still computationally expensive, which makes it impractical for large-scale graphs.
To address this, we propose learnable encoders that effi- ciently approximate low-frequency embeddings. In Sec- tion 4.2, we introduce these encoders to extract global low-frequency information, which enhances an early perfor- mance. However, despite the presence of these encoders, we observe that a pure MSE-bbase reconstruction objective eventually reintroduces high-freque frequeeey noise (evidences in Figure 4).
Lyec = Ex [A(1) [x,, 4,C, C) - x (10) where x, is the noised input at time i, X the denoiser and C a conditioning term described next.
Although do regress the full adjacency, struc- tural patterns are indispensable for high-quality represen- tations (HaoChen et al., 2021). Instead of predicting every edge entry, we distil the global, low-frequency information of the graph Laplacian and feed to the denoiser as context:
To mitigate this, Section 4.3 proposes a new multi-scale loss that emphasizes low-frequency reconstruction, better align- ing generation with recognition tasks. Additionally, Sec- tion 4.1 details the reconstruction objective, and Section 4.3 introduces denoising decoder. The SDMG framework is illustrated in Figure 5.
C
topology LF
feature
where Eg¹ and EX are encoders for graph topology and node features, respectively, which will be detailed in Seciton 4.2.
5

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
This design lets the model exploit long-range structural regularities without being overwhelmed by the highly imbal- anced edge-level target. By defining C' as concatenation function, our model incorporates the low-frequency compo- nents of both node features and adjacency matrices as con- ditioning information during reconstruction, as described in Section 4.2.
Although Equation (12) filters high-frequency information early in training. the MSE loss still guides the model to- ward exact element-wise matching, which reintroduces high- frequency details (Figure 4). This motivates to introduce a new learning objective to address this issue, which will be discussed later in Equations (15) and (16).
4.3. Denoise Decoder and the MSS Learning Objective
4.2. Low- Low-Frequency Component Encoders
Denoise Decoder. In standard diffusion models, a denoise decoder reconstructs a predicted sample x. from a noisy version x by minimizing the MSE between Xo and the clean sample Xg. While we also use a denoise decoder for reconstructing clean data, our approach introduces new objective that treats different frequency components, particularly low-frequency structural signals, with varying priorities.
Graph Topology Low-frequency Information Encoder Eg. We aim to approximate the first q eigenvectors of the normalized Laplacian É using a neural network func- tion &g. Specifically, the encoder Egg, implemented as multi-layer perceptron, takes node-level structural features (e.g. random-walk positional encodings derived from the adjacency matrix (Rampášek et al., 2022)) and outputs an embedding matrix A E Nxq, The key insight is that by minimizing an appropriate loss, we can recover the low- frequency eigenvectors (z)) The optimization objective for the encoder is then:
To implement this, we adopt a U-Net architecture X (Li et al., 2024; Dhariwal & Nichol, 2021), where the recon- structed output is given by: X x (x,, t, C(E Eg) As shown in Figure 5(d), the U-Net consists of multiple residual blocks and downsampling layers, followed by mirrored structure of residual blocks with upsampling lay- ers. We denote the upsampling portion of the network as NN = {NNTP, .N ,.....N?? NN,"). The outputs from these upsampling layers are collected as:
min C(A): = ( A-ATA AER X9
AAT |
(11)
Based on the classical low-rank approximation results (Eckart- Youuart-Yuug---e Moougy theorem (Eckart & Young, 1936)), this objective ensures that A approximates the smallest q eigenvectors of the graph Laplacian up a scaling factor.
H (H H,. HJ),
(13)
and concatenated to form the final representation:
Node Feature Low-frequency Information Encoder Eg. In contrast to adjacency matrices, feature matrices are typi- cally much smaller in large networks, making it computa- tionally feasible to extract low low-frequency information from node features using standard graph neural networks (GNNs). Previous work (Dwivedi et al., 2023) suggests that the it- erative application of the normalized adjacency matrix can serve as low-pass filter. Inspired by this, we use Graph Attention Network (GAT) (Veličković et al., 2018) to model the low-frequency components of node features, represented by X = Eg (G.X).
H Concatenation/ (H, (=)))
(14)
This multi-scale feature extraction is particularly beneficial for capturing different levels of structural information across the graph (Dhariwal & Nichol, 2021). More details on the U-Net architecture are provided in Appendix D.
Multi-Scale Smoothing Learning Objective. Existing diffusion models minimize element-wise discrepancies be- tween the reconstructed &o and the clean signal Xo. as- suming that higher fidelity leads to better representations. However, as shown in Section 3, this MSE-based objective often overemphasizes irrelevant high-freyuency details for downstream tasks.
By extracting these low-frequency components from both the graph topology (Ea²) and node features (Ef), ), we obtain necessary conditioning information for our diffusion model. Specifically, these components are concatenated construct the loss function, and Equation (3) is rewritten as:
To address this, we introduce the Multi-Scale Smoothing (MSS) loss, which focuses on preferentially reconstructing low-frequency, global features. Starting with the MSE- based objective in Equation (12), we define the MSE loss as:
Lime E, (0 Exi1no |xx(xx, 1, ClEd (8)) xo ||] (12)
where &n (x,, E denotes the recon- 1. structed signal based on the noisy input x, and conditioning from the lwwfrreuuecy encoders. C(æ, y) represents the concatenation function, the implementation details are pro- vided in Appendix D.
RMSE = 11:00 - xx11 (xx, C(E EE) -
-xulli xo||²
The MSS loss is formulated by focusing on cosine similarity between feature embeddings at different scales, specifically
6

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Table Node classification accuracy (%) ) of various methods. Best and second-best results are bolded and underlined, respectively.
CATEGORY
METHOD
CoRA
CITESEER
PUBMED
OGEN-ARXIV
COMPUTER
PHOTO
GCN
81.510.5
70.3
79.010.4
71.723.0
86.5
92.4110.2
Supervised
GAT
83.010.7
72.5
79.010.3
72.110.1
86.910.3
92.610.4
NODE2VEC
74.8
52.3
80.3
84.39
89.67
Random Walk
DEEPWALK
75.7
50.5
80.5
85.68
89.44
DGI
82.3#0.6
71.8+0.7
76.8+0.6
70.3=0.2
84.0=00.5
91.6400.2
MVGRL
83.5+0.6
73.3+0.5
80.1
70.3=0.5
87.5+0.1
91.7+00.1
BGRI.
82.7:0.6
71.1
79.611..5
71.620.1
89.7.0.3
92.9
INFOGCL
83.5+0.3
73.5
79. 10.2
71.2=0.2
88.740.4
93.110.1
CCA-SSG
84.0+0.4
73.1
81.010.44
71.2
88.7
93.110.1
GPT-GNN
80.1+1.0
68.4+11.6
76.3+10.8
GRAPHMAE
84.2+0.4
73.4:0.4
81.1+0.4
71.8=0.2
88,6
93.6:t0.2
Self-Supervised
GRAPHTCM
81.510.5
72.8
77.240.5
54.7=0.2
84.9
92.11:0.2
VGAE
76.3+0.2
66.8+0.2
75,8+0.4
66,4=0.2
85.8+0.3
91,5+002
SP-GCL
83.2#0.1
71.9
79.220.7
68.3=0.2
89.7
92.5
GRAPHACL
84.240.3
73.6+0.2
82.0+0.2
71.7+0.3
89.8+0.3
93.3++022
DSSL
83.5100.
73.2.0..
81.3100.3
69.910.4
89.2
93.110.3
DDM
83.110.3
72.110.4
79.610.9
71.3
89.811.2
93.8.10.2
SDMG (w/ mask)
84.3+00.5 83.6110.6
73.9+0.4
80.0++0,5
72.1=0.3
91.6=+0.2
94.7=10.2
Our
SDMG
73.2
80.0
70.620.2
90.4.10.2
94.1100.2
over multiple hops in the graph. The MSS objective is given by:
cosine similarity a better alignment criterion for downstream tasks. ii) Second, the MSS loss enforces multi-scale consis- tency across neighborhood scales. By considering features aggregated over multiple hops, this approach emphasizes low-frequency signals that are stable across larger graph regions and preserve erucial global structural information. A recent work, GraphMAE, introduces per-node scaled- cosine eгror (SCE) loss (Hou et al., 2022); however, un- like SCF, MSS aligns representations over multiple hops, thereby explicitly encouraging global low-frequency struc- ture.
hop-1
RMSS= = (xx: &n)) 101
II (n ".
(15)
where S(-) denotes cosine similarity, and h and h rep- resent the feature embeddings of the target node to aggre- gated over its k-hop neighborhood. The weight wa controls
the influence of each hop. Specifically, h(* and ( are the original and reconstructed feature representations of the target node th, aggregated with the features from its h--hop neighborhood, respectively. This can be formalized as:
Mathematically, we show that this multi-scale smoothing objective preferentially penalizes errors in low- cow-freuencyy components, forcing the model to reconstruct global struc- tural information more effectively:
H ik) (Anom)* x,
H (k) (A X
Parameter settings and analysis of Wyy and are given in Appendices C.2 and E. This formulation ensures that node features are progressively aggregated across multiple hops, capturing multi-scale and smooth structural information. Thus, the objective in Equation (12) is rewritten
Theorem 4.1. Minimizing the loss function in Equation (15) based on X encourages the model to reconstruct more low- frequency features with a low-pass filter g g(Xa)
2
Proof Sketch. See Appendix A.2 for the complete proof. Intuitively, multi-scale neighborhood consistency empha- sizes slowly varying, global features (low-frequency com- ponents), while allowing more flexibility in reconstructing high- fegh-freeuency details that contribute less to the aggregated embeddings over larger neighborhood sizes.
mmss E, [x()) Ex, xo RMSs]
(16)
The MSS loss leverages two key ideas enhance repre- sentation learning. i) First, it uses cosine similarity for pairwise reconstruction, aligning feature directions instead of exact values. This is particularly effective for tasks like node classification, where the angular similarity between node representations is more important than precise value- by-value matching. Nodes belonging to the same class typically occupy similar directions feature space, making
Empirically, we find that MSS leads to stronger represen- tation quality and an improved classification accuracy, as detailed the experimental Section Together with the low-frequency encoder loss ((A) in Equation (11) and Cass in Equation (16), the final loss is Lioss L(A) + Lmss-

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Table Graph classification accuracy (%) of various methods. Best and second-hest results are bolded and underlined, respectively. *L" denotes data that is out of memory or not included in the original paper.
CATEGORY
METHOD
IMDB-B
IMDB-M
PROTEINS
COLLAB
MUTAG
GIN
75. 14511
52.3428
76.2
80.2+1.9 78.942.3
89.4=5.6
Supervised
DIFFPOOL
72.6#3.9
75.1
85.0=10.3
NODE2VEC
50.20::0.90
70
57.49
12.63=10.20
Random Walk
SUB2VEC
55.26=1154
36.70+0.80
53.03=5.55
15.05
GRAPH2VEC
71.10=0054
50.4440.87
73.30=0.05
83.15+9..2
INFOGRAPH
73.03+0087
49.69+0.53
74.44=0.31
70.65+1.13
89.01+1.13
GRAPHCL
71.14=0044
0..5
74.39=0:45
71.36
86.8041.34
JOAO
70.21
49.20:10.7
74.55=0.41
69.50
87.35
GCC
72.0
49.4
78.9
Self-supervised
MVGRI.
74.20-10.70
51.20+40.50
1.
GRAPHMAE
75.52-+0.66
51.63++0.52
75.30=0.39
80.32::0.46
88. 18.19
INFOGCL
75.10:10.90
51.40+0.80
10.00+1.30
91.20+11.0
SIMGRACE
71.30:10.77
75.35=0.09
71.72:1088
89. 41.01
DDM
74.05=20.17
52.02+00.9
00.51
80.70+0.18
90.15 1540.66
Our
SDMG
76.03
12.4
73.17
82.23
91.58:0..3 1.0.8
5. Experiments
and apply it to the clean node features Xo via element-wise multiplication:
5.1. Experimental Setup
Datasets. We evaluate our approach on node-level and graph-level benchmarks. For node classification, we use six datasets: three citation networks (Cora, CiteSeer, PubMed (Sen et al., 2008)), two co-purchase graphs (Photo, Computer (Shchur al., 2018)), and the large-scale arXiv dataset from the Open Graph Benchmark (Hu et al., 2020a). For graph classification, we use five benchmarks: IMDB-BR IMDB-M, PROTEINS, COLLAB, and MUTAG (Yanardag & Vishwanathan, 2015). For graph classification tasks, node degrees serve as the initial attributes, which are then one- hot encoded for processing. More details are presented in Appendix C.3
Xg
X
mask.
(17)
Here, some feature entries become zero before being fed into the conditional encoder. Consequently, when the denoising decoder receives X,, portion of the features has already been masked out. Importantly, requires reconstructing the enfire node features, including positions that were masked out and never observed by the decoder. This compels the model to learn a more holistic approximation of the data distribution, filling in missing dimensions even when they are not visible during encoding.
training objective still
Results. Table shows the performance of SDMG and its variant SDMG (w/ mask). Specifically, relative the pio- neering diffusion model-based graph SSL approach, DDM, SDMG consistently outperforms it on 5 datasets, highlight- ing the effectiveness of our framework. In particular, the masked variant (SDMG w/ w/mask) achieves slightly higher ac- curacy on most benchmarks, likely because random feature masking forces the denoiser model missing dimensions and thus learn more robust, holistic node representations, while reducing over-fitting to spurious details. Moreover, SDMG demonstrates competitiveness when compared to fully supervised approaches and advanced self-supervised methods, highlighting that our diffusion perspective on graph representation learning is effective.
Baselines and Settings. We compare method with unsupervised and supervised methods, using accuracy as the primary metric. Our experiments are conducted on 4 NVIDIA H100 GPUs. Further details on baselines, and experimental protocols are provided in Appendix C, with additional results in Appendix E.
11
5.2. Node Classification
Masking Strategy for Node Classification. Inspired by Soft Diffusion (Daras al., 2022) and Ambient Diffu- sion (Daras et al., 2024) in computer vision, we introduce masking process for node classification tasks. Concretely, given masking rate we sample each entry of mask vector mask E {0, 1 i.idd. from a Bernoulli(p) distribu- tion. Formally, mask; ~ Bernoull(pp), Vi 1,...,d, d,
5.3. Graph Classification
To further validate the effectiveness of our approach, we con- duct extensive experiments on graph classification tasks. As reported in Table 2, SDMG achieves the best performance
'Our implementation is available at:
https///hthabbbom/////tUUUSSSSD
8

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
on four out of five datasets. This result indicates the poten- tial of our SDMG in driving strong representation learning performance. Moreover, similar to the node classification setting observations, SDMG consistently surpasses fully supervised baselines and self-supervised methods based on contrastive learning, generative modeling, or random walks, highlighting our framework's practical utility and broad applicability.
This study connects spectral graph theory and advances in diffusion models, with much potential for future exploration of adaptive, frequency-aware objectives in graph diffusion learning.
A Acknowledgements
Chao Gao and Dongpeng Hou acknowledge the National Natural Science Foundation of China (Nos. 62261 1 136549, 62471403), the Technological InnovationTeamm of Shaanxi Province (No. 2025RS-CXTD-OA the International coop- eration project of Shaanxi Province (No. 2025GH-YBXM- 017).
5.4. Ablation Study
Table 3. Results of ablation study. Classification accuracy (%) for models: without low-frequency encoders and MSS loss (w/o both), without low-fre freerueqecn encoders (w/ LE), without MSS loss (w/o MSS), and full model.
Junyou Zhu and Frank Hellmann gratefully thank the Fed- eral Ministry of Education and Research (03SF0766), the German Federal Ministry for Economic Affairs and Climate Action (03E11092A), and the German Research Foundation (DFG) (KU 837/39-2).
Dataset
w/o both
w/o LE
w/o MSS
Full
Photo
83.65
87,42
93.46
94.73
Computer
76.42
85.86
89.46
91.64
Langzhou He and Philip S. Yu are supported in part by NSF under grants 111-2106758, and POSE-2346155
IMDB-B
74.05
74.53
75.20
76.03
MUTAG 90.15
90.53
90.40
91.58
All authors gratefully acknowledge the Ministry of Re- search, Science and Culture (MWFK) of Land Brandenburg for supporting this project by providing resources the high performance computer system at the Potsdam Institute for Climate Impact Research.
We perform an ablation study to validate the effectiveness of our low-frequency encoders (I.F) and the proposed multi- scale smoothing (MSS) loss. Table 3 compares: (i) our full model, (ii) w/o LE (replacing the low-freques wwwffreuency encoders with MLPs), (iii) w/o MSS (replacing MSS with MSE), and (iv) w/o both (removing both).
Impact Statement
Our findings show that eliminating either LE or MSS de- grades the classification accuracy, indicating each module's s unique contribution. Low-frequency encoders extract global patterns that enhance downstream performance, and our MSS loss prevents overfitting fig-frrquency details. Fur- thermore, dropping both modules significantly reduces ac- curacy, highlighting their complementary roles in boosting representation quality. Although removing MSS causes a smaller accuracy drop than removing the LF encoders, MSS still closes a substantial portion of the gap between "w/o both" and the full model.
Our work introduces an unsupervised framework that lever- ages diffusion probabilistic models learn robust, low- frequency representations on graphs. In principle, our ap- proach could reduce reliance on large labeled graph datasets, benefiting domains that lack substantial annotations (c.g.. drug discovery, social network analysis). We believe this has a potential positive societal impact, as it may accelerate research and development where quick and accurate graph insights required.
On the other hand, generative techniques, including diffu- sion models, pose general risks if used without proper over- sight, such as generating misleading or adversarial graph data. However, our method is primarily designed for repre- sentation learning rather than standalone graph generation, mitigating immediate risks of malicious misuse. We en- courage practitioners combine our approach with security checks or domain-specific safeguards if it is deployed in sensitive applications (e.g., financial networks, healthcare). Overall, this research aims to advance the field of machine learning on graphs and does not introduce new, specific eth- ical or societal threats beyond those commonly associated with generative modeling.
6. Conclusion
In this paper, we introduced the Smooth Diffusion Model for Graph Representation Learning (SDMG), which inte- grates low-frequency component encoders for node features and graph topology, along with a novel Multi-Scale Smooth- ing (MSS) loss. By prioritizing wwwffrquency information, SDMG effectively captures the global structure of graphs and reduces an overemphasis on sigh-frequency noise, re- sulting in better graph representation learning. Our experi- mental results demonstrate that SDMG outperforms existing methods, particularly in tasks sensitive to structural informa- tion, such as node classification and graph reconstruction.

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
References
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Ben- gio, Y., and Bresson, X. Benchmarking graph neural networks. Journal of Machine Learning Research, 24 (43):1-48, 2023.
Adhikari, B.. Zhang, Y., Ramakrishnan, N., and Prakash,
B. A. Sub2vec: Feature learning for subgraphs. In Ad- vances in Knowledge Discovery and Data Mining, pp. 170-182. Springer International Publishing, 2018.
Eckart, C. and Young, G. The approximation of one matrix by another of lower rank. Psychometrika. 133):2112118, 1936.
Bo, D., Wang. X., Shi, C., and Shen, H. Beyond low- frequency information in graph convolutional networks. In Proceedings of the AAAI conference artificial intel- ligence, volume 35, pp. 3950-3957, 2021.
Fang, T., Chow, W., Sun, Y., Han, K., Ma, L., and Yang. Y. Exploring correlations of self self-super tasks for graphs. In Proceedings of the 41st International Conference on Machine Learning, 2024.
Cai, F., Liu, C.,, Gerten, D., Yang, S., Zhang, T., Li, K., and Kurths, J. Sketching the spatial disparities in heatwave trends by changing atmospheric teleconnections in the northern hemisphere. Nature Communications, 15(1):
Fu, X., Gao, Y., Wei, Y., Sun, Q., Peng, H., Li, J., and LI, X. Hyperbolic geometric latent diffusion model for graph generation. In Proceedings of the 41 International Conference on Machine Learning, 2024.
2024a.
Cai, Z., Wang, X., and Zhang, M. Latent graph diffusion: unified framework for generation and prediction on graphs. arXiv preprint arXiv:2402.0255/ 02518, 2024b.
Gao, C., Zhu, J., Zhang, F., Wang. Z., and Li, X. A novel rep- resentation learning for dynamic graphs based on graph convolutional networks. JEEE Transactions on Cybernet- ics, 53(6):359,,311,, 2023a.
Chen, D., Xue, S., Chen, L.,, Wang, Y., Liu, Q. Wu, S., Ma, Z.-M., and Wang, L. Graffe: Graph representation learning via diffusion probabilistic models. 2025.
Gao, S., Zhou, P., Cheng, M.-M., and Yan, Masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 23164 231733, 2023b.
Chen, X., He, J., Han, X., and Liu, L. Efficient and degree- guided graph generation via discrete diffusion modeling. In Proceedings of the 40th International Conference Machine Learning, volume 202, pp. 138554610 2023.
Grover, A. and Leskovec, J. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discov- ery and data mining, pp. 855-864, 2016.
Chen, X., Liu, Z., Xie, S., and He, K. Deconstructing denoising diffusion models for self superviee learning. arXiv preprint ar arXiv:2401.1 20V:2401.14444 14404, 2024a.
Chen, Y., Fang, Y., Zhang. Y., Ma, C., Hong, Y., and King, I. Towards effective top-n hamming search via bipar- tite graph contrastive hashing. IEEE Transactions on Knowledge and Data Engineering, 2024b.
Haefeli, K. K., Martinkus, K., Perraudin, N., and Wat- tenhofer, R. Diffusion models for graphs benefit from discrete state spaces. arXiv preprint arXiv:2210.01549. 2022.
Cho. H., Jeong, Jeon, S., Ahn, S., and Kim, W. H. Multi- resolution spectral coherence for graph generation with score-based diffusion. Advances in Neural Information Processing Systems, 36, 2024.
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T. Prov- able guarantees for self-supervised deep learning with spectral contrastive loss. Advances in neural information processing systems, 34:5000-500 5011, 2021.
Daras, G., Delbracio, M., Talebi, H., Dimakis, A., and Milanfar, P. Soft diffusion: Score matching with general corruptions. Transactions on Machine Learning Research, 2022.
Hassani, K. and Khasahmadi, A. H. Contrastive multi- view representation learning on graphs. In International conference on machine learning, pp. 4116-4126. PMLR, 2020.
Daras, G., Shah, K., Dagan, Y., Gollakota, A., Dimakis, A., and Klivans, A. Ambient diffusion: Learning clean distributions from corrupted data. Advances in Neural Information Processing Systems, 36, 2024.
He, L., Wang, S., Wang, J., Gao, C., and Tao, L. Integrating global features into neural collaborative filtering. In Inter- national Conference Knowledge Science, Engineering and Management, pp. 926-336. Springer, 2022.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 2021.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-688 6851, 2020.
10

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Hoang, N., Machara, T., and Murata, T. Revisiting graph neural networks: Graph filtering perspective. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 8376-838 06638,83.1188 IEEE, 2021.
Kipf, T. N. and Welling, M. Variational graph auto-encoders. arXiv preprint /r 2016b.
Kong. L.., Cui, J., Sun, H., Zhuang, Y., Prakash, B. A., and Zhang, C. Autoregressive diffusion model for graph gen- eration. In International conference on machine learning, pp. 17391-17408, PMLR, 2023.
Hou, D., Gao, C., Li, X., and Wang, Z. Dag-aware varia- tional autoencoder for social propagation graph genera- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 8508-8516, 2024a.
Li, Y., Wang, H., Jin, Q., Hu, J., Chemerys, P., Fu, Y., Wang, Y., Tulyakov, S., and Ren, J. Snapfusion: Text-to-image diffusion model mobile devices within two seconds. Advances in Neural Information Processing Systems, 36, 2024.
Hou, D., Gao, C., Wang, Z., Li, X., and Li, X. Random full- order-coverage based rapid source localization with lim- ited observations for large-scale networks. IEEE Transac- tions on Network Science and Engineering, 11(5):4213- 4226, 2024b.
Liu, G., Inae, E., Zhao, T., Xu, J., Luo, T., and Jiang. M. Data-centric learning from unlabeled graphs with diffu- sion model. Advances in neural information processing systems, 36:21039 166220099-11 2023a.
Hou, D., Gao, C., Wang, Z., and Li, X. Fgssi: A feature- enhanced framework with transferability for sequential source identification. IEEE Transactions on Dependable and Secure Computing, 2025.
Liu, G., Xu, J., Luo, T., and Jiang, M. Graph diffusion trans- formers for multi cunti-conditional molecular generation. Ad- vances in neural information processing systems, 2024a.
Hou, Z., Liu, X., Cen, Y., Dong, Y., Yang. H.,, Wang, C.,, and Tang, J. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 194-604, 2022.
Liu, N., Wang. X., Bo, D., Shi, C., and Pei, Revisiting graph contrastive learning from the perspective of graph spectrum. Advances in Neural Information Processing Systems, 35:2972-2998 2022.
Hu, W., Fey, M., Zitnik, M.. Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 13-2233, 2020a.
Liu, Y., Wang, X., Wang, X., Wang, Z., and Kurths, J. Dif- fusion source inference for large-scale complex networks based on network percolation. IEEE Transactions on Neural Networks and Learning Systems, 2023b.
Hu, Z., Dong, Y., Wang, K., Chang, K.-W., and Sun, Y. Gpt- gnn: Generative pre-training of graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. pp- 1857-1867, 2020b.
Liu, Y., Wang, X., Wang, X., and Wang, Z. Fast out- break sense and effective source inference via minimum observer set. JEEE/ACM Transactions on Networking, 2024b.
Hudson, A., Zoran, D., Malinowski, M., Lampinen, A. K., Jaegle, A., McClelland, J. L., Matthey, L.., Hill, F,, and Lerchner, A. Soda: Bottleneck diffusion models for representation learning, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23115-23122 2024.
Luo, T., Mo, Z., and Pan, J. Fast graph generation via spectral diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3496-335 -3508, 2024.
Martinkus, K., Loukas, A., Perraudin, N., and Wattenhofer, R. Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. In In- ternational Conference on Machine Learning, pp. 15159- 15179. PMLR, 2022.
Jo, J., Lee, S., and Hwang, S. J. Score-based generative modeling of graphs via the system of stochastic differen- tial equations. In International conference on machine learning, pp. 10362-10383, PMLR, 2022.
Minello, G., Bicciato, A., Rossi, L.., Torsello, A., and Cosmo, L. Generating graphs via spectral diffusion. In The Thirteenth International Conference Learning Representations, 2025.
Kalantidis, Y. Sariyildiz, M. B., Pion, N., Weinzaepfel, P., and Larlus, D. Hard negative mixing for contrastive learning. Advances in neural information processing systems, 33:21798-21809 2020.
Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., and Jaiswal, S. graph2vec: Learning distributed representations of graphs. arXiv preprint 17Xiv:/70700500 2017.
Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. In International Conference on Learning Representations, 201 6a.

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Niu, C., Song, Y., Song, J., Zhao, S., Grover, A., and Ermon, S. Permutation invariant graph generation via score-based generative modeling. In International Conference on Ar- rificial Intelligence and Statistics, pp. 4474 4484. PMLR, 2020.
Tian, C., Tao, C., Dai, J., Li, H., Li, Z.,, Lu, L., Wang, X., Li, H., Huang. G., and Zhu, X. ADDP: Learning general representations for image recognition and generation with alternating denoising diffusion process. In The Twelfth International Conference on Learning Representations, 2024.
Pan, Z., Chen, J., and Shi, Y. Masked diffusion as self-supervised representation learner. arXiv preprint NrXiv:2308.06620 2023.
Veličković, P., Cucurull, G., Casanova, A., Romero, A.. Liò, P., and Bengio, Y. Graph attention networks. In International Conference on Learning Representations, 2018.
Perozzi, B., AI-Rfou, R., and Skiena, S. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowl- edge discovery and data mining, pp. 701-710, 2014.
Veličković, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax. In Interna- tional Conference Learning Representations, 2019.
Preechakul, K., Chatthee, N., Wizadwongsa, S., and Suwa- janakorn, S. Diffusion autoencoders: Toward a meaning- ful and decodable representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10619-10629, 2022.
Vignac, C., Krawezuk, I., Siraudin, A., Wang, B., Cevher, V., and Frossard, P. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv: 14734, 2022.
Qiu, J., Chen, Q., Dong, Y.. Zhang, J., Yang. H.,, Ding, M., Wang, K., and Tang, J. Gee: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowl- edge discovery & data mining, pp. 1150-1160, 2020.
Wang. H., Zhang, J., Zhu, Q., Huang, W., Kawaguchi, K., and Xiao, X. Single-pase contrastive learning can work for both homophilic and heterophilic graph. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Wang. Y., Schiff, Y., Gokaslan, A., Pan, W., Wang, F,
Rampášek, L., Galkin, M., Dwivedi, V. P., Luu, A. T., Wolf, G., and Beaini, D. Recipe for a general, powerful, seal- able graph transformer. Advances in Neural Information Processing Systems, 35: 14501-1455,, 2022.
De Sa, C., and Kuleshov, V. Infodiffusion: Represen- tation learning using information maximizing diffusion models. In International Conference on Machine Learn- ing, pp. 36336-36354, PMLR, 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffuuson models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Wei, C., Mangalam, K., Huang, P.-Y., Li, Y., Fan, H., Xu, H., Wang, H., Xie, C., Yuille, A., and Feichtenhofer, C. Dif- fusion models as masked autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16284-16294, 2023.
Sen, P., Namata, G., Bilgic, M., Getuor, L., Galligher, B., and Eliaasi-RRd,, T. Collective classification in network data. Al magazine, 29(3):93 93, 2008.
Wu, Y. and Verdú, S. Functional properties of minimum mean-square error and mutual information. IEEE Trans- actions on Information Theory, 58(3))1229-113333 1301, 2011.
Shchur, O., Mumme, M., Bojehevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. arXiv preprint 181 2018.
Xiang, W., Yang, H., Huang, D.,, and Wang, Y. Denois- ing diffusion autoencoders are unified self-supervised learners. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2023.
Song. Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic diffferential equations. In International Conference on Learning Representations, 2021.
Xiao, T., Chen, Z.., Guo, 7.,, Zhuang. 7.,, and Wang, S. Decoupled self-supervised learning for graphs. Advances in Neural Information Processing Systems, 35:620-634, 2022.
Thakoor, S., Tallec, C., Azar, M. G., Azabou, M., Dyer, E. L., Munos, R., Veličković, P., and Valko, M. Large- scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations, 2021.
Xiao, T., Zhu, H., Chen, Z., and Wang, S. Simple and asym- metric graph contrastive learning without augmentations. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Xiong, P., Schnake, T., Gastegger, M., Montavon, G., Muller, K. R., and Nakajima, S. Relevant walk search for explaining graph neural networks. In International Con- ference on Machine Learning, pp. 2023.
Zhou, C., Wang, X., and Zhang, M. Unifying generation and prediction graphs with latent graph diffusion. Ad- vances in Neural Information Processing Systems, 2024.
PMLR,
Zhu, J., Li, X., Gao, C., Wang. Z.,, and Kurths, Unsuper- vised community detection in attributed networks based mutual information maximization. New Journal of Physics, 23(1 I): 1 13016, 2021.
Xu, D., Cheng, W., Luo, D., Chen, H., and Zhang, X. In- fogel: Information-awes aware graph contrastive learning. Ad- vances in Neural Information Processing Systems, 34: 30414-30425, 2021.
Zhu, J., Wang. C.,, Gao, C., Zhang, F., Wang, Z., and Li, X. Community detection in graph: An embedding method. IEEE Transactions Network Science and Engineering, 9(2):699-702, 2022.
Xu, K. Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In International Conference on Learning Representations. 2018.
Zhu, J., Gao, C., Yin, Z., Li, X., and Kurths, J. Propagation structure-aware graph transformer for robust and inter- pretable fake news detection. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 4652 4663, 2024.
Yanardag, P. and Vishwanathan, S. Deep graph kernels.
In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp- 1365-1374, 2015.
Yang, R., Yang, Y., Zhou, F., and Sun, Q. Directional diffu- sion models for graph representation learning. Advances in Neural Information Processing Systems, 36, 2024.
Ye, M., Wu, L., and Liu, Q. First hitting diffusion mod- els for generating manifold, graph and categorical data. Advances in Neural Information Processing Systems, 35: 17280-27292 2022.
Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., and Leskovec, J. Hierarchical graph representation learning with differentiable pooling. Advances in neural informa- tion processing systems, 31, 2018.
You, Y., Chen, T., Sui, Y., Chen, T., Wang. Z., and Shen, Y. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.
Yu, J., Xu, T., Rong. Y., Bian, Y., Huang, J., and He, R. Rec- ognizing predictive substructures with subgraph informa- tion bottleneck. IEEE Transactions Pattern Analysis and Machine Intelligence, 46(3):1650-116) 1663, 2024a.
Yu, Y., Wang, X., Zhang, M., Liu, N., and Shi, C. Prov- able training for graph contrastive learning. Advances in Neural Information Processing Systems, 36, 2024b.
Zhang, H., Wu, Q. Yan, J., Wipf, D., and Yu, S. From canonical correlation analysis self-supervised graph neural networks. Advances in Neural Information Pro- cessing Systems, 34:76-89, 2021.
Zhang, Z., Zhao, Z., and Lin, Z. Unsupervised represen- tation learning from pre-trained diffusion probabilistic models. Advances neural information processing sys- tems, 1122117-22131 2022.
13

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
A. Proof of Theorems.
A.I. Proof of Theorem 3.1.
Theorem. Consider an encoding Z of X with a bounded capacity 1(X; Z) S. C, where 1(3-) denotes mutual information. Assume that X takes values in the unit ball of an Euclidean space. Let Y be a target variable be predicted from X, and
assume X be decomposed X (X X,) such that X, is relevant to Y and X is superfluous predicting Y given
X,, that is (XX;; O. Then we have:
The minimal mean square error (mmse) for predicting X from Z is bounded
mimse (nm)s 2 var var(x) - 1 [ZZ - var(x) 1 (1(;xX,, (
(18)
The information that the encoding Z carries on the variable is bounded by
(Z.Y) S C (Zz,
(19)
Therefore, optimizing the encoding Z in order to minimize manse ( is partially misaligned with optimizing the encoding for (ZZ The former benefits from an increase in encoded superftuous information, the latter is penalized by
Proof. The first equation is Theorem 10 of (Wu & Verdú, 2011) plus the chain rule. For the second, first note that Z is an encoding of X, thus T(Z; (()) O, and X₂ is conditionally independent of Y. Then have by the chain rule that
T(Y: 2,, IY;;
I(Y: z|x, x,)
=0 x
(20)
As Y is conditionally independent of Z and X,, is also conditionally independent of Z and we have 2( Thus + X, + a Markov chain, and by the data processing incquality we have:
0.
I(Z:Y)
IZ -2, - IZ; X < 1C-
-
(21)
A.2. Proof of Theorem 4.1.
Theorem. Minimizing the loss function in Equation (15) based on X encourages the model to reconstruct more low- frequency features with a low-pass filter g(As) = S.
Proof. Since L = =I- Agorn is symmetric and positive semidefinite, it admits an eigenvalue decomposition:
i UAUT
(22)
where U NxN is an orthogonal matrix of eigenvectors T D) and A diag(A, dg.... .a.....a. Ax) contains the eigenvalues 0 < 2. Therefore, the k-step aggregation becomes:
X (Anor))* X I (I-1)² x
(23)
(I- UT-UAUT>*
(24)
=(U(I- (U(I-A)UT) A)UT)* 1xx
(25)
=U(1 (U(1-A)4 x
(26)
AU(--A)'
(27)
14

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Here, X = U x denote the graph Fourier transform of X. This expression indicates that each frequency component X, is scaled by g\,,) (1 }.)* Since 0 I X S 2, the magnitude 1 - >₂1 S 1.
For low-frequency components (small A) (1 - X²) is close 1, so g(},) remains near even for large k. preserving these components. For high-frequency components (large λd), |1 - X is small, and g\x,) diminishes rapidly as k increases, attenuating these components.
By minimizing loss function involving X, the model is encouraged to reconstruct the features emphasizing the low- frequency components of X, effectively acting as low-pass filter that suppresses high-frequency noise.
B. Related Works
Graphs emerge naturally in a wide range of physical (Zhu et al., 2021), and social phenomena (Zhu et al., 2024). In climate science, Cai al. use teleconnection graphs to reveal spatial disparities in Northern Hemisphere heat-wave trends (Cai et al., 2024a). For spreading processes on large interaction networks (Hou et al., 2025; 2024a;b), Liu et al. propose percolation- based algorithm for diffusion-source inference (Liu et al., 2023b), while subsequent study by the same authors introduces a minimum-observen strategy that enables faster outbreak sensing (Liu et al., 2024b). Against this backdrop, learning expressive graph representations has become a central problem in machine learning (Xiong et al., 2023). Below we review the advances most relevant to our work and explain how our Smooth Diffusion Model for Graphs (SDMG) both builds on and extends these directions.
B.1. SuperSupe Graph Learning.
Driven by the goal of removing dependency on labeled data, self-supervised learning (SSL) has rapidly gained traction. Two predominant strategies emerge:
(i) Contrastive SSL. Contrastive methods seek to learn invariant representations by generating multiple views or aug- mentations of the input and ensuring that embeddings of positive examples (e.g., different augmented versions of the same node or subgraph) are similar, while pushing apart representations of negatives (Gao et al., 2023a). Early works such as DGI (Veličković et al., 2019) and InfoGraph (Xu et al., 2021) rely on by corrupting node features and graph topology to create challenging negative samples. Subsequent approaches explore diverse augmentation schemes, ranging from random-walk sampling (Hassani & Khasahmadi, 2020; Qiu et al., 2020) to feature masking shuffling (You et al., 2020), to mitigate semantic damage from aggressive perturbations. Recent methods like BGRL (Thakoor et al., 2021) discard explicit negatives to simplify training, while GraphCI. (You et al., 2020) systematically designs graph-level transformations. (Chen et al., 2024b) proposes effective graph contrastive operations in the discrete hamming space. Despite promising results, contrastive methods can be sensitive to the choice of augmentations and sampling strategies, risking suboptimal performance when augmentations are misaligned with downstream tasks.
objectives, often
(ii) Generative SSL. Generative strategies instead learn by masking or corrupting part of the graph data (features or edges) and reconstructing the masked information. VGAE (Kipf & Welling, 2016b) adopts a variational autoencoder to recover adjacency signals, while GraphMAE (Hou et al., 2022) implements a masked autoencoder to reconstruct node features with a noise-robust objective. Other approaches, such as GPT-GNN (Hu et al., 2020b), follow an autoregressive paradigm analogous to language models, iteratively predicting node or edge attributes in large-scale or heterogeneous graphs. Although these generative methods effectively capture structural and semantic properties, many give equal priority to all frequency components of the graph signals, potentially overfitting ligh-frequeney noise. Moreover, they often employ heuristic corruption strategies that may not align perfectly with subsequent learning objectives, creating opportunities for improvement in balancing reconstruction fidelity and downstream performance.
B.2. Diffusion Models for Graphs.
Background on Diffusion Models. Diffusion probabilistic models have shown remarkable success in image and text domains by gradually injecting noise into the data, then learning to reverse this process via a denoising network (Ho et al., 2020; Song et al., ;021; Dhariwal & Nichol, 2021; Rombach et al., 2022). Most such methods rely on continuous noise perturbations in the pixel or latent space, enabling high-quality sample generation and powerful representation learning (Tian et al., 2024; Hudson al., 2024). Other works, for example (Pan et al., 2023), propose using mask-based strategies to
15

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
learn image representations. However, extending these approaches to graph-structured data poses unique challenges due discrete topologies and node attributes that may not align neatly with Gaussian noise models.
Diffusion Models for Graph Generation. Early attempts at graph-focused diffusion primarily address graph generation rather than representation learning (Niu et al., 2020; Jo et al., 2022; Luo et al., 2024; Zhou et al., 2024; Liu et al., 2023a; 2024a; Chen et al., 2023; Ye et al., 2022; Kong et al., 2023; Fu et al., 2024). For instance, Niu et al. (2020) introduce graphs, while Jo et al. (2022) employ stochastic differential equations
score-based methods to create permutation-invarte (SDEs) to handle continuous adjacency matrices and node attributes. Yet, these continuous approaches can struggle with fundamentally discrete structures. To mitigate this, Haefeli et al. (2022) design discrete diffusion model specific to unattributed graphs, and Vignac et al. (2022) propose DiGress to progressively add or remove edges and nodes through a discrete noise schedule. Several recent generators also operate in the spectral domain: SPECTRE Martinkus et al. (2022) combines GANs with spectral conditioning to model low-frequency structure, while GGSD Minello et al. (2025) applies a diffusion process to a truncated set of eigenpairs before reconstructing the graph adjacency. Such efforts underscore the versatility of diffusion processes for generating novel graphs (e.g., molecular or social), but they seldom target downstream classification other tasks that require learned representations.
Transition to Representation Learning. While several diffusion models now integrate autoencoder-like architec- tures (Preechakul et al., 2022; Wang et al., 2023; Zhang et al., 2022; Wei et al., 2023), exploit latent-variable spaces for more expressive generation (Gao et al., 2023b), or explore decoder-only models (Xiang et al., 2023; Chen et al., 2024a), their objectives remain rooted in learning high-quality representations of images. Only recently have researchers begun to explore diffusion models explicitly for graph representation learning. One example is DDM (Yang et al., 2024), which adapts denoising framework to node-level encoding and taps intermediate denoising layers for learned embeddings. Despite this conceptual shift, DDM's default mean wawssss-uuuueeeeeeeeeee error objective can overemphasize high-frequene, fluctuations, echoing known pitfalls in graph neural networks (GNNs) (Kipf& & Welling, 20 Veličković et al., 2018; He et al., 2022) where fow-frequency signals are often most relevant to classification. Additionally, concurrent studies have explored the potential of diffusion models for graph self-supervised learning in discrete (Chen et al., 2025) or continuous spaces (Cai et al., 2024b). however, these approaches still face the same challenges as DDM.
In this work, we address the shortcomings of conventional denoising objectives by selectively reconstructing crucial low- frequency information, thereby aligning the diffusion process with the frequency characteristics of real-world graphs. Our method, SDMG, bridges the gap between generative diffusion models and task-oriented representation learning, resulting in embeddings that better capture structurally significant and semantically discriminative features for downstream tasks.
B.3. Frequency Analysis in Graph Learning
A complementary line of work studies the role of graph frequencies in shaping GNN performance (Kipf & Welling, 2016;; Bo et al., 2021). Low-pass filtering has been identified as a key factor in preserving globally meaningful features (Hoang et al., 2021). However, prior attempts often rely on architectural choices (Hoang et al., 2021) (e.g., stacking many convolutional layers) or contrastive learning-based heuristics (Liu et al., 2022) (e.g., adding adjacency-basee smoothing) to approximate low-frequency filtering. In contrast, SDMG explicitly capitalizes on frequency insights, introducing a reconstructions. By doing so, our model not
novel multi-scale smoothing objective that naturally favors nower-frequent only encourages the diffusion process to ignore irrelevant high-frequency components but also offers a more direct and theoretically principled route to extract globally consistent features. To the best of our knowledge, this is the first work to enhance biffusion-model-bassee graph SSL from a frequency perspective.
C. Experimental setting
C.1. Baselines
We compare our proposed SDMG against a diverse range of representative graph representation learning methods from three primary categories: supervised learning, random walk-based, and self-supervised learning.
Supervised Methods. These approaches train Graph Neural Networks using labeled data. GCN (Kipf & Welling, 2016a) and GAT (Veličković et al., 2018) learn node embeddings via spectral filtering and attention mechanisms, respectively. While they achieve competitive results with sufficient annotations, performance may degrade in large-scale or noisier settings
16

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
with scarce labels. For graph-level classification, GIN (Xu al., 2018) leverages a sum-based aggregation that is powerful in graph isomorphism tests, whereas DiffPool (Ying et al., 2018) introduces differentiable pooling scheme to hierarchically coarsen graphs. Both methods heavily rely on labeled supervision and are less applicable in unlabeled or weakly labeled scenarios.
Random Walk-Based Methods. These methods typically generate node sequences via random walks and then learn embeddings akin to word2vec. Examples include node2vec (Grover & Leskovec, 2016), DeepWalk (Perozzi et al., 201 4), Sub2Vec (Adhikari et al., 2018), and graph2vec (Narayanan et al., 2017). While they are unsupervised and straightforward to implement, they primarily capture local topological patterns. Higher-order or global dependencies may be overlooked, limiting their effectiveness in more complex or heterogeneous graphs.
Self-Supervised Methods. To remove the reliance on manually labeled data, a wide spectrum of graph self-supervised learning approaches has been proposed. We group these methods into three main branches based on their training objectives:
(i) Contrastive Methods. By contrasting positive and negative instances, these models encourage consistent embeddings under various augmentations or subviews. DGI (Veličković et al., 2019) uses a global-local alignment, MVGRL (Hassani & Khasahmadi, 2020) fuses information across multiple graph views, BGRL (Thakoor et al., 2021) introduces a bootstrap strategy under asymmetric encoder-predictt setup, InfoGCL et al., 2021) integrates information-theorret principles into the contrastive framework, CCA-SSG (Zhang et al., 2021) blends canonical correlation analysis with multi-view graph augmentations, SP-GCL (Wang et al., 2024) proposes a single-past pass solution catering to both homophilic and heterophilic graphs. GraphACL (Xiao et al., 2024) adopts an augmentation-fiee frre scheme with asymmetric contrast, and DSSL (Xiao et al., 2022) refines the contrastive objective by decoupling representation components.
(ii) Generative Reconstruction. Another line of work uses partial masking or autoencoding strategies learn robust graph embeddings. GraphMAE (Hou et al., 2022) extends masked autoencoders to graphs by masking node features, VGAE (Kipf& & Welling, 2016b) leverages variational autoencoder reconstruct graph structure, GraphTCM (Fang et al., 2024) exploits multi-task learning to capture richer correlations among self-supervised objectives, while GPT-GNN (Hu et al., 2020b) and GCC (Qiu et al., 2020) follow large-scale pre-training paradigm, showing adaptability in heterogeneous or complex graph scenarios.
(iii) Certusion-Based Modeling. Although still in its early stage for graph representation learning, diffusion models show promise. DDM (Yang et al., 2024) an initial attempt that leverages a denoising diffusion process in node feature space to reconstruct the original input; however, it largely focuses high-frequency details that do not necessarily benefit downstream recognition tasks. Our SDMG extends this line of work by explicitly prioritizing low-frequency information and mitigating irrelevant high-frequency components, aligning the generative diffusion objective more closely with discriminative goals.
full-spectrum MSE objective. Such an approach can overfit
Summary and Key Differences. Our SDMG inherits the generative perspective yet departs from traditional "reconstruct- everything" paradigms by emphasizing smooth, global (low-frequency signals that correlate more strongly with downstream tasks. This targeted design combats the common pitfall of overfitting to frgguereyeenc noise, yielding more discriminative representations and consistent accuracy gains across a variety of node- and graph-level tasks.
C.2. Hyper-paramter Configurations
The hyperparameters employed in our experiments are detailed in Tables 4 and 5. Note that we did not extensively fine-tune these hyperparameters, suggesting that further optimization could potentially enhance the experimental results.
C.3. Datasets
In this section, we provide a comprehensive overview of the datasets used in our experiments for both node-level and graph- level tasks. These benchmarks are widely adopted in the graph representation learning community, and they encompass diverse domains and scales to thoroughly evaluate the effectiveness and generality of our proposed SDMG framework.
Node-Level Datasets. We employ six standard node classification benchmarks: three citation networks (Cora, CiteSeer, PubMed (Sen et al., 2008)), two co- co-phrc graphs (Amazon Photo, Amazon Computer (Shchur et al., 2018)). and the large-scale arXiv dataset from the Open Graph Benchmark (Hu et al., 2020a).
17

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Table 4. Statistics and hyper-parameters for graph classification datasets.
Dataset
IMDB-B
IMDB-M
COLLAB
PROTEINS
MUTAG
# graphs
1000
1500
5000
1113
188
Dataset Statistics
# classes
2
3
2
2
Avg. nodes
19.8
13.0
74.5
13.0
17.9
feat_drop
0.4
0.4
0.4
0.2
0.2
attn_drop
0.4
0.2
num head
2
4
Hyper-parameters
num hidden
128
512
512
512
512
hop hop-weights
5
5
5
4
[0, 3. 1. 1]
[1. 1. 1.1.1.1.111
11,2, 1,
[1, 2, 1.
[1.5, 1. 1,
learning_rad
le
le
le
3e
3c
norm
LayerNorm Sigmoid
LayerNorm Lincar
LayerNorm Const
LayerNorm Lincar
LayerNorm
beta schedule
Sigmoid
Table 5, Statistics and hyper- for node classification datasets.
Dataset
Cora
Citeseer
PubMed
Ogbn-arxiv
Computer
Photo
# nodes # edges
2708
3327
19717
169343
13752
7650
Dataset Statistics
5429
4732
44338
1166243
245861
119081
# classes
6
40
10
8
num hidden
512
1024
1024
512
512
512
learning rate
4e 4
2
2e 2e-4
2e
2e
4e -4
Hyper-parameters
num hop hop-weights norm beta_schedule
2
4
[1. 0.5]
[1. 0.5]
[1. 1. 1. IJ LayerNorm Const
[1. I, 1] LayerNorm Linear
[1. I. 1. BatchNorm Quad
[1. 2, 2. 1] BatchNorm Sigmoid
LayerNorm Sigmoid
LayerNorm Linear
Cora, CiteSeer, and PubMed are classical citation networks where each node represents scientific publication and edges denote citation relationships. Nodes assigned bag-of-words features derived from document abstracts, and class labels correspond to academic topics.
Amazon Photo and Amazon Computer co-purchase networks from Amazon, where products (nodes) linked if they tend to be purchased together. Node features capture product descriptions and categories, with labels reflecting product classes
OGB-arXiv is large-scale citation graph from the Open Graph Benchmark. Each node is a paper from arXiv, edges represent citation relationships, and node features are derived from paper abstracts. The labels correspond to the primary subject area of the paper (e.g., LS. MI.).
These datasets vary size, average node degrees, and feature sparsity, enabling a comprehensive assessment how well our method generalizes across different scales and graph topologies.
Graph-Level Datasets. We further evaluate method five widely used benchmarks for graph classification: IMDB-B, IMDB-M, PROTEINS, COLLAB, and MUTAG (Yanardag & Vishwanathan, 2015).
IMDB-B and IMDB-M are movie collaboration datasets. Each graph corresponds to an ego-network of actors or directors, with a classification label indicating genre or thematic category.
PROTEINS is protein interaction dataset, where each graph represents a protein structure. Nodes indicate secondary structure elements, and edges denote spatial or sequential proximity. Labels characterize protein functions or enzyme classes.
COLLAB is derived from collaboration networks in scientific domains. Nodes are authors, edges represent co- authorship, and graph labels reflect the research field or academic community.
18

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning MUTAG consists of small molecular graphs, where each node is an atom and edges represent chemical bonds. The binary classification label indicates mutagenic effect given organism.
For these graph-level datasets, node degrees are used as the initial attributes and then one-hot encoded for processing, following standard protocols in prior studies (Yanardag Vishwanathan, 2015). For both the node-level and graph-level benchmarks, we adopt the commonly used public splits ensure fair comparison with existing baselines (Yang et al., 2024).
D. A Architecture of Denoise U-Net
U-Net with Graph-Aware Conditioning. As illustrrated in Figure 5(d), our denoising decoder adopts a U-Net-like structure that combines a contracting (down-sampling) path with an expansive (up-sampling) path. Standard U-Net architectures are typically designed for Euclidean domains (e.g., images), which can limit their ability to capture topological information in graphs. To address this, we enhance U-Net by (i) injecting graph-structured conditional information (extracted from our low-frequency encoders) and (ii) using GNN or MLP layers inside the U-Net to better preserve graph structural patterns.
Implementation for Graph Classification. For graph-level tasks, the contracting path employs GAT layers, which denoise target node features by aggregating representations from neighboring nodes. This strategy leverages attention mechanisms to focus on the most relevant parts of the graph during down-sampling. In the expansive path, the final GNN layers act as the decoder, projecting the denoised node features into latent representation space and further smoothing them among neighbors. Afterward, we feed these latent features into an MLP to produce reconstructed outputs. In essence, the decoder's final representations already integrate the encoder activations through skip connections, allowing richer synthesis of local and global graph signals.
Implementation for Node Classification. For node-level tasks, replace the GNN layers in both the contracting and expansive pathways with more lightweight MLPs, which reduce computational overhead without compromising performance. Again, we only use the representations from the decoder side, since the skip connections already channel valuable encoder activations.
Fusion of Low-Frequency Information. To ensure that low-frequency structural signals are emphasized, we fuse the noisy data x, with the low-frequency encodings of topology (A) and node features (X), both derived from our encoders EA and Eg. Specifically:
Graph classification tasks:
h (1+1) = h A * X,
(28)
where denotes element-wise multiplication.
Rationale. In graph-level tasks, each node representation needs to capture both local and global context that contributes the overall graph embedding. Element-wise multiplication, being stronger gating mechanism, preserves only those features that consistently appear across h A, and X. This forces the model to highlight global information shared among the node itself and its low-freuueccy topology/feature conditioning, making it well-suited for graph classification.
Node classification tasks:
h (11)
h
A X.
(29)
Here, the signals are combined by additive fusion.
Rationale. For node-level tasks, we more interested in preserving each node's specific feature contributions, while still injecting complementary low-frequency topology and feature signals. An additive fusion scheme allows h()) to retain node-specific details while gradually incorporating the smoothing effects of A and A' in a less restrictive way compared to dlement-wise multiplication. This suits node classification, where fine-grained differences at the node level often matter.
These fusion strategies ensure that the model explicitly incorporates the global (low-frequency) graph structure and feature information into each layer in a manner tailored to the prediction task at hand.
In summary, our enhanced U-Net architecture, combined with carefully designed GNN/MLP layers and the masking process, substantially improves the denoising capabilities on graph-structured data. By incorporating low-frequency structural priors
19

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
through conditional encoders, method avoids overfitting high-frequency noise and achieves robust performance in both node-level and graph-level representation learning tasks.
E. Parameter Analysis: Effect of hop Number
In our Multi-Scale Smoothing (MSS) objective (Equation (15)), the parameter hop controls the maximum neighborhood distance over which node representations are compared. Intuitively, each increment in hop extends the receptive field of the similarity terms S(b,), (k) h thereby encouraging the denoising model to maintain consistency in increasingly larger neighborhoods around the target node vo.
To investigate the influence of hop, we conduct node classification experiments on Computer and Photo, varying hop from 5. The rest of the hyperparameters remain fixed isolate the effect of hop. When hop=0, the loss reduces to comparing only the target node features Xo and its denoised counterpart &o. As hop increases, higher-order neighborhood embeddings (k) ho
come into play, guided by weights {w } to emphasize or de-emphasize each scale.
Results and Observations. Figure 6(a) and (b) illustrate how classification accuracy progresses with increasing hop. We observe a rapid improvement at low hop values: moving from hop=0 to hop=1 or 2 yields a substantial accuracy jump, indicating that incorporating even limited neighborhood information helps the model suppress high-frequency noise and maintain local smoothness. As hop grows or 4, there are stable gains beyond ho suggesting that moderate expansions to the receptive field further refine denoised embeddings and reinforce valuable low-frequency components. Finally, we also find potential diminishing returns hop=5, where performance gains become marginal, especially Photo, possibly due to increased redundancy or noisy distant neighbors.
Takeaways. Increasing hop generally benefits the diffusion based denoising process by capturing richer global information. However, the optimal setting appears lie in sweet spot where neighborhood expansions enhance low-frequency signal capture without overwhelming the model with uninformative high-frequency details from distant regions. Consequently, tnning hop can be an effective strategy for balancing the trade-off between local precision and global context in our MSS objective.
0.920
0.948
0.915
0.945
Accuracy
0.910
Accuracy
0.942
0.905
0.939
0.900
0.895
0.936
0.890
0.933
3
5
0
2
3
5
hop Number
hop Number
(a) Computer
(b) Photo
Figure 6. Classification accuracy versus mask ratio, with each bar presenting accuracy for mask ratios from 0.0 0.9.
F. Limitations and Future Work
While SDMG demonstrates strong performance across multiple benchmarks, our current formulation also reveals a few limitations. First, the framework explicitly prioritizes ore-freyynncy information, potentially underutilizing mid- or high- frequency components that might prove beneficial in certain tasks (e.g., detecting anomalies fine rre-grained community boundaries). Thus far, we assume that high-frequency signals are predominantly noise rather than source of discriminative cues. Extending SDMG to adaptively filter reconstruct different frequency bands-rather rather than imposing a strict low- frequency bias-may further boost representation quality in domains where subtle high-frequency details matter.
Additionally, although our model avoids computing exact graph spectra by using approximate encoders (thereby mitigating the cubic complexity of full eigendecomposition) some components- such as repeated neighborhood aggregations could
20

---

Smoothing Your Diffusion Models for Powerful Graph Representation Learning
still become costly extremely large dynamically evolving networks. Investigating scalable incremental strategies for updating low-frequency encoders in streaming scenarios would therefore be natural direction to handle real-time graph data.
Moving forward, future research could refine the frequency perspective by selectively reconstructing mid- or high-frequency signals based on task relevance, or by learning which frequency bands are essential in an end-to-end manner. Another intriguing avenue involves unifying our smoothing objective with other forms structural priors, such as motif- or subgraph-level constraints, capture richer relational information. Finally, incorporating partial or noisy labels-whhe they are available-couul yield superviid extensions of SDMG that seamlessly balance generation and discrimination, ultimately leading to more flexible and robust graph representation learning.
21


Reviews:
[{"review_id": "ynA1tpg4b8", "reviewer": "Reviewer_P6bv", "summary": "The authors study the problem of graph representation learning (GRL) via diffusion models. The authors argue that current graph diffusion models for representation learning are sub-optimal due to their focus on the high-frequency signals. However previous literature, including a preliminary study by the authors, show that low frequency signals are more important for GRL. As such, the authors design a newer diffusion model that emphasizes lower graph frequencies. This is done through the use of two components. The first is a lowe frequency encoder that attempts to to only synthesize the low frequency dignals of the node features and graph structure. The second component is a multi-scale smoothing loss that places more emphasis on reconstructing lower frequency signals. The authors report their performance on node and graph classification tasks, showing good performance.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for taking the time to review our manuscript and for the suggestions that have helped clarify our original contributions. Due to its overarching importance, we would like to start with Weakness 2:\n\n**Novelty** \n\nOur work is not on GRL as such, but GRL for use in a downstream classification task. As highlighted by Reviewers 9RUT and DKTW we are the first to consider potential misalignment and possible solutions in this context. Specifically：\n\n1. **Identifying and analyzing the misalignment between “graph generation” and “graph representation”.**  \n   - Existing diffusion-based GRL typically seeks to match the entire frequency spectrum, including irrelevant details. From both **theoretical and empirical** perspectives, we show that this reduces performance on downstream tasks.\n\n2. **Proposing a novel **multi-scale smoothing (MSS) loss** that mitigates misalignment between generation and discriminative representation.**  \n   - Even if one applies known LF filters (e.g., by truncating high frequencies or using a GNN), relying on MSE for element-wise reconstruction reintroduces high-frequency noise. (see Figure 4 and Theorem 3.1). Our MSS loss overcomes this by enforcing stronger penalties on LF errors throughout the diffusion steps. This mechanism is new and, from a **theoretical** standpoint (Theorem 4.1), leads the reconstruction toward LF signals without entirely suppressing useful high-frequency components.\n\nThus, we emphasize that the **key contribution** is **not** about which LF encoder one employs (we use a mature and efficient GNN simply because it is practical). Rather, it is **the combination of** LF and our MSS that resolves the inherent misalignment between generation and representation. \n\nWe thank you for your comments and will revise our manuscript to reflect this more clearly.\n\n**Question 1 and Issue 1: Performance Without Masking**\n\nBelow are our node/graph classification results without any masking. As shown, our method consistently improves over DDM on all datasets, with especially substantial gains in graph classification.\n\n*Node Classification* \n|Method|Cora|Citeseer|Pubmed|OGB-Arxiv|Computer|Photo|\n|-|-|-|-|-|-|-|\n|DDM|83.1|72.1|79.6|71.3|89.8|93.8|\n|SDMG w/o mask|83.6|73.2|80.0|71.7|90.4|94.1|\n\n*Graph Classification* \n|Method|IMDB-B|IMDB-M|PROTEINS|COLLAB|MUTAG|\n|-|-|-|-|-|-|\n|DDM|74.05|52.02|71.61|80.07|90.15|\n|SDMG w/o mask|76.03|52.05|73.17|82.23|91.58|\n\n**Question 2 & Issue 2: Statistical results (*p*-value)** \n \nIn our paper, we report the widely used mean and standard deviation as standard performance statistics. However, we are also happy to include more statistical tests (e.g., Wilcoxon Signed-Rank Test) with p-values.  In general, p < 0.05 indicates statistically significant improvements; while some datasets (e.g., Cora) show p > 0.05, most meet p < 0.05.\n\n|SDMG (Ours) vs.|Computer|Photo|Cora|MUTAG|COLLAB|IMDB-B|\n|-|-|-|-|-|-|-|\n|DDM|0.00098|0.00488|0.01367|0.00195|0.00098|0.00195|\n|GraphMAE|0.00098|0.00098|0.21582|0.00098|0.00098|0.02441|\n\n**Question 3 & Weakness 1: Why Diffusion Model Benefits**  \n\nThanks for your comment. We do not merely “insert” a low-frequency (LF) encoder and a multi-scale smoothing (MSS) module into generative models. Rather, we exploit the diffusion process’s iterative noise-injection–denoising scheme: at each timestep t, varying noise levels enable our MSS term to progressively emphasize coherent LF signals. This contrasts with single-step models (e.g., GraphMAE), which lack such iterative noise scheduling. While our MSS approach can be generalized elsewhere, the table below shows that it yields larger gains in our multi-step diffusion framework, aligning with our core viewpoint. \n||photo|computer ||photo| computer|\n|-|-|-|-|-|-|\n|GraphMAE|93.6|88.6|SDMG w/o MSS|93.4|89.4|\n|GraphMAE+MSS|93.9|89.4|SDMG|94.7|91.6|\n|Relative Improvement|0.32%|0.90%|Relative Improvement|1.36%|2.44%|\n\n**Question 4 & Issue 3: Runtime Efficiency**  \nOur method does not suffer from excessive runtime overhead (training + inference)—in fact, it often converges faster. By prioritizing global smooth signals (e.g., via our MSS), SDMG quickly encodes downstream-relevant information and requires fewer total epochs. All experiments were run on a single NVIDIA H100.\n\n|Method|Cora (epochs)|Citeseer (epochs)|Pubmed (epochs)|\n|-|-|-|-|\n|GraphMAE|13.32s (1500)|3.58s (300)|20.69s (1000)|\n|DDM|13.12s (800)|11.23s (350)|10.35s (300)|\n|SDMG|5.88s (200)|9.39s (150)|15.23s (100)|\n\n**Weakness 3: Notation Clarification** \n\nThank you for the suggestion. As noted (page 6, line 290), $\\mathcal{E}^x_{\\theta}$ is a GAT encoder for LF node features.  $f = \\mathcal{E}^A_{\\phi}$ is an MLP for topology. $\\mathcal{A}$ is a learnable embedding matrix approximating the LF spectrum. $f$ can take various graph-structure inputs (e.g., adjacency), and we use random-walk positional encodings for preserving LF information. We will expand these details (including the mask strategy) in the final version.", "claims_and_evidence": "The claims are mostly well supported by evidence. This includes both the preliminary study (Finding 1 and 2 in Section 3) and both proposed components (see the ablation study in Table 3).\n\nHowever, I think the authors should note that the assumption that only the low frequency signals are crucial doesn't hold true for all graphs. For example, for heterophilic graphs, it's known that incorporating high frequency signals is helpful [1].\n\n[1] Luan, Sitao, et al. \"Revisiting heterophily for graph neural networks.\" Advances in neural information processing systems 35 (2022): 1362-1375.", "methods_and_evaluation_criteria": "The baselines and evaluation make sense for this task and are commonly used for GRL.", "theoretical_claims": "The theoretical claims in Section 3 look good to me.", "experimental_designs_or_analyses": "The experimental results and analyses are mostly fine.\n\nHowever, I think there are a few crucial issues:\n1. For node classification, the authors use an additional masking strategy to improve performance. This is quite problematic in my opinion, as Figure 6 shows it can have quite a large effect on performance (I assume mask\\%=0 corresponds to an ablation of it.) Without it, the performance is very similar to DDM, another diffusion based approach which as far as I can tell doesn't use any masking. To me, this calls into question if most of the improvement of SDMG over DDM on node classification is due to this inclusion.\n2. The performance improvement over simpler methods is quite small and often seems not statistically significant. For example, GraphMAE can achieve only slightly worse performance than SDMG despite being much simpler. As such, I recommend including whether each of the results are better than baselines in a statistically significant way (i.e., include the p-values).\n3. No efficiency experiments are shown. It is well known that diffusion models are quite slow. As such, it's important for the authors to show the runtime needed for training + inference of SDMG. Because even if a model can achieve better performance, poor efficiency can make them impractical to realistically use. As such, detailing its efficiency against simpler methods like GraphMAE is important.", "relation_to_broader_scientific_literature": "I think this paper is relevant for the field of graph representation learning. Specifically the emphasis on the low frequency signals is a useful insight.", "essential_references_not_discussed": "A number of references are missing.\n\nFor graph generative models, multiple models consider the graph spectrum. This includes [1, 2] (note that [2] was available since early 2024). To be clear, these methods are not designed for GRL. However, they are still generative models that operate on the graph spectrum and should be cited. Furthermore, these methods focus on the lowest k eigen-values/vectors. Of particular interest is [2] which is a diffusion model. The paper should be updated with a discussion of these papers.\n\nFurthermore, the use of the $q$ smallest eigenvectors of the laplacian is quite common in graph literature. See [3]. From my understanding SMDG relies on approximation of these eigenvectors, but the motivation and principle remains the same.\n\n\n[1] Martinkus, Karolis, et al. \"Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators.\" International Conference on Machine Learning. PMLR, 2022.  \n[2] Minello, Giorgia, et al. \"Generating Graphs via Spectral Diffusion.\" The Thirteenth International Conference on Learning Representations, 2025.  \n[3] Rampášek, Ladislav, et al. \"Recipe for a general, powerful, scalable graph transformer.\" Advances in Neural Information Processing Systems 35 (2022): 14501-14515.", "other_strengths_and_weaknesses": "There are a few notable weaknesses with the paper in its current form.\n\n1. I find the emphasis on diffusion models to be a little strange. As noted by the authors, the idea that low frequency information is most helpful for graph learning is very well known in Graph ML. Specifically, many works have drawn the connection between GNN and low pass filters. The contributions in this paper reflect thus reflect an interest in focusing on low frequency information. However, this has nothing to do with diffusion. As such, both the LE and MSS components can be included in other architectures as well and are not specific to diffusion models. For example, I see no reason why they can't be incorporated into a method like GraphMAE (please correct me if I'm wrong). I'm therefore quite confused by the framing of this paper as I'm unsure what it has to do specifically with diffusion models. To me, it seems like it would apply to any generative methods.\n\n2. Building on the previous weakness, I find the novelty to thus be quite low. This is particulary true in regards to the encoders. $\\mathcal{E}\\_{\\theta}^x$ is just a GNN which is used by all frameworks. $\\mathcal{E}\\_{\\phi}^x$ considers an approximation of the $q$ smallest eigenvectors. Many works consider pre-computing the $q$ smallest eigenvectors of the laplacian and using them as input to a GNN/Graph-Transformer [1]. Therefore, these contributions offer very little novelty over existing methods.\n\n3. In my opinion, some crucial information is ommitted from the main text and either not mentioned entirely or put in the appendix. The main text should be self-contained (within reason of course). Some examples include: **(a)** What is the design of $\\mathcal{E}\\_{\\phi}^x$ exactly? In 4.2, the authors say that they use a neural network function $f$? What is the design of $f$, what is it's input? Furthermore, how is $\\mathcal{A}$ defined, is it simply a set of learnable embeddings? **(b)** For node classification, the authors use an additional masking strategy to improve performance. However, this is not mentioned in the main text at all. I expound on this more in the \"Experimental Designs Or Analyses\" section. \n\n\n[1] Rampášek, Ladislav, et al. \"Recipe for a general, powerful, scalable graph transformer.\" Advances in Neural Information Processing Systems 35 (2022): 14501-14515.", "other_comments_or_suggestions": "1. Please include a mention of the node masking in the main text.\n2. Please explicitly define $\\mathcal{E}_{\\phi}^x$.", "questions_for_authors": "1. What is the performance of SDMG on node classification w/o the node masking? This allows for a fairer comparison with DDM.\n2. Can you include the p-values of SDMG's improvement over other methods like DDM and GraphMAE?\n3. Can the contributions in this paper be applied to other generative frameworks? If not, why is it specific to diffusion models?\n4. Can you show a runtime comparison of SDMG compared to other methods (e.g., GraphMAE)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mza5CArKO9", "reviewer": "Reviewer_Q4ro", "summary": "The authors introduce a new diffusion-based self-supervised framework for graph representation. It addresses the issue that minimizing generation-based learning objectives can overfit high-frequency noise instead of capturing important global structures. To overcome this, the authors propose (1) learnable encoders that approximate low-frequency components of both node features and graph topology, and (2) a new multi-scale smoothing (MSS) loss emphasizing multi-hop similarity for more semantically meaningful information. Experiments show the proposed method outperforms baselines in node-level and graph-level tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback. We appreciate your time and have addressed your comments below.\n\n**Question 1: Clarification on noise addition in Figure 1.**\n  \nThe primary goal of Figure 1(a) is to illustrate the misalignment between generative reconstruction objectives and representation learning. As shown, reconstructing only a small fraction of information (e.g., the lowest 5% frequency components) is sufficient to achieve competitive downstream performance. Conversely, reconstructing the full frequency spectrum (100%), as most current methods do, results in performance degradation. We attribute this to diffusion models allocating their limited information capacity to unnecessary reconstruction of high-frequency noise.\n\nTo further confirm this hypothesis, we performed the experiment shown in Figure 1(b), where we explicitly added high-frequency Gaussian noise with variance $\\sigma^2 = 5$, specifically sampling from $\\mathcal{N}(0, 5)$. The rationale is that, if the diffusion model could effectively prioritize reconstructing task-relevant (low-frequency) information under limited capacity, its performance should remain robust despite the addition of irrelevant high-frequency noise.\n\nIndeed, in Figure 1(b), we observe that models focusing solely on reconstructing a small fraction of low-frequency information remain robust, maintaining strong downstream performance even after noise injection. In contrast, reconstructing the full frequency spectrum leads to a significant performance drop under added noise conditions. This clearly indicates that under limited capacity, enforcing full reconstruction exacerbates the misalignment between generation and representation. Our theoretical analysis in Theorem 3.1 further supports and explains this phenomenon.\n\nThank you for your insightful comment. We will revise the final version accordingly.\n\n**Question 2: Clarification on Interpretability implications of focusing on low-frequency signals.**\n\nLow-frequency components summarize global structural information and thus can offer valuable interpretive insights in **certain** applications. For instance, in relatively homogeneous networks such as social networks, low-frequency signals help explain group memberships in clustering tasks by highlighting shared global characteristics among nodes. However, interpretability is inherently task- and context-dependent. In applications like biological anomaly detection or extreme event prediction, relying solely on low-frequency information may not provide sufficient interpretability, as critical insights could reside in higher-frequency or local features.\n\n**Question 3: Clarification on the Aggregation Mechanism in MSS loss.**\n\nThe construction of $\\mathbf{h}$  in the  $R_{MSS}$ loss (Eq. 13) indeed employs an aggregation strategy similar to those used in standard GNN layers. Note that, we leverage this aggregation within our proposed MSS loss specifically to capture multi-scale reconstruction information across different neighborhood ranges. By explicitly comparing node representations at multiple neighborhood scales, our MSS loss uniquely encourages the model to prioritize smooth, global (low-frequency) signals during reconstruction, which differentiates it from conventional aggregation strategies that primarily aim for node-level feature updates.\n\n**Weakness: Clarification on Frequency selection and potential extensions.**\n\nWe acknowledge the reviewer’s point regarding tasks that might benefit from high-frequency information. While our current approach intentionally emphasizes low-frequency signals due to their effectiveness in mitigating the misalignment between generation and representation, we do not exclude the potential value of mid- or high-frequency components. Instead, we view our method as providing a foundational perspective that future studies can adapt or extend based on specific task requirements. Future work may incorporate adaptive frequency selection mechanisms, which could better accommodate scenarios such as anomaly detection or extreme event prediction.", "claims_and_evidence": "The paper’s main claims on the importance of low-frequency components and the effectiveness of SDMG are backed by thorough experiments and theoretical arguments across multiple benchmark datasets.", "methods_and_evaluation_criteria": "Yes. The authors verify their method on widely accepted node and graph classification benchmarks (Cora, Citeseer, PubMed, etc.).", "theoretical_claims": "The authors provide two core theorems on how low-frequency reconstruction aligns with improving representations. The proofs are clearly structured and do not show obvious errors; they seem consistent with known results in the field.", "experimental_designs_or_analyses": "The experimental design and analyses mainly follow standard practices (DDM in 2024) and appear valid with no evident methodological flaws.", "relation_to_broader_scientific_literature": "The paper extends prior diffusion-based generative models into frequency-aware graph representation learning. It bridges concepts from spectral GNNs with recent advances in diffusion-based self-supervision. This integration builds on findings about overfitting to high-frequency noise in graph tasks and proposes a new approach by introducing novel multi-scale smoothing loss with low-frequency encoders. This work provides a novel perspective for existing diffusion model-based graph learning works.", "essential_references_not_discussed": "No clearly essential prior works appear to be omitted.", "other_strengths_and_weaknesses": "Strengths:\n\nThe paper’s theoretical and experimental analysis effectively demonstrates how pure MSE-based reconstruction can diverge from learning discriminative representations.\n\nApplying diffusion models to graph representation learning is an emerging yet crucial research topic that remains relatively underexplored.\n\nThe fusion of a spectral perspective with diffusion modeling introduces a novel angle for graph learning field.\n\nWeaknesses:\n\nThe multi-hop neighborhood weights and choice of how many low-frequency components to encode may require careful tuning on different graphs.\n\nMainly emphasizing low-frequency signals might not be optimal for tasks that rely on high-frequency information (e.g., anomaly detection or graph isomorphism), and although the authors briefly mention the potential value of mid/high-frequency components, the current approach remains mainly focused on low-frequency prioritization.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Can you provide more details on the experimental setup in Figure 1? Specifically, why did you add noise in Figure 1(b), and what variance did you use for the Gaussian noise?\n\nIn real-world applications such as biomedicine, interpretability can be crucial. Do you think the key frequencies, like the low-frequency components shown to be critical, could offer interpretive insights into the model’s outputs?\n\nIn the $R_{MSS}$ loss proposed in Eq. (13), is the construction of $h_0$ the same as in existing GNN layers, or does it employ a new aggregation mechanism?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sZ7NajtvXy", "reviewer": "Reviewer_9RUt", "summary": "The authors reveal that purely generation-oriented objectives can conflict with recognition goals, demonstrating that excessive non-smooth-frequency reconstruction can harm representation quality. Specifically, they systematically investigate how reconstructing different parts of the graph frequency spectrum affects downstream classification tasks. Their findings show that reconstructing the full spectrum significantly decreases classification performance. Due to the computational burden of direct spectral decomposition, the authors propose approximate methods, such as a novel low-frequency encoder and a multi-scale smoothing loss, focusing on reconstructing the spectral components most relevant to downstream tasks. Both theoretical analysis and empirical results demonstrate the effectiveness of the proposed approaches.\n\n## update after rebuttal\nThe authors have adequately addressed my concerns; therefore, I would like to maintain my positive evaluation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive and positive feedback; we appreciate your insights and have addressed your comments below.\n\n**Question 1 and Weakness 1: Clarification on Masking Strategy Motivation.**  \n\nThe masking strategy has two primary motivations: enhancing robustness and promoting learning discriminative global representations. Recent literature (Daras et al., 2022) on diffusion-based models highlights that carefully designed masking or corruption processes encourage the model to selectively prioritize the most meaningful structural information during reconstruction. Specifically, masking introduces controlled information degradation, forcing the model to reconstruct from limited context, which aligns naturally with our objective of focusing on smooth, globally relevant graph structures. This process prevents the model from simply memorizing trivial local or high-frequency noise features. Consequently, such a masking strategy guides the model to leverage global graph topology, improving performance on downstream tasks through more robust and semantically meaningful representations.\n\nWe have also demonstrated the performance of our model without masking; please refer to our response to Reviewer P6bv, Question 1, for detailed comparisons. Based on your suggestion, we will include further clarifications regarding this aspect in the final version of the manuscript.\n\n**Question 2: Clarification on Frequency Component Definition.**\n\nThank you for the insightful question. Yes, in Section 3 (the Investigation Section), the \"5% lowest frequency components\" indeed refers specifically to the eigenvalues of the Laplacian matrix derived from the normalized adjacency matrix, which naturally captures the global structure of the graph.\n\nWe adopt the Laplacian spectrum because it reflects the smooth, global signals of the graph. Please note that we perform the graph Fourier transform only in the Investigation section to validate our hypothesis. In the Method section, we instead propose an approximate computation method for extracting low-frequency information, which is more efficient and scalable.\n\n**Question 3: How to extend SDMG beyond classification, such as social community detection or traffic flow prediction.**\n\nExtending our framework to tasks such as social community detection or traffic flow prediction would involve a few targeted modifications that build on our core approach of low-frequency emphasis and multi-scale smoothing. For social community detection, one could replace the classification head with a clustering-oriented output by incorporating a clustering loss (e.g., modularity maximization or a spectral clustering regularizer) to directly optimize the learned embeddings for community structure, or by using the generated representations as input to an external clustering algorithm. Additionally, integrating community-aware regularizers into the diffusion process could further enhance the capture of global structural patterns. \n\nFor traffic flow prediction, a regression task with significant temporal dynamics, the output layer would need to be restructured to produce continuous predictions (using, for instance, a mean squared error loss), and it would be beneficial to integrate temporal modeling components (such as recurrent layers, temporal convolutional networks, or attention mechanisms) into the denoising decoder. These modifications would allow our method to adapt its emphasis on smooth, global features and multi-scale consistency to effectively address the unique challenges of these different tasks.\n\n**Weakness 2: Clarification on Balancing MSS Loss and Low-Frequency Encoders.**\n\nThank you for your insightful comment. While Table 3 shows a larger improvement from introducing the low-frequency encoders compared to the MSS loss alone, we believe these components have distinct yet complementary roles. The low-frequency encoders provide a robust initial approximation of the global, low-frequency signals from the graph topology and node features, which are critical for guiding the denoising process. In contrast, the MSS loss fine-tunes the representations by enforcing multi-scale consistency and preventing the reintroduction of high-frequency noise during training. Although its standalone impact may seem smaller, the MSS loss is essential for maintaining overall representation quality over time. Moreover, adding extra low-frequency encoding (e.g., additional GNN layers) to the final node representations would likely be redundant, as our ablation studies show that the best performance is achieved when both the low-frequency encoders and the MSS loss are used together.\n\nThank you again for your helpful comment. We will incorporate the necessary modifications in the final version.", "claims_and_evidence": "The submission’s claims are validated by evidence from both theoretical analysis and extensive experiments.", "methods_and_evaluation_criteria": "The proposed methods make sense for this task. The evaluation criteria rely on standard graph and node classification datasets, which are commonly used benchmarks in representation learning.", "theoretical_claims": "I checked the proofs for both theorems. They are clear and logically sound, and I found no issues.", "experimental_designs_or_analyses": "The experiments are clear and sound. However, I was a bit confused about the masking strategy experiments presented in the Appendix because I didn’t fully understand their motivation. Please see my questions for more details.", "relation_to_broader_scientific_literature": "Comment 1: As far as I know, DDM (Yang et al., 2024) is the first diffusion model-based graph representation learning method. However, it focuses on generation-based reconstruction and may suffer from misalignment with recognition tasks. Unlike DDM, the submission work shifts the focus to low-frequency signals, effectively aligning the diffusion process with downstream objectives.\n\nComment 2: While low-frequency features have been shown to benefit existing GNN-based classification (Hoang et al., 2021; Liu et al., 2022), this submission work shows that traditional low-pass filters (like GNN layers) do not adequately capture these signals within diffusion models. The paper further addresses this gap by proposing a novel multi-scale smoothing loss, which convinced me that the proposed method is innovative in graph representation learning.", "essential_references_not_discussed": "I did not notice any crucial references missing.", "other_strengths_and_weaknesses": "Strengths 1: It’s the first work to address the key challenge of misalignment between reconstruction and classification in the graph domain.\n\nStrengths 2: The introduction of the multi-scale smoothing loss and low-frequency encoders is innovative and interesting.\n\nStrengths 3: The work demonstrates an interesting finding that existing low-frequency filters in the denoise decoder do not prevent the reconstruction of irrelevant high-frequency features, which motivates the new learning objective.\n\nWeakness 1: A minor weakness is that there is not enough discussion about the motivation for the masking strategy. I detail it more in the \"Questions\" part.\n\nWeakness 2: As shown in Table 3, the improvement provided by introducing low-frequency encoders appears to be greater than that from the MSS loss. This observation is interesting, but the paper lacks sufficient exploration and explanation of this point, which might raise concerns about the proposed learning objective. Moreover, these observations lead me to wonder whether applying additional low-frequency encoding (such as GNN layers) to the final node representations might be beneficial, or if the design of the MSS loss (Equation 13) already serves the function of GNN layers?", "other_comments_or_suggestions": "Some figure captions (e.g., Figure 2) have small font sizes that might affect legibility.", "questions_for_authors": "Question 1: What is the motivation behind the masking strategy? Specifically, how does the masking interact with the low-frequency reconstruction process, and why is it expected to improve performance on node-level tasks?\n\nQuestion 2: If I understand correctly, Section 3 refers to the \"20% lowest low-frequency components\" as the first 20% of the full spectrum. Does this imply that, in Equation (6), $q$=$d$*0.2? If not, please explain how $q$ is calculated.\n\nQuestion 3: What modifications would be needed to extend the framework beyond classification, such as social community detection or traffic flow prediction?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VpbKQ7UVGf", "reviewer": "Reviewer_DktW", "summary": "The paper presents a new diffusion-based approach for self-supervised graph representation learning (SDMG). Rather than reconstructing all graph details, which often leads to the overfitting of high-frequency noise, the authors provide both theoretical and empirical evidence that focusing on low-frequency components yields more robust, globally informative representations. To this end, SDMG integrates the proposed low-frequency encoders for node features and graph topology along with a new multi-scale smoothing loss that enforces consistency across multiple neighborhood scales. Extensive experiments on node-level and graph-level benchmarks demonstrate that this targeted reconstruction strategy leads to state-of-the-art performance, while ablation studies validate the contributions of each component, demonstrating the method’s novelty and effectiveness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable input and positive comments, and we have carefully addressed your comments in our responses below.\n\n**Question 1: Clarification on the Meaning of “–” Symbol on Table 1.**  \n\nIn Table 1, the “–” symbol indicates that the corresponding method either exceeded the available memory, had prohibitive computation time on that particular dataset, or that the original authors did not report performance on that dataset. We will clarify this notation in the revised manuscript to avoid any ambiguity.\n\n**Question 2: Clarification the U-Net Representations on Equations (11) and (12).**  \n\nFor Equations (11) and (12), we only concatenate the activations from the U-Net’s up-sampling layers. However, the information from the down-sampling layers is fully preserved through skip connections that directly link these layers to the up-sampling path. As a result, although our final representation is built solely from the up-sampling activations, it inherently incorporates the local features captured during down-sampling, ensuring that both local and global information is maintained without redundancy. Thank you for pointing this out, and we’ll revise the final version to make this clearer.\n\n**Question 3: Clarification on the Notation of Encoding “Z” and Node Representation “H”.**\n\nIn Theorem 3.1, the encoding “**Z**” typically refers to the intermediate representation produced by our graph encoder. For example, mapping the input graph to a low-dimensional space. On the other hand, “**H**” denotes the final node representation extracted from the activations of the U-Net. In the context of our paper, these two representations can be considered equivalent. Based on your suggestion, in the revised version, we will harmonize our notation by clearly stating that H is derived from **Z**, ensuring consistency throughout the paper.\n\n**Weakness 1: Clarification on Gaussian Noise Intensity Used in Figure 1.** \n \nWe appreciate your observation regarding the noise intensity. In our experiments for Figure 1, we sample Gaussian noise from a normal distribution, $\\mathcal{N}(0, 5)$, to simulate realistic high-frequency perturbations. We agree that specifying this value explicitly would improve clarity, and we will include these details in the final version. Furthermore, please refer to our response to *Reviewer Q4ro*, Question 1, where we provide a detailed explanation of the motivation behind Figure 1.\n\n**Weakness 3: Clarification on Reconstructing Only Node Features.**  \n\nThank you for your insightful comment. As shown in Figures 3(b) and 3(d), we demonstrate that reconstructing the smooth aspects of the graph structure benefits downstream tasks. Accordingly, in our work we also introduce a low-frequency encoder for graph topology to extract global, smooth structural information. Thus, when reconstructing node features, our SDMG essentially leverages both topology and node feature information to capture the distribution of node features. Nonetheless, we agree that an alternating reconstruction strategy for node features and graph topology is an interesting idea, and we plan to explore this approach in future work.", "claims_and_evidence": "The paper’s main claims are supported by solid experimental evidence.", "methods_and_evaluation_criteria": "Yes, the methods and evaluation criteria are suited to the problem.", "theoretical_claims": "Yes, I checked the proofs for Theorem 3.1 and Theorem 4.1. They make sense to me: Theorem 3.1 provides strong theoretical support for the observed misalignment between generation and representation learning, while Theorem 4.1 directly validates the equivalence between the proposed MSS loss and a low-frequency filter.", "experimental_designs_or_analyses": "Yes, I reviewed all experimental designs and analyses, particularly the preliminary experiments and ablation studies, and found them to be sound with no significant issues. For example, the conclusions in Figures 1 and 3 are robust and intuitively reasonable since, for recognition tasks, reconstructing every detail is not always beneficial. Recent work in computer vision is just beginning to address this issue from different perspectives, and I am pleased to see such promising prospects in graph tasks. The only minor issue is that the authors should summarize the findings or conclusions more clearly and concisely in the captions of figures and tables.", "relation_to_broader_scientific_literature": "The paper’s key contribution is being the first to reveal the misalignment between graph generation and recognition. Previously, researchers trained generative diffusion models and then extracted the learned representations for classification or prediction. By uncovering this misalignment, the paper could point to a new direction for diffusion model–based graph representation learning.", "essential_references_not_discussed": "The paper provides sufficient related work to understand its key contributions. \n\nI have one minor suggestion: consider discussing a recent vision paper [R1] in ICML 2024 that explores the relationship between recognition and generation. Although that work comes from a different domain and uses a different perspective, it shows that representation and recognition can misalign in visual data. This insight could help readers better understand the misalignment observed in graph data.\n\n[R1] Balestriero, Randall, and Yann LeCun. \"How Learning by Reconstruction Produces Uninformative Features For Perception.\" in ICML 2024.", "other_strengths_and_weaknesses": "**Other Strengths:**\n\n1. I find this work interesting as it highlights a new potential direction for representation learning—designing a suitable loss function to encourage the reconstruction of key aspects rather than all details, which appears promising for improving downstream tasks like classification and prediction.\n\n2. Unlike existing diffusion-based methods, this paper is the first to demonstrate that the graph generation optimization objective is not entirely suited for graph downstream tasks.\n\n3. Instead of using a standard generation-oriented learning objective, the authors propose a new loss function that effectively prioritizes the reconstruction of global and smooth information.\n\n**Other Weaknesses:**\n\n1. In Figure 1, the authors claim that after adding Gaussian noise, the model focusing on a narrow low-frequency band remains robust, while the model with full-spectrum reconstruction collapses in performance. However, the paper does not specify the noise intensity (e.g., the value of σ), which may raise concerns about the robustness.\n\n2. The related work section is placed in the appendix, which might confuse readers who are not familiar with diffusion models or graph learning; this issue could be addressed in the camera-ready version.\n\n3. The current approach focuses only on reconstructing node features, which may limit the model’s ability to capture topological geometry. Exploring the reconstruction of both graph topology and features, for example, by alternating reconstruction [R2], might allow the model to learn more meaningful structural-semantic information。\n\n[R2] Jo, Jaehyeong et.al,. \"Score-based generative modeling of graphs via the system of stochastic differential equations.\" In ICML 2022.", "other_comments_or_suggestions": "In page 15, line 773, one citation is not being displayed correctly.", "questions_for_authors": "1. In Table 1, what does the “–” symbol represent? Does it indicate that the method exceeded available memory or that the computation time was prohibitive? \n2. In Equations (11) and (12), did you only concatenate the activations from the U-Net’s up-sampling layers, or did you also include representations from the down-sampling layers? If I understand correctly, the down-sampling information is preserved solely via skip connections into the final representation?\n3. In Theorem 3.1, is the encoding “Z” equivalent to the node representation “H” used elsewhere in the paper? If so, it might be clearer to use consistent notation to avoid confusion.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents a diffusion based self-supervised method for graph representation learning that can be applied to downstream classification tasks. The main message of the work is that generation-based learning objectives may be misaligned from downstream objectives and can focus too much on high-frequency signal or noise which is not useful. Hence, focusing the representation learning on low-frequency signal better aligns the representation learning with downstream objectives.

Most reviewers appreciated this message and found the paper to be a meaningful contribution, with solid theoretical contributions and good experimental validation. One reviewer noted by the end of discussion that the novelty of the contributions may be limited, since the separation of low and high frequencies is common in graph learning, and that performance increases were overall modest. Other reviewers saw these points more favorably, and in my view are not issues that preclude acceptance. Several reviewers asked for better motivation for the masking strategy, and to clarify how much of the performance improvement is attributed to masking as opposed to other components, which the authors did provide during the discussion.

Given the majority consensus that the contribution is sound and valuable, I am recommending acceptance.
Decision: Accept (poster)