Generating  Graphs  via Spectral Diffusion
Authors: Giorgia Minello
Alessandro Bicciato
Luca Rossi
Andrea Torsello
Luca Cosmo
Venue: ICLR 2025 Poster (2025)
Keywords: graph neural networks
laplacian
eigendecomposition
spectrum
diffusion model
generative model
Primary area: generative models

Abstract: In this paper, we present GGSD, a novel graph generative model based on 1) the spectral decomposition of the graph Laplacian matrix and 2) a diffusion process. Specifically, we propose to use a denoising model to sample eigenvectors and eigenvalues from which we can reconstruct the graph Laplacian and adjacency matrix. Using the Laplacian spectrum allows us to naturally capture the structural characteristics of the graph and work directly in the node space while avoiding the quadratic complexity bottleneck that limits the applicability of other diffusion-based methods. This, in turn, is accomplished by truncating the spectrum, which, as we show in our experiments, results in a faster yet accurate generative process, and by designing a novel transformer-based architecture linear in the number of nodes. Our permutation invariant model can also handle node features by concatenating them to the eigenvectors of each node. An extensive set of experiments on both synthetic and real-world graphs demonstrates the strengths of our model against state-of-the-art alternatives.

Published as a conference paper at ICLR 2025
GENERATING GRAPHS VIA SPECTRAL DIFFUSION
Giorgia Minello
Alessandro Bicciato
Ca' Foscari University giorgia www.iia..iieeeeeeme www..mmmeeeiiieeeeeeeemmmmmmeeee
Ca' Foscari University alessandro www.iatosuniie........
Luca Rossi
Andrea Torsello
The Hong Kong Polytechnic University luca. rossi @ yu edu.hk hk
Ca' Foscari University
andrea torseltorellll wow.eeloouunieee... it
Luca Cosmo
Ca' Foscari University
luca. coreacccse@@eieeccoo
ABSTRACT
In this paper, we present GGSD, a novel graph generative model based on 1) the spectral decomposition of the graph Laplacian matrix and 2) diffusion process. Specifically, we propose to use a denoising model to sample eigenvectors and eigenvalues from which matrix. Using the Laplacian spectrum allows us to naturally capture the structural characteristics of the graph and work directly in the node space while avoiding the quadratic complexity bottleneck that limits the applicability of other diffusion- based methods. This, turn, accomplished by truncating the spectrum, which, as we show in our experiments, results in faster yet accurate generative process, and by designing a novel transformer- based architecture linear in the number of nodes. Our permutation invariant model can also handle node features by con- catenating them
reconstruct the graph Laplacian and adjacency
eigenvectors of each node. An extensive set of experiments both synthetic and real-world graphs demonstrates the strengths of our model against state-of oftic-ff-th-arrrt alternatives.
INTRODUCTION
Generating realistic graphs by learning from distribution of real-world graphs has gained increas- ing attention from researchers in many fields due to its wide range of applications. For instance, synthetic graph generation plays crucial role in drug design Homez-Boobaane et al. (2018): Li
ctt al. (201 8a); You et al. (2018a) as well as network science Watts & Strogatz (1998) Leskoved al. 2010); Albert & Barabási 2002)).
Seminal graph generation approaches date back to the 1960s and rely on simple stochastic processes, limiting their ability to capture complex dependencies seen in real-world networks. For example, the Barabási-Alberrt Albert & Barabasi (2002) and Kronecker Leskovec et al. (2010) graph models specifically designed to generate graphs belonging specific families and lack the ability to learn directly from observed data. While these models may excel in capturing a set of predefined properties, they are often unable represent wider range aspects observed in real-world graphs. In addition, in several domains, network properties are largely unknown, which further limits the applicability of these techniques. For instance, the Barabási-Albert model Albert & Barabási] (2002) allows to create graphs that exhibit the scale-free nature found in empirical degree distributions, however it is unable to capture other facets real-world graphs, e.g., community structure. While flurry of new models attempting address these shortcomings have been introduced by the network science community (see Drobyshevskiy & Turdakov (2019) for a recent review), these methods often lack the ability leam to mimic the characteristies of given dataset. This in turn limits the expressivity and fidelity of generated graphs and thus the range of possible applications of graph generative models.

---

Published as a conference paper at ICLR 2025
In this paper, we introduce new model for Generating Graphs via Spectral Diffusion (GGSD). The ideas underpinning our approach are 1) to represent the graph using the eigendecomposition of its ceffuusion-bacttt approach to learn to sample sets of eigenvalues and
Laplacian matrix and 2) to use eigenvectors from which a graph adjacency matrix can be reconstructed. Doing this allows us to work directly in the space of nodes while overcoming the computational bottleneck (quadratic in the number of graph nodes) of other methods that follow a similar approach Vignac et al. (2022). By limiting the number of eigenvalues and eigenvectors used to reconstruct the graph adjacency matrix, we reduce the complexity of the iterative denoising process to be linear with respect to the number of nodes while, at the same time, having a representation tablet to encapsulate graph structural characteristics. Moreover, unlike other models conditioned on spectral representations MMattnnus et al. (022), our model also allows us to robustly condition the generation of new graphs on desired spectral properties (subsets of eigenvalues and/or eigenvectors) at inference time.
The remainder of this paper is structured as follows. Section reviews the related work, while Sections 3 introduces the necessary background on denoising diffusion models. We introduce our graph generative model in Section 4 and we present the experimental evaluation against state-of- the-art alternatives in Section 5 Finally, Section conclludes the paper.
2 RELATED WORK
In contrast to the image and text domains, where the development of generative models is well understood and established, graphs introduce a series of additional challenges.
The first issue is the non- con-unqquessss of graph representations, i.e., if a graph contains n nodes, there exist up to n! possibly distinct adjacency matrices that serve equivalent representations of the same graph, since there is no reason to prefer a particular node order. Ideally, a generative model should assign equal probability to each of these n!! adjacency matrices. Another crux lies in the size of the output space, which is quadratic in the number of nodes, and that quickly becomes a bottleneck when dealing with large graphs. Graph generative models should also be able consider the existence of dependencies and relationships between nodes and edges, rather than treating them as independent, e-g., in social networks the likelihood of two nodes being connected is often higher when they have common neighbors. Finally, standard machine learning techniques designed for continuously differentiable objective functions are unsuitable to be directly applied discrete graph structures Guo & Zhao (2022).
Seminal graph generative model approaches seek to address these problems, yet focus only on the generation of graphs displaying limited set of structural characteristics. These initial methods rely on identifying common characteristics in real-world graphs, such as degree distribution, graph SaaffiiiennFaaauuttttt Faloutsos (2008), and then generate synthetic graphs through the application of a set of heuristic rules Leskovec et al. (2010); Leskovee & Faloutsos (2007); Erdös] et al. (1960); Albert & Barabási] (2002). Although these models can produce synthetic graphs with the given desired features, they are limited in their ability to generate node features as well as novel structural patterns.
diameter, and clustering coefficient]
A breakthrough in this field has been marked by the recent progress in deep learning models such Variational Auto Encoders (VAEs) Kingma & Welling (2013), Recurrent Neural Networks (RNNs) Zaremba et al.) (2014) and Generative Adversarial Networks (GANs) [Goodfellow et al. 2014 In this context, we encounter models commonly referred to as general-purpose deep graph generative models, such as GraphRNN You et al. (2018)) and GRAN Liao et al. (2019), which ex- ploit deep architectures to learn the graphs distribution. Even though they represent a step forward in the field of generative graph models, most of them are limited by exclusively focusing on the graphs structure. Further, approaches of this type adopt evaluation metrics based only on graph statistics, like degree distribution or clustering coefficients, and thus overlook omit the assessment of the generated node features.
Node and edge features are instead considered in a number of methods developed specifically for the generation of molecules, indeed one of the most promising application scenario for modern graph generation approaches. Models falling in this domain. referred to molecule graph generative models, exploit deep architectures such as GAN in De Cao & Kipf (2018) or RNN in Popova et al. (019) as well as other generation strategies (e.g., graph normalizing flows] Luo al. 2021) or the
2

---

Published as a conference paper at ICLR 2025
combination of different approaches. For instance, Shi et al. (2020) combines the advantages of both autoregressive and flow-based methods.
Nevertheless, there other deep learning approaches beyond molecule graph generative models that are capable of generating graphs with node and edge features even though the evaluation itself often still based on molecule generation task. For instance (iimoovsky & Komodakis (2018) and Grrver et al.] (2019) propose general deep generative models for graphs based on variational autoencoders. The main drawback of these architectures is that they are specialized and limited to small-scale graphs with low-dimensional feature space Yoon et al. (202)).
Another category of graph generative models takes cues from the score-based generative modeling work of Song & Ermon (2019) to define diffusion models for graphs. For instance, in Huang et al. (2022) the authors propose forward diffusion process, specifically a continuous-time generative diffusion process for permutation invariant graph generation. Similarly, Niu et al. (2020) introduce different diffusion model named Edge-wise Dense Prediction Graph Neural Network (EDP-GNN) which uses Gaussian noise and uses thresholding to address the issue of generating discrete valued adjacency matrix from continuous values. Crucially, the proposed method cannot fully capture node- edge dependencies. A similar score-based generative model for graphs, where both node features and adjacency matrix are created, is presented in Jo et al. (2022). Finally, Vignac et al. (2022) suggest alternative approach where a discrete diffusion process is used to generate graphs with discrete node and edge features. This is similar to Haefeli et al. 2022). however the latter can only be applied to unattributed graphs.
More recently, Martinkus et al. (2022) with their SPECTRE network and Luo et al. (2023) take a different approach by considering the graph spectra, thus leveraging the inherent ability of the low frequency portion of the spectrum to capture global structural characteristics of the corresponding graph. Although similar to our method, SPECTRE focuses on generating an adjacency matrix condi- tioned on a set of eigenvectors, which may or may not have been generated themselves. Our method instead only generates eigenpairs from which the adjacency matrix recovered. As a result, unlike SPECTRE, our method is capable of generating graphs that respect a set of given spectral properties (see Subsection (.4).. GSDM [uo et al.] (2023). on the other hand, proposes to reduce the complex- ity by performing diffusion just the eigenvalues and optionally on the node features, while the eigenvectors used to reconstruct the final adjacency matrix are uniformly sampled from the training set. DiGress Vignac et al. (2022) is also closely related to model, however its complexity is quadratic in the number of nodes of the graph, making it unsuitable to work on large graphs.
Our approach is also related to recently introduced latent graph diffusion models, which employ an autoencoder architecture to map nodes and edges of a graph to latent space where the diffusion process takes place Yang al. (2024): Zhou et al. (2024). While these approaches aim to learn low-dimensional embedding of the graph nodes, we rely instead on the well-established concept of spectral embedding Luo al. (2003), with eigenvectors providing low-dimensional embedding of the graph nodes and eigenvalues capturing global structure information.
3 DENOISING DIFFUSION MODELS
Denoising Diffusion Probabilistic Models (DDPMs) are a class of generative models inspired by considerations from -quiiliiiir thermodynamics. In particular, diffusion models in deep learn- ing were first introduced in Sohl-Dickstein et al. (2015) yet popularized only in 2020 HH et al. (2020). They operate by iteratively introducing noise to an input signal and then learning to de- noise it thus generating new samples from the corrupted signals. Specifically, the idea is to destroy the structure in data distribution through an iterative forward diffusion process (noising) and then leam reverse diffusion process (denoising). This reverse process restores structure in the data, thus yielding a tractable generative model of the data.
Given a data point sampled from real but unknown data distribution Xo q(x), we define forward noising process producing a sequence of noisy samples x1,...,X0 .y as a Markov Chain given by g (Xi,..., XT| xo) = II (Xt. x;-1), with the diffusion kernel defined as:
q (x, xx-1) = N (xs: V 1,,11)
(1)
3

---

Published as a conference paper at ICLR 2025
Craphi
Spectral Diflusio
Ps
1
1r ArT
s, A
1 Ap-1
Go. X
PPGN Generator
A( 3o)
f
Ee Bytc&
(same model)
PPGN Disommintto
PPCNN Generator
A( Agl
Lipes
(#t, Sel
Figure 1: GGSD pipeline. During the spectral diffusion process (left) the neural network trained to predict the denoising steps for the eigenvectors é and eigenvalues À of the graph Laplacian. The second stage of our method is the graph predictor (right), where we train a Provably Powerful Graph Network (PPGN) Maron al.] (2019) (similar to what was done in SPECTRE Martinkus et al. 2022 ) Given the eigenvalues and eigenvectors generated, it predicts the adjacency matrix.
Note that, if we define ā₁ II'm (1 (a), we can reformulate Eq. as a single step q(x, xx) N (xe Vaxx, (1 ā.)
(2)
In the reverse diffusion process, the goal is to recreate the true sample from a Gaussian noise input XT ~ (0, I) by sampling from y (x,-1 x,), the true denoising distribution. In order to run the reverse diffusion process, need to learn model pa, often referred as score model, to approxi- mate these conditional probabilities. As Feller (1949) showed, in the case Gaussian distributions the diffusion process reversal has the same functional form of the forward process. From this it follows that the reverse diffusion process kernel can be defined as
po (x+-1 xx) = (X (40 (), ()),,()))
where 9 are the parameters of the reverse diffusion kernel at each time step, which can be learned using neural network. If fix the variance need to learn the distance between the means of two Gaussian distributions, ie., between the noise added in the forward process and the noise predicted by the model. This leads variational lower bound loss expressed in terms of the Kullback-Leibler (KL) divergence between the posterior of the forward process and the parameterized reverse diffusion process.
constant Br (i.e. Σo(x,,
B,T), only
4
OUR METHOD
Consider an undirected unweighted graph 9 = (), where is the set of n nodes connected by the edge set E. Recall that for graph with adjacency matrix A, the graph Laplacian L is defined as L = D A, where D is the diagonal degree matrix. Finally, let & and A be the orthonormal matrix of eigenvectors (as columns) and the diagonal matrix of eigenvalues given by the cigendecomposition L = 4 respectively. In the following sections, we use À to denote the vector of eigenvalues of the graph Laplacian.
The fundamental intuition underpinning our model is that we can represent the graph connectivity with (possibly a subset of) the eigenvectors & and the corresponding eigenvalues À of the graph Laplacian. The connection between the spectrum of the graph Laplacian and the structural properties of the underlying graph is well known and studied. For example, well established that the low frequency portion of the spectrum captures the global structural characteristic of the graph, while the high frequencies are essential in the reconstruction local connectivity patterns Chung (1997).
Fig. I shows overview of the proposed pipeline. This is made of two main components, namely 1) a spectral diffusion process that generates set of eigenvalues and eigenvectors from which an approximation of the Laplacian matrix can be reconstructed and 2) graph predictor, which outputs a binary adjacency matrix from the (noisy) Laplacian reconstruction. The two components are discussed in detail in the following subsections.
4

---

Published as a conference paper at ICLR 2025
L layers
Φ
MLP
Cross Attention
MLP
Ex
X
A
X
convid
Cross
convid + Ex
Attention
The score model takes as input the noisy eigenvector matrix and eigenvalues at time and predicts the noise the data to be used in the denoising step. The node feature eigenvectors & projected through MLP d dimensional space. The sequence of le eigenvalues is given as input to a 1D convolutional layer, which outputs d features for each eigenvalue. Both eigenvectors and eigenvalues go through series of L layers composed of two multi-heat cross- aross-attention blocks, one updating the eigenvectors conditioned by the eigenvalues and one updating the eigenvalues conditioned the eigenvectors. After each layer, apply residual block @, which adds to the layer input the updated values scaled and shifted by time-dependent factors. Finally, & and X are projected & dimensional space through an MLP and a ID convolution.
Figure 2:
4.1 SPECTRAL DIFFUSION
Moving from the Laplacian its spectrum reduces the double row-, column-covarian with respect to node permutations of the Laplacian matrix, to a single covariance over the rows of the eigenvec- tor matrix. To address this covariance, represent the eigenvector matrix embedding of the nodes, i.e., we interpret the i-th component of the j-th eigenvector as the j-th component of the i-th node embedding, alternatively, add the eigenvalues À as global graph descriptor. We can then define the reverse diffusion step as Po 1 A 1 A,) = - (1 A 1 pe (2, A., t). of
series of spectral
the rows of 4 as vectors. To this
(4)
where the normal distribution is over the product set of the spectral embeddings and the global spectral descriptor.
Following DDPM Ho et al. (2020). train a neural network to predict the denoising step. We design the backbone of our spectral diffusion process of Eq. 4 as a neural network composed of sequence of layers containing pair of multi-head attention blocks Vaswani al. (2017), one operating on the eigenvectors conditioned on the eigenvalues and one operating on the eigenvalues conditioned by the cigenvectors. This choice allows to achieve both node permutation invariant model and to let the eigenvectors and eigenvalues condition each other on the prediction of the de- noising step. Moreover, this model can easily handle node features X by simply concatenating them to the cigenvectors of each node. Fig. [ shows the overall structure of the proposed neural network. Note that the convld layer is applied the eigenvalues, which are invariant with respect to node permutations, while the diffusion process itself acts in an invariant way on node embeddings, which are permutationally covariant. As a consequence, the whole process permutationally invariant.
Crucially, our model allows us to reduce the memory footprint of the diffusion component from O(n²) to (kn), where denotes the number graph nodes, by fixing the maximum number of eigenvectors to k; resulting in a faster generative process. For this reason, for larger graphs, we perform diffusion on a subset of l eigenvectors.
this case, after the denoising diffusion process, with the corresponding subset of eigenvalues X, which allows
we obtain subset & of columns for an approximated reconstruction of L & A es L, from which the adjacency matrix can be inferred. In addition, since the cigenvector associated with the null eigenvalue does not contribute to the reconstruction of the Laplacian matrix, we can safely ignore it in the generation process.
Note that the optimal number of eigenvalues/vectors (k) to use is not fixed. We determined its range through preliminary analyses on the SBM and Planar datasets (see Appendix E), and general, we treat it as sepee--prraettr of the model. Interestingly, our experiments reveal that the eigenvec- tors corresponding to the smallest eigenvalues (lowest frequencies) do not consistently offer more information about connectivity result in better reconstructions of the original adjacency matrix.
5

---

Published as a conference paper at ICLR 2025
Table 1:
Comparison with other graph generative models using MMD metries (the smaller, the better) on synthetic datasets.
Community -maall
Planar
Stochastic Block Model (SBM)
Deg. 4
Clus.
Spect 4
4:
Deg. 4
Clux. 4
Spect.
Orb.
Deg. 4
4
Spect. +
Orh.
GraphRNN
0.0271
0.1072
0.0520
0.1469
0.0096
0.2985
0.0389
1.4022
0.0178
0.0151
0.0104
0.0351
GRAN
0.0013
0.0843
0.0282
0.0201
0.0202
0.2985
0.0248
0.1 0.1964
0.0135
0.0149
0.0034
0,0352
DiGress
0.0096
0.1035
0.0506
0.0372
0.0005
0.0178
0.0020
0.0115
0.0166
0.0246
0.0066
0.1327
GSDM
0,0099
0.0446
0.0131
0.0155
0.0220
0.0222
0.0096
0.0371
0.2295
0.2280
0.1578
0,2876
GDSS
0.0107
0.1060
0,0450
0.0356
0.0701
0,3025
0.0403
1.0345
0,2658
0.0442
0.0551
0,2780
SPECTRE
0.0079
0.1067
0.0460
0,0250
0.0008
0,0859
0.0147
0.0058
0.0044
0.0140
GGSD
0.0016
0.0590
0.0153
0.0142
0.0007
0.1881
0.0125
0.0047
0.0005
0.0115
0.0045
0.0289
4.2
GRAPH PREDICTOR
The main drawback of considering a subset of the eigenvectors is the introduction of noise on the reconstructed adjacency matrix. We adopt a strategy similar to the one proposed in Mar- tinkus et al.] (2022) to predict a binary adjacency matrix starting from a noisy reconstruction. We train a -laaer Provably Powerful Graph Network (PPGN) Maron et al. (219), which takes input the generated eigenvectors & scaled by the square root of the eigenvalues À as node features as well as the noisy adjacency matrix À D L and predicts the binary adjacency matrix
A( Φ A) o (PPGN A, .
(5)
where a is a sigmoid activation function and PPGN, is sequence of PPGN layers. We train this network with two losses: 1) a reconstruction loss encouraging the adjacency matrix predicted from the reduced ground-truth eigenvectors Ttt and eigenvalues Aye to match the corresponding input adjacency matrix Agt, and 2) adversarial loss on the generated adjacency matrix A(O, Å), i.e.,
Lree= = BCE(C gl. A(@ gt, )gt)) and Ludv = (DggD((A (D(gt)) + log(1 D(A( q ()))
(6)
where BCE is the standard binary cross entropy loss and D is a discriminator network composed of a sequence of PPGN layers followed by a global pooling for the final graph-level classification. Note that, unlike in SPECTRE SPECTRE/Martinuss Martinkus et al. (2022), this refining step is not generative, meaning that the output is deterministic and depends solely on the input eigenvectors/values
5 EXPERIMENTAL EVALUATION
Datasets. We compare the performance of our model against that of state-of-the-ar alternatives on both synthetic and real-world datasets. In line withh prior work, we use three synthetic datasets and two real-world datasets. The synthetic datasets we consider are (i) Community-small (12 < [V| < 20). (ii) Planar (V) = 64), and (iii) Stochastic Block Model (SBM) (2-5 communities and 20-40 nodes per community). The real-world datasets are both from the molecular domain, namely (i) Proteins (100-500 nodes) Dobson & Doig (2003) and (ii) QM9 (9 nodes) Ruddigkeit et al. (2012);
Ramakrishnan et al. (2014). Detailed descriptions of all datasets can be found in Appendix A
Evaluation Metries. We assess the ability of the models to generate graphs with structural char- acteristics close to those of the training graphs by following the methodology outlined in [iao et al. (2019), which aims to address the difficulties of measuring likelihoods when evaluating autoregres- sive graph generative models reliant on orderings. In particular, we adopt the approach proposed by You et al. (2018b) and Li et al. (2088)) and used by many others Krawczuk et al. (2020) as well. Our evaluation centers on contrasting the distributions of graph statisties between the gen- erated and actual graphs. Specifically, we consider the following key graph statistics: 1) degree distribution (Deg.), 2) clustering coefficient (Clus.), 3) eigenvalues of the normalized graph Lapla- cian (Spec.), and 4) the occurrence frequency of all 4-node orbits (Orb.). Moreover, for QM9, follow the literature and evaluate the quality of the generated graphs by computing the validity of the generated molecules, their uniqueness, and their novelty w.r.t. to the molecules in the training set and vicerversa Samanta et al.] (2020); Guo & Zhao (2022). Detailed information on the evaluation metries used can be found in Appendix

---

Published as a conference paper at ICLR 2025
Baselines. We evaluate the effectiveness of our model by comparing it against a number of well- known graph generative models as well as some recently developed deep graph generative models. In particular, consider G You et al. (018b). GRAN Liao et al. (2019), DiGress Vignac et al. (2022), (ESECTEE[Martiken et al. (2022), GDSSJJo et al. 2022) and GSDMLum et al. 2023))
For the molecule generation task we also include Graph VAF Simonovsky & Komodakis (0018).
Experimental Setup. To maximize the robustness of the experimental results, we follow a slightly different experimental setup compared to previous works. Specifically, for the synthetic datasets, we decided to create larger set of test graphs: 200 graphs for Planar and SBM, and 100 graphs for community-small, Accordingly, we let each model generate an equivalent number of graphs (200 for Planar and SBM, 100 for community-sman number of graphs in the Proteins dataset (see Appendix (A), we also followed different and more robust protocol to evaluate the generated graphs on this dataset. Rather than utilizing single subset of the dataset as test set, we created 10 folds (identical for each method) allowing to report the average of each metric (# standard deviation) over the 10 folds. Further detailed information on the model settings and training setup, both for our model and the baselines, is provided in Appendix D
to compute the MMD measures. Due to the limited
5.1 EVALUATING THE GENERATED GRAPHS
Synthetic Datasets The experimental results for community-small, SBM, and Planar are shown in Table [] In this table, report the MMD metrics for the graph statistics, where the smaller the statistics, the better. The results of our method (GGSD) are chosen from those obtained using either the lower or upper range of eigenvalues. Specifically, for the Planar, the results refer to the 16 smallest eigenvalues, whereas for the community-small dataset and SBM we used the largest ones, 8 and 32 respectively. The best performance is highlighted in bold, while the second-best value is underlined. Overall, our model consistently achieves the best or second-best results across all datasets, with the exception of Clus. in Planar and Spect. in SBM and Planar. We posit that the lower performance on planar graphs may be related to the behaviour of the eigenvectors of this type of graphs. Note in fact that there no clear class structure in this dataset but rather the graphs are related by the (hard) property of planarity. Indeed, graphs with similar spectra can lie on opposite sides of the discrimination boundary, iee, between planar and non-planar graphs. As such, the addition removal of an edge connecting local substructures can easily break the planarity of the graph without significantly affecting its spectral representation. Finally, for Planar and SBM, we also evaluated the quality of the generated graphs in terms of validity, uniqueness, and novelty in Appendix C showing the ability of our method to generate graphs that are at the same time valid, unique, and novel.
Real-world Datasets Results for real-world dataset generations on Proteins and QM9 are reported in Table 2 left and right, respectively. Also in this series of experiments, we achieve good perfor- mance. Again, we selected the best results from either the highest or lowest frequencies. Notably, for the Proteins dataset, we utilized the 16 smallest eigenvalues, while for QM9, we used the entire spectrum. The Proteins dataset especially challenging due the size of the graphs which can reach 500 nodes. For this dataset, GGSD ranks as the best in all metrics Table (left). QM9, on the other hand, composed by small graphs of up to 9 nodes, with both node and edge features. In Table 2 (right), it is important to note that the last column is of particular interest as it summarizes all values, where we emerge as the second top performer.
5.2 GRAPH PREDICTOR ABLATION
Given that the "Graph Predictor" is trained using discriminative loss, one may think that it is this component that is doing all the "heavy lifting" of the graph generation task, while the "Spectral Diffusion" may be just producing noisy data.
To assess that this is not the case, we designed an experiment training our model to predict all the eigenvectors and eigenvalues datasets allows us to generate all the eigenvectors & and eigenvalues À and reconstruct the exact Laplacian L $ and the (almost) binary adjacency matrix A D L, where D is the diagonal of the Laplacian. To obtain a binary adjacency matrix, we further threshold A to get the actual edges of the generated graph (i.e., every entry above 0.5 is considered an edge).
the community-small dataset. The small size of the graphs in this
7

---

Published as a conference paper at ICLR 2025
Table 2: Comparison on real datasets with other graph generative models. Left: Proteins, using MMD metrics (the lower, the better) and mean + standard deviation (over 10 folds). Right: QM9, based on vV)aliidiyy (U)niqueness, and (N)ovelty metrics (the higher. the better). Results denoted by and are taken from Martinkus et al. (0222) and Vignac et al. (2022) respectively.
Proteins
QM9
Deg. 4
Clas.
Spoct
Orbit 4
Val.t
V.& U.F
V&U&NT
GraphRNN
0.0065+0.000
0.1658=0.0088
0.0170:10.0009
0.8142-000277
GraphVAE"
0.5570
0.4200
0.2610
GRAN 0.0569:0.0088
0.1622::0.009
0.0146:10.0007
0.3430=-00016
DiGress
0.9900
0,9523
0.3180
GSDM
0,3792==00044
0,4653#0.0088
0.3024=20.0032
0.9589=0.0225
GDSS
0.8335
0.8281
0.7257
GDSS
0.0633=000063
0.4169::00089
0.0706:10.0021
0.8168=0000188
SPECTRE"
0.8730
0.3120
0.2910
SPECTRE
0.0062:10.0021
0.09/88 1000077
0..066.10.0004
0.0328 0.0039
GGSD
0.966
0.864
0.847
GGSD
0.0003
0.065611.00066
0.0059110.0007
0.029610.0006
the technique we just described, in which the graph is constructed directly from the generated o and À without using the "Graph Predictor"; 2) "Noise + Predictor": we give input the "Graph Predictor" noise drawn from a Gaussian distribution; 3) "Diff. + Prediction": this is the full model used in all the other experiments. For each configuration, we provide results for two cases: 1) using the full set of eigenpairs all eigenbasis to train the model, and 2) using subset of the eigenpairs model trained using the largest eigenvalues and their corresponding eigenvectors.
In Table 3 show three different configurations of our method: 1) "Only Diffusion":
Reconstructing the graph directly from the Laplacian (referred to as "Only Diffusion") using the complete spectrum yields the best results. Conversely, using either random noise or the reconstructed Laplacian as input to the Predictor results in significantly inferior outcomes. This suggests that the "Spectral Diffusion" part of the network is responsible for the actual generation process. On the other hand, if we consider truncated eigenbase for the training phase, the Predictor becomes useful as it helps to refine the results further.
To provide comprehensive overview, we also provide some qualitative examples in Figure com- paring the eigenvectors generated by the diffusion module of GGSD with those of the Laplacian computed the adjacency matrix predicted by the PPGN module. In smaller graphs, the eigenvec- tors are almost perfectly preserved, while only minor local differences emerge in larger graphs. In contrast, the generative approach used by SPECTRE fails maintain the relationship between the conditioning eigenvectors and the final generated graph.
5.3 EIGENVECTORS ORTHOGONALITY
The general framework of DDPM cannot guarantee the orthonormality of the generated eigenvec- tors. While in principle this might pose problem, in practice we observed that this property is well preserved in the generations. In Appendix F we present both quantitative and qualitative analyses to evaluate the orthogonality of the generated eigenvectors, demonstrating that they exhibit approxi- mate orthonormality. Moreover, in order to test if having an exact orthonormality of the eigenvectors brings any advantage, tried reproject the final generated eigenvectors an orthonormal ma- trix through QR decomposition before the PPGN predictor step. As reported in Figure { we did not observe a clear benefit in having orthonormal basis. We argue that, even if orthonormality is a well-known characteristic of the cigendecomposition of the graph Laplacian (indeed, of the eigen- decomposition of any symmetric matrix), it is probably not the most important (nor essential) to guarantee good reconstruction of a valid graph Laplacian, which exhibits more complex properties that need to be learned directly from data.
Table 3: Ablation of the Graph Predictor network on the community-smi small dataset.
Full Set of Eigenpairs
First Eigenpairs
Deg. L
Clus. 4
Spect, 4
Orh. J
Deg. 4
Clus. J
Spect 4
Orh.
Only Diffissio
0.00134
0.05484
0.01574
0.00514
0.00936
0.09753
0.04341
0.01255
Noise Predictor
0.04428
0.11129
0.05691
0.46051
0.06681
0.12330
0.05972
0.45137
Dift. + Predictor
0.00562
0,07887
0.02232
0.00942
0.00423
0.05464
0.01832
0.00866
8

---

Published as a conference paper at ICLR 2025
GGSD
SPECTRE
Computed on A
Generaled Computed
Generated &
33
04
Q6
97
08
22
Computed on A
Generated Computed
Ganerate +
Ø3
Q4
Q6
Ø7
ds
Ø3
Q4
Figure Left column: Comparison of the eigenvectors generated by the diffuusio module of GGSD with the cigenvectors recomputed on the Laplacian computed on the adjacency matrix pre- dicted by the PPGN module on the Community (top) and SBM (bottom) datasets. The models have been trained with the 8 smallest eigenvectors. In the smaller dataset (Community) the diffusion generates nearly perfect eigenvectors. In the more challenging SBM dataset we can notice that generated eigenfunction are slightly different from the one computed on the predicted graph while preserving the overall structure. Right column: comparison of the interpolated cigenvectors that SPECTRE uses condition the PPGN module to the actual cigenvectors of the generated graph. In this case the eigenvectors structure is completely lost the generative process.
SBM
Gencrated
Orthoganal
Orthogonal
Avange
6-
Numbor of elgonpairs
Numbor of agenpairs
Figure 4: Performance analysis without (Generated) and with (Orthonormal) reprojecting the gen- orthonormal basis. The average enror represents the mean degradation of metrics between the generated graphs and the training set. We report both the mean and the standard deviation eигог bars 10 generations of 200 graphs. Specifically, Degree, Cluster, and Spectral metrics are calculated between the generated graphs and the test set, then normalized by the metrics between the training and test sets.
erated eigenvectors
5.4 SPECTRAL CONDITIONED GENERATION
The spectrum of the Laplacian plays an important role in many applications, from graph classifi-
cation and mesh analysis Cosmo et al. (020) 2022); Bai et al. (2015); Hu et al. (2014); Minello et al. 2019); Gasparetto al. (201 5a b) to reconstructing the underlying geometry of triangulated 3D shape Cosmo et al. (2019) Marin et al. (001);; Moschella et al. 2022) and to define universal adversarial attacks Rampini et al. (021). Being able to generate graph given spectrum is thus important feature of a generative method. We pose the graph generation problem conditioned on a sequence of eigenvalues and/or eigenvectors inpainting problem Lugmayr al. (2022).

---

Published as a conference paper at ICLR 2025
Donor Graph
GGSD
SPECTRE
gen. comm.
2
GGSD
À of 2
76
commn
SPECTRE
À of comm.
93
0
À of comm.
96
0
Figure 5 & Table 4: Conditioning of the generation on the first 3 smallest eigenvectors as an in- painting task using RePaint Lugmayr et al. (2022) (Figure). Number of communities in the graphs generated with spectrum conditioning (Table). Higher values should appear in the bold diagonal.
Eigenvalues conditioned generation In this setup, the eigenvectors at time - 1 are computed according Eq. while the eigenvalues are derived from the target ones through the diffusion process of Eq. 2 i.e.
A,- (V a.) --11 NV 1: (e.. )r; IITI)
To validate the generation conditioned on the cigenvalues, we generate graphs of the SBM data distribution by fixing the number of nodes as the average number of nodes of graphs containing and 3 communities (70 nodes). We randomly choose one graph with 2 communities and one graph with 3 communities from the test set, and we consider their spectra. We use these to condition the generation of two sets of 100 graphs, for the and 3 communities eigenvalue sequences, respectively. The results reported in Table 4 show that spectral conditioning is able to influence the properties of the generated graphs, while the spectral conditioning provided by SPECTRE fails do so.
Eigenvectors conditioned generation We adopt a similar strategy to condition on subset of the eigenvectors. In this case, the portion of k' known eigenvectors & [<'u..... the] time are derived from the target ones through the forward diffusion process, while the remaining eigenvectors क' = [0 Φg
and the eigenvalues are computed according to Eq. The three groups of graphs in Figure 5 show the donor graphs (left) from which the first three eigenvectors were computed, the graphs conditioned on the given eigenvectors using GGSD (center) and SPECTRE (right). The color is the 2D color encoding of of the three first eigenvectors manually selected to highlight the different clusters. The colors from the donor graph have then been transported on the generated graphs' corresponding node (same node index). While GGSD is able preserve the community structure encoded by the given eigenvectors, in SPECTRE this information is completely lost, causing nodes from the same community in the donor graph to randomly spread over different communities of the generated graph. This may be due to the particular generation mechanism of SPECTRE. Specifically, SPECTRE learns a set of reduced orthogonal bases during training, which left and right-rotated according (219)) network based on some input (generated) eigenvalues. This requires an alignment of the graphs the learned bases, which makes the training more complex. Our approach on the other hand is fully covariant.
rotation matrix predicted by PointNetST [Segol & Lipman
CONCLUSION
We have introduced GGSD, diffusion-based generative model for graphs where the spectrum of the graph Laplacian is used to retain structural information while reducing the computational com- plexity. Our approach has a number of advantages, from the ability to directly generate eigenvectors and eigenvalues, to the possibility of naturally encapsulate node feature information as well con- ditioning the generation on target spectral properties.
Our model suffers from two main limitations. Firstly, while using low/high frequencies to recon- struct the spectrum is theoretically grounded, in practice it would be interesting allow the model to select the most informative frequencies for a given dataset. We will explore this possibility in future work. Secondly, while the diffusion model is linear, the bottleneck of our model the PPGN- based predictor, which has quadratic complexity. In the future, it would be interesting investigate alternative methods using a sparse representation of the adjacency matrix reconstructed from the Laplacian.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
L.C. and A.B. are supported by the PRIN 2022 project n. 2022AL45R2 (EYE-FLAI, CUP H53D2300350-0001), G.M. acknowledges financial support from the European Union NextGen- erationEU in the framework of the iNEST Interconnected Nord-Est Innovation Ecosystem (iN- EST ECS CCC_00000043- CUP H43C22000540066, In this regard, the views and opinions expressed are solely those of the authors and do not necessarily reflect those of the European Union, nor can the European Union be held responsible for them. A.T.'s work was partially supported by the project "Perturbation problems and asymptotics for elliptic differential equations: variational and potential theoretic method" funded by the program NexiGenerationEU and by MUR-PRIN, grant 2022SENI73.
REFERENCES
Réka Albert and Albert-László Barabási. Statistical mechanics of complex networks. Reviews of modern physics, 74(1):47, 2002.
Lu Bai, Luca Rossi, Andrea Torsello, and Edwin R Hancock. A quantum jensen- shannon graph kerne for unattributed graphs. Pattern Recognition, 48(2):344- 355, 2015.
Fan RK Chung. Spectral graph theory, volume 92. American Mathematical Soc., 1997.
Luca Cosmo, Emanuele Rodola, Jonathan Masci, Andrea Torsello, and Michael M Bronstein. Matching deformable objects in clutter. In 2016 Fourth international conference on 3D vision (3DV), pp. 1-10. IEEE, 2016.
Luca Cosmo, Mikhail Panine, Arianna Rampini, Maks Ovsjanikov, Michael M. Bronstein, and Emanuele Rodolà. Isospectralizztion, or how to hear shape. style, and correspondence. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 7529-7538. Computer Vision Foundation IEFE, 2019. doi:
10.1109/CVPR.20190077.
Luca Cosmo, Giorgia Minello, Michael Bronstein, Luca Rossi, and Andrea Torsello. The average mixing kernel signature. Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision - ECCV 2020, pp. 1-17, Cham, 2020. Springer International Publishing. ISBN 578-3-030-58565-5.
Luca Cosmo, Giorgia Minello, Michael Bronstein, Emanuele Rodolà, Luca Rossi, and Andrea Torsello. 3d shape analysis through a quantum lens: the average mixing kernel signature. In- ternational Journal of Computer Vision, 13066):144--149,, 2022.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs.
arXiv preprint arXiv: 180.. 2018
Hubert de Fraysseix and Patrice Ossona de Mendez. Trémaux trees and planarity. European Jour- nal of Combinatorics, 33(3):279-22) 293,, 2012. ISSN 0195-6698. doi: https://doi.or//0...00 1016//j.jjj..jj......... 2011.09.012. URL https: wwww//wwwww..m/ www.scciiiiiiiieeeeeceocieeiieee rect. com /ttp///ennnc/aatcc//...ll Topological and Geometric Graph Theory.
S01 001195668810000 1001 000098811010000
Paul D Dobson and Andrew Doig. Distinguishing enzyme structures from non-enzymes without
alignments. Journal of molecular biology, 330 2003.
Mikhail Drobyshevskiy and Denis Turdakov. Random graph modeling: A survey of the concepts.
ACM computing surveys (CSUR), 1-36) 2019.
Paul Erdōs, Alfréd Rényi, et al. On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 17-), 1960.
Ludwig Fahrmeir, Thomas Kneib, Stefan Lang, Brian Marx, Ludwig Fahrmeir, Thomas Kneib, Stefan Lang, and Brian Marx. Regression models. Springer, 2013.
Christos Faloutsos. Graph mining: Laws, generators and tools. Lecture Notes in Computer Science, 5012:1, 2008
11

---

Published as a conference paper at ICLR 2025
W. Feller. On the theory of stochastic processes, with particular reference to applications. In Pro- ceedings of the [First] Berkeley Symposium on Mathematical Statistics and Probability, pp. 403- 432. 1949.
Andrea Gasparetto, Giorgia Minello, and Andrea Torsello. A non-parametric spectral model for graph classification. In International Conference on Pattern Recognition Applications and Meth- ods, volume 2. pp- 312-319. SciTePress, 2015a.
Andrea Gasparetto, Giorgia Minello, and Andrea Torsello. non-parametric spectral model for shape retrieval. In 2015 International Conference on 3D Vision, pp.
IEEE, 2015b.
Rafael Bambez,, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez- Lengelin,, Dennis Sheberla, Jorge Ryan P Adams, and Alán Aspuru-Guzik, Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268 22)6266-276, 2018.
Timothy Hirzel,
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Aditya Grover, Aaron Zweig, and Stefano Ermon. Graphite: Iterative generative modeling of graphs. In International conference on machine learning, pp. 2434-2444. PMLR, 2019.
Xiaojie Guo and Liang Zhao. systematic survey on deep generative models for graph generation.
IEEE Transactions Pattern Analysis and Machine Intelligence, 45(5):5370-53900 2022.
Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, and Roger Wattenhofer. Dif- fusion models for graphs benefit from discrete state spaces. arXiv preprint arXiv:2210.01549, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-668 6851, 2020.
Nan Hu, Raif M Rustamov, and Leonidas Guibas. Stable and informative spectral signatures for graph matching. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pp. 2305-2312, 2014.
Han Huang. Leilei Sun, Bowen Du, Yanjic Fu, and Weifeng Lv. Graphgdp: Generative diffusion processes for permutation invariant graph generation. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 201-210. IEEE, 2022.
Jachyeong Jo, Seul Lee, and Sung Ju Hwang. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning, pp. 10362-10383, PMLR, 2022.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. 2013.
arXiv preprint
Igor Krawezuk, Pedro Abranches, Andreas Loukas, and Volkan Cevher. Gg- igggaan& A geometric graph generative adversarial network. 2020.
Jure Leskovec and Christos Faloutsos. Scalable modeling of real graphs using kronecker multipli- cation. In Proceedings of the 24th international conference on Machine learning, pp. 497-504, 2007.
Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, and Zoubin Ghahramani. Kronecker graphs: an approach to modeling networks. Journal of Machine Learning Research, 11(2), 2010.
Yibo Li, Liangren Zhang, and Zhenming Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10:1-24, 2018a.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. arXiv preprint / 0333433244.0 2018b.
12

---

Published as a conference paper at ICLR 2025
Renjie Liao, Yujia Li, Yang Song. Shenlong Wang, Will Hamilton, David K Duvenaud, Raquel Urtasun, and Richard Zemel. Efficient graph generation with graph recurrent attention networks. Advances in neural information processing systems, 32, 2019.
Andreas Lugmayr, Martin Danelljan. Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. 2022.
Bin Luo, Richard C Wilson, and Edwin Hancock. Spectral embedding of graphs. Pattern recog- nition, 36( (00::2)222220000 2003.
Tianze Luo, Zhanfeng Mo, and Sinno Jialin Pan. Fast graph generation via spectral diffusion. IEEE Transactions Paltern Analysis and Machine Intelligence, 2023.
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. Graphdf: A discrete flow model for molecular graph generation. In International Conference Machine Learning, pp. 7192-7203. PMLR, 2021.
Riccardo Marin, Arianna Rampini, Umberto Castellani, Emanuele Rodolà, Maks Ovsjanikov, and Simone Melzi. Spectral shape recovery and analysis via data-driven connections. Iní. J. Comput. Vision, 129 129(10):275-2766) oct 2021. ISSN 0920-5691. doi: 10070s1 1263-021-014922-21
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably powerful graph networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL pttp:://proeeeinss..errrsssssss Pap.re..PPpppp....
files/
Karolis Martinkus, Andreas Loukas, Nathanaël Perraudin, and Roger Wattenhofer: Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. Interna- tional Conference on Machine Learning, pp. 15159-15179. PMLR, 2022.
Giorgia Minello, Luca Rossi, and Andrea Torsello. the von neumann entropy of graphs. Journal of Complex Networks, 7(4):491-511 2019.
Luca Moschella, Simone Melzi, Luca Cosmo, Filippo Maggioli, Or Litany, Maks Ovsjanikov, Leonidas Guibas, and Emanuele Rodolà. Learning spectral unions of partial deformable 3d shapes. Computer Graphics Forum, volume 41, pp. 407-417. Wiley Online Library, 2022.
Chenhao NIu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Per-
mutation invariant graph generation via score-based generative modeling. In International Con- ference Artificial Intelligence and Statistics, pp. 4474-4484. PMLR, 2020.
Tiago P. Peixoto. Hierarchical block structures and high-resolution model selection large net- works. Phys. Rev. X, 4:01 1047, Mar 2014. doi: 10. 1..110//hysReex...010000 URL https: 7/link www.kaaa.orrgooooooooo http:////...11111111//eeeeee 0110047
Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev. Molecularrnn: Generating realistic molecular graphs with optimized properties. arXiv preprint arXiv: 13372, 2019.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
Arianna Rampini, Franco Pestarini, Luca Cosmo, Simone Melzi, and Emanuele Rodola. Universal spectral adversarial attacks for deformable shapes. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pp. 3216-3226. 2021.
Emanuele Rodolà, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In Computer graphics forum, volume 36, pp. 222-236. Wiley Online Library, 2017.
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean- Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. 17. Journal of chemical information and modeling, 52(1 28:2664 2875, 2012.
13

---

Published as a conference paper at ICLR 2025
Bidisha Samanta, Abir De, Gourhari Jana, Vicenç Gómez, Pratim Chattaraj. Niloy Ganguly, and Manuel Gomez- Modriguo. Nevae: A deep generative model for molecular graphs. Journal of machine learning research, 21(114):1-33, 2020.
Nimrod Segol and Yaron Lipman. On universal equivariant networks. In International Confer- ence on Learning Representations, 2019.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382, 2020.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning-CCANN 2018: 27th International Conference Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part I 27, pp. 412-422. Springer, 2018.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2266 PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL
occp files/paper/2017/
Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pas- cal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.147: 2022.
Duncan J Watts and Steven H Strogatz. Collective dynamics of worlll nature, 393 (6684):440-443) 1998.
Ling Yang. Zhilin Huang, Zhilong Zhang, Zhongyi Liu, Shenda Hong, Wentao Zhang, Wenming Yang. Bin Cui, and Luxia Zhang. Graphusion: Latent diffusion for graph generation. IEEE Transactions on Knowledge and Data Engineering, 2024.
Minji Yoon, Yue Wu, John Palowitch, Bryan Perozzi, and Russ Salakhutdinov. Graph generative model for benchmarking graph neural networks. 2023.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018a.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphrnn: Generat- ing realistic graphs with deep auto-regressive models. In International conference machine learning, pp. 5708-5717. PMLR, 2018b.
Wojeiech Zaremba, Ilya Sutskever, and Oriol Vinyals. Recurrent neural network regularization. arXiv preprint arXiv: 1409.24099222 292999.999.............. 2014.
Cai Zhou, Xiyuan Wang, and Muhan Zhang. Unifying generation and prediction on graphs with latent graph diffusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
14

---

Published as a conference paper at ICLR 2025
Planar
SBM 2
SBM 3
SBM 4
SBM 5
Proteins
Figure 6: Visual comparison between training set graph samples and generated graph samples pro- duced by GGSD. Each column represents graph type (Planar, Stochastic Block Model with 2,3,4, and communities, and Proteins). Top row (Original): training set graphs. Bottom row (Generated): graph generated by GGSD.
A DATASETS.
We utilize five commonly used datasets for graph generative tasks. Some examples the graphs contained in these datasets alongside similar graph generated by GGSD are shown in 6
Sommunity-small: synthetic dataset consisting of 100 random community graphs, with [12, 20] nodes.
Stochastic Block Model (SBM): A synthetic dataset from Martinkus et al. (2022) comprising 200 Stochastic Block Model graphs, each with random selection nodes within each community. The probability of edges between communities is set at 0.3, while the probability of edges within communities is set 0.05.
communities and 20 to 40
Planar: A synthetic dataset from Martinkus et al. (0022) of 200 planar graphs, each containing 64 nodes. Graphs are created through the application of Delaunay triangulation to a randomly and uniformly placed set of points.
QM9 Ramakrishnan et al. (2014): RRudigkeet al. (2012: This real-world dataset comprises 134k organic molecules with maximum of 9 heavy atoms (carbon, oxygen, nitrogen, and fluorine). By following (Simonovsky & Komodakis (2018) we allocate 10k molecules for validation, 10k for testing, and the rest for training.
Dobson & Doig (2003): The dataset encompasses 918 protein graphs, each ranging from 100 to 500 nodes. In this representation, each protein is depicted as a graph, with nodes correspond- ing to amino acids. Nodes are connected by an edge if they are within a distance of less than Angstroms from each other.
Proteins
B EVALUATION METRICS
B. SERTISTICS-BASEE
We consider the following key graph statistics: degree distribution (Deg.), clustering coefficient (Clus.), and the occurrence frequency of all 4-node orbits (Orb.). The deviation of these metrics between the generated graphs and the actual ones measured using the maximum mean discrep- ancy (MMD) You et al. (2018b). In its initial formulation, the computation of the MMD relied on the Earth Mover's Distance (EMD) and as result was very slow. For this reason, as suggested in Liao et al. (2019), we use the total variational (TV) Gaussian kernel. This in turn significantly accelerates the evaluation process while maintaining consistency with EMD. In addition to assess- ing node degree, clustering coefficient, and orbit counts, also extend our evaluation to include a spectral analysis (Spect.). following Liao et al. (2019). This involves computing the eigenvalues of the normalized graph Laplacian, quantized to approximate probability density. The spectral
15

---

Published as a conference paper at ICLR 2025
Table 5: Comparison with other graph generative models based on validity, uniqueness, and novelty metrics (the higher the better) on synthetic datasets.
Planar
Stochastic Bluck Model (SBM)
Val.t
Uniq.
Nout
V&LRN f Val.1
Uniq.
Nov.
VitU& f
GrahRRNN
1.00
GRAN
0.03
1.00
1.00
0.03
0.20
1,00
1,00
0.20
DiGress
0.70
0.98
0.95
0.65
0.13
0.98
1,00
0.13
GSDM
0.85
0.50
0.28
0.12
0.08
0.88
0.50
0.04
GDSS
0.00
0.00
0.01
1,00
1,00
0.01
SPECTRE
1.00
1,00
GGSD
0.15
1.00 1.00
0.15
0.49
1.00
1,00
0.49
comparison offers insights into the global properties of the graphs, complementing the local graph statistics emphasized by previous metrics.
B.2
QUALDTY-AAED
The validity is determined by the ratio of valid molecules to all generated molecules. For molecule graphs (QM9), the validity in molecule generation represents the percentage of chemically valid molecules based on specific domain rules. We measure it using RDKit anitization For Planar graphs, we use the NetworkX python library based on the left-right planarity test (de Fraysseix & Ossona de Mendez 2012). For the stochastic block model graphs (SBM), we use a Monte Carlo and greedy heuristic for the inference of the stochastic block model parameters (as implemented by cdlib (Peixoto 2014)). We consider graph valid if its probability, assessed by a Wald test (Fahrmeir et al. 2013), of having been generated by the SBM model with inter-edge probability of 0.3 and intra-edge probability of 0.005 higher than 0.9, and if the number of communities in the range 2-5 with a number of nodes for each community between 20 and 40.
The novelty gauges the percentage of valid graphs that are not sub-graphs of the training set, and vice versa. It checks if the model has successfully learned to generalize to unseen graphs and it considers two graphs identical if they are isomorphic.
The uniqueness is defined as the ratio of unique samples to valid and novel samples, measuring the level of variety during sampling. To calculate uniqueness, generated graphs that are sub-graph isomorphic to others are initially removed, and the remaining percentage represents uniqueness. For instance, if a model generates 100 identical graphs, the uniqueness is 1/100- 1%.
The product of these three metrics is referred to as V&U&N (or VUN) and summarizes the ability of the method to generate graphs that are at the same time novel, unique, and valid.
C VUN EXPERIMENTS
Table presents the results on the quality of the synthetic graphs generated for the Planar and SBM datasets, analyzing the metries of validity, uniqueness, novelty, and their combination. As highlighted in Section 5.1 our method achieves 100% uniqueness and novelty on both datasets while showing the second-best score in terms of VUN. Specifically, for SBM, the gap of SPECTRE and our method with respect to other methods is considerable, due to a higher validity score compared to competitors. On the other hand, the validity score of our method on the Planar dataset is lower than the pop-prforriig method (DiGress) of some margin. This behavior is not unexpected and can be explained by the insensibility of the eigenvectors to small local changes of the topology as discussed in Section [5.]] It is worth noticing how GSDM, despite showing the best validity performance on planar graphs, is not able to generate novel graphs, with a novelty score of just 28% in Planar, and 50% in SBM. This does not come as surprise, since in GSDM the cigenvectors used to reconstruct the final adjacency matrix are uniformly sampled from the training set. This limits the generative power of the method, since most of the information about the graph connectivity is contained in the
https:: fttps::/www..e http://wwww...ti.o/oo
Hook. html
16

---

Published as a conference paper at ICLR 2025
eigenvectors. As such, the obtained graphs are not actually generated but rather slight modifications of the training set graphs.
D MODEL SETTINGS AND IMPLEMENTATION DETAILS
For all datasets except for QM9, we retrained the models using the configurations recommended by the authors. When no recommended ssper-parameters setups or model weights were available, we explored the space of hyper- apper-parameters tuning them according to the ranges mentioned for other datasets. Finally, for our method (GGSD), we use the k largest/smallest eigenvalues of the unnor- malized Laplacian. The values of k are experimentally determined as explained in Appendix We stress that these are only fraction of the full set of eigenvectors.
We used the unnormalized Laplacian since it yields an easier graph reconstruction by simple thresh- olding (as explained in Section 5.2) In order to handle potential scaling issues, we simply normal- ized the eigenvalues and eigenvectors based on the training data so as to reflect a normal distribution.
For the training of the diffusion model, we split each dataset into 90% train and 10% test, and we train the Spectral Diffusion on the whole dataset for 100k epochs, using early stopping on the reconstruction loss. We performed a grid search the number of layers between 6, 9 and 12, and selected the best model according to the degree metric computed from the graphs reconstructed directly from the eigenvectors/values and the graphs of the training set. The sampling has been done using DDIM with 200 steps. Moreover, we generate each sample 4 times and keep the one with the lower deviation from orthogonality.
For the training of the Graph Predictor, we used the same splits of the Spectral Diffusion, and trained for 100k epochs. We performed early stopping by comparing the degree distribution of the generated graphs with the training graphs. We used 6 PPGN layers and 3 PPGN layers for the Graph Predictor and the discriminator network respectively, except for QM9 in which also the Graph Predictor is composed of three layers. For QM9, we let the Graph Predictor to generate also edge features, similarly to Martinkus et al. (2022).
For all datasets, following the observations in Appendix E we train both Spectral Diffusion and Predictor on the 16 smallest and 32 largest eigenpairs and select the final model according to the best average metrics on the validation set.
In order to guarantee the reproducibility of both our model architecture and results, we have made our code accessible on online public repository ?
E NUMBER OF EIGENVECTORS
To evaluate which part of the spectrum is more relevant and the proper number of eigenvectors to use, we performed experiment. We trained our model focusing on either the smallest eigenvalues (Smallest) or the largest ones (Largest), while gradually increasing the corresponding number of cigenvectors taken into consideration. The results obtained from two synthetic datasets, Planar and SBM, are shown in Figure 7 Regarding the optimal number of cigenvectors, it appears that too many eigenvectors do not yield the best results, either considering low or high frequencies. Specifically, in the case of the Planar dataset, both smaller and larger eigenvalues exhibit the best performance with 16 eigenvectors. However, for SBM, while the optimal count for lower frequencies remains at 16, for higher frequencies, it increases to 32. These results are not entirely surprising, considering the inherent trade-off between the diffusion model's capability to manage high high-dimensional data and the quantity of information (number of cigenvectors) accessible to the Predictor. All in all, it should be noted that the selection of the number of eigenvectors can generally be regarded as a model hyperparameter for optimization, acknowledging its potential dependence on the specific dataset.
Building on the findings outlined above, in our experiments we employed the 32 largest eigenvalues and the 16 smallest eigenvalues and their corresponding eigenvectors for all datasets, except for the community-small dataset, for which we utilized the top 8 largest/smallest eigenvalues, and QM9, for which used the full set of eigenpairs.
httpps:/ thu.i com cosmo/GG81
17

---

Published as a conference paper at ICLR 2025
Planar
SBM
17.5
Smallest
15.0
Largest
10.0
2.5
ampairs
of elgenpairs
Figure 7: Performance analysis with varying numbers of eigenpairs and different spectrum parts, for the SBM (left) and Planar (right) datasets. The average error represents the mean degradation of all metrics between the generated graphs and the training the standard deviation as error bars on 10 generations of 200 graphs. Specifically, we consider the Degree, Cluster, and Spectral metries. For each metric, the degradation computed as the ratio between (1) the MMD value computed between the generated graphs and the training graphs and (2) the MMD computed between the test graphs and the training graphs. value of indicates that the generated graphs exhibit the same statistical difference wrt the training graphs the test set graphs do. The ratios computed for each metric are then averaged to get single value indicating the quality of the generation.
We report both the mean and
F EIGENVECTORS ORTHOGIONAITYY STUDY
We conducted some experiments to provide both quantitative and qualitative analyses of the or- thogonality behavior of the generated eigenvectors. In Figure X (left). we show how the generated eigenvectors deviate from forming eigenvectors between and 12 for Community, and between and 64 for SBM. This choice re- flects the fact that the graphs in the Community dataset have between 12 and 20 nodes, while in the case SBM we observed that using more than 64 eigenvectors appears to lead degrada- tion in performance (see Figure 8). As expected, increasing the number of generated eigenvectors introduces greater deviations from orthogonality, which aligns with our findings in Appendix [ In Figure g (right), we provide qualitative examples by comparing the generated eigenvectors with those computed from the adjacency matrix predicted by the PPGN. We can observe that in simpler datasets, such as the Community dataset, the eigenvectors are perfectly aligned. In more challenging datasets, such as SBM, while the alignment is not exact, the overall correspondence remains good and significant.
orthonormal basis. Here we vary the number of generated
Community
SBM
Community
SBM
0.55
0.00
0.224
4.00
6.95
6.000
Sumbe stngemm2204
the two bar plots show the deviation of the eigenvectors generated by GGSD from an orthonormal basis on two datasets for different numbers eigenpairs. The deviation from or- thogonality is computed as the average root mean squared difference of the inner product of the eigenvectors with the identity matrix. i.e. RMS((2) 1 2o 1 with :2 being the elementwise square operator and & the number of cigenvectors. Right: Qualitative results show- ing the inner product of the generated eigenvectors.
Figure 8: Left:
18

---

Published as a conference paper at ICLR 2025
10
11
12
13
14
15
16
SBM
1.11 2.54
3.62
4.88
6.65
6.88
7.11
7.42
7.51
7.75
7.47
7.84
7.76
7.85
8.26
8.13
Community
1.09
2.97
3.41
4.18
4.71
5,46
5.77
6.13
6.46
6.79
6,87
7.54
Table 6: Average Dirichlet energy computed on 500 generated graphs from the SBM and Commu- nity datasets, trained the lower part of the spectrum (smaller eigenvalues).
The two experiments described above demonstrate that the generated eigenvectors exhibit the smoothness property and are often very similar, sometimes nearly identical, the eigenvectors com- puted from the final predicted graph. To further investigate this, we computed the Dirichlet energy of the generated approximate eigenvectors on the final generated graph. We observed a consistent pattern, with the energy increasing as the eigenvalues grow larger. In Table we report the average Dirichlet energy computed from 500 generated graphs the SBM and Community datasets. We ex- clude the first eigenvector (A = 0) from the analysis, as it does not contribute to the reconstruction of the Laplacian, and the diffusion process is trained only on non-zero eigenvalues.
G RUNTIME COMPARISON
Table 7: For each dataset, every method generates 100 graphs. We report the total generation time in seconds.
Planar
SBM Comm.
Proteins
QM9
GraphRNN
3.30
4.19
3.16
16.87
GRAN
8.58
48.43
2.90
205.47
DiGress
859.64
3882.30
70.49
OOM
68.
GSDM
10.71
31.51
9.74
160.09
GDSS
81.10
160.88
456.30
3177.03
1.32
SPECTRE
1.71
3.63
0.72
OOM
0.06
GGSD
9.51
18.63
4.41
124.07
2.50
Table showss the total generation time (in seconds) for our method and the baseline methods across the datasets analyzed in this paper. For fair comparison, generated 100 graphs for each dataset and method. These experiments were conducted on a computer equipped with an AMD Ryzen 3700X processor, 64GB of RAM, and NVIDIA RTX 3070 8GB graphics card. We achieve significant speedup compared to DiGress, thus showing that we are able to overcome the computa- tional bottleneck of diffusion-bers based methods which, unlike GGSD, work on diffusion space that is quadratic in the number of nodes of the graph. Additionally, our hardware configuration could not register the time for large datasets like Proteins due their space complexity. While method outperforms GDSS in terms of speed, it is less efficient than simpler algorithms such as GRAN and GraphRNN.
H STABILITY OF THE SPECTRAL DECOMPOSITION OF THE GRAPH LAPLACIAN
In the spectral graph theory literature, the instability of the Laplacian has become true-ism. Yet. this claim requires further qualification as several spectral approaches have shown to be robust even under severe deformation Rodolà et al.] (2017); Cosmo et al. (2016). In general, random structural perturbation can cause major topological changes which will reflect on the eigenvectors and eigen- values of the Laplacian quite dramatically, but it strongly depends on the location of the actual perturbation, and it is linked with small gaps in the eigenvalues.
From spectral perturbation theory, we that under a perturbation 8, long as the eigenvalues and remain distinct, the eigenvalues of the perturbed Laplacian L. L E are perturbed by
19

---

Published as a conference paper at ICLR 2025
quantity
A Re of Ed:
(8)
while the eigenvectors are perturbed by A = &B. Here the matrix B (biy) is defined as:
EΦj
bi Xj A
As consequence, the mixing can become large even for small perturbations if the gap between the eigenvalues is small, and in general only eigenvectors with close eigenvalues will mix a significant way. All this being said, this characterizes what happens when perturb the graph, which is not what is happening here. By recreating the spectrum through a stable diffusion process, the perturbation is in the spectrum, and, in general, small perturbations of the spectrum do cause major topological changes in the structure (which, as we said, are associated with large spectral variations). Let say that the eigenvectors are perturbed by factor of A and the eigenvalues by a factor of AA, then the reconstructed Laplacian is
L = ④ + ( + + -0) OAΦ" ODA PAΛD" T + A
order error terms
$A^Λサ" + "
A -
(10)
Il order terms
III order ertor term
which varies smoothly with noise and does not have elements at the denominator that force the terms to explode. Indeed, we have not observed topological instabilities any the datasets considered in this study.
20


Reviews:
[{"review_id": "1yBlETUsHM", "reviewer": "Reviewer_D9zK", "summary": "This article introduces a denoising diffusion model for generating graphs  which will look alike graphs of a training dataset. The parameters of the model are learned in a supervised way from a dataset of graphs, and the output is a Laplacian which can be converted to a graph. The model combines 2 elements in its architecture: a first step to learn the process of spectral diffusion (the diffusion mostly connect the eigenvectors and eigenvalues to the one after 1 time step of diffusion -- the model tries to learn the way back, ie. the reverse diffusion process) and a second step which predicts the graph (which an architecture of GNN, here a PPGN from Maron et al., 2019) to generate the Laplacian (and hence the graph). The idea is that the first step provides a noisy version of the Laplacian matrix (e.g., without orthogonality of the eigenvectors ; or with a reduced number of eigenvectors and eigenvalues) and that the second step helps to recover a correct Laplacian (hence graph). The first step, relying of the general idea of denoising some diffusion model to generate new samples (from the initial work of Sohl-Dickstein et al., 2015 and ho et al., 2020); for graphs, diffusions are operated on the set of eigenvectors and eigenvalues of the Laplacian which are considered (classically) as embedding of the graphs. \n\nThe article combines ideas coming from other works (with many points inspired by the structure SPECTRE of Martinkus et al, 2022; or by DiGress from Vignac et al., 20222). Yet the present work comes with some novelties which are well explained. For instance, the diffusion is done on a limited number $k$ of pairs of eigenvectors and eigenvalues, to reduce the memory cost of the model ; the architecture of the neural networks of the 1st step (the learning of the reverse diffusion) is original (with attention heads from eigenvalues to eigenvectors and the converse way as well). The one for the 2nd step relies on existing previous work, yet it is shown in ablative studies that it works well and is needed for good performance.\nThese new elements are well integrated and explained. An evaluation is carried out on both synthetic datasets, and on real-world datasets of molecular graphs, and compared to 5 or 6 baselines (depending on the experiment), both with some metrics about the relevance of the general structure of the obtained graphs, and some inspections about validity, uniqueness and novelty of the generated graphs seen as molecules. An ablation study is conducted and some additional remarks are made (about orthogonality of the obtained eigenvectors, about the use of the method to generate graphs given a target spectrum for Laplacian, and about runtime (in appendix).", "questions": "Here are some remarks and questions which can help to improve the work:\n\n* 1) the name \"GRASP\" is already used by several previous work in domains related to the present one: \"GRASP: Graph Alignment through Spectral Signatures\" of J Hermanns et al., 2021  ; or the GraSP toolbox for Graph Signal Processing (popularized thanks to the A. Ortega's book which uses it). I advise the authors to adopt a different name.\n\n* 2) The 2nd paragraph, p.1 l. 033-042 is not really true, and particularly naive: there are now tons of works to generate random models of graphs with a variety of properties (quoting the Albert and Barabasi model from >20 years ago don't make justice to what is done in the complex network, or network science, community). What is lacking, is most of the time is the precise knowledge of which feature has to be controlled and tuned to a dataset. The present approach which tunes a model to a specific dataset is relevant because of that.\n\n* 3) p.2 l. 071: why should a generative model \"assign equal probability to each of these n! adjacency matrices.\" ? There can be various ways of building probabilities for graphs and why permutation does matter that much ?\n\n* 4) Section 4: I am not certain of the usefulness of that part. This is very basic things which would be incorporated in a part with notations (such a part is missing), and the properties recalled here should be recalled while introducing the work.\n\n* 5) p4, eq. (4): this is the reverse diffusion step, right ?\n\n* 6) Section 5: it would be clearer to have a subsection about step 1 (possibly including 5.1 as a paragraph), then a section about step 2, and maybe a last one about the loss function and how training is done. By the way, even if these questions of training and the two loss functions are inspired by ESPRIT, it would be worth to detail that more (using the space liberated by the removal of section 4).\n\n* 7) Experimental validations in Section 6 is good albeit limited due to a small number of datasets. For real-world datasets, it would be good to have examples which are not only related to molecules. \n\n* 8) Performance themselves is decent, yet, as told above: Some more lines should be devoted to understand why that, and what works better in some other works for some cases.\n\n* 9) in 6.2: why only a threshold of 0.5 is considered ? Given that real-world graphs are often sparse, one could expect a different natural  threshold to obtain desired sparsity.\n\n* 10) In 6.3, are the authors sure of their remark of l. 459 : \"...orthonormality..... (indeed, of the eigen-decomposition of any matrix\" ? There are matrices with eigendecomposition and non-orthogonal eigenvectors. Your remark holds only for normal matrices.\n\nEdit after revision and discussions: Rating set to 8: accept, good paper", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This a  quite good article, with a good presentation and interesting ideas.\nFor me, the strengths of the article are:\n\n* The article is well written, which comprehensive explanations for the method and good presentations of the architecture, the numerical experiments, the ablation study and some inspections of features of the method in 6.3 and 6.4. \n\n* The method builds on previous works, yet it does not feel incremental but more like a thoughtful construction to improve on ESPRIT, DIGRess or other related works.\n\n* The scope is relevant, because it is indeed difficult to build random models of graphs from a limited dataset of graphs to generate new relevant samples. (See however my remark 2 underneath).\n\n* The ablation study is convincing in showing why the two steps are better than only one of the two. Also, there is an interesting discussion and experimental study to see how many pairs of eigenvectors/eigenvalues of $L$ are needed to build a graph and which ones (highest or lowest eigenvalues) are the most appropriate. This element can be explored more in future work and this is an original finding of this article.", "weaknesses": "Some weaknesses are:\n\n* The scope is not large, and the work can be seen as somehow incremental. However, these increments are good enough for an article.\n\n* Experimental validations in Section 6 are correct albeit limited due to a small number of datasets. For real-world datasets, it would be good to have examples which are not only related to molecules. Currently, the applications seem too specific.\n\n* The baselines used are ok, yet many results are not computed anew and taken from the published articles. Given the modest number of examples, and the small number of baselines (6 at most), the authors could have tried to re-implement all of them for better control of the reproducibility of the baselines and comparison to the present work.\n\n* Performance is decent, yet not far above (or not above in some cases) the competitors. Some more lines should be devoted to understand why that, and what works better in some other works for some cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7pJsmQ3G3U", "reviewer": "Reviewer_sjdg", "summary": "This paper considers the problem of realistic graph generation. The\nproposed approach focuses on the spectral properties of the generated\ngraphs, that is, the eigenvectors and eigenvalues. Roughly speaking,\nthe method learns a diffusion denoiser that acts on the eigenvectors\nand eigenvalues jointly, in a way that respects desired equivariance\nproperties of graph generative models. A key step in reducing the\ncomplexity of this procedure is to restrict the diffusion model to a\nsmall number of eigenvectors, allowing for a linear complexity with\nrespect to the size of the graph. This reduced representation is\naccounted for using a graph neural network.", "questions": "I would be interested to hear the authors' response to my main point in the weaknesses section: what sorts of graphs do you think the method, as it is presented in the paper, would have a hard time generating?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper is for the most part well-written and easy to\nunderstand. Indeed, a strength of the proposed approach is that the\narchitecture is not too different from methods that are already\npopular in the literature, so I envision practitioners and other\nresearchers having an easy time reimplementing this.\n\nThe experimental results seem quite good. I was particularly impressed\nby the fact that GRASP was able to capture graph statistics well, even\nthough the method starts with spectral features. The experiments were\ncarefully selected to demonstrate various aspects of the model's\nbehavior, which provided insight into the problem beyond just claiming\n\"SOTA\" -- indeed, most of the weaknesses that I was thinking of as I\nread Sections 1-5 were addressed directly in Section 6. Bravo!", "weaknesses": "I did not find this paper to suffer from any glaring weaknesses, but I\ndo have one concern about the idea of using a small portion of the\nspectrum to reconstruct the entire graph structure. As pointed out,\ndifferent parts of the spectrum correspond to different aspects of the\ngraph structure, that is, local vs. global features. Of course, this\nis remarked upon as a limitation by the authors, but I would\nappreciate some more discussion on the sorts of graphs that can be\ngenerated in light of this.\n\nSuppose we have a family of graphs that statistically vary in both\ntheir global and local properties, in a way where those properties\n(global vs. local) do not have strong correlations. I would imagine\nthat such a family of graphs could not be captured by merely choosing\nto restrict generation to the lowest or highest set of eigenpairs. I\nsuspect that this concern points to a deeper question about spectral\ngraph theory than is within the scope of this paper, but I would still\nbe interested to hear from the authors what sorts of graphs they\nexpect the proposed method is able to capture.\n\nFor instance, SBMs are largely characterized by their global\nstructure, where the local connections follow an Erdos-Renyi pattern\n-- thus, it makes sense to generate such graphs by focusing on the\nlower spectrum. On the other hand, expander graphs are known to be\nvery sparse, while also exhibiting certain properties that are global\nin nature -- such as being well-connected in some sense. It would be\nhelpful to have some experiments to see if such a class of graphs\ncould be generated by the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WnWD7FtNnw", "reviewer": "Reviewer_MC9j", "summary": "The authors propose a new algorithm for generating graphs based on spectral decomposition by using diffusion models for the resampling of eigenvectors and eigenvalues, which has been crucial in network analysis.", "questions": "Q1: From my understanding, the authors replace the GAN module in the pipeline of SPECTRE with the diffusion model. Therefore, it would be better to highlight the difference of the two methods. For example in section 6.4, is there any explanation why GRASP works better in preserving the network community when conditional on the spectral than SPECTRE, since both methods are based on spectral decomposition? In addition, in SPECTRE it was mentioned that conditional on the spectra the performance (section 5.1 therein) can be boosted. How is the performance of GRASP in terms of MMD compared to SPECTRE?\n\nQ2: For methods based on spectral decomposition, it is key to choose the number of non-zero eigenvalues k. I noticed that in SPECTRE relatively smaller k such as 2 and 4 can achieve good performance, while for GRASP large k's are needed. Specifically, for the dataset, QM9 SPECTRE only used k=2, while GRASP used all the eigenvalues.  How is the performance of GRASP compared to that of SPECTRE when the number of non-zero eigenvalues is the same? How will the number of non-zero eigenvalues affect the computational time? \n\nQ3: Could other fast sampling algorithms of diffusion models such as DEIS, DPM-Solver++, UniPC, and so on be leveraged to improve the quality or speed of GRASP?\n\nQ4: Figure 8 is a bit confusing. The first two graphs in the right panel seem to be the same. Some explanations may help the readers to understand why k=12 for the community and k=64 for the SBM are compared. \n\nQ5. For Figure 7, it is unclear what the authors mean by ‘average errors’.  In the caption, it is said that Degree, Cluster, and Spectral metrics are calculated.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Using the Laplacian spectrum allows us to naturally capture the structural characteristics of the graph and work directly in the node space while avoiding the quadratic complexity bottleneck that limits the applicability of other diffusionbased methods.", "weaknesses": "1) The motivation of the method is not clear. Why replacing the GAN module could lead to better results?\n2) How to choose non-zero eigenvalues $k$. See also my Q2.\n3) Some figures are not clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4pRl70llYX", "reviewer": "Reviewer_mxAE", "summary": "An approach using spectral properties for efficient graph diffusion followed by a reconstruction module.", "questions": "1. The number of eigenvalues and eigenvectors being used, and using either large or small ones seem to be critical for this method - since SBM, Planar, QM9 use very different settings. Would that require lots of tuning to find the proper one? Explanations about your choice of different graph datasets can be helpful for people not familiar with graph spectra.\n\n2. I fail to understand the information in Table 4. What do the values in the Table represent?\n\n3. Have you considered or experimented using other information besides graph spectrals for diffusion?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Spectral information is critical for graph topology, and leveraging it directly for diffusion is an interesting approach to explore.\n2. The given ablations are interesting to read and follow.\n3. The sampling speed is quick, unlike previous diffusion models.\n4. My main concerns while reading the paper are mentioned in the conclusion as limitations, which is clear and closes some of my issues.", "weaknesses": "1. Experimentally, some key metrics are missing for synthetic datasets, namely the VUN for the planar and for the SBM dataset. I also have some concerns about the VUN metrics for QM9 since the uniqueness does not seem to be very high while the novelty is outstanding. Both metrics do not overlap totally but reflect the diversity of generation from different perspectives. Would the codes be released later (till my review the given anonymous repo is empty) for checking this technically?\n\n2. The introduction / related work emphasizes a lot on traditional graph generation - which may not be the most critical or related work here. More background for the spectra-based method (SPECTRE is included but it would help if there are discussions with more relevant works) can help to clarify the storyline. Generally, the writing/clarity should be improved. Another branch of method that may be related to GRASP is the 'latent diffusion' for graphs - where people may encode a graph to node features, and, different from GRASP, they may denoise based on those learned features and reconstruct the graph based on it. This method in terms of structure is very similar, and the comparison of using learned graph features, and using spectral information directly is an interesting question to check or mention in the writing.\n\nWill consider increasing the score with the concerns being addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QR4xwKxdge", "reviewer": "Reviewer_sbAq", "summary": "The paper proposes a graph generative model based on the denoising diffusion probabilistic model (DDPM). The diffusion process is performed on the graph spectral domain.", "questions": "See \"Weaknesses\".", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The work combines DDPM and graph spectral decomposition for the proposed generative model. An advantage is that complexity might be reduced as it is necessary to consider the entire graph spectrum. Numerical results demonstrate that the model is effective.", "weaknesses": "1. Though combining DDPM and graph spectral decomposition is new to me, there are already works that use DDPM to generate graphs  (e.g., DiGress) and works that use SDE diffusion on graph spectrum (e.g., GSDM). In my opinion, this paper is experimenting with a different combination of diffusion approach and signal domain, which can produce useful results but lacks significant novelty.\n2. The model uses the graph eigendecomposition and performs diffusion on the eigenvectors and eigenvalues. However, it is well-known that eigendecomposition (of Laplacian) is highly unstable. I think the authors should theoretically address this issue.\n3. Using a part of the spectrum has the advantage of reducing complexity. However, information is lost. What is the balance between these two factors? Is it true that most of the high-frequency components are not important?\n4. Is it possible for the diffusion process to generate eigenvectors and eigenvalues that cannot be obtained from the eigendecomposition of any graph? \n5. For some datasets (e.g., QM9), the proposed method does not seem to show a clear advantage with a few performance metrics, even though the entire graph spectrum is used. One may gain more insights if the authors can also show the results when a partial graph spectrum is used.\n6. How to choose $k$? Is there a principled approach?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)