Title: Multi-resolution Spectral Coherence for Graph Generation with Score-based Diffusion
Venue: NeurIPS 2023
Accepted: yes
Keywords: graph wavelet transform

multi-scale wavelet filtering

graph generation

diffusion model
Abstract: Successful graph generation depends on the accurate estimation of the joint distribution of graph components such as nodes and edges from training data. While recent deep neural networks have demonstrated sampling of realistic graphs together with diffusion models, however, they still suffer from oversmoothing problems which are inherited from conventional graph convolution and thus high-frequency characteristics of nodes and edges become intractable. To overcome such issues and generate graphs with high fidelity, this paper introduces a novel approach that captures the dependency between nodes and edges at multiple resolutions in the spectral space. By modeling the joint distribution of node and edge signals in a shared graph wavelet space, together with a score-based diffusion model, we propose a Wavelet Graph Diffusion Model (Wave-GD) which lets us sample synthetic graphs with real-like frequency characteristics of nodes and edges. Experimental results on four representative benchmark datasets validate the superiority of the Wave-GD over existing approaches, highlighting its potential for a wide range of applications that involve graph data.

Multi-resolution Spectral Coherence for Graph Generation with Score-based Diffusion

Abstract

Successful graph generation depends on the accurate estimation of the joint distribution of graph components such as nodes and edges from training data. While recent deep neural networks have demonstrated sampling of realistic graphs together with diffusion models, however, they still suffer from oversmoothing problems which are inherited from conventional graph convolution and thus high-frequency characteristics of nodes and edges become intractable. To overcome such issues and generate graphs with high fidelity, this paper introduces a novel approach that captures the dependency between nodes and edges at multiple resolutions in the spectral space. By modeling the joint distribution of node and edge signals in a shared graph wavelet space, together with a score-based diffusion model, we propose a Wavelet Graph Diffusion Model (Wave-GD) which lets us sample synthetic graphs with real-like frequency characteristics of nodes and edges. Experimental results on four representative benchmark datasets validate the superiority of the Wave-GD over existing approaches, highlighting its potential for a wide range of applications that involve graph data.

1 Introduction

The flexible representation of graphs lets us investigate a variety of real-world phenomena such as social networks [8,39], traffic flows [43,45], molecule structure [18,19,34,44], and brain networks [2, 22,28]. The rich expressive power of graphs inherits from embodying various combinations of nodes and edges within an irregular structure. However, the analysis of graph data is challenging due to the arbitrary and heterogeneous structure, as it becomes difficult to characterize robust patterns both within a graph and across different graphs for a downstream machine learning task.

The same issue arises in graph generation, where the objective is to learn the underlying graph distribution from training samples and thereby synthetic graphs with similar characteristics to the training data can be generated, i.e., sampled. For this, accurately estimating the joint distribution of the measurements from node features and graph structures is essential. Recent graph generative methods [13,35,37] have been successful in sampling realistic node and edge features, however, their generated samples often fail to embed the true relationships especially given in sparse connections between nodes. This problem often occurs as existing methods derive separate embeddings of nodes and edges considering them as different entities [37]. Moreover, successive layers of graph convolution [13,35] aggravate this issue by overly averaging signals on graphs, such that local details (i.e., high-frequency characteristics) become intractable.

Key ideas. Notice that the edge signals, e.g., edge weights and connectivity, should be coherent to node signals as the edges should properly explain the relationships between the nodes. Based on this homophily assumption, to address the aforementioned issues, we propose a novel generative model that precisely captures the dependency between nodes and edges in multiple resolutions on local-to-global structures. This is realized via spectral graph wavelet transform (SGWT) [9], which

(a)

(b)

(c)

(d)

(e)

(f)

Q

1

Q

O

14.3

O

0

0.9

O

O

1

1

0

23.9

2.0

-9.5

1.1

0.9

-0.2

1

14.3

0.9

O

O

Q

0

-O

O

-9.5

■

O

O

Q

0.2

O

enables the filtering of graph signals at a specific level of granularity. We transform the node and edge signals into a common graph wavelet space and obtain their underlying cross-characteristics, i.e., spectral coherence, via their inner product at different scales. We expect that sparse connections between nodes (i.e., higher variation) can be captured with high fidelity as high-frequency components in the spectral space. Fig. 1 shows an example of such an effect with wavelet transforms on edges, which accentuates certain connectivity and disconnectivity over a template graph. These accentuated (dis-)connectivities at specific scales can be characterized by a generative model with multiresolution wavelet filtering. Also, we further show that the scale-specific node and edge features computed in the spectral space can be easily and jointly obtained in a spatial graph convolutional form by applying wavelet filtering to either nodes or edges for efficiency.

Proposed Framework. We leverage this spectral coherence to model the joint distribution of graph node and edge features embedded in a common graph wavelet space with a score-based diffusion model [36]. As opposed to conventional approaches [13] where the scores for the joint distribution of nodes and edges are modeled with graph multi-head attention (GMH) [1] involving graph convolutions of the raw node and edge signals, our model calculates the scores over multiple scales of nodes and edges to capture different frequency characteristics. With diffusion and reverse denoising processes in multi-resolution, our Wavelet Graph Diffusion Model (Wave-GD) accurately estimates the joint distribution of the node and edge features preserving their graph frequency characteristics.

Contributions. To this end, we summarize our main contributions as follows: 1) To obtain representative graph characteristics of different fidelity, we propose a novel concept to capture spectral coherence between nodes and edges in multiple scales in a common graph wavelet space. 2) Furthermore, we show that spectral coherence can be obtained in the graph domain as a spatial graph convolutional formulation, which allows us to perform the SGWT only on the edge signals for efficiency. 3) We propose a training scheme on the multi-resolution at which the optimal joint frequency characteristics of graph nodes and edges can be captured from training samples. The Wave-GD was tested on four representative benchmarks for graph generation which demonstrated outperforming results over state-of-the-art baselines. Extensive experiments on real-world, synthetic, and biochemical graph datasets validate the generality of the method, suggesting its potential for diverse applications with graph samples.

2 Related Work

Deep graph generative models aim to learn the underlying distribution of node and edge signals from the training set. They can be categorized into (1) autoregressive models that construct graphs by the iterative generation of a node or an edge signal at each step or (2) one-shot models that simultaneously update all the signals at once.

Autoregressive models. Autoregressive models are widely used for graph generation since they are based on a straightforward way to iteratively update the graph structure conditioned on an incomplete graph generated so far. Prior works have parameterized the autoregressive models using recurrent neural network (RNN) [29,41,42], variational autoencoder (V AE) [11,12], and normalizing flow [20,34]. However, they suffer from high complexity since the required number of updates to generate the graph grows with the size of the graph. Furthermore, their generation is sensitive to the order of nodes being generated, i.e., autoregressive models may assign different probabilities for the construction of the same graph executed in a different way.

One-shot models. One-shot models alleviate the issues of autoregressive models by simultaneous generation of node and edge signals. This potentially reduces the complexity of generation since the number of generation steps no longer depends on the graph size. Furthermore, their generation is often permutation invariant and assigns the same probability for generating a graph regardless of the permutation. Researchers have considered parameterizing these models using generative adversarial network (GAN) [4], VAE [21], normalizing flow models [23,44], and diffusion models [13,25,37].

Diffusion models. In particular, diffusion models are recently catching attention due to their extraordinary capability in learning the joint distributions of graph components such as nodes and edges. They are based on (1) defining a forward diffusion process to sequentially corrupt a graph using noise distributions and (2) training a reverse diffusion process parameterized by a graph neural network to reconstruct the original graph. Existing works mainly differ by defining the diffusion processes in the continuous space [13] or the discrete space [10, 25, 37].

3 Generating Graphs with Robust Spectral Characteristics

3.1 Preliminary: Graph Wavelet Transform

Let G = ( X , A ) be an undirected graph, where X ∈ R N × F represents F -dimensional node features for N nodes and A ∈ R N × N is an adjacency matrix representing connectivity among the nodes. The graph Laplacian is defined as L = D -A , where D is a diagonal degree matrix. As the L is real and positive semi-definite, it has a set of orthonormal eigenvectors U = [ u 1 , u 2 , ..., u N ] and corresponding non-negative real eigenvalues Λ = diag ( λ 1 , ..., λ N ) . Decomposing this Laplacian as L = UΛU T , the connectivity, complexity, and spectral properties of the graph are characterized. Specifically, its eigenvectors and eigenvalues identify the frequencies and mode of vibrations of a graph. Eigenvectors with small eigenvalues represent large connected components in a graph structure and slow-varying signals among nodes. On the other hand, eigenvectors associated with large eigenvalues indicate the sparse and disconnected signals in the graph [22,26,40].

To capture localized characteristics of signals on arbitrary graph structures, the spectral graph wavelet transform [9] is defined by constructing a set of graph wavelet bases ψ s = ( ψ s 1 , ψ s 2 , ..., ψ s J ) with J scales where ψ s = U k ( s Λ ) U T . The ψ s is a realization of a kernel function k ( · ) in the spectral domain localized with δ n in the graph space, which captures the local characteristics of the graph at each node. The scale s selects a specific bandwidth in the spectral space which corresponds to the range of locality in the original graph space. The choice of wavelet basis, i.e., mother wavelet, depends on the shape of k ( · ) , which may vary depending on a target task such as smoothing or band-pass filtering. In this work, we used both types of kernel functions: (1) a band-pass kernel k ( s ) = sxe -sx to capture both low and high frequency of graphs and (2) a low-pass kernel k ( s ) = e -sx to capture cluster-like and smoothed features from graphs.

Using the bases ψ s , graph wavelet transform projects a signal x (e.g., node feature or connectivity) from the graph domain into the spectral domain as

which yields a wavelet coefficient W x ( s ) . This wavelet coefficient is a scale-filtered signal defined in the wavelet space. When admissibility condition on k ( · ) is satisfied [24], the inverse transform completely reconstructs the signal x by projecting W x ( s ) back to the graph domain as follows:

with an admissibility constant C k = ∫ ∞ 0 k ( x ) 2 x dx < ∞ .

3.2 Adaptive Multi-Resolution Representation of Nodes and Edges

To capture both fine and coarse graph features from an irregular graph structure, the filtering operation is performed over multiple scales. The multi-resolution representations of graph nodes and edges are derived by decomposing the graph signals X and A with scales { s i } J i =1 using (1), which result in { W X ( s i ) } J i =1 and { W A ( s i ) } J i =1 in the spectral domain . These are wavelet coefficients that contain different levels of local and global details of graph node and edge signals in a multi-resolution fashion.

To obtain the multi-resolution representations of these graph signals in the graph space , the inverse transform in the Eq. 2 is performed on the filtered coefficients using the bases ψ s at the same scales.

(a)

Diffusion

process

(b) Overview of the model architecture.

ForwardSDE

→ReverseSDE

Scale-filtering

XT

AT

A

↑1

↑1

X1

A1

Ai

Xo

Ao

—A

Node

Adjacency

Scale-filtered

Features

Matrix

Adj.Matrix

Graph

At

X

Xθ,t

Vxlogpt(Xt,At)

Forward SDE

Reverse SDE

VAtlogpt(Xt,At)

A

shared

2

→VAs logpt(Xt,At)

ForwardSDE

Inverse

<Ao,s >=

=

WA(s)

Transform

As the Eq. 2 is the superposition of multi-resolution representation of x over scales at s ∈ [0 , ∞ ) , it allows us to define a scale-filtered signal in the graph space at specific s as

where k 2 ( s Λ) = diag ( k 2 ( sλ 1 ) , ..., k 2 ( sλ N )) is a diagonal matrix [14, 22]. Therefore, the X and A at multiple resolutions in the spatial domain are defined as { X s i } J i =1 = { U k 2 ( s i Λ ) U T X } J i =1 and { A s i } J i =1 = { U k 2 ( s i Λ ) U T A } J i =1 , respectively. Within our proposed framework, we make these scales trainable such that the local-to-global characteristics can be obtained adaptively.

3.3 Refining Spectral Coherence between Nodes and Edges in Multi-resolution Fashion

As the graph features X and A should be coherent, i.e., connectivities should explain the relationship among features from different nodes, we hypothesize that they should share similarities even when observed from different resolutions. To capture the key components that are shared on both edges and nodes at multi-resolution, we measure the underlying cross-characteristics between nodes and edges in each scale in the spectral space. We quantify their coherence as a dot product as W A ( s ) · W X ( s ) and use this spectral coherence as a feature that represents a graph at a specific resolution s , which is used to estimate the score function in multiple resolutions later in Section 3.5.

Lemma 1. Let G be an undirected graph, with a node signal X and a symmetric adjacency matrix A . Given wavelet coefficients W A ( s ) and W X ( s ) , which are spectral representations of X and A at a specific scale s , a dot product of these wavelet coefficients is equivalent to the product between the scale-filtered edge signals and the node signals in the graph space, i.e., W A ( s ) · W X ( s ) = A s X .

By Lemma 1, we can efficiently extract multi-resolution coherence by applying the filtering only on the edges. Using the J number of A s along with the given data G = ( A , X ) , we introduce our simple yet effective graph generative framework through the stochastic differential equations (SDEs) in Sec. 3.4 and explain how Lemma 1 is used in the generative model in Sec. 3.5. The proof of Lemma 1 is given in Appendix A.

3.4 Wave-GD: Wavelet Graph Diffusion Model via Multi-resolution

Now we describe our graph generative model which captures the dependency of node and edge on both local and global structures with adaptive spectral filtering. The overall description of Wave-GD is given in Fig. 2, which demonstrates (a) multi-resolution diffusion strategy and (b) overall model architecture. In particular, we enhance the existing diffusion models for graphs [13] using the spectral coherence introduced in Lemma 1. The building blocks of our model are as follows: 1) the forward

diffusion to corrupt the graph distribution in a continuous domain, 2) the reverse diffusion to recover the original graph, and 3) the training objective for our model.

Multi-resolution forward diffusion process. Building upon the work of GDSS [13], we propose a new diffusion process that sequentially corrupts a graph structure at different levels of resolution based on the Itˆ o SDE [32]. As the forward diffusion process is modeled by a SDE which describes a process of random variables, { X t , { A s i t } J i =0 } with A s 0 t = A t at timestep t ∈ [0 , T ] are diffused as follows:

where f X : R N × F → R N × F and f A s i : R N × N → R N × N are drift functions, σ X ,t and σ A s i ,t are scalar diffusion coefficients, and w X and w A s i are standard Wiener processes for X t and { A s i t } J i =0 , respectively. As this diffusion process is simulated at varying levels of granularity in a multi-resolution scheme, it is likely that certain frequency characteristics are accentuated and the highlighted information will be preserved from corruption. In other words, at an arbitrary timestep t , the scale-filtered edges { A s i t } J i =1 are highly likely to contain richer information of edges for score estimation than the given A t as their edge signals are adaptively controlled and highlighted by trainable scales.

Learning the reverse diffusion process. Let G t = ( X t , A t ) be a given graph at timestep t and { G s i t = ( X t , A s i t ) } J i =0 be a set of graphs for multiple scales at the same timestep. Given the multiresolution forward diffusion process, we generate graphs from solving the associated SDE backward in time. To be specific, the reverse diffusion process can be formulated as follows:

where p t ( G t ) and p t ( G s i t ) denote the distribution of the forward diffusion process evaluated at time t . This reverse-time SDE recovers the original data distribution p 0 ( X 0 , { A s i 0 } J i =0 ) with the standard reverse-time Wiener processes ¯ wX and ¯ wA s i in which the direction of time is reversed as d ¯ t .

However, since the score functions of the joint distribution, i.e., ∇ X t log p t ( G t ) and ∇ A s i t log p t ( G s i t ) , are unknown, we train a neural network to estimate them using the denoising score matching loss [38]:

where p 0 t ( · ) denotes the transition distribution from p 0 to p t , X θ,t , A ϕ,t are score-based models which are graph neural networks trained to approximate the true score functions. The λ X ( t ) , λ A ( t ) , λ A s i ( t ) are time-varying scaling factors for individual loss terms and are defined as in [36].

Both our method and GDSS perform denoising score-matching to the partial scores of X t and A t . However, our method additionally models the joint probability space of X t and A s i t via SDEs. This is realized by the loss in Eq. 7, and this operation allows a model to flexibly estimate the complex dependency between nodes and edges with multi-resolution SGWT. Also, note that the scales of SWGT { s i } J i =1 in Eq. 7 are trainable so that multi-level granularities that characterize a graph distribution can be adaptively captured during training.

Remark. While we solve the reverse SDEs for { X t , { A s i t } J i =0 } in the training , we only need to solve the reverse diffusion processes (5) only for the node signal X and the non-filtered graph A = A s 0 in the sampling phase. In other words, we do not generate { A s i } J i =1 in the sampling phase. Also note that the parameters of the networks A ϕ,t are shared across different scales so that they can learn generalizable representations over different resolutions. As the shared parameters in the score function do not overfit to a specific scale, it also promotes stable training.

3.5 Score-based Joint Density Estimation with Multi-Resolution Coherence

Finally, we describe the score-based models X θ,t and A ϕ,t to better capture the spectral coherence over different resolutions. In particular, we revise the graph multi-head attention (GMH) [1] layer to better incorporate the spectral coherence between the edge and the node attributes. In the original GMHlayer, node and edge attributes were encoded into query Q , key K , and value V as follows:

where W Q , W K , W V are trainable weights. This layer is repeated for each { s i } J i =1 as follows:

which incorporate spectral coherence at a specific scale s i as given in Lemma 1. Using Q s i , K s i , V s i as the new query, key, and value for the GMH layer, one can incorporate the spectral coherence when training W 's. Consequently, the score-based models consist of multiple GMH layers followed by a multi-layer perceptron (MLP). We used the original GMH layer for the parameterization of X θ,t , and the modified GMH layer is used for A ϕ,t . Note that this parameterization preserves the permutation invariance of the original GMH layer, hence our score-based models are also permutation invariant.

4 Experiment

In this section, we quantitatively and qualitatively evaluate our method in comparison to various recent graph generative methods on four benchmark datasets. Ablation studies are also introduced to empirically analyze the roles of individual components within Wave-GD.

4.1 Datasets

We evaluated Wave-GD on four public datasets with varying sizes and characteristics, demonstrating its robustness and generalizability to generate high-quality graphs for diverse graph domains. Egosmall [33] consists of 200 small real sub-graphs from the Citeseer network dataset with 4 ≤ N ≤ 18 . Community-small consists of 100 randomly generated synthetic graphs with 12 ≤ N ≤ 20 . The graphs are constructed by two equal-sized communities and each community is generated by E-R model [5] and 0.05 N inter-community edges are added with uniform probability as in previous works [17, 25]. Grid consists of randomly generated 100 standard 2D grid graphs with 100 ≤ N ≤ 400 . As all nodes are arranged in a regular lattice, the maximum number of edges per node is 4. QM9 [31] is a molecular dataset with 133,885 molecules that are represented by attributed graphs with 1 ≤ N ≤ 9 , four node types, and three edge types.

4.2 Experimental Setup

For all datasets, we used 80% of the whole data as a train set and the rest 20% as a test set with the same split as in [13, 42]. As in [13], we sampled 10,000 molecules from the QM9 dataset, and for the remaining generic datasets, we sampled an equal number of data as the number of test data in each respective dataset. Also, the predictor-corrector sampler (PC-sampling) proposed in [36] was used to sample data with 1000 predictor and 1000 corrector steps.

We set two wavelet kernels k ( s ) : one is a low-pass filter k ( s ) = e -sx that captures signals in the low-frequency and the other k ( s ) = sxe -sx is a band-pass filter which is 0 at the origin. To obtain the combined effect of the low-pass and band-pass filters using a total of J scales, edges at one scale are filtered using the low-pass filter. For the remaining J -1 scales, the band-pass filter is applied. This approach enables selective extraction and processing of different frequency components across the spectral domain. The number of scales J was set to 11, 6, 4, and 6 for the Ego-small, Community-small, Grid, and QM9 datasets, respectively. More detailed experimental settings are given in Appendix B.

4.3 Experiment on Generic Graphs

To assess the versatility of Wave-GD, we used both real-world and synthetic datasets with varying sizes and characteristics. Experimental results on these diverse generic datasets demonstrate the robustness of Wave-GD in sampling graphs of various characteristics.

Baselines and metrics. For experiments on generic graph datasets (i.e., Ego-small, Community-small, and Grid), we compared the performance of our proposed method with the following autoregressive and one-shot graph generation methods: DeepGMG [16], GraphRNN [42], GraphAF [34],

-: results not reported in the original manuscripts, † : results obtained with the official public code.

GraphDF [20], GraphVAE [35], GNF [17], EDP-GNN [25], GDSS [13] and DiGress [37]. To evaluate the generation quality of the generated graphs, we used Maximum Mean Discrepancy (MMD) [7] to statistically measure similarities between probability distributions of sampled data and the real test data. Following prior works [13, 42], we compared distributions of three different graph statistics-degree, clustering coefficient, and orbit counts-to compute MMD between generated and test graphs.

Main results. Table 1 shows the comparisons of degree, clustering, orbit, and their averaged MMDs between the graphs generated from baselines and our method. The results show that our model significantly outperforms all baseline models for all metrics on Ego-small and Community-small datasets. Specifically, on the Ego-small dataset, Wave-GD demonstrated far smaller average MMDs by a margin of 0 . 108 over GraphVAE and at least 0 . 007 lower than DiGress. For the Communitysmall data, our method exhibited up to 0 . 601 lower MMD compared to GraphVAE and at least 0 . 024 better than GDSS. For the Grid dataset, our method outperformed all baselines in terms of orbit MMDand outperformed all one-shot baselines for the averaged MMD. Note that, while one-shot models struggle to generate large graphs due to their intricate and complex structures, our method achieved comparable results to GraphRNN on large grid graphs (with 100 ≤ N ≤ 400 ). This highlights the effectiveness of our approach in learning graphs with varying sizes and structures through multi-resolution learning.

Table 2 shows converged scales for different datasets. Interestingly, the scales of the large Grid dataset (with 100 ≤ N ≤ 400 nodes) were generally smaller compared to the Ego-small ( 4 ≤ N ≤ 18 ) and Community-small ( 12 ≤ N ≤ 20 ) datasets. Specifically, the converged scales of Ego-small ranged within [20 , 47] and the scales of Grid ranged within [12 , 36] . Note that the smaller scales capture local graph features with higher frequencies. Therefore, the results demonstrate that capturing local and detailed graph representations is more critical when dealing with complex and large graphs, in

—

urs

Train

HO

0.8889

0.9231

0.9231

0.9

0.9286

0.9286

0.9412

0.9412

0.9

0.8947

0.8889

0.9474

Ours

(a) Degree

(b) Clustering

(c) Orbit

(d) Average

0.08

0.14

0.02

0.08

0.07

GDSS—Ours

0.12

GDSS—Ours

GDSS—Ours

0.07

GDSS

一

Ours

0.06

0.1

0.015

0.06

MMD

0.05

MMD

0.08

MMD

MMD

0.05

0.04

0.03

0.06

0.01

0.04

0.03

0.02

0.04

0.005

0.02

0.01

0.02

0.01

0

0

0

0

1000

2000

3000

4000

5000

1000

2000

3000

4000

5000

1000

2000

3000

4000

5000

1000

2000

3000

4000

5000

Epoch

Epoch

Epoch

Epoch

comparison to smaller graphs. In other words, for the Ego-small and Community-small datasets, features in relatively low frequency (e.g., cluster-related features) were captured with larger scales.

Experiments with larger sample size. To provide a more definitive assessment of the generated sample quality, we conducted additional experiments by generating a larger number of samples compared to the test data, as in previous works [13, 17, 25]. For this, we sampled 1024 graphs for evaluating the MMD metrics for the Ego-small and Community-small datasets. As shown in Table 3, the lower MMDs of Wave-GD demonstrate superior performance to the baselines on Ego-small. For the Community-small dataset, GraphRNN performed better than ours on the averaged MMD, however, our method still surpasses recent score-based models [13,25] and normalizing flow model [17] with at most 0 . 066 margin. Along with these quantitative results, we present the actual visualization of the generated samples from Wave-GD in Fig. 3, which are highly similar to the training samples.

Stability analysis. In addition to MMDs, we present a stability analysis of Wave-GD in Fig. 4. The figure presents a comparison of the averaged MMDs between GDSS and our method on the Ego-small dataset. Specifically, between 2000-5000 epochs, while the difference between the maximum and minimum values of the averaged MMD of GDSS is 0 . 016 , Wave-GD shows better stability with a difference of ∼ 0 . 0018 at the same epochs, indicating that our method can much more robustly generate high-quality samples.

4.4 Experiment on Molecular Graphs

In addition to the generic graphs, we evaluated Wave-GD on a molecular benchmark dataset, which contains complex chemical bonds between atoms (i.e., nodes) and edges. By evaluating Wave-GD on

† : results obtained with the official public code.

the molecular graph generation task, we validated its ability to capture the complex dependencies between nodes and edges. Notably, Wave-GD successfully generated plausible graphs while adhering to the chemical valency rule, ensuring the realistic representation of molecular structures.

Baselines and evaluation metrics. In this experiment, we compared the results from seven baselines and Wave-GD on a real molecular QM9 dataset. Among the baselines, two are autoregressive flowbased models, namely GraphAF [34] and GraphDF [20], which employ discrete latent variables. Five others are one-shot generative models, which include MoFlow [44], GraphEBM [18], EDP-GNN [25], GDSS [13], and DiGress [37]. Following the standard procedure as in [20,34], the molecule data were kekulized by the RDKit library [15] and hydrogen atoms were removed.

We evaluated the quality of generated molecular samples with six metrics as in [13]: 1) Validity is the fraction of valid molecules that hold chemical rules. In our experiment, we did not apply any valency correction or edge resampling. 2) Novelty measures the proportion of valid molecules that are not present in the training set. 3) Uniqueness is the proportion of generated molecules that are not duplicated. 4) Neighborhood subgraph pairwise distance kernel (NSPDK) [3] is the MMD between the generated molecules and test molecules which considers both the nodes and edges for evaluation. 5) Fréchet ChemNet Distance (FCD) [30] evaluates the distance between distributions of the training and generated sets using the activations of the ChemNet layer. 6) Time quantifies the duration required to generate 10,000 molecules in the RDKit molecule format. In addition to these metrics, we also report the average values of validity, novelty, and uniqueness to present a unified evaluation over various aspects of the generated graphs.

Results. As shown in Table 4, our model outperformed all baselines on the average of validity, novelty, and uniqueness and showed the second-best results on validity, uniqueness, and NSPDK. To be specific, our method showed at most 27 . 49% p gain over GraphEBM and 18 . 06% p margin over DiGress in terms of the averaged statistic. Note that, DiGress performed better than our model in NSPDK and FCD, however, its lower novelty (35.31%) inherently leads to lower FCD and NSPDK scores. As the molecular generation task aims to create previously unseen molecular structures, novelty may be a more critical measure on which Wave-GD performed better.

In general, Wave-GD outperformed most of the baselines, excelling not only in generation quality but also in generation speed. It was faster than all autoregressive models, showing 442 × speed up compared to GraphDF and 21 × speed up compared to GraphAF. Moreover, our method requires a shorter generation time than discrete-time diffusion models. For example, our method was 36 × faster than EDP-GNN and 5 × faster than DiGress. These results demonstrate that employing a continuous-time diffusion process for converting graphs into noise and vice versa is considerably more efficient compared to the discrete-step noise perturbation utilized in EDP-GNN and DiGress.

4.5 Ablation Studies

Ablation study on oversmoothing. In Fig. 5, we present the effect of the number of GMH layers (i.e., depth D ) on the quality of generated samples. With larger D , while the sample quality of GDSS is decreased with high averaged MMDs (blue line) due to the oversmoothing, the average MMD of Wave-GD (yellow line) is consistently low representing high fidelity in the generated samples. This is because Wave-GD captures diverse graph frequency characteristics by adaptive multi-resolution filtering, which allows the model to preserve fine-grained details even in deeper layers.

0.12

Degree

Clustering

Orbits

Ours

GDSS

0.09

1

M

0.06

M

0.03

Q

2

3

5

6

Ablation study on the number of scales. In Table 5, we report degree, cluster, orbit MMDs, and their averaged value on the Community-small by changing the numbers of scales J . Also, validity, uniqueness, novelty, and their averaged value are presented on the QM9 dataset for multiple J 's. All experiments were replicated three times and the averaged results are reported. We observed that the averaged MMD slightly depends on the number of scales, however, the overall quality of generated samples is still better than existing baselines in most settings.

4.6 Limitations

Although Wave-GD is robust to varying numbers of scales as shown in Table 5, the choice of J differs across datasets which should be carefully chosen by a user. Also, as Wave-GD performs multi-resolution diffusion, it requires larger resources than conventional models during training. However, the diffusion processes are performed in parallel so there is no overhead in time. Moreover, as sampling is performed without multi-resolution filtering, the generation process is still quite efficient.

5 Conclusion

In this paper, we proposed Wave-GD which leverages spectral dependencies between node and edge signals to better characterize their joint distributions via a score-based diffusion model. By capturing their multi-resolution coherence, the model is able to generate graphs of high-fidelity preserving frequency characteristics of the graphs from the training samples. Extensive validation on various datasets and superior performance of Wave-GD highlights its significant potential for various application domains for graph modeling and generation.

Acknowledgement. This research was supported by NRF-2022R1A2C2092336 (40%), IITP-2022-000290 (40%), IITP-2019-0-01906 (AI Graduate Program at POSTECH, 10%), IITP-2022-2020-001461 (ITRC, 10%) from South Korea, and NSF IIS CRII 1948510 from the U.S.

References

Baek, J., Kang, M., et al.: Accurate learning of graph representations with graph multiset pooling. International Conference on Learning Representations (2021)

Cho, H., Wu, G., et al.: Mixing temporal graphs with MLP for longitudinal brain connectome analysis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 776-786. Springer (2023)

Costa, F., De Grave, K.: Fast neighborhood subgraph pairwise distance kernel. In: International Conference on Machine Learning. pp. 255-262 (2010)

De Cao, N., Kipf, T.: MolGAN: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973 (2018)

Erdös, P., Rényi, A.: On random graphs i. Publicationes Mathematicae (Debrecen) 6 , 290-297 (1959)

Grenander, U., Miller, M.I.: Representations of knowledge in complex systems. Journal of the Royal Statistical Society: Series B (Methodological) 56 (4), 549-581 (1994)

Gretton, A., Borgwardt, K.M., et al.: A kernel two-sample test. The Journal of Machine Learning Research 13 (1), 723-773 (2012)

Grover, A., Zweig, A., et al.: Graphite: Iterative generative modeling of graphs. In: International Conference on Machine Learning. pp. 2434-2444. PMLR (2019)

Hammond, D.K., Vandergheynst, P., et al.: Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis 30 (2), 129-150 (2011)

Hui, K.H., Li, R., et al.: Neural wavelet-domain diffusion for 3d shape generation. In: SIGGRAPH Asia 2022 Conference Papers. pp. 1-9 (2022)

Jin, W., Barzilay, R., et al.: Junction tree variational autoencoder for molecular graph generation. In: International Conference on Machine Learning. pp. 2323-2332. PMLR (2018)

Jin, W., Barzilay, R., et al.: Hierarchical generation of molecular graphs using structural motifs. In: International Conference on Machine Learning. pp. 4839-4848. PMLR (2020)

Jo, J., Lee, S., et al.: Score-based generative modeling of graphs via the system of stochastic differential equations. In: International Conference on Machine Learning. pp. 10362-10383. PMLR (2022)

Kim, W.H., Kim, H.J., et al.: Latent variable graphical model selection using harmonic analysis: applications to the Human Connectome Project (HCP). In: Computer Vision and Pattern Recognition. pp. 2443-2451 (2016)

Landrum, G., et al.: RDKit: Open-source cheminformatics software. 2016. URL http://www. rdkit. org/, https://github. com/rdkit/rdkit (2016)

Li, Y., Vinyals, O., et al.: Learning deep generative models of graphs. arXiv preprint arXiv:1803.03324 (2018)

Liu, J., Kumar, A., et al.: Graph normalizing flows. Advances in Neural Information Processing Systems 32 (2019)

Liu, M., Yan, K., et al.: GraphEBM: Molecular graph generation with energy-based models. arXiv preprint arXiv:2102.00546 (2021)

Liu, Q., Allamanis, M., et al.: Constrained graph variational autoencoders for molecule design. Advances in Neural Information Processing Systems 31 (2018)

Luo, Y., Yan, K., et al.: GraphDF: A discrete flow model for molecular graph generation. In: International Conference on Machine Learning. pp. 7192-7203. PMLR (2021)

Ma, T., Chen, J., et al.: Constrained generation of semantically valid graphs via regularizing variational autoencoders. Advances in Neural Information Processing Systems 31 (2018)

Ma, X., Wu, G., et al.: Learning multi-resolution graph edge embedding for discovering brain network dysfunction in neurological disorders. In: Information Processing in Medical Imaging. pp. 253-266. Springer (2021)

Madhawa, K., Ishiguro, K., et al.: GraphNVP: An invertible flow model for generating molecular graphs. arXiv preprint arXiv:1905.11600 (2019)

Mallat, S.: A wavelet tour of signal processing. Elsevier (1999)

Niu, C., Song, Y., et al.: Permutation invariant graph generation via score-based generative modeling. In: International Conference on Artificial Intelligence and Statistics. pp. 4474-4484. PMLR (2020)

Opolka, F., Zhi, Y.C., et al.: Adaptive gaussian processes on graphs via spectral graph wavelets. In: International Conference on Artificial Intelligence and Statistics. pp. 4818-4834. PMLR (2022)

Parisi, G.: Correlation functions and computer simulations. Nuclear Physics B 180 (3), 378-384 (1981)

Park, J., Hwang, Y., et al.: Convolving directed graph edges via Hodge Laplacian for brain network analysis. In: International Conference on Medical Image Computing and ComputerAssisted Intervention. pp. 789-799. Springer (2023)

Popova, M., Shvets, M., et al.: MolecularRNN: Generating realistic molecular graphs with optimized properties. arXiv preprint arXiv:1905.13372 (2019)

Preuer, K., Renz, P., et al.: Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. Journal of Chemical Information and Modeling 58 (9), 1736-1741 (2018)

Ramakrishnan, R., Dral, P.O., et al.: Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data 1 (1), 1-7 (2014)

Rogers, L.C.G., Williams, D.: Diffusions, Markov processes and martingales: Volume 2, Itô calculus, vol. 2. Cambridge University Press (2000)

Sen, P., Namata, G., et al.: Collective classification in network data. AI Magazine 29 (3), 93-93 (2008)

Shi, C., Xu, M., et al.: GraphAF: a flow-based autoregressive model for molecular graph generation. International Conference on Learning Representations (2020)

Simonovsky, M., Komodakis, N.: GraphVAE: Towards generation of small graphs using variational autoencoders. In: International Conference on Artificial Neural Networks. pp. 412422. Springer (2018)

Song, Y., Sohl-Dickstein, J., et al.: Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations (2021)

Vignac, C., Krawczuk, I., et al.: DiGress: Discrete denoising diffusion for graph generation. International Conference on Learning Representations (2023)

Vincent, P.: A connection between score matching and denoising autoencoders. Neural Computation 23 (7), 1661-1674 (2011)

Wang, H., Wang, J., et al.: GraphGAN: Graph representation learning with generative adversarial nets. In: Association for the Advancement of Artificial Intelligence. vol. 32 (2018)

Xu, B., Shen, H., et al.: Graph wavelet neural network. International Conference on Machine Learning (2019)

You, J., Liu, B., et al.: Graph convolutional policy network for goal-directed molecular graph generation. Advances in Neural Information Processing Systems 31 (2018)

You, J., Ying, R., et al.: GraphRNN: Generating realistic graphs with deep auto-regressive models. In: International Conference on Machine Learning. pp. 5708-5717. PMLR (2018)

Yu, J.J.Q., Gu, J.: Real-time traffic speed estimation with graph convolutional generative autoencoder. IEEE Transactions on Intelligent Transportation Systems 20 (10), 3940-3951 (2019)

Zang, C., Wang, F.: MoFlow: an invertible flow model for generating molecular graphs. In: ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 617-626 (2020)

Zhang, Q., Chang, J., et al.: Spatio-temporal graph structure learning for traffic forecasting. In: Association for the Advancement of Artificial Intelligence. vol. 34, pp. 1177-1185 (2020)

Appendix

In this appendix, we present 1) the proof of Lemma 1 given in the main paper, 2) additional implementation details of Wave-GD, 3) standard deviations of MMDs for generic datasets, 4) additional ablation studies on the number of scales on Ego-small and Grid datasets, 5) holistic architecture of two score-based models X θ and A ϕ , and 6) qualitative comparison of generated samples for all datasets which were not included in the main manuscript due to the page limit.

A Proof of Lemma 1

Lemma 1. Let G be an undirected graph, with a node signal X and a symmetric adjacency matrix A . Given wavelet coefficients W A ( s ) and W X ( s ) , which are spectral representations of X and A at a specific scale s , a dot product of these wavelet coefficients is equivalent to the product between the scale-filtered edge signals and the node signals in the graph space, i.e., W A ( s ) · W X ( s ) = A s X .

Proof. With ψ s = U k ( s Λ ) U T as the basis, the wavelet coefficients of the edge and node signals are given as follows:

As the k ( s Λ ) is a diagonal matrix and U is a set of orthonormal eigenvectors, the dot product of W A ( s ) and W X ( s ) is computed as

From Eq. (3) in the main paper, A s = ψ s · W A ( s ) = U k 2 ( s Λ ) U T A , and Eq. (11) becomes

as A s is symmetric. Therefore, a dot product of wavelet coefficients of the edge and node signals can be replaced with a spatial graph convolution form in the graph domain by applying an inverse transform only to the edge coefficients.

B Additional implementation setting

We utilized the PyTorch framework to implement Wave-GD and trained the model using a single NVIDIA GeForce RTX 3090 GPU. In Table 6, we provide details of the implementation settings of Wave-GD. For a fair comparison, we followed the same settings of data splitting and node features as in [13, 17, 25, 42]. For generic datasets, the node features are initialized as a one-hot encoding of the degrees and the generated adjacency matrices are quantized with the operator 1 x> 0 . 5 to obtain binary edges. As in [13], we used two types of SDEs: Variance Preserving (VP) SDE and Variance Exploding (VE) SDE [36] for diffusion processes. To solve the SDEs, we used the predictor-corrector sampler (PC sampler) [36] with either the Euler-Maruyama (EM) predictor or the reverse diffusion predictor (Rev), which discretizes the reverse-time SDE. As the corrector for the PC sampler, Langevin MCMC [6,27] was employed.

C Generation results with standard deviation on generic datasets

In Table 1 and Table 3 of the main manuscript, we reported the mean MMDs of three independent runs on generic datasets, using different model parameter initializations. In Table 7, we provide the standard deviation of degree, clustering, and orbit MMDs, as well as the time required to generate samples for each generic dataset. The generation time slightly increases in the experiments where more samples (i.e., 1024 samples) are generated on Ego-small and Community-small datasets. Furthermore, as the graph becomes more complicated with more nodes and edges, the generation time also increases accordingly.

D Ablation studies on the number of scales

Along with Table 5 in the main paper, we report the ablation studies on the number of scales J for Ego-small and Grid datasets in Table 8. All experiments with different J were replicated three times and their averaged results were reported. We found that the average MMDs on the Ego-small slightly differ depending on the J and that of Grid is relatively consistent along the J 's. In general, the number of scales affects the sample quality depending on the datasets, however, the averaged MMD of Wave-GD is still better than existing methods in most settings.

E The architecture of the score-based models of Wave-GD

GraphMulti-HeadAttention(GMH)

Q

=

GCN(At,0,Xt,0)

Attention

Q：

= GCN(At,D-1,Xt,D-1)

Attention

At0

Xt0

→iK=GCN(At,0,Xt,0)→

V

=GCN(At,0,Xt,0)

MLF

P

MLE

-K：

V

=GCN(At,D-1,Xt,D-1)

=GCN(At,D-1,Xt,D-1)→

MLF

Vx.logpt(Xt,At)

！

iQ = GCN(At,0,Xt,0)

Attention

Q = GCN(At,D-1,Xt,D-1)

1

Attention

→iK=GCN(At,0,Xt,0)→

MLP

MLE

→..→↓K=GCN(At,D-1,Xt,D-1)→

P

ML

At.0

0X

iV = GCN(At,0,Xt,0)

{V = GCN(At,D-1,Xt,D-1)

VA,logpt(Xt,At)

ishared

shared

iQ = GCN(At.0,Xt,0)

Attention

Q = GCN(At,D-1,Xt,D-1)

Attention

K=GCN(A0，Xt.0)→

MLP

→..→K=GCN(At,D-1,Xt,D-1)→

MLP

→A.。I.Il A,D→

iV = GCN(Ato,Xt,0)

IV = GCN(At,D-1,Xt,D-1)

At,0

0'x

VAs logpt(Xt,A)

Here we illustrate the architecture of score-based models X θ,t and A ϕ,t in Figure 6 and Figure 7, respectively, which are described in Section 3.5 in the main manuscript. Basically, we utilized graph multi-head attention (GMH) [1] as in GDSS [13], which is an iterative attention mechanism using graph convolutions as query, key, and values.

Given H t, 0 = X t , the attention operation GMH ( · ) is performed D times to estimate the partial scores ∇ X t log p t ( X t , A t ) and {∇ A s i t log p t ( X t , A s i t ) } J i =0 as follows:

respectively, where [ · ] represents the concatenation of GMH outputs and H t,d +1 = GNN ( H t,d , A t,d ) .

As shown in Figure 7, the score-based model A ϕ is shared across the multi-resolution edges and the given adjacency matrices. During training , it estimates the partial scores in parallel along the J +1 graphs using a graph convolution with node features X in the GMH layers, so that the model is able to capture generalized spectral coherence across different resolutions. However, in the sampling process, only the partial score of A t (i.e., ∇ A t log p t ( X t , A t ) ) is estimated from A ϕ as generating samples of multi-resolution edges is not our aim. For the other score-based model X θ , it produces the partial score ∇ X t log p t ( X t , A t ) in both training and the sampling processes as it does not rely on multi-resolution approach.

F Qualitative evaluation of generated samples

F.1 Comparison with state-of-the-art baselines on generic datasets

In Figures 8, 9, and 10, we qualitatively compare the generation results of Wave-GD on the generic datasets with two state-of-the-art diffusion models for graph generation: DiGress [37] and GDSS [13]. The visualized graphs demonstrate the superiority of Wave-GD which closely simulates training data compared to the SOTA baselines.

Train

Ours

A区米米丨丨

DiGress

S

AN

GDSS

S

Train

Ours

DiGress

GDSS

Train

Ours

GDSS

F.2 Comparison with state-of-the-art baselines on QM9 dataset

In Figure 11, we provide the visualization of training datasets and generated samples of Wave-GD for the molecular dataset. Along with the visualization, a quantitative comparison is also provided by measuring Tanimoto similarity based on Morgan fingerprints, which are calculated by the RDKit library [15]. For each generated molecule, the pairwise similarity between the training sample is calculated. As shown in the figure, Wave-GD successfully generates high-quality molecules that closely resemble the training data, whereas other baseline methods produce molecules that deviate from the training distribution.

n

a

0.9286

0.9412

0.9286

0.9412

0.9412

0.9

0.9474

0.9474

0.9375

0.9286

0.9231

0.9412

O

OH

DiGress

0.5263

0.5

0.381

0.4583

0.4706

0.45

0.6087

0.52

0.5238

0.5714

0.5556

0.5

0.8

0.8333

0.6667

0.5882

0.5789

0.5

0.6818

0.4091

0.4348

0.6

0.7143

0.5789

GDS

HO

OH

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Wave-GD, a score-based generative model, to generate graphs with high fidelity. By capturing the dependency between nodes and edges at multiple resolutions in the spectral space, it claims it overcomes the over-smoothing problem and achieves real-like frequency characteristics of nodes and edges.\n\nSTRENGTHS:\n1. The paper is well-written and easy to follow.\n2. The paper's motivation is clear, and the paper has shown some limitations that previous models may have.\n3. The idea of the paper is simple and straightforward.\n\nWEAKNESSES:\n1. The experiment is not sufficient to support the proposed method.\n\n(1) lack of baselines: some baselines that utilize spectral information/graph characteristics are missing. It would be great to see the author compare them.[1,2,3]\n\n(2) lack of more robust graph datasets: Ego-small and Community-small graphs are too simple, I think over-smoothing can't be a major issue for graphs in such scales. And how the \"multi-resolution\" should be defined in such small graphs? I suggest the authors run experiments on more complex graphs such as Planar graphs, SBM graphs, and large networks if possible with hundreds of nodes. \n\n(3) I suggest the author also include DiGress in Figure 5 -- since it also utilizes spectral information.\n\n(4) Another comment in Figure 5: it's not clear whether the performance drop when increasing #GNN layers is really due to the over-smoothing problem, more investigation should be conduct to further prove this phenomenom. \n\n2. While the author propose to also diffuse on the edge weights obtained from SGWT, there are also other ways to define the edge importance (e.g., edge conductance). Justiifcation should be provided why SGWT is the chosen over others.\n\n[1] Luo, Tianze, Zhanfeng Mo, and Sinno Jialin Pan. \"Fast Graph Generative Model via Spectral Diffusion.\" arXiv preprint arXiv:2211.08892 (2022).\n\n[2] Martinkus, Karolis, et al. \"Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators.\" International Conference on Machine Learning. PMLR, 2022.\n\n[3] Chen, Xiaohui, et al. \"Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling.\" arXiv preprint arXiv:2305.04111 (2023).\n\nQUESTIONS:\nPlease address the weakness above and:\n\n1. How exactly can the SGWT addresses the over-smoothing problem?\n\n2. Can you give a concrete example why it can capture \"multi-resolution\" of the graph, and can you provide a formal definition of the multi-resolution.\n\n3. what's the motivation of conducting the experiment with larger sample size?\n\nLIMITATIONS:\n1. The result on molecule generation is not that expressive.\n\n2. The contribution may be insignificant: there are many GNN design that may alleviate the over-smoothing problem. Any baseline model with a better score networks may overcome this limitation easily.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work the authors tackle the problem of graph generation learning where the goal is to learn the key features of a set of graphs and be able to generate graphs with similar properties. To that extend, the authors extend the GDSS (Jo 2022) method through an additional loss term (Eq. 7). This loss term encourages the employed GNNs to learn to reconstruct spectrally modified matrices A^s_i in addition to the normal adjacency matrices. The spectrally modified matrices are obtained through an SVD/PCA like approach where some spectral properties of the adjacency matrix are accentuated. This spectral accentuation is learnable as part of the training procedure.\nThe authors evaluate their approach on two real world and two synthetic datasets also used in previous studies. In terms of MMD (Mean maximum deviation) their approach frequently outperforms other methods employed for the task of graph generation learning. On molecule data, the employed procedure still ranks amount the best.\n\nSTRENGTHS:\nThe authors provide an interesting extension to an existing approach (GDSS), which allows to better learn the scales that are important for a specific set of graphs. The greatly increased stability during training compared to GDSS (Figure 3) seems promising. The paper contains extensive comparison to other methods. The presented approach is on par/outperforms these other methods on the presented datasets. Most computational overhead is in the training phase, inference is as fast as GDSS.\n\nWEAKNESSES:\nOverall the core weakness of the paper is its presentation, which has a lot of room for improvements and weak support for the main claims of the paper (last point).\n\nThe presentation of the SDE learning (lines 155-179) is not understandable without reading the GDSS paper. I think it could be drastically improved by highlighting the differences/ improvements in comparison to GDSS rather than repeating all the definitions/equations.\n\nThe description of Figure 2a) could be much improved as it is not well understandable without reading the rest of the paper first.\n\nSome of the equations and notation does not further the main cause of the paper: Section 3.1 beyond the first paragraph can almost entirely be cut. The introduced transformation $X^{s_i}$ on the edge signal X is not really used.\n\nLemma 1 seems to be misplaced as it introduced (lines 138ff) quite far away from where it is used (lines 190ff). Also it seems Lemma 1 is likely not a new result.\n\nThe bold highlights in Table 1 are not correct, in column “orbit”, three other methods outperform the presented approach but are not highlighted in bold. Also in lines 289f the text mismatches the table.\n\nIt is also unclear from the text how much hyperparameter tuning was done to achieve the results presented.\n\nLast but not least, the experimental section shows, from an applicant's point of view, that the proposed changes lead to empirical performance increase on the graph generation task. On the other hand the experimental section does not well support whether the introduced changes actually had the desired effect of learning different “scales” better. \n\nIt might as well be, that the introduced changes simply increase training robustness. More on this in the “Questions” section. It seems that one would need additional support for the main claim.\n\nQUESTIONS:\nIt would be interesting to see which scales lambda (as in lines 219, 220) have actually been obtained from the training. Do they differ significantly for different datasets? Would an equidistant choice of lambda be sufficient?\nSimilarly it would be interesting to see the relative strengths of the lambda_A/ lambda_A^{s_i} (as in eq.7). From a theoretical perspective (thinking of spectral clustering) one would assume that mostly low frequency modes are relevant for the community dataset, is that really the case?\nLastly, it would be interesting to see whether the performance increase obtained from the spectrally filtered adjacency matrix is any better than just introducing A^{s_i}\n\nLIMITATIONS:\nsee questions above", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a graph generative approach that leverages diffusion models and wavelet theory. The key concept revolves around utilizing the wavelet transform of the adjacency matrix across various scales, and learning a joint backward diffusion process that remains valid at all considered scales simultaneously. Consequently, the proposed approach exhibits a multi-resolution characteristic. The proposed approach is evaluated in the graph generation task using four benchmarks and is compared against autoregressive and one-shot approaches from previous works.\n\nSTRENGTHS:\nThe adaptation of diffusion models to the graph generation field is a timely and intriguing topic, considering the challenges posed by the discrete nature of graph data.\n\nThe experimental evaluation provides substantial support for the claims made in the paper. The proposed approach outperforms recent methods on three benchmark in the graph generation task and achieves comparable results in molecule generation task.\nThe experimental setup and the metric considered for the evaluation are clearly presented.\n\nWEAKNESSES:\nHaving to choose the parameter *J* without any insight or guidance for different datasets can be a drawback in practical applications. I would suggest to further investigate the relationship between performance and *J* among different datasets, along with the impact of graph statistics on the optimal *J*. Also, it is not clear which how many scales have been used to obtain the results reported in Tables 1-3.\n\nThe time complexity of the proposed approach, as indicated in Table 3, could present challenges in certain settings. Further discussion or potential mitigation strategies for addressing this issue would be beneficial.\n\nClarity could be improved in certain sections. For instance, Figure 1's purpose is unclear to me, and the role of spectral coherence between nodes and edges in the proposed approach needs better explanation. Providing a clear definition of the scale (s) domain and kernel function (k) in the preliminary sections would also help readers understand the concepts more quickly.\n\nQUESTIONS:\nIt would be helpful if the authors could provide clarification on why only $A^s$ and not $X^s$ is considered for learning the diffusion process.\n\nRegarding Table 4, should the lower numbers indicate better results? In the QM9 part of the table, higher numbers are bold. Is my interpretation wrong?\n\nThe following paper, which explores diffusion in the wavelet coefficient space for 3D shape generation, could be an interesting reference: \\\n_Hui, Ka-Hei, et al. \"Neural wavelet-domain diffusion for 3d shape generation.\" SIGGRAPH Asia 2022 Conference Papers. 2022_\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper claims the node feature and graph topology are not coherent in most previous generative graph models and high-frequency signals in node features and graph topology may neglect during the generation process. Therefore, they propose a Wavelet graph diffusion model (Wave-GD) with score-based diffusion. Specifically, it uses different graph wavelet bases to get graph signals in different frequency ranges. The overall model diffuses node features, the original graph, and the adjacency matrix constructed by graph wavelet bases. To improve the coherence between node features and graph topology, the score-based models are based on graph multi-head attention layers, which take the product of node features and adjacency matrices including the original one and ones learned by different bases, which alleviates the gap between node features and graph structures. Performance on three small synthetic datasets and one real-world molecular dataset show that the proposed model can generate graphs that are not only realistic graphs in shape but also obey the chemical rules in high fidelity.\n\nSTRENGTHS:\n1.\tThe novel part is the paper considers high frequency and discovers the coherence between node features and graph structures in diffusion models but still uses a simple dot product to solve it.\n2.\tThe empirical results on real-world molecular datasets show the effectiveness in terms of generating realistic graphs with high fidelity and high novelty in a relatively fast time. \n3.\tThe model enjoys high flexibility regarding different tasks where they need frequency graph signals at different scales.\n\nWEAKNESSES:\n1.\t\"nodes and edges\" in line 26 is misleading, it would be better if you mention \"node\" means \"node features\" and \"edges\" means \"graph structures\" in the introduction and then use \"node\" and \"edge\" for simplicity.\n2.\tMulti-resolution and coherence are two major claims in your paper, but it is not verified that coherence improves performance. \n3.\tthe proposed model is limited to small graphs due to the decomposition in spectral graph wavelet bases.\n\nQUESTIONS:\nQ1: Would you give some experimental analysis that can show if the main contribution comes from coherence or multi-resolution?\n\n\nQ2: how do you define $s$ for graph wavelet bases?\n\nLIMITATIONS:\nYes. It may produce molecules that may harm the body.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}