SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} W _ {\mathbf {A}} (s) \cdot W _ {\mathbf {X}} (s) = (\mathbf {U} k (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {A}) ^ {T} (\mathbf {U} k (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {X}) \\ = \mathbf {A} ^ {T} \mathbf {U} k ^ {T} (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {U} k (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {X} \\ = \mathbf {A} \mathbf {U} k (s \boldsymbol {\Lambda}) k (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {X} \tag {11} \\ = \mathbf {A} \mathbf {U} k ^ {2} (s \boldsymbol {\Lambda}) \mathbf {U} ^ {T} \mathbf {X}. \\ \end{array}
$$
From Eq. (3) in the main paper,  $\mathbf{A}^s = \psi_s\cdot W_{\mathbf{A}}(s) = \mathbf{U}k^2 (s\mathbf{\Lambda})\mathbf{U}^T\mathbf{A}$ , and Eq. (11) becomes
$$
\begin{array}{l} W _ {\mathbf {A}} (s) \cdot W _ {\mathbf {X}} (s) = \left(\mathbf {A} ^ {s}\right) ^ {T} \mathbf {X} \tag {12} \\ = \mathbf {A} ^ {s} \mathbf {X}. \\ \end{array}
$$
as  $\mathbf{A}^s$  is symmetric. Therefore, a dot product of wavelet coefficients of the edge and node signals can be replaced with a spatial graph convolution form in the graph domain by applying an inverse transform only to the edge coefficients.
B Additional implementation setting
We utilized the PyTorch framework to implement Wave-GD and trained the model using a single NVIDIA GeForce RTX 3090 GPU. In Table 6, we provide details of the implementation settings of Wave-GD. For a fair comparison, we followed the same settings of data splitting and node features as in [13, 17, 25, 42]. For generic datasets, the node features are initialized as a one-hot encoding of the degrees and the generated adjacency matrices are quantized with the operator  $1_{x > 0.5}$  to obtain binary edges. As in [13], we used two types of SDEs: Variance Preserving (VP) SDE and Variance Exploding (VE) SDE [36] for diffusion processes. To solve the SDEs, we used the predictor-corrector sampler (PC sampler) [36] with either the Euler-Maruyama (EM) predictor or the reverse diffusion predictor (Rev), which discretizes the reverse-time SDE. As the corrector for the PC sampler, Langevin MCMC [6, 27] was employed.
Table 6: Hyperparameters of Wave-GD for all datasets. We provide hyperparameters and detailed settings for training two score-based models  $\mathcal{X}_{\theta}$  and  $\mathcal{A}_{\phi}$  and diffusion processes with SDE.
Hyperparameter	Ego-small	Community-small	Grid	QM9
Xθ	Number of attention heads	4	4	4	4
Number of initial channels	2	2	2	2
Number of hidden channels	8	8	8	8
Number of final channels	4	4	4	4
Number of GCN layers	5	5	7	3
Hidden dimension	32	32	32	16
Aφ	Number of scales J	11	6	4	6
Number of attention heads	4	4	4	4
Number of initial channels	2	2	2	2
Number of hidden channels	8	8	8	8
Number of final channels	4	4	4	4
Number of GCN layers	5	5	7	3
Hidden dimension	32	32	32	16
SDE for X	Type	VP	VP	VP	VE
Number of sampling steps	1000	1000	1000	1000
βmin	0.1	0.1	0.1	0.1
βmax	1.0	1.0	1.0	1.0
SDE for {Asi}j=0	Type	VP	VP	VP	VE
Number of sampling steps	1000	1000	1000	1000
βmin	0.1	0.1	0.2	0.1
βmax	1.0	1.0	0.8	1.0
Solver	Type	EM	EM + Langevin	Rev. + Langevin	Rev. + Langevin
SNR	-	0.05	0.1	0.2
Scale coefficient	-	0.7	0.7	0.7
Train	Optimizer	Adam	Adam	Adam	Adam
Learning rate	8 × 10-4	5 × 10-3	1 × 10-3	2 × 10-3
Weight decay	1 × 10-4	1 × 10-4	1 × 10-4	1 × 10-4
Batch size	128	128	8	1024
Number of epochs	5000	5000	6000	300
Exponential Moving Average	-	-	0.999	-
C Generation results with standard deviation on generic datasets
In Table 1 and Table 3 of the main manuscript, we reported the mean MMDs of three independent runs on generic datasets, using different model parameter initializations. In Table 7, we provide the standard deviation of degree, clustering, and orbit MMDs, as well as the time required to generate samples for each generic dataset. The generation time slightly increases in the experiments where more samples (i.e., 1024 samples) are generated on Ego-small and Community-small datasets. Furthermore, as the graph becomes more complicated with more nodes and edges, the generation time also increases accordingly.
Table 7: Generation results of Wave-GD on the generic datasets. The results are the mean MMDs of three runs and their standard deviation.
Ego-small
# of test data	# of samples	Degree	Cluster	Orbit	time (s)
40	40	0.012 ± 0.001	0.010 ± 0.001	0.005 ± 0.001	40.03
1024	0.010 ± 0.001	0.018 ± 0.003	0.005 ± 0.002	40.95
Community-small
20	20	0.007 ± 0.002	0.058 ± 0.004	0.002 ± 0.0	89.80
1024	0.016 ± 0.003	0.077 ± 0.006	0.001 ± 0.002	94.75
Grid
20	20	0.144 ± 0.004	0.004 ± 0.001	0.021 ± 0.001	733.79
D Ablation studies on the number of scales
Along with Table 5 in the main paper, we report the ablation studies on the number of scales  $J$  for Ego-small and Grid datasets in Table 8. All experiments with different  $J$  were replicated three times and their averaged results were reported. We found that the average MMDs on the Ego-small slightly differ depending on the  $J$  and that of Grid is relatively consistent along the  $J$ 's. In general, the number of scales affects the sample quality depending on the datasets, however, the averaged MMD of Wave-GD is still better than existing methods in most settings.
Table 8: Ablation studies on the number of scales.
Ego-small	Grid
J	Deg.	Clus.	Orb.	Avg.	J	Deg.	Clus.	Orb.	Avg.
8	0.019	0.016	0.011	0.015	3	0.142	0.003	0.034	0.059
9	0.016	0.001	0.015	0.011	4	0.144	0.004	0.021	0.056
10	0.015	0.008	0.005	0.009	5	0.161	0.001	0.015	0.059
11	0.012	0.010	0.005	0.009	6	0.168	0.005	0.015	0.062
12	0.018	0.016	0.013	0.016	-	-	-	-	-
E The architecture of the score-based models of Wave-GD
Figure 6: The architecture of  $\mathcal{X}_{\theta}$  that estimates the partial score  $\nabla_{\mathbf{X}_t}\log p_t(G_t)$ .
Figure 7: The architecture of  $\mathcal{A}_{\phi}$  that estimates the partial scores  $\nabla_{\mathbf{A}_t} \log p_t(G_t)$  and  $\{\nabla_{\mathbf{A}_t^{s_i}} \log p_t(G_t^{s_i})\}_{i=1}^J$ .
Here we illustrate the architecture of score-based models  $\mathcal{X}_{\theta ,t}$  and  $\mathcal{A}_{\phi ,t}$  in Figure 6 and Figure 7, respectively, which are described in Section 3.5 in the main manuscript. Basically, we utilized graph multi-head attention (GMH) [1] as in GDSS [13], which is an iterative attention mechanism using graph convolutions as query, key, and values.
Given  $H_{t,0} = \mathbf{X}_t$ , the attention operation  $\mathrm{GMH}(\cdot)$  is performed  $D$  times to estimate the partial scores  $\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$  and  $\{\nabla_{\mathbf{A}_t^{s_i}}\log p_t(\mathbf{X}_t,\mathbf{A}_t^{s_i})\}_{i = 0}^J$  as follows:
$$
\mathcal {X} _ {\theta , t} \left(G _ {t}\right) = \operatorname {M L P} _ {\theta} \left(\left[ \left\{\operatorname {G M H} _ {\theta} \left(H _ {t, d}, \mathbf {A} _ {t, d}\right) \right\} _ {d = 0} ^ {D} \right]\right), \tag {13}
$$
$$
\mathcal {A} _ {\phi , t} \left(G _ {t} ^ {s _ {i}}\right) = \operatorname {M L P} _ {\phi} \left(\left[ \left\{\operatorname {G M H} _ {\phi} \left(H _ {t, d}, \mathbf {A} _ {t, d} ^ {s _ {i}}\right) \right\} _ {d = 0} ^ {D} \right]\right),
$$
respectively, where  $[\cdot ]$  represents the concatenation of GMH outputs and  $H_{t,d + 1} = \mathrm{GNN}(H_{t,d},\mathbf{A}_{t,d})$
As shown in Figure 7, the score-based model  $\mathcal{A}_{\phi}$  is shared across the multi-resolution edges and the given adjacency matrices. During training, it estimates the partial scores in parallel along the  $J + 1$  graphs using a graph convolution with node features  $\mathbf{X}$  in the GMH layers, so that the model is able to capture generalized spectral coherence across different resolutions. However, in the sampling process, only the partial score of  $\mathbf{A}_t$  (i.e.,  $\nabla_{\mathbf{A}_t} \log p_t(\mathbf{X}_t, \mathbf{A}_t)$ ) is estimated from  $\mathcal{A}_{\phi}$  as generating samples of multi-resolution edges is not our aim. For the other score-based model  $\mathcal{X}_{\theta}$ , it produces the partial score  $\nabla_{\mathbf{X}_t} \log p_t(\mathbf{X}_t, \mathbf{A}_t)$  in both training and the sampling processes as it does not rely on multi-resolution approach.
F Qualitative evaluation of generated samples
F.1 Comparison with state-of-the-art baselines on generic datasets
In Figures 8, 9, and 10, we qualitatively compare the generation results of Wave-GD on the generic datasets with two state-of-the-art diffusion models for graph generation: DiGress [37] and GDSS [13]. The visualized graphs demonstrate the superiority of Wave-GD which closely simulates training data compared to the SOTA baselines.
Figure 8: Non-curated samples generated by Wave-GD, DiGress, and GDSS trained on the Ego-small dataset.
Figure 9: Non-curated samples generated by Wave-GD, DiGress, and GDSS trained on the Community-small dataset.
Figure 10: Non-curated samples generated by Wave-GD, DiGress, and GDSS trained on the Grid dataset.
F.2 Comparison with state-of-the-art baselines on QM9 dataset
In Figure 11, we provide the visualization of training datasets and generated samples of Wave-GD for the molecular dataset. Along with the visualization, a quantitative comparison is also provided by measuring Tanimoto similarity based on Morgan fingerprints, which are calculated by the RDKit library [15]. For each generated molecule, the pairwise similarity between the training sample is calculated. As shown in the figure, Wave-GD successfully generates high-quality molecules that closely resemble the training data, whereas other baseline methods produce molecules that deviate from the training distribution.
Figure 11: Visualization of the sampled graphs on the QM9 dataset with maximum Tanimoto similarity. For each graph, we display the one-to-one similarity to the real training samples at the same location in the upper panel.