SlowGuess/NeurIPS_2023_OCR huggingface dataset:
D4Explainer has a search space of  $\mathcal{O}(N^{2K})$  for modifying  $K$  edges in an  $N$ -nodes graph, which is larger than previous counterfactual explainers that only consider deleting edges. By framing the explanation task as a generation problem, the space complexity of each layer in D4Explainer is reduced to  $\mathcal{O}(N^2)$ . The time complexity is  $\mathcal{O}(N^3)$  due to the matrix multiplication. Despite the large search space, the complexity of D4Explainer is still acceptable and faster than some generation-based explanations [10]. Runtime and more complexity analysis are given in Appendix E.6. Furthermore, we directly recover the terminal explanation  $\tilde{G}_0$  in the training procedure, rather than intermediate  $G_t$ , which greatly increases the efficiency of D4Explainer. The Denoising Model can also be trained in parallel under different noise levels without iterative optimization from  $t = 0$  to  $t = T$ .
We test the proposed approach to explain the performance of node classification models and graph classification models. Dataset statistics and classifier information are summarized in Appendix E.1.
Table 1: CF-ACC AUC and Fidelity (FID) AUC of D4Explainer and baseline explainers over eight datasets. We report AUC values computed over 10 modification ratios from 0 to 0.3. The best result is in bold and the second best result is underlined.
Models	BA-Shapes	Tree-Cycle	Tree-Grids	Cornell	BA-3Motif	Mutag	BBBP	NCII
CF-ACC	FID	CF-ACC	FID	CF-ACC	FID	CF-ACC	FID	CF-ACC	FID	CF-ACC	FID	CF-ACC	FID	CF-ACC	FID
Random	0.251	0.261	0.260	0.281	0.337	0.375	0.138	0.172	0.404	0.452	0.192	0.256	0.073	0.113	0.288	0.352
GNNExplainer	0.473	0.444	0.652	0.580	0.672	0.622	0.075	0.120	0.250	0.253	0.450	0.449	0.212	0.241	0.375	0.443
SAExplainer	0.773	0.773	0.405	0.408	0.547	0.542	0.199	0.241	0.474	0.500	0.300	0.338	0.110	0.133	0.421	0.446
GradCam	0.552	0.570	0.637	0.613	0.590	0.578	0.138	0.189	0.459	0.495	0.202	0.250	0.274	0.301	0.467	0.488
IGExplainer	0.208	0.240	0.198	0.226	0.308	0.372	0.233	0.281	0.440	0.474	0.231	0.280	0.159	0.183	0.347	0.389
PGExplainer	0.361	0.357	0.353	0.322	0.293	0.340	0.128	0.204	0.320	0.323	0.208	0.313	0.233	0.282	0.338	0.366
PGMExplainer	0.208	0.210	0.242	0.214	0.128	0.237	0.206	0.274	0.212	0.213	0.128	0.251	0.105	0.154	0.348	0.390
CXPlain	0.125	0.168	0.245	0.220	0.222	0.274	0.132	0.180	0.235	0.239	0.187	0.305	0.067	0.131	0.489	0.484
CF-GNNExplainer	0.773	0.728	0.812	0.718	0.537	0.527	0.328	0.297	0.302	0.304	0.797	0.751	0.623	0.632	0.715	0.674
D4Explainer	0.838	0.828	0.917	0.862	0.905	0.832	0.623	0.559	0.912	0.922	0.765	0.675	0.781	0.739	0.737	0.690
Node classification. For synthetic datasets, we use BA-Shapes, Tree-Cycle, Tree-Grids [6]. There exists a motif that plays an important role in the model's prediction. The node labels are determined by the structural roles. We train a vanilla GCN for synthetic datasets, achieving over  $95\%$  accuracy on each synthetic dataset. Additionally, we use Cornell [52] dataset, a highly heterophilous real-world webpage graph. Wherein, more complex relationships exist between a node and its neighbors, thus posing a more significant challenge to the explanation tasks. We train an EGNN [53], which is specifically designed for heterophilous graphs, achieving  $83\%$  accuracy on Cornell.
Graph classification. We use one synthetic dataset, BA-3Motif [54] and three real-world molecule datasets, Mutag [55, 56], BBBP [57] and NCI1 [58] for graph-classification task explanation. BA-3Motif contains 3 graph classes: graphs with cycle motif, grid motif, and house motif. Mutag, BBBP, and NCI1 are molecular datasets where nodes represent atoms and graphs represent molecules. Specifically, the chemical functionalities of molecules determine the graph labels. We train a vanilla GCN for BA-3Motif, BBBP, and NCI1. For the Mutag dataset, GIN [59] is used as the target GNN.
Baselines. For the counterfactual explanation task, we take the same baseline setup as CF-GNNExplainer[13] and involve more recent state-of-the-art explainers as our baselines, including GNNExplainer [6], SAExplainer [29], GradCam [21], IGExplainer [22], PGExplainer [8], PGMExplainer [9], and CXPlain [60]. For the methods that are originally designed for the factual explanation, we construct a subgraph with the least important edges as the counterfactual explanation. For the model-level explanation task, we compare with XGNN [15], which is a state-of-the-art model-level explanation method for GNNs. More implementation details are given in Appendix E.3.
5.2 Counterfactual Explanations
Metrics. Following evaluation protocols of prior works [13, 36], we adopt Counterfactual Accuracy, Fidelity, and Modification Ratio (MR) as our metrics. Let  $G^{o}$  and  $G^{c}$  denote the original input graph and generated counterfactual graph, respectively.  $\mathcal{G}$  is the test dataset. Counterfactual Accuracy is defined as the proportion of generated explanations that change the model's prediction, CF-ACC = 1 - 1/|G| ∑G°∈G(1(Hg=c - hg)). Fidelity measures the change in output probability over the original class, i.e., Fidelity = 1/|G| ∑G°∈G[f(G{o})[hg] - f(Gc)[hg]]. Modification Ratio refers to the proportion of changed edges as MR = (# of deleted edges + # of added edges)/|E|. Higher counterfactual accuracy and fidelity with lower modification ratios indicate better performance.
Results. CF-ACC and Fidelity are sensitive to the modification ratio, we thus compute the areas under CF-ACC curve and Fidelity curve over 10 different modification ratios from 0 to 0.3. We run 10 different seeds for each approach and report the average in Table 1. As can be seen from the table, D4Explainer achieves the best performances on seven out of eight datasets, with especially strong CF-ACC AUC values ( $>90\%$ ) on Tree-Cycle, Tree-Grids, and BA-3Motif. Notably, D4Explainer consistently works well on explaining both node classification and graph classification tasks, while the efficacy of baselines is unstable across datasets. For instance, most baselines fail to generate effective counterfactual explanations for complex graphs with multiple motifs or heterophilous edge relations, e.g., Cornell and BA-3Motif.
To further investigate the relation between CF-ACC and the modification ratio, we show the change of CF-ACC w.r.t. modification ratios from 0 to 0.3 in Figure 4, where the X-axis is in the log scale.
Figure 4: CF-ACC Curves of all explainers over different modification ratios from 0 to 0.3. The  $x$ -axis is shown in the log scale. CF-ACC tends to increase as the modification ratio increases in general.
Table 2: MMD distances between the generated explanations and test graphs. We report MMD distances of degree distributions (Deg.), cluster coefficients (Clus.), spectrum distributions (Spec.), and the summation (Sum.). We bold the best value and underlined the second-best value.
Models	Mutag	BBBP	NCI1
Deg.	Clus.	Spec.	Sum.	Deg.	Clus.	Spec.	Sum.	Deg.	Clus.	Spec.	Sum.
RandomCaster	0.1593	0.0247	0.0417	0.2257	0.1693	0.0072	0.0397	0.2162	0.1847	1.9769	0.0404	2.2020
GNNExplainer	0.1614	0.0002	0.0409	0.2025	0.1615	0.0002	0.0395	0.2012	0.1577	0.0005	0.0405	0.1987
SAExplainer	0.0940	0.0032	0.0412	0.1384	0.1594	0.0032	0.0402	0.2028	0.189	0.0002	0.0408	0.2300
GradCam	0.1122	0.0083	0.0416	0.1621	0.0699	0.0026	0.0384	0.1109	0.1638	0.0003	0.0404	0.2045
IGExplainer	0.1292	0.0000	0.0411	0.1703	0.0908	0.0000	0.0394	0.1302	0.4288	0.0002	0.0398	0.4688
PGExplainer	0.1475	0.0002	0.0418	0.1895	0.2014	0.0018	0.0403	0.2435	0.1937	0.0000	0.0396	0.2333
PGMExplainer	0.1800	0.0002	0.0419	0.2221	0.1916	0.0003	0.0403	0.2322	0.2199	0.0000	0.0404	0.2603
CXPlain	0.1734	1.2706	0.0417	1.4857	0.1768	0.0001	0.0394	0.2163	0.1629	0.0001	0.0404	0.2034
CF-GNNExplainer	0.1172	0.0000	0.0380	0.1552	0.0870	0.0001	0.0393	0.1264	0.1224	0.0001	0.0404	0.1629
D4Explainer	0.1172	0.0000	0.0244	0.1416	0.0530	0.0000	0.0331	0.0861	0.1006	0.0000	0.0353	0.1359
As illustrated in Figure 4, D4Explainer consistently achieves the highest CF-ACC with the smallest modification ratio (see the right side of the X-axis). Especially for Tree-Cycle and BBBP dataset, D4Explainer obtains a significant boost compared to the baselines. It demonstrates that D4Explainer can generate counterfactual explanations that can strongly influence the prediction of the target GNN and reflect the effective counterfactual properties.
5.2.1 In-Distribution Evaluation
To evaluate the in-distribution property of the generated explanations, we adopt the maximum mean discrepancy (MMD) to compare distributions of graph statistics between the generated counterfactual explanations and original test graphs. Following the evaluation setting in prior works [61, 19, 20, 62, 44], we use Gaussian Earth Mover's Distance kernel to compute MMDs of degree distributions, clustering coefficients, and spectrum distributions. Smaller MMDs mean that the two distributions are more similar and close, which indicates a better in-distribution property.
Results. Table 2 shows the MMD results on three real-world molecular datasets. We observe that D4Explainer outperforms baselines in general. Especially for BBBP and NCI1 datasets, D4Explainer achieves the lowest MMD distances across all metrics. The MMD results verify the effectiveness of D4Explainer in capturing the underlying distribution of datasets and generating in-distribution and more faithful explanations. We refer to Appendix E.4 for more results.
5.2.2 Additional Faithfulness Aspects
Explanation Diversity Evaluation. We evaluate the diversity of counterfactual explanations in Figure 5 and Appendix E.5. The first row shows the original graphs. The second row shows the generated counterfactual explanations by CF-GNNExplainer [6], where only edge deletion is allowed. With edge addition, D4Explainer is capable of generating alternative counterfactual explanations from a different perspective. As can be found from Figure 5, there are two main approaches to generating counterfactual explanations. The first one is deleting determinant edges and destroying the original motif, thus greatly influencing the model's prediction. The second one is converting the original motifs to truly counterfactual motifs through both deleting and adding essential edges. Previous methods can only produce the first type of counterfactual explanations, while D4Explainer makes the second approach possible and successful, leading to alternative and diverse counterfactual
explanations. We ascribe the success to the special training mechanism of D4Explainer. The intrinsic stochasticity in the forward process allows D4Explainer to take as input a sequence of noisy versions of the original graph, instead of a singular input graph. This enlarges the search space of possible counterfactual explanations for D4Explainer.
Figure 5: Counterfactual explanations comparison. Red labels represent the motifs in the graph.
Figure 6: Top-K accuracy w.r.t. noise levels on BBBP dataset
Robustness Evaluation. To evaluate the robustness of all methods, we compare the counterfactual explanations produced on the original graph and its perturbed counterpart, respectively. A robust model would predict the same explanation for both inputs. Following previous setup [36], we identify the  $K$  most relevant edges in the original counterfactual explanation and compute the fraction of these edges present in the explanation of its noisy version, denoted by Top-  $K$  Accuracy. We apply noise by randomly adding or removing edges with probability  $\sigma$ . A consistent  $20\%$  modification ratio is used across all methods. Results on BBBP dataset are shown in Figure 6. We observe that D4Explainer outperforms all baselines over different noise levels from 0 to  $10\%$ . We restrict that  $\sigma < 10\%$ , as the larger noise may cause the noisy graph to switch the predicted label. Overall, results in Figure 6 verify D4Explainer's strong ability to generate consistently effective counterfactual explanations despite the noise. See Appendix E.7 for complete results.
5.3 Model-level Explanations
In each step of the reverse sampling, we denoise  $K$  candidate graphs from the noisy graph and select a temporary explanation. Following the setting in XGNN [15], we qualitatively evaluate the generated explanations with different pre-defined numbers of nodes  $N$ , shown in Figure 7.  $p$  denotes the target class probability predicted by the GNN. A higher  $p$  indicates higher explanation confidence. We observe that D4Explainer can produce more determinant graph patterns with nearly  $100\%$  confidence for synthetic datasets, e.g., BA-shapes and BA-3Motif.
Figure 7: Qualitative evaluation of D4Explainer
Quantitative evaluation. We adopt the target class probability  $p$  and Density as the quantitative metrics. Density measures the sparsity level of the explanations, which is defined as Density =  $|\mathcal{E}| / |\mathcal{V}|^2$ , where  $\mathcal{E}$  and  $\mathcal{V}$  denote the set of edges and nodes in the explanation. Quantitative comparisons between XGNN and D4Explainer under different numbers of nodes are shown in Table 3. Hyperparameter sensitivities of the  $K$  (number of candidates in each step) and  $T$  (number of reverse sampling steps) are shown in Table 4. The results are averaged over 100 generated model-level explanations without
any regularization constraints in the selection policy. We find that (1) D4Explainer is capable of generating sparse and succinct model-level explanations with high target class probabilities, even without any regularization constraints on the explanation size. The superiority can be attributed to our distribution learning objective. However, it is worth noting that training graphs might be noisy and densely self-connected in some real-world applications. A regularization constraint can be easily plugged into the selection policy if required by downstream tasks; (2) smaller  $T$  and  $K$  both degrade the performance and quality of model-level explanations, which further emphasize the effectiveness of candidates and multi-step sampling. In the implementation, we ensure  $K \geqslant 20$  and  $T \geqslant 50$  for a balance between the quality and time complexity.
Table 3: Quantitative comparison in terms of probability and density with different numbers of nodes.
# nodes	Mutag	Tree-Cycle
6	7	8	5	6	7
Ours	Prob.	0.832	0.856	0.920	0.991	0.995	0.989
Density	0.278	0.327	0.315	0.400	0.381	0.343
XGNN	Prob.	0.523	0.824	0.875	0.968	0.989	0.992
Density	0.537	0.479	0.437	0.400	0.390	0.367
Table 4: Hyperparameter sensitivity in model-level explanation generation
Mutag (N=6)	Tree-Cycle (N=6)
Prob.	Density	Prob.	Density
(1) K = 10, T = 50	0.799	0.314	0.987	0.372
(2) K = 20, T = 10	0.524	0.284	0.991	0.388
(3) K = 20, T = 50	0.812	0.295	0.994	0.361
(4) K = 20, T = 100	0.832	0.278	0.992	0.325
(5) K = 30, T = 50	0.823	0.287	0.997	0.361
6 Conclusion and Broader Impacts
In this work, we propose D4Explainer, a novel generative approach for counterfactual and model-level explanations based on a discrete denoising diffusion model. By framing the explanation problem as a distribution learning task, D4Explainer can generate more reliable explanations with better in-distribution property, diversity and robustness. Additionally, D4Explainer can simultaneously perform model-level explanations with a pre-trained denoising model.
While denoising diffusion models show promise for explaining Graph Neural Networks (GNNs), they face potential scalability concerns on large graphs. Additionally, the explanations rely on the specific GNN architecture, limiting their generalizability across different GNN models. This work has dual social impacts. It enhances the transparency and interpretability of GNNs. However, it is vital to acknowledge the limitations and potential risks of relying solely on these explanations. They may not always capture the complete causal relationships in complex graph structures, which could lead to unintended consequences, reinforce biases, or make incorrect assumptions about the model's behavior. Looking ahead, an interesting direction for future research is to consider the node attributes and edge attributes during the explanation generation, e.g., by performing diffusion processes over continuous features.
[1] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
[2] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[3] Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. CoRR, 2020.
[4] Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10772-10781, 2019.
[5] Enyan Dai, Tianxiang Zhao, Huaisheng Zhu, Junjie Xu, Zhimeng Guo, Hui Liu, Jiliang Tang, and Suhang Wang. A comprehensive survey on trustworthy graph neural networks: Privacy, robustness, fairness, and explainability. arXiv preprint arXiv:2204.08570, 2022.
[6] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, pages 9240-9251, 2019.
[63] Paul Erdős, Alfréd Rényi, et al. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1):17-60, 1960.
[64] Asim Kumar Debnath, Rosa L Lopez de Compadre, Gargi Debnath, Alan J Shusterman, and Corwin Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2):786-797, 1991.
[65] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[66] Zhiyuan Li and Sanjeev Arora. An exponential learning rate schedule for deep learning. arXiv preprint arXiv:1910.07454, 2019.
The main notations used throughout this paper are summarized in Table 5.
Table 5: Summary of the notations
Notation	Description
G	An input graph / computational graph for a given node
G0	Original graph i.e., G
X0	Node features
t	The timestep of the forward diffusion
Gt	Noisy graph at timestep t
f	A well-trained GNN classifier to be explained
ŶG	The label of graph G predicted by f
Gc	Counterfactual explanation of G
{C1,···,Cl}	Class set of the input graphs
Gm	Model-level explanation for a certain class
q(Gt|Gt-1)	Forward diffusion process
pθ(·|Gt)	Denoising model
G0	Reconstructed clean graph
Lf	Counterfactual loss
Ldist	Distribution loss
Gr	Graph at timestep t in the reverse sampling process
p	Explanation confidence
K	Number of candidates in each step for the model-level explanation
T	Number of reverse sampling steps for the model-level explanation
N	(Predefined) number of nodes in the target model-level explanation
B Explanation Setting
The counterfactual explanation for a prediction highlights the smallest change to the original instance that changes the prediction. It is a post-hoc step after the model is designed and well-trained.
Definition 1. (Counterfactual Explanation) Given a well-trained classifier  $f$  that predicts the label  $\hat{Y}_G$  for an instance  $G$ , a counterfactual explanation consists of an instance  $G^c$  such that the prediction on  $G^c$  is different from the original  $\hat{Y}_G$  on  $G$ , such that the difference between  $G$  and  $G^c$  is minimal.
It can be formulated as an optimization problem that minimizes the mutual information [6, 13]:
$$
\underset {d (G, G ^ {c}) <   K} {\operatorname {a r g m i n}} M I \left(\hat {Y} _ {G}, G ^ {c}\right) \Leftrightarrow \underset {d (G, G ^ {c}) <   K} {\operatorname {a r g m a x}} H \left(\hat {Y} _ {G} \mid G ^ {c}\right) \Leftrightarrow \underset {d (G, G ^ {c}) <   K} {\operatorname {a r g m i n}} - \mathbb {E} _ {\hat {Y} _ {G} \mid G ^ {c}} \left[ \log \left(1 - P _ {f} \left(\hat {Y} _ {G} \mid G ^ {c}\right)\right) \right] \tag {5}
$$
where  $MI(\cdot)$  is the mutual information function,  $H(\cdot)$  is the entropy function and  $H\left(\hat{Y}_G \mid G^c\right) = -\mathbb{E}_{\hat{Y}_G|G^c}\left[\log P_f\left(\hat{Y}_G \mid G^c\right)\right]$ .  $P_f\left(\hat{Y}_G \mid G^c\right)$  denotes the probability for the  $\hat{Y}_G$  label given the counterfactual explanation  $G^c$ , predicted by  $f$ .  $d(G, G^c)$  measures the proximity between the original  $G$  and counterfactual explanation  $G^c$ , which can be specified by the number of changed edges (including the removed edges and newly added edges). An ideal counterfactual explanation should be similar to the original graph, therefore,  $K$  is applied as a constraint over the proximity.
Definition 2. (Model-level Explanation) Given a well-trained GNN classifier  $f$  and a set of graph  $\mathcal{G}$  that is predicted as the same label  $C_i$  by  $f$ , the model-level explanation for the target class  $C_i$  is a recurrent and determinant graph pattern that leads to the certain prediction made by  $f$ .
Formally, the model-level explanation for the target class  $C_i$  can be formulated as  $G^m = \operatorname{argmax}_G P_f(C_i|G)$ , where  $P_f(C_i|G)$  can be computed by the probability for the class  $C_i$  predicted by the well-trained GNN  $f$ . Meanwhile, the model-level explanations should be recurrent in the given graph set. Typically, the model-level explanations should adhere to the distribution of the input graphs to be representative of the graph characterizations [16].
C Diffusion Process
C.1 Discrete Diffusion Process for Graph
Forward diffusion process. Let  $t \in [0, T]$  denote the timestep of the diffusion process, which is also a noise level indicator. Let  $A_t$  denote the one-hot version of the adjacency matrix at timestep  $t$ , where the row vector  $a_t^{ij} \in \{0, 1\}^2$  is a 2-dimensional one-hot encoding of the  $ij$ -th element  $a_t^{ij}$  in the adjacency matrix  $A_t$ . The forward diffusion process is a Markov chain that progressively transforms the input graph into pure noise. The forward transition probabilities can be represented by a transition matrix  $Q_t \in \mathbb{R}^{2 \times 2}$ , where the  $rs$ -th element  $Q_t^{rs} = q(a_t^{ij} = s|a_{t-1}^{ij} = r)$ . For example,  $Q_t^{01}$  indicates the probability of being absent at timestep  $t - 1$  and transitioning to being present at timestep  $t$  for each edge. Therefore, the transition matrix  $Q_t$  can be represented as
$$
\boldsymbol {Q} _ {t} = \left( \begin{array}{c c} 1 - \beta_ {t} & \beta_ {t} \\ \beta_ {t} & 1 - \beta_ {t} \end{array} \right), \tag {6}
$$
where  $1 - \beta_{t}$  models the probability that an edge state does not change at timestep  $t$  (e.g., remaining present or remaining absent in the graph). With the transition matrix and one-hot encoding  $\pmb{a}_{t}^{ij}$ , the forward diffusion process can be written as  $q(\pmb{a}_{t}^{ij}|\pmb{a}_{t-1}^{ij}) = \mathrm{Cat}(\pmb{a}_{t}^{ij};\pmb{P} = \pmb{a}_{t-1}^{ij}\pmb{Q}_{t})$ , where  $\mathrm{Cat}(\pmb{x};\pmb{P})$  is a categorical distribution over the one-hot vector  $\pmb{x}$  with probability vector  $\pmb{P}$ .
Multi-step diffusion. The formulation of Equation 6 allows for computing multiple-step diffusion from  $\pmb{a}_0^{ij}$  to  $\pmb{a}_t^{ij}$  directly in a closed form, by  $q(\pmb{a}_t^{ij} | \pmb{a}_0^{ij}) = \mathrm{Cat}(\pmb{a}_t^{ij}; \pmb{P} = \pmb{a}_0^{ij} \bar{\pmb{Q}}_t)$ , where  $\bar{\pmb{Q}}_t = \prod_{i=1}^t \pmb{Q}_i$ . Additionally,  $\bar{\pmb{Q}}_t$  can also be represented as a symmetric matrix like Equation 6, with  $\beta_t$  being replaced by
$$
\bar {\beta} _ {t} = \frac {1}{2} - \frac {1}{2} \prod_ {i = 1} ^ {t} \left(1 - 2 \beta_ {i}\right). \tag {7}
$$
In the implementation, we only perform multi-step diffusion. We uniformly sample  $\bar{\beta}_t$  in the range of  $[0, 0.5]$  to control the level of noise.
Graph-level expression. The forward diffusion process is independently performed over all of the edges in the full adjacency matrix. Therefore, the graph-level diffusion  $q(G_{t}|G_{t - 1})$  is the product of element-wise categorical distributions as