Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces DiGress, a discrete diffusion-based generative model for graphs. The noise model sequentially corrupts the graph structure by modifying the categories of edges and nodes. The denoising model is a Transformer that leverages additional features. Experiments on graph and (conditional) molecule generation demonstrate the efficacy of DiGress.
strength_and_weaknesses: Strengths
- The paper also introduces a continuous version (ConGress) and uses it for ablation purposes, showing that the discrete version produces better results.
- The proposed method yields good results for general graph generation, outperforming SPECTRE (ICML, 2022).

Weaknesses
- The proposed method does not advance the current art --- widely used molecule generation methods (e.g., JT-VAE) outperform the proposed approaches.
- Experiments on conditional generation are somewhat weak, with no extensive comparison with other baselines.
clarity,_quality,_novelty_and_reproducibility: Overall, the paper reads well and tackles a relevant problem (one-shot molecule generation). Nonetheless, the technical novelty is limited as the work follows the framework by Austin et al., (2021), with improvements mainly coming from adopting marginal probabilities as prior and structural features. Regarding reproducibility, the authors have made their code available, and the paper provides sufficient details about the method and its training methodology.

A few questions/comments:
- It requires some effort to understand some parts of the formulation. For instance, the idea of predicting the clean graph at each step $t$ isn't adequately motivated and doesn't fit into the formulation introduced in Section 2. Also, the paper uses $u$ and $v$ to refer to node and edge distributions, which I found unusual. 
- ZINC represents one of the main benchmarks to evaluate molecule generation. Any specific reason for not considering it? 
- What are the numbers for VUN (valid, unique & novel) for the experiments in Tables 3 and 4?
- Tables 3 and 4 miss some baselines --- e.g., GraphAF (https://arxiv.org/pdf/2001.09382.pdf).
- While the paper shows the idea is promising, the results are not strong. For instance, in Table 3, JT-VAE (a contribution from 2018) outperforms DiGress and ConGress. Also, LSTM on SMILES representation (used in the REINVENT tool, for instance) beats the proposal. - - Thus, the paper doesn't seem to advance the current art.
- Could the same additional features be used to boost the performance of the baselines?
- Experiments on conditional generation are weak. What are the current SOTA methods for these tasks? How do the proposed methods perform on different properties?
- Typo: $G^t$ instead of $G_t$ in Algorithm 1.
summary_of_the_review: Although I have concerns regarding clarity and performance gains over current art, this is an interesting contribution to one-shot molecular generation. Thus, I am inclined to accept the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents DiGress, a diffusion model for graph generation. Apart from using graph-based architectures, the key idea is to start with a noise model that preserves the marginal distribution of node and edge types rather than using uniform noise. Then the denoising network is augmented with structural features. Finally, conditional generation is enabled by guidance procedures. Empirical results shows that the performance of DiGress matches the performance of autoregressive models on some molecular datasets.
strength_and_weaknesses: Strength:
- The paper is written clearly and the idea behind non-uniform diffusion noise addition makes sense, and is empirically confirmed.
- The proposed architecture handles node-level, edge-level and graph level features and has some nice properties, such as equivariance, and exchangeability.
- The diffusion model framework allows for easy use of classifier-based / classifier-free guidance for conditional generation.

Weakness:
- The method does not deal with growing graphs like in autoregressive models. 
- The sampling procedure is not entirely clear (in algorithm 3). While the distribution $p(G^{t-1} | G^t) p(y | G^{t-1})$ can be sample from Langevin dynamics, it is not sure if that is actually used -- more likely it is similar to the sampling procedure of diffusion models, which would not guarantee that $G^{t-1}$ is sampled from that distribution described above. 
- Some of the experimental results seem odd. In Table 1. the V.U.N. for ConGress is 0% meaning that it generates nothing that is valid, unique and novel graphs. In Table 4, the validity of Congress is 0% and FCD is also 0, which seems quite odd because FCD is better if it is lower (that is also a typo? also KL div is pointed upwards as well).
- The performance between ConGress and DiGress differ by a lot, which seems odd as usually the discrete case converges to continuous case as number of timesteps goes to infinity.
clarity,_quality,_novelty_and_reproducibility: Clarity (High): the paper is written clearly in general. 

Quality (High): The paper has extensive method and experiment sections that are generally well written. 

Originality (High): while training diffusions on categorical data is not exactly new, it is interesting to choose a different forward distribution. This is also well motivated in the context of graph generation. 

Reproducibility (High): code is provided.
summary_of_the_review: The method section of the paper is written nicely, but there are some confusions surrounding the experimental sections.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a discrete diffusion model for graph generation by combining discrete diffusion [Austin 2021] and graph transformer [Dwivedi and Bresson 2021]. Its main novelties are:
 - The diffusion model is discrete: sampling of node and edge features is performed after adding noise.
 - The denoising network is a graph transformer. It is permutation equivariant and generalizes to graphs of different sizes (sec 3.3). 
 - Since "having no edge" is encoded as a categorical edge feature, choosing a uniform noise model does not preserve the sparsity of the graph. Instead, the paper proposes a noise model that preserves the marginal distribution of node and edge types during.
 - Conditional generation is made possible through regressor guidance.
strength_and_weaknesses: === Strength ===

This paper fills a known gap in the literature, between discrete diffusion [Austin 2021] and diffusion for graph data [Niu 2020]. As a result, the approach is well-motivated, and contributes greatly to the literature on deep graph generation. The model is analyzed thoroughly, especially on permutation equivariance and invariance. The ablation studies are thorough: the benefit of discrete (over continuous) diffusion, marginal (over Gaussian) noise, and conditional (over unconditional) generation are well-established.

=== Minor weaknesses ===

What is z in the algorithms?

I understand the omission of graph transformer network, but the discussion about sampling after adding noise should not be omitted. After all, it is the main reason why the diffusion model is "discrete."

I would like to see an evaluation on training on small graphs and testing on large graphs, similar to those in "Learning the Travelling Salesperson Problem Requires Rethinking Generalization".
clarity,_quality,_novelty_and_reproducibility: Clarity and quality: I was left confused about how the graph transformer network maintains a discrete graph before checking the code. Other than that, the paper is generally well-written; the literature review is thorough and complete.
Novelty: Using discrete diffusion in graph generation is technically novel. Conditioning a one-shot generation model on a subgraph is, to the best of my knowledge, new to the literature.
Reproducibility: I briefly checked the code. It appears to be well written.
summary_of_the_review: The paper is well-written, the contribution is novel, and should be of interest to a broad graph generation and diffusion audience. The method proposed is well-motivated and the empirical evaluation is thorough. I recommend accept.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduces a diffusion model for graph generation, referred to as DiGress. Building on top of the discrete diffusion framework of Austin et al. (2021), DiGress sequentially corrupts the graph by modifying the categories of edges and nodes. To reconstruct the graph, it borrows the graph transformer [Dwivedi and Bresson 2021] in the denoising reverse diffusion process. Different from previous discrete diffusion that starts the reverse from a uniform noise, DiGress starts with a noise model that preserves the marginal distribution of node and edge types. In addition, DiGress enables conditional generation with regressor guidance. The efficacy of DiGress is demonstrated with experiments on graph and (conditional) molecule generation, with better performance than autoregressive models on some molecular datasets. All reviewers are in favor of accepting the paper.
justification_for_why_not_higher_score: The paper can be viewed as a relatively straightforward combination of two existing techniques: the discrete diffusion of Austin et al. (2021) and graph transformer of Dwivedi and Bresson (2021). A major theoretical or methodological contribution will help improve the score.
justification_for_why_not_lower_score: As well summarized by Reviewer 7EF5: "This paper fills a known gap in the literature, between discrete diffusion [Austin 2021] and diffusion for graph data [Niu 2020]. As a result, the approach is well-motivated, and contributes greatly to the literature on deep graph generation. The model is analyzed thoroughly, especially on permutation equivariance and invariance. The ablation studies are thorough: the benefit of discrete (over continuous) diffusion, marginal (over Gaussian) noise, and conditional (over unconditional) generation are well-established."
Decision: Accept: poster