Title: Graph Diffusion that can Insert and Delete
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Networks

Generative Models

Deep Learning
Abstract: Generative models of graphs based on discrete Denoising Diffusion Probabilistic Models (DDPMs) offer a principled approach to molecular generation by systematically removing structural noise through iterative atom and bond adjustments. However, existing formulations are fundamentally limited by their inability to adapt the graph size (that is, the number of atoms) during the diffusion process, severely restricting their effectiveness in conditional generation scenarios such as property-driven molecular design, where the targeted property often correlates with the molecular size. In this paper, we reformulate the noising and denoising processes to support monotonic insertion and deletion of nodes. The resulting model, which we call GrIDDD, dynamically grows or shrinks the chemical graph during generation. GrIDDD matches or exceeds the performance of existing graph diffusion models on molecular property targeting despite being trained on a more difficult problem. Furthermore, when applied to molecular optimization, GrIDDD exhibits competitive performance compared to specialized optimization models. This work paves the way for size-adaptive molecular generation with graph diffusion.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces GRIDDD, a novel discrete Denoising Diffusion Probabilistic Model (DDPM) for graph generation, which supports size-adaptive graph generation through monotonic insertions and deletions of nodes (atoms). Existing graph diffusion models have a fixed graph size throughout the generation process, limiting their application in tasks like property targeting and optimization, where graph size (number of atoms) is crucial. GRIDDD addresses this limitation by dynamically adjusting graph size during generation. It is evaluated on two tasks: property targeting (e.g., molecular weight, dipole moment) and property optimization (improving existing molecules). GRIDDD demonstrates superior performance on benchmark datasets (QM9 and ZINC-250k), particularly excelling in property targeting and optimization tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. **Originality:** GRIDDD introduces a novel extension to DDPMs for graph generation by allowing node insertion and deletion during diffusion, significantly advancing beyond fixed-size models and enabling practical applications in molecular property targeting.\n\n2. **Technical Contribution:** GRIDDD’s dynamic graph size adjustment during generation provides a substantial improvement in molecular generation tasks, effectively maintaining valid chemical structures throughout the process.\n\n3. **Empirical Results:** The comprehensive experiments demonstrate GRIDDD’s superior performance on widely-used datasets (QM9 and ZINC-250k), achieving notable results in both property targeting and optimization, especially in enhancing LogP and QED scores.\n\n4. **Clarity of Methodology:** The detailed technical explanations clearly illustrate how node insertions and deletions integrate into the diffusion process, including transition matrices and insertion-time predictions.\n\nWeaknesses:\n\n1. **Computational Overhead:** GRIDDD's insertion and deletion operations introduce significant computational overhead. The paper lacks concrete optimizations (e.g., sparse graph updates, mixed-precision training) or quantification of additional computational costs (training time, memory usage, FLOPs).\n\n2. **Split Molecules:** GRIDDD occasionally generates more split molecules due to late-stage node insertions. Further exploration to address this issue through additional constraints or algorithmic modifications is needed.\n\n3. **Limited Generalization to Larger Graphs:** GRIDDD’s effectiveness decreases significantly when generating molecules larger than its training distribution. Enhanced methods for improving generalization capabilities to larger graph sizes would be valuable.\n\nQUESTIONS:\n1. **Split Molecules Issue:** Could you elaborate on the causes behind split molecule generation? Have you considered additional constraints to mitigate this?\n\n2. **Computational Overhead:** Have you explored optimization techniques (e.g., sparse updates, gradient checkpointing) to address computational overhead?\n\n3. **Larger Graphs Generalization:** How might GRIDDD be adapted to effectively handle larger molecules beyond current limitations?\n\n4. **Impact of Hyperparameters:** Could you provide sensitivity analyses regarding hyperparameters such as noise schedules and insertion/deletion probability distributions?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
GRIDDD extends discrete DDPMs for graphs by introducing mechanisms for node insertion and deletion within the forward (noising) and reverse (denoising) diffusion processes. It enables step-wise monotonic node insertions and removals during diffusion, and is specifically designed to change the graph size dynamically throughout the generative process, providing the necessary flexibility to incorporate conditional information more effectively.

STRENGTHS AND WEAKNESSES:
Pros
Addresses a Critical Limitation: Directly tackles the inability of prior graph DDPMs to change graph size, which is vital for many real-world molecular design tasks (e.g., property-driven design where size correlates with properties).

Improved Performance in Property Targeting: GRIDDD matches or exceeds the performance of state-of-the-art graph diffusion models (DiGress, FreeGress) on molecular property targeting benchmarks (QM9 and ZINC-250k) in terms of Mean Absolute Error (MAE) and chemical validity.

Cons
Assumption of Mutual Independence: The reverse process factorizes probabilities assuming mutual independence of nodes and edges, which might be a simplification given the highly interconnected nature of molecules.

QUESTIONS:
Failure Analysis: For cases where GRIDDD shows slightly lower validity (e.g., QM9, μ target), what are the common types of invalid molecules generated? Understanding these failure modes could guide future improvements.

LIMITATIONS:
Sensitivity to Hyperparameters: How sensitive is GRIDDD's performance to the choice of hyperparameters, particularly pmin, pmax, D, w, and the λ weights in the loss function? Are there general guidelines for tuning these for new datasets?
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method that allows to vary the number of nodes in graph discrete diffusion models. In particular, this method allows to insert and delete nodes throughout both the forward and reverse processes. This is relevant for problems such as property optimisation, where the objective is to modify a molecule, parsed to a graph, such that given properties of the molecule are optimised, while still remaining structurally close to the initial molecule. The newly introduced diffusion process is combined with pre-existing classifier-free conditional guidance to target the tasks of property targeting, property optimisation, and out-of-distribution sampling.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The writing of the paper is clear. \n2. The targeted problems (property targeting, property optimisation, out-of-distribution sampling) are relevant. The proposed method outperforms existing alternatives for property optimisation.\n3. While the idea of varying the dimensionality of the generated instance by the diffusion model is not novel in general [1], neither for discrete state-spaces in particular [2], this work is the first, to the best of my knowledge, employing successfully such idea for graph diffusion models. \n4. *Empirical results*: GrIDDD clearly outperforms existing alternatives for property optimisation setting. This method can also be used to ensure better out-of-distribution graph generation in terms of number of nodes, as its training allow the model to learn some representations over graphs that have more or less nodes than the ones in the training set (contrarily to vanilla discrete diffusion processes). \n5. *Reproducibility*: the authors provide the code.\n\n**Weaknesses**:\n1. Empirical performance for property targeting is not remarkable for property targeting, as it only dominates existing methods (FreeGress) in 2 out of 5 different settings.  \n2. The paper does not compare/discuss to alternative graph continuous diffusion models (that model graph diffusion over continuous state-spaces), and their alternatives for varying dimensionality across diffusion process [1].  \n3. As mentioned by the authors in their limitations, this more intricate diffusion process incurs computational overhead. An objective analysis of this aspect of the contribution is of utmost importance for and correct evaluation of the contribution, e.g., to better understand the scalability of the method to larger molecular datasets.\n4 While the utility of the method for property optimization is clear, I have some concerns regarding the pertinence of the proposed method for the other experimental settings.\n\n\n**Minor weaknesses**:\n1. The formatting of Table 1 and Figure 3 should be improved. \n2. Still regarding reproducibility, it would benefit the paper to include a discussion on hyperparameter selection, particularly for the newly introduced hyperparameters. While the remaining ones follow prior work, a description of how the new parameters were chosen would strengthen the reproducibility and transparency of the experimental setup.\n\n\n[1] - Trans-Dimensional Generative Modeling via Jump Diffusion Models, Campbell et al., NeurIPS 2023\n\n[2] - Beyond In-Place Corruption: Insertion and Deletion in Denoising Probabilistic Models, Johnson et al., ICML 2021 Workshops\n\nQUESTIONS:\n1. The removal of the edges connected to deleted nodes causes discontinuities in the edge diffusion processes as well. Did the authors analysed this component? Any relevant observation?\n2. \"One problem of using a single deletion type is that nodes will loop in the absorbing state indefinitely once taken the DEL type, while the reverse process is supposed to revert all deletions into normal atom types\" (l. 150-152). This is not true for the marginal noise model employed in the diffusion process as, contrarily to masking discrete diffusion processes, it allows transition between every state at any timestep. Therefore, it seems to me that the motivation for the definition of the state Del* is incorrect. While I still see the importance of using the Del* state to signal to the model at sampling time to also predict for that node, the motivation of such formulation has to be modified. Please clarify.\n3. The node and edge classes assigned at insertion time by sampling from the marginal distribution of nodes and edges, respectively, can be further motivated by the fact that the usual marginal noise model preserves marginal distributions across the forward process. Without this motivation, seems an ad-hoc trick. Did you try anything different?\n4. An important baseline to incorporate consists of a DiGress implementation with, as similar to edges, a \"non-existing\" node class (equivalent to DEL) and let the process unroll. Edges connected to non-existing nodes should be treated consistently with how edges connected to DEL/DEL* nodes are handled.\n This process also allows to vary the number of nodes throughout the diffusion process without incurring into extra computational costs neither the more intricate diffusion process formulation of this paper. A superior performance to this baseline would clearly fundament the utilisation of the proposed diffusion process, with the additional model.\n\nLIMITATIONS:\nYes, the authors explicitly address the limitations of the paper and potential future directions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript presents a method, GRIDDD, that extends the DIGRESS method for discrete graph diffusion by monotonic node insertion or removal. Original DIGRESS method shows excellent performance in molecular generation tasks, but number of nodes of the graph have to be specified beforehand. For some applications, e.g., drug design, one aims to generate a molecule with specific properties and the size of the molecule might not be known beforehand. The GRIDDD method allows to change the number of atoms during the diffusion process; in combination with classifier guidance this makes it possible to optimize the generated molecule towards a certain property. For this, the manuscript introduces new transition states for deleting nodes, a scheduling function that determines when and how likely a deletion/insertion happens as well as an auxiliary neural network that predicts how many nodes to delete at a specific time step.\n\nSTRENGTHS AND WEAKNESSES:\nFirst of all, the limitation that graph size has to be specified beforehand is not as critical as it seems. One can simply start from multiple graph sizes and sample all of them, deciding for the best graph afterwards. This is more time consuming than changing graph size during denoising, though. Another possibility is to learn a predictor that estimates an optimal graph size for the given problem. This is done by FREEGRESS and the evaluation shows that the GRIDDD method performs more or less the same for most problems. Only for molecular weight, the GRIDDD method outperforms FREEGRESS clearly. However, this is a property that completely depends on graph size and might be somewhat arbitrary and, in practice, rather uninteresting.\n\nI would argue that the most interesting problems for generative models are sampling of similar molecules that optimize certain properties (e.g., given an already known drug, find another one that is highly similar but has better water solubility). For such tasks the size of the optimized/generated molecule is often close to the input molecule anyways. The interesting question here would be rather: can I start from the input molecule and only add a little bit of noise to ensure that the generated molecule is close. Here, a method like GRIDDD could be advantageous, as it allows, in theory, to add new nodes rather late in the diffusion process. I'm not sure if the \"Property optimization\" section goes in this direction, as the description does not mention if they start from a random latent or from a input molecule.\n\nThe decision to use monotonic insertions and deletions make the underlying markov chain time irreversible. My feeling is that this might introduces a lot of problems and it also makes the underlying math very complicated. I wonder if there is any advantage in the way GRIDDD introduces insertions and deletions. The original DIGRESS model keeps the number of nodes fixed, but it already allows for arbitrary insertions and deletions of edges. The way it's done is that edges just have a special \"delete\" state which is treated similarly to their \"single bond\", \"double bond\" and so on states. During diffusion, edges can be inserted and deleted. The same would also be possible with atoms. One would still have to decide for a \"maximum number of atom\" constraint at the beginning, but besides that no changes of the method are necessary to allows node insertions/deletions. Clearly, there are a lot of disadvantages of such a method. For example, edges could connect deleted atoms. But it is already possible that more than 4 edges connect to a single carbon atom and the model just learns to prevent this. In my opinion the GRIDDD method should evaluate against this very simple baseline.\n\nFinally, I found several small errors in the manuscript\n- the negative sign in the probability density function zeta has to be removed, otherwise, all probabilities are negative\n- the scaling parameter w has to be inverted, i.e. the manuscript states to use w=0.05 but they probably mean w=1/0.05. Otherwise, the exponential becomes extremely large and the zeta function very steep.\n- equation 2 is missing an absolute, otherwise the probability is maximum when all atoms are deleted and minimum if all atoms are inserted. But the maximum should be at zero insertions/deletions.\n\nQUESTIONS:\nAs stated above, I see the main advantage of this method in the theoretical possibility to insert nodes/edges at a very late timestep. Otherwise, I could just use the original DIGRESS method (either by inserting \"DEL\" nodes in the beginning, or by simply adding nodes in the middle of the diffusion process. But the latter will probably give me worse results as there is not enough time to properly denoise them). I do not really understand the experimental setup of the \"property optimization\" section. Do you start from a input molecule and only add a low/middle amount of noise to guarantee that the generated molecule is close to the input molecule? Or do you generate new molecules from random latent and just keep the ones that are close to the input molecule? I think an evaluation would be interesting where the input molecule is noised by a lower amount of timesteps (e.g. t=100) and then denoised with a classifier-guidance.\n\nSecond, as I stated above it's relatively trivial to implement node insertions/deletions in DIGRESS, although this naive implementation is probably not as good as a proper implementation. I would still be interested in how well such a baseline method performs in your evaluations.\n\nFinally, I just do not understand why you need two different DEL states. Can you explain this in more detail please?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-17"}