Graph Diffusion that can Insert and Delete
Authors: Matteo Ninniri
Marco Podda
Davide Bacciu
Venue: NeurIPS 2025 poster ()
Keywords: Graph Neural Networks
Generative Models
Deep Learning
Primary area: deep_learning
TL;DR: We develop a discrete graph diffusion denoising model that dynamically grows or shrinks a chemical graph during generation.

Abstract: Generative models of graphs based on discrete Denoising Diffusion Probabilistic Models (DDPMs) offer a principled approach to molecular generation by systematically removing structural noise through iterative atom and bond adjustments. However, existing formulations are fundamentally limited by their inability to adapt the graph size (that is, the number of atoms) during the diffusion process, severely restricting their effectiveness in conditional generation scenarios such as property-driven molecular design, where the targeted property often correlates with the molecular size. In this paper, we reformulate the noising and denoising processes to support monotonic insertion and deletion of nodes. The resulting model, which we call GrIDDD, dynamically grows or shrinks the chemical graph during generation. GrIDDD matches or exceeds the performance of existing graph diffusion models on molecular property targeting despite being trained on a more difficult problem. Furthermore, when applied to molecular optimization, GrIDDD exhibits competitive performance compared to specialized optimization models. This work paves the way for size-adaptive molecular generation with graph diffusion.

Graph Diffusion that can Insert and Delete
Matteo Ninniri
Marco Podda
Department Computer Science University of Pisa 56127 Pisa (Italy)
Department of Computer Science University of Pisa
56127 Pisa (Italy) marco podda@unipi.it
matteo www.iniiiniiiiiimmmm unipi.it
Davide Bacciu
Department of Computer Science
University of Pisa 56127 Pisa (Italy) davide bacciuuunipi.it
Abstract
Generative models of graphs based on discrete Denoising Diffusion Probabilistic Models (DDPMs) offer a principled approach to molecular generation by system- atically removing structural noise through iterative atom and bond adjustments. However, existing formulations are fundamentally limited by their inability adapt the graph size (that is, the number of atoms) during the diffusion process, severely restricting their effectiveness conditional generation scenarios such property-driven molecular design, where the targeted property often correlates with the molecular size. this paper, we reformulate the noising and denoising processes to support monotonic insertion and deletion of nodes. The resulting model, which we call GRIDDD, dynamically grows or shrinks the chemical graph during generation. GRIDDD matches exceeds the performance of existing graph Diffusion Models on molecular property targeting despite being trained on more difficult problem. Furthermore, when applied to molecular optimization, GRIDDD exhibits competitive performance compared to specialized optimization models. This work paves the way for size-adaptive molecular generation with graph diffusion.
Introduction
Generating molecules conditioned on predefined structures or properties central endeavor in computational chemistry, with applications ranging from de novo drug design (You et al. 2018a to materials discovery (Zhao et al., 2202). Depending on the conditioning information, we distinguish two key learning tasks: property targeting, aimed at generating molecules endowed with prespecified properties; and property optimization, where the goal to edit given molecule improve target property while retaining its core structure. Unlike continuous data, generating graphs must account for their discrete and combinatorial connectivity. For molecules particular, chemical constraints invalidate most atom-bond and atom-aton combinations, making this problem difficult and, therefore, actively researched. Despite these challenges, deep graph generators [Faez et al. [202T] have achieved striking success in the mboveomentioned tasks, due their ability to approximate complex molecular distributions and their flexibility in incorporating conditioning information.
Based on the way they decode a graph from latent representation, deep generative models for molecules can be broadly categorized as autoregressive or one-shot (Zhu et al. 2022). Autoregressive models build the graph sequentially (atom-by aitom-by-atoo by-moommooommmamm as in You et al. [208b], or geegment-b--fagmeee
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

CH
CH2
CH2
NH;
CH2 CH2
Figure 1: Two qualitative examples of the proposed model (GRIDDD) when generating molecules from the QM9 dataset. In the top row, from left to right, we show subset of the denoising process to generate a molecule starting from latent graph with two atoms (which are extremely rare in QM9). GRIDDD successfully inserts six more nodes to obtain sample resembling the training set's distribution. In the bottom row, we start from a latent with 18 atoms instead (not present in QM9). GRIDDD manages to delete nine atoms and obtain valid molecule. Notice that, unlike current DDPMs for graphs, the graph size is changed dynamically during denoising.
as in Jin et al. [2018b). but suffer from order dependence and linearly-scaling sampling time. One- shot methods generate all nodes and edges in a single pass enabling parallel sampling, but struggle to learn high-order interactions. Denoising Diffusion Probabilistic Models (DDPMs) [Ho et al. 2020 for graphs bridge these approaches by progressively removing Gaussian or discrete noise to recover the molecular graph. Indeed, DDPMs preserve permutation invariance and sampling parallelism while iteratively refining long-range interactions (e.g.. ring closures), which traditional one-shot decoders are forced to learn in single transformation. Moreover, DDPMs can be easily adapted to conditional generation with classifier-based [Dhariwal and Nichol 2021 or classifier-free [Ho and Salimans 2022 guidance.
One draw of Diffusion Models (not limited to graphs) is that the sample size remains fixed throughout the generative process. While for different modalities this is less of concern (e.g.. for images it is equivalent to fixing the resolution beforehand), it becomes relevant when generating molecules, and more in general when modeling combinatorial data. In practical terms, this restriction implies that the model by design cannot add or remove atoms during generation, leading to two major drawbacks: in property targeting, it is not possible to make the molecular structure responsive to properties which correlate with the size, e.g., when generating molecules with a specific molecular weight; b) in property optimization, the model cannot optimize a given property by adapting the generated structure. Existing methods circumvent this limitation by sampling the graph size before the generative process starts. For example, Vignac et al. [2023a] and [Hoogeboom et al. [2022] sample different graph sizes from the empirical distribution of the training dataset, while Ninniri al. [2024] use an auxiliary classifier predict the graph size from the conditioning information. Though these workarounds are sufficient for property targeting, they become ineffective for property optimization, where size is strictly related to the structure to optimize. Ketata al. (2025) adapts to
property optimization by sampling additional nodes in the reverse process, but does not incorporate this procedure into the training process.
Motivated by these considerations, our primary contribution is generalization of the standard discrete diffusion process on graphs. Our approach enables step-wise monotonic node insertions and removals during diffusion, and is specifically designed to change the graph size dynamically throughout the generative process, providing the necessary flexibility incorporate conditional information more effectively. We implement this novel formulation as a model called GRIDDD (short for Graph Insert-Delete Discrete Diffusion). An intuitive example of how GRIDDD generates molecules is provided in Figure [ and ZINC-250k), where it consistently performs on par or better than the state of the art in terms of approximating the target property while keeping high chemical validity, despite having been trained on more difficult problem. When applied to molecular optimization, GRIDDD convincingly outperforms other molecular optimizers, achieving higher average improvement and optimization success rate. To the best of our knowledge, this is the first work that allows size-adaptive molecular generation with graph diffusion. Our code is available at
We test GRIDDD on property targeting in two widely used benchmarks (QM9

---

2 Background and related works
Notation We represent molecules with atoms as graphs GX, where X Raxa is the matrix of one-hot encoded atom types (over a possible atom types), and E E Raxuxb is an adjacency (or edge) tensor that encodes both the bond connectivity and the one-hot encoded bond types (over b possible bond types while considering the absence of bond type). We denote as 2; E R" the i-th row of X and as Eij E R2 the slice of E along the first and second dimensions. Oftentimes, we use the terms "nodes" and "edges" to refer to atoms and bonds, respectively, Conditioning vectors will be generally denoted as y E R d being a positive integer.
additional bond
2.1 Graph Generative Models for Property Targeting and Optimization
Several models have recently been developed for property targeting. mostly based on graph diffusion since it is particularly suited for conditional generation. For example, DiGress [Vignac et al. 2023a uses a categorical diffusion process on nodes and edges, and a graph transformer denoiser to learn generative distribution over molecular graphs. DiGress allows conditional sampling through bassifier-based guidance, by training an auxiliary regressor to steer the generation towards the target properties. FreeGress (Ninniri et al.) (024) builds upon DiGress and proposes classifier-fres free approach for conditional generation that injects the guide directly into the denoiser during training, greatly improving property-targetinn accuracy.
Molecular property optimization mainly achieved through translation models and optimization models. VJTNNs (Jin et al.) 2018b), Seq2Seq models [He et al. [02T], and HierG2G (Jin et al. 2020] all work by translating between different types of molecular representations, ranging from junction trees to SMILES strings. While these models achieve good results, they require a dataset of pairs of similar molecules with specific properties, which are of limited availability. At the same time, they are not designed to generate new molecules from scratch. Optimization models are usually designed for molecular generation. GCPN (You et al., 2018a] and JT- [Jin et al. al.] [0188] employ different techniques, ranging from reinforcement learning to junction trees, generate data, and are easily extensible to perform property targeting and optimization.
2.2 Discrete Denoising Difffusion Probabilistic Models
DDPMs consist of an un-parameterized forward process q(x' (x'-1] and a parameterized reverse process Po (x"""(=²). The forward process progressively corrupts the initial data point x transform- ing into a noise-like sample x²' The reverse process, trained to denoise these samples, aims to reconstruct æ by sequentially removing the added noise. Sampling from a trained DDPM involves drawing " from the noise distribution and applying the reverse process iteratively over T steps to obtain a new sample 5" ~ q(x). While in the original formulation of DDPMs the noise distribution Gaussian, we focus on discrete DDPMs where noise is injected through specially crafted transition matrices Q', with [Q')y = p(x² = jl j|=--1 = i), encapsulating the probability of switching from category to category
step of the forward process. Crucially, cumulative transitions from an arbitrary timestep to t can be expressed as a matrix Q2j- = I Q'. The reverse process (o xx) is computed by marginalization over the possible categories of the input æ.
2.3 Insert and delete operations for Diffusion Models
The idea of insert and delete operations for discrete Diffusion Models originates from Johnson et al. [2021]. which, similarly to us, proposed to gradually insert or delete nodes both during the forward and reverse process. However, there are some major differences with our work. Firstly, our solution is designed for graphs, instead of textual data. Secondly, they resort to complex edit summaries to compute the various probability distributions involved, while we simplify the computation of the posteriors by generalizing the denoising process to account for nodes that have yet to be deleted or inserted through the computation. Some continuous-time Diffusion Models [Campbell et al.) 2023 can increase the entries in the data point during the reverse process, but can not remove them, making them unsuitable for property optimization. At the same time, they are better suited for continuous data with positional dependencies, rather than graphs.
3

---

3 Graph Insert-Delete Discrete Diffusion
We now introduce our main contribution. The backbone of GRIDDD is reformulation of the standard discrete denoising diffusion for graphs to support the use of monotonic node insertions and deletions, which we describe in this section.
Preliminaries. Our end goal is to obtain a denoised graph G = (X", E*) with n nodes from
latent noisy graph with G (X",, EE)) with n" nodes. To change size from n² to n we monotonically delete (if n² > n²)) or insert (if n² < n²) |A" nodes, where A = n - , as the reverse process advances. This gives full control over the graph size at any given moment. The timesteps when insertions and deletions are performed are sampled from the normalization of the absolute value of the logistic distribution's 's probability density function ç (1):
<(11)
w(e 1)2
The function is scaled in a way such that 5 D indicates the timestep in which the function maximized, while w is the scale parameter (from a visual standpoint, it controls the steepness of the curve). During training, the final size n' that the number of
with C' - 0. The parameter
training sample with size n² will assume is sampled from discrete distribution nodes owo (nz) In our work, we defined it as follows, and then normalized it to sum to one:
h (n) Pmax
Pmin
Pmax
7mmax
where Tmmx the maximum size that graph may assume (which can be to be higher than the size of the largest graph in the dataset), while Pmin and Pmax are hyperparameters. Intuitively, the highest probability Pmax occurs when n = n²", and it linearly decreases to Pmin with a rate equal to Min-PPac as the absolute difference between 72 and n increases.
3.1 Forward process
The discussion above implies that we have three different forward processes, depending on whether AT > 0, A < O, or A = O. We start by describing the latter, which is the simplest and serves as basis for the deletion and insertion cases. Building on previous works [Vignac et al. 2023a Ninniri
et al. 2024 when A = 0, the forward process is implemented as:
GG G¹ = (x Q'x E'--
Focusing on the nodes X (edges are defined analogously), the associated transition matrix is:
Q'x = a' Ax + (1 (1- (l)
where Ax I, Bx = 1, with m.x being the vector of the marginal distribution of the atom types computed on the training set, and {a.)i=1...r being set of noise schedulers, chosen such that 01 = 1, ar = 0, and the intermediate elements gradually decrease from one to zero. Multi-step transitions from an arbitrary timestep s can be computed as matrix Q defined as:
Φx
II i=++1
Q' a
Ax + (1- (11-a11)BB
where d+
with 00
11 a² = cos ( (++0)" (notice the hyperparameter vz). 1+s
Monotonic deletions (AT < 0). To introduce deletions, we build on Johnson et al. 2021 and treat them an additional atom type DEL. This is similar to absorbing diffusion [Kong et al.) 2023), in which the forward process consists of progressively switching all atom types to the deletion type; our strategy is more general, as we allow for nodes and edges to change category during the forward and reverse processes. During the reverse process, nodes need to be reinserted at step whenever they were deleted during the forward process the same timestep (that is, if they transitioned to type DEL at forward step t). Consequently, such nodes are expected to revert to

---

D
D
D
0 0
A
B
A
(1)
B
(2)
0
1
0
D
D
D
D
0
C
0:0:1 0 0.1:0"1
(3)
D
0:1:0
(4)
00'0000
0.1:0
0.100
Figure 2: The matrices employed in the computation of Q and Q 2 For spacing reasons, the states DEL and DEL have been shortened and D*
proper (non-DEL) type at the next step / 1. However, since the DEL state is absorbing (meaning p(a' き DEL x = DEL) = 0), such transition is forbidden: once a node deleted, it must remain deleted up the end of the forward process, doing otherwise would violate monotonicity. To enable node reinsertion, we introduce an auxiliary transient type DEL*, designed such that p(a' = D deterministic, and p(x'" E (DEL, DEL Hx DEL") ) = 0. This ensures that a node is in state DEL during the reverse process, it only switch guarantees that nodes deleted at step can be reinserted at step t 1, thereby preventing the absorbing DEL state from blocking node recovery. To ensure that the final graph size is consistent with the fixed at the start of the forward process, preventing that too many nodes switch into the deletion state, employ hybrid scheme, in which no A nodes undergo the standard forward process (described by Eq. 4). while A nodes use different transition matrix defined as:
DEL*) 1, i.e. the transition from DEL to DEL during the forward process is
proper (non -EL) type. This mechanism
Q = c(t) (a' A" + B²) (1 ((t) C
(6)
Above, A and B" are A and B augmented to account for the deletion types, C* describes the transition from a normal type to a deletion (see Figure 2 and (()) is the integral of Ç (1) described in Equation Since <' (t) is a probability distribution, ((t) can be seen as its cumulative distribution function; therefore, at time T, Q C If chosen such that ((T 1) 0. we enforce exactly
| nodes with DEL type at timestep T. We refer the reader to Appendix A.1 for more details on
the function ç' used in this study. Computing the cumulative transition matrix Q ffs -Ii Q requires an additional matrix D' (also displayed in Figure 2
t s
(
1-1 1/s (1- (1)) +11- D.. A (1- B² +
(7)
where 2" Ii (()). Intuitively, D" represents state where all nodes are immediately switched to state DEL. If a node is set to category DEL or DEL", so are the edges associated with it.
Monotonic insertions (A > 0). Node insertions are defined implicitly (i.e., without a specific INS type) since by design, node is effectively inserted in the graph, or "activated", only at timestep + 1 if the model sampled as insertion time. The critical operation after inserting nodes to establish which category they belong to. Our solution is to sample the inserted node's 's state from the marginal distribution mx the graph itself, which can be precomputed with no significant computational overhead (for more information, refer Appendix A.2). Once node is inserted, so are the edges connecting it to the rest of the graph. Their initial state is computed analogously using mg
From here drop the subscript X for brevity. Equations 6 6-9 are defined analogously for E.

---

3.2 Reverse process
The reverse process starts from a randomly initialized graph and gradually removes noise. However, since our main objective is to perform conditional generation, we also input the conditioning vector y using classifier-free guidance (Ho and Salimans 2022]. We factorize the reverse process as the product of the individual reverse probabilities of the nodes and edges (assuming mutual independence. which we discuss in Appendix (B.4):
Po Po GG" GG', .))
II
pa( y)
II 6.JEU,...m'-1
Po
(e---1 (llet (ely. v).
(8)
In previous works NNnniri et al. 2024]. the posterior for the nodes was implemented with the aid of a neural network po a x y) tasked to predict the true atom and bond types given their noisy categories and the guide. However, this no longer works in our setting, as we cannot predict nodes that did not exist when / = but were inserted subsequently during the forward process. To address this issue, we let the main model predict, alongside G", the activation time & of each node and introduce the following generalized posterior:
g(>²1 (xx 1g((""
x x)
po(a" (x'
po(a = x|x0 y)
q q<<1|x>
x)
MEX
The activation time of the nodes belonging to the original graph is assumed be zero. Notice that when q(== [<" = x)- 0, then pe pax |x',, y) is set to zero as well. The edge posterior is computed analogously. Below, we describe how to extend the base setting to support insertions and deletions.
Deletions. During the reverse process, nodes deleted during the forward process need to be re- inserted. We do with auxiliary neural network g (G') that predicts, given the latent noisy graph G', the number of DEL ' to add before passing the model to the main neural network that predicts the original graph GD given G The auxiliary network is trained alongside the main model by masking out the nodes set to DEL in G after computing po (G G* y). which reduces the training time significantly. Notice that Is is trained only when Ç (f) > 0, since it is not possible to insert nodes otherwise.
Insertions. Conversely to the deletion case, nodes inserted during the forward process need to be deleted. To do this, we let the main model predict, alongside Go, the activation time é of each node. Then, at timestep i, we remove from the graph the nodes predicted to have been inserted at step t. The activation time of the nodes belonging the original graph is assumed to be zero
3.3 Training and sampling
Training. During the learning phase, our goal is to train the model by corrupting each dataset sample Go = (x", E²) into G = (x* E'",, with t < T, sampled from a uniform distribution, using the forward process. Having been subject insertions or deletions, X might contain only a subset of the nodes in X (if we deleted nodes) or include a new set of nodes (the ones that have been inserted up to step 1). We train the main model to predict the values that the nodes xt and edges E had at their respective activation times S (which may be equal to zero if the element belongs to the original graph) and denote these prediction X=0 and E=0 At the same time, we predict the activation times themselves, which we denote as S Finally, we train the auxiliary model (G to predict the correct amount NoøLL of DEL"s s that were present in G which we denote as NOEL.. The final loss is a sum of Cross-Entropy (CE) terms of the targets Xt Eet S*, and nom against their respective predictions:
(10) where Ax. AE, ÅE, and λPEL are hyperparameters that control the weight of each term. If we are training a model to condition on a specific set of properties y. it also fed at training time an auxiliary term e which, with a probability equal to 1 - p. P being an hyperparameter, will be equal to y. while in the remaining cases it will assume the value of a parametrized placeholder y. which is randomly initialized. This technique is called conditional drepout and is shown to produce better results in conditional generation (Ho and Salimans 2022]. Algorithm T in Appendix A.3 describes the new training process.
AxCE x=*) + >=CEE E² + AsC ,S") + ADEL CE(BDE- NOEL.)

---

Sampling, During sampling, we start from random latent G* sampled from the marginal distri- butions on the nodes and edges m and m g. The size of a sample is also chosen from the marginal distribution of the nodes in the training set. At each step, we first insert as many DEL" nodes as predicted by the auxiliary neural network. Then, we feed the current latent Go to the main model to predict GHO That is, the values that each node and edge had at their respective activation timesteps, as well as the activation timesteps themselves. Afterwards, we delete the nodes that are predicted to have been inserted at timestep t. Finally, we use these information to compute Equation 9 and sample from it the node and edge values at step - to obtain G We then repeat the operations using it as the new G and keep doing so for total of T steps until obtain the final graph Go. In the case in which we are performing conditional generation on a property y. the term pa (a xx') is re-elaborated follows:
l Po 8 = x=/x: y) = po(x x "er,,) y) + )(po = x x||= y) po po = x: x//=, y)),
(11)
where À is hyperparameter controlling the influence of the conditioning vector y. Algorithm 2 Appendix A.3 describes the sampling process in detail.
in
4 Experiments
Here, we detail the experiments to evaluate GRIDDD on different property targeting and optimization tasks. Our code is based on MiDi [Vignac et al.) 2023b] which, in turn, is based on DiGress. In all our experiments, we have T = 500 diffusion timesteps. The function ç' (t) is parameterized with w = 0.05 and D = T 250. Similarly to MiDi, we use = 1 for the node matrices' noise schedulers and v 1.5 for the edge matrices. Ax and A are set, respectively, to 1 and 2. We set the hyperparameters pmin and Pmax in Equation respectively as 0.2 and 1.
Datasets. Following previous works [Ninniri al. 2024 Vignac al., 2023a , we used QM9 [Ramakrishnan et 2014), dataset of 133k molecules made by up to 9 non-hydrogen atoms, and 250k- a collection of 250k drug-like molecules selected from the ZINC dataset [Irwin and Shoichet 2005]. On ZINC-250K, the molecules were first preprocessed by removing stereochemistry information and infrequent non-neutrally-charred atoms, leaving only N+ and O-, These atoms have been treated as standalone atom types. Dataset splits are described in Appendix B.1
4.1 Property Targeting
We adhere to the setup of Ninniri et al. [2024. where they extract 100 property vectors from the test set and generate, for each one of them, TO molecules. Then, they compute the Mean Absolute Error (MAE) between the target properties y and the estimated properties of the generated molecules ÿ:
100
MAE( 1000
y Visl-
(12)
On QM9, we targeted the Dipole Moment μı and the Highest Occupied Molecular Orbital (HOMO) properties, while on ZINC-250k we targeted the Log-Partition coefficient (LogP). the Quantitative Estimation Drug-likeness (QED), and the molecular weight (MW). Since MW strongly correlates with the number of nodes of the molecular graph, it provides natural and intuitive testbed to assess the ability to flexibly adapt the graph size, which is precisely what GRIDDD was designed for. Our baseline for comparison is discussed in Appendix B.3
4.2 Property Optimization
We use the test suite by Jin al. 2020]. which consists of three experiments on the ZINC-250K dataset. The objective is to generate molecules that improve on certain property while meeting similarity constraint. To optimize a molecule, we corrupt it for 100 steps before starting the denoising process, setting the property vector as the target value that the optimized molecule should have. In the LogP experiment, we optimize the 800 molecules with the lowest LogP among those in the test set. For each of them, we sample 20 candidates starting from different latents and only keep the one with the largest improvement among the ones with a fingerprint Tanimoto similarity with the starting molecule equal to or above threshold 6. We report the average of these improvements
7

---

across the 800 molecules, with d E {0.4,0.6) In the QED experiment, we optimize 800 molecules from the test set with QED the range [0.7, 0.8). For each molecule, we sample 20 candidates from different latents. We consider the optimization successful if at least one among the candidates has QED in the range [0.9, 1.0) and a fingerprint Tanimoto similarity with the starting molecule 2 0.4. We report the success rate across the 800 molecules. In the DRD2 experiment, we optimize for the biological activity against the dopamine type 2 receptor, selecting 800 molecules from the test set with DRD2 activity score < 0.05. For each molecule, we sample 20 candidates from different latents. We consider the optimization successful if at least one among the candidates has a DRD2 score 2 0.5 and fingerprint Tanimoto similarity with the starting molecule 2 0.4. We report the success rate across the 800 molecules. Our baseline for comparison is discussed in Appendix B.3
4.3 Out-of-distributioo sampling
To assess whether GRIDDD is able generate valid molecules outside the training distribution, we train an instance of DiGress and one of GRIDDD to generate samples unconditionally on QM9. However, during training, we force GRIDDD to insert up 14 nodes the molecular graph, despite the fact that QM9 features molecules with up to 9 atoms. Then, during generation, we force both models to generate molecules with up to 15 atoms (that is, above GRIDDD's 's training capacity).
Experimental setup. The denoising network shares similar architecture to FreeGress, and is described in Appendix [B.2 For QM9, we use the same hyperparameters employed by the authors. The auxiliary neural network used to predict the number of DEL's also uses the exact same setup, with the exception that it only uses one layer. For ZINC-250K, we also use the same hyperparameters FreeGress except for the number of layers, which we set 10 instead of the original 12 to approximately the same number of parameters. The size of the linear layer used to predict the activation time is 256. In QM9, we the guidance scale X 3, while in ZINC-250K we set = 2. These values have been chosen since they were the ones offering the best results for FreeGress. All experiments have been performed on an n Vidia A100 GPU with 80 GBs of VRAM (two on ZINC-250k).
5
Results
Here, we detail the experimental results obtained by GRIDDD. A visual collection of samples obtained from the model is available in Appendix D.3
5.1 Property targeting
QM9. Results are displayed in Table As can be seen, GRIDDD stably achieves state-of off the-hh-an art performance. When conditioning on μ., it improves MAE by 14% at the expense of a slightly lower validity. Conversely, when conditioning of a slightly higher MAE. For ablation purposes, we also include an unconditional model in the comparison, which is unable optimize the properties satisfactory degree. A discussion of the failure cases in QM9 is presented in Appendix D.4
HOMO, the model achieves better validity at the expense
Table 1: Results of property targeting QM9.
HOMO
Method
MAE +
Val.
MAE +
Val. 1
Unconditional
1.68 + 0.15
91.5 %
0.95 0.10
91.5 %
DiGress
0.80 + 0.07
82,5 %
0.61 4 0.07
91.2 %
FreeGress
0.74 + 0.08
83.7 %
0.32 + 0.04
90.1 %
GRIDDD
0.66 + 0.07
79.0 %
0.37 + 0.07
94 %
ZINC-250k.
Results are shown in Table 2 On ZINC-250k, GRIDDD once again achieves compa-
rable or better MAE, while improving validity rates two cases out of three. Of particular interest the MW experiment, as this property strongly correlates with the graph size. Note that FreeGress'
8

---

results have been obtained using an auxiliary neural network to predict the optimal number of atoms the molecule should have given the target MW, information which used to set the graph's size before starting the denoising process. GRIDDD is capable of halving FreeGress" MAE without any prior knowledge of the graph size distribution. summary, GR GRIDDD shows an improved capacity in generating valid molecules without sacrificing the MAE. What is truly relevant in these results is the fact that GRIDDD is capable of obtaining them despite being trained on a more difficult task, showing that our solution has smoother tradcoff between validity and accuracy.
Table 2: Results of property targeting the ZINC-250K dataset.
LogP
QED
MW
Method
MAE 4
Val.
MAE
Val.
MAE +
Val. f
Unconditional
1.52 + 0.12
86.1 %
0.15 + 0.01
86.1 %
74.16 6.71
86.1 %
DiGress
0.74 + 0.08
74.6 %
0.15 + 0.01
85.1 %
20.92 + 2.90
40.4 %
FreeGress
0.17 + 0.01
84.9 %
0.04 + 0.01
84.9 %
8.96 + 1.93
79.7 %
GRIDDD
0.19 + 0.02
87.9 %
0.04 + 0.00
87.2 %
4.89
0.57
84.2%
5.2
Out-of-distribution sampling
Results are displayed in Figure 3 While both models perform similarly with up to 12 atoms, the validity rates of DiGress steeply decrease on larger graph sizes. Notably, GRIDDD still generates 35% of valid molecules with 15 nodes, despite being trained on latents with up to 14 nodes. This shows how our model is fairly usable -iiittiiittiitttionn sampling, as it is able to learn graph sizes that do not appear in the training dataset.
100
Validity (%)
DIGRESS
GRIDDD
50
0
10
11
12
13
14
13
Molecule size (atoms)
Figure 3: Validity results of out-of-distribution sampling on QM9.
5.3 Property optimization
Results are summarized in Table Regarding the LogP property optimization, GR GRIDDD outperforms all competitors in both settings in terms of improvement. In particular, when the similarity threshold 0.6, it achieves an average improvement almost twice as high as the second best. In the other two tasks, GRIDDD scores remarkably better, especially in the QED task where it improves 5 times the success rates of GCPN. An additional comparison with the method by Ketata et al. (2025) (under a slightly different experimental setup) provided in Appendix D.I showing that GRIDDD performs comparably or better than methods that use 3D information to optimize molecules.
5.4
Computational considerations
The insertions and delctions performed by GRIDDD inevitably introduce computational overhead during training, leading to slightly increased training durations compared to methods that do not employ these operations. In particular, our analysis indicates that it is 30% slower than FreeGress (see Appendix C for an extended discussion). However, its flexibility in adapting the number of denoising steps to the task makes it generally faster sample from (as detailed in Appendix C.2).
Ablation. test the effective influence of insert and delete operations over the optimization process, we performed the same experiments with GRIDDD but disabling insertions and deletions.

---

Table
Benchmark on graph property optimization.
LogP (sim 2 0.4)
LogP (sim 2 0.6)
QED (sim 2 0..4)
DRD2 (sim
2 0.4)
Method
Improv. t
Divt
Improv. f
Div.
Succ.t
Div.
Succ.t
Div.
JT-VAE
1.03 1.39
0.28 + 0.79
%
3.4
(G-VAB
0.61
1.09
0.25 0.74
4.8 %
GCPN
2.49 1.30
0.79 + 0.63
9.4 %
0.216
4.4 %
0.152
GRIDDD
2.70 + 0.94 0.482
1.33 + 0.61 0.280
45.1 %
0.283
5.0 %
0.121
The results show that while the success rate remains relatively unchanged in the LogP and DRD2 experiments, it significantly drops to 33.8% when optimizing QED, likely because the QED score is a function of the molecular weight (and thus, correlates with the number of atoms). We conclude that, similarly to targeting MW, controlling the graph size substantially increases success rates.
naive form of insertion and deletion operations can be achieved implicitly on any graph DDPM by working with a large graph size (even beyond the largest in the training data), treating excess nodes/edges as padding nodes marked by a special PAD class which allows their removal at the end of the denoising process, Clearly, this method is slower to train and sample from, as it always works with maximally sized graphs. To check that this approach also less accurate, we compared two DiGress variants that implement node and edge padding against GRIDDD: the first only assigns the PAD category padding nodes, while the second also assigns it to their associated edges. All models were trained on the QM9 dataset for 250 epochs, after which we sampled 100 molecules each without conditioning. The results are displayed in Table As possible to see, while the baseline generates 3% more valid molecules than GRIDDD, they severely underperform in every other metric.
Table 4: Comparison of GrIDDD with DiGress- based padding variants that perform implicit in- sertions and deletions. Val: validity, Avg NC: average number of connected components, Max NC: maximum number of connected components, NSC: number of graphs sampled with single connected component, XCE (ECE): validation cross- over nodes (edges).
Model
Val
Avg NC
Max NC
NSC
XCE |
ECE |
GrIDDD
0.97
1
1
100
0.44
0.32
Nodes PAD
1
1.56
8
69
0.47
0.37
All PAD
1
1.2
4
84
0.48
0.38
In Appendix D.2 we conduct sensitivity analysis on some critical hyperparameters of GRIDDD, showing that results are robust to their choices.
6 Conclusions
We introduced GRIDDD, a discrete DDPM which supports the insertion and deletion of atoms during the denoising process. This addresses one of the major limitations of graph DDPMs, which cannot change the graph size throughout the generative process. In future studies, we aim to study a simplified approach that does need auxiliary network predict when insert node during the denoising process, and whether the distribution of the inserted nodes can be learned rather than employing predefined heuristics. Lastly, since the methodology we propose quite general. we foresee its applications to domains such as, e.g., vector floor-plan generation (Shabani et al., 2023
Limitations. During generation, we observed that the two neural networks composing GRIDDD may occasionally conflict, simultaneously performing an insertion and a deletion at the same timestep. which is theoretically illegal in our framework. GRIDDD also tends to produce more split molecules compared to FreeGress and DiGress. This is likely because nodes inserted late in the denoising process are challenging to connect before denoising concludes. Incorporating additional input features, such as the current number of connected components, could potentially mitigate this issue.
10

---

Acknowledgments
This work has been partially supported by EU-EIC EMERGE (Grant No. 101070918) and PNRK, PE00000013, "FAIR Future Artificial Intelligence Research", Spoke 1, funded by European Commission under NextGeneration EU programme (CUP: B53DD2302620000,
References
Andrew Campbell, William Harvey, Christian Weilbach, Valentin De Bortoli, Tom Rainforth, and Arnaud Doucet. Trans-dimensional generative modeling via jump diffusion models, 2023. URL
www./og////babaaoeoo..o /205. 16261
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. Principal neighbourhood aggregation for graph nets. In Advances in Neural Information Processing Systems, volume 33, pages 13260-13271. 1. Curran Associates, Inc., 2020.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Infor- mation Processing Systems, 2021. URL https: wttp:///ewweeteeeetooooooooaaaaa
Faezeh Faez, Yassaman Ommi, Mahdieh Soleymani Baghshah, and Hamid R. Rabiee. Deep graph generators: A survey. IEEE Access, 9:106675-106702, 2021. 3098417.
doi: 10. 10.1E09/CCEESS.2022
Jiazhen Hc, Huifang You, Emil Sandström, Eva Nittinger, Esben Jannik Bjerrum, Christian Tyrchan, Werngard Czechtizky, and Ola Engkvist. Molecular optimization by capturing chemist's s intuition using deep neural networks. J. Cheminformuatics, 13(1):26, 2021. doi:
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop
DGMs Applications, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Associates, Inc., 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3D. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 8867-8887. PMLR, 17-23 Jul 2022.
John Irwin and Brian Shoichet. Zinc a free database of commercially available compounds for virtual screening. Journal of Chemical Information and Modeling, 45(1):177-182, 2005. doi: 10.1021/c1049714:
wttp:////...11111////00111 PMID: 15667143.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2323-2332. PMLR, 10-15 Jul 2018a.
Wengong Jin, Kevin Yang. Regina Barzilay, et al. Learning multimodal graph-to-grape translation for molecular optimization. ICLR, 2018b.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Leurning, ICML'20. JogR. 2020.
Daniel D. Johnson, Jacob Austin, Rianne van den Berg, and Daniel Tarlow. Beyond in-place corruption: Insertion and deletion in denoising probabilistic models. In ICML Workshop Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. URL.
https://openreviae http://wwww.we.ooooomo//o///coee

---

Mohamed Amine Ketata, Nicholas Gao, Johanna Sommer, Tom Wollschläger, and Stephan Gün- nemann. Lift your molecules: Molecular graph generation in latent euclidean space. In The Thirteenth International Conference on Learning Representations, 2025.
Lingkai Kong, Jiaming Cui, Haotian Sun, Yuchen Zhuang, B. Aditya Prakash, and Chao Zhang. Autoregressive diffusion model for graph generation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt. Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 17391-17408. PMLR, 2023. URL http://diww..a...ppeeee/////////
https::/prooceeddngg mlr. ppress/2202 wttps////222k/oog2bbbbb html
Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L. Gaunt. Constrained graph variational autoencoders for molecule design. In Neural Information Processing Systems, 2018.
URL httppes//ap...sssstttccccooaa org/ Perg/CorpusID.. 43924438
Matteo Ninniri, Marco Podda, and Davide Bacciu. Classifier free graph diffusion for molecular property targeting. In Albert Bifet, Jesse Davis, Tomas Krilavičius, Meelis Kull, Eirini Ntoutsi, and Indré Zliobaité. editors, Machine Learning and Knowledge Discovery in Databases. Research Track, pages 3 318-335, Cham, 2024. Springer Nature Switzerland. ISBN 978-3-031-70359-1
Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. doi: w2t.://a....2....11....
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1(1), August 2014. ISSN 2052-4463. doi: 10.1038/sdata.20144 2014.22
Mohammad Amin Shabani, Sepidehsadat Hosseini, and Yasutaka Furukawa. Housediffusion: Vector floorplan generation via a diffusion model with discrete and continuous denoising. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 5466 5475. IEEE, June 2023. doi: 10. 20099//vpP2229.22230035555 URL http:t/::/dxx doi. org 10. 1109/ CVPR52729 2023. 00529
Clement Vignac, Igor Krawezuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. In The Eleventh International Confer- ence on Learning Representations, 2023a.
Clement Vignac, Nagham Osman, Laura Toni, and Pascal Frossard. Midi: Mixed graph and 3d
denoising diffusion for molecule generation. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pages 560-576. Springer, 2023b.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018a.
Jiaxuan You, Rex Ying, Xiang Ren, et al. GraphRNN: Generating realistic graphs with deep auto- regressive models. In Proceedings of the 35th International Conference on Machine Learning. volume 80 of Proceedings of Machine Learning Research, pages 5708-5711 PMLR, 10-15 Jul 2018b.
Yong Zhao, Edirisuriya M. Dilanga Siriwardane, Zhenyao Wu, Nihang Fu, Mohammed Al-Fahdi, Ming Hu, and Jianjun Hu. Physics guided deep learning for generative design of crystal materials with symmetry constraints. upj Computational Materials, 9(1), March 2023. ISSN 2057-3960. doi: doi org 10. 1038/ s41524-144444 020-2444-220000722200000
10.1038//41522-0223 020-00023------00
URL
Yanqiao Zhu, Yuanqi Du, Yinkai Wang, Yichen Xu, Jieyu Zhang, Qiang Liu, and Shu Wu. A survey on deep graph generation: Methods and applications. In The First Learning on Graphs Conference, 2022. URL http:///ppppneeeeeeeeeffrroorree
12

---

A Additional details on the methodology
A.1 Delete scheduler and insert/delete timestep distributions
We show possible realization of the delete scheduler ((1) and the insert/delete timestep distribution
(t) in Figure
1.0
0.8
0.000
0.006
0.4
0.002
0.0
0.000
200
(b) < (t)
(a) ((t)
Figure 4: The functions ((t) and its derivative 5' (t) for w 0.05 and T = 500. - ((t) is effectively the "delete scheduler", and represents the probability that a node selected for deletion has already been deleted at timestep l. Ç (l) represents instead the probability that node chosen for deletion will switch to category DEL" exactly at timestep
A.2 Distribution the labels of the inserted nodes and edges
During the forward process, GRIDDD assigns labels to the newly inserted nodes and edges using marginal distribution these labels computed on the individual training sample. Such distributions be computed just once before starting the training process and can also be saved for multiple training sessions. We have also tried different distributions during our preliminary studies, such always inserting the least frequent node/edge class, well as sampling them from uniform probability, but the sample's 's marginal distribution gave us better results.
A.3 Training and sampling algorithms
Here, we provide the pseudocode describing the procedures to train (Algorithm) I and sample from (Algorithm GRIDDD.
B
Additional experimental details
B.1
Dataset splits
QM9. On QM9, the training is made of the first 100000 samples in the dataset. The test is made of 10% of the overall data, and the remaining data is used to make the validation set.
ZINC-250k. The training set uses the first 80% of the data. The remainder is equally split between the validation set and the test set.
B.2 Neural architecture
Both the main model and the auxiliary model employed to predict the number of DELs are based on FreeGress' architecture, The model is composed of a stack of Graph Transformer layers. receives as input the node features matrix X, edge features matrix E, and vector encoding the timestep / and optional extra features. These inputs are augmented with conditioning information y (denoted with the + superscript in Figure 5a and processed through the transformer stack.
13

---

Algorithm 1 Pseudocode of the training algorithm
Require: g
dataset of graphs
Require: T. the number of denoising timesteps.
Require: pe. the main model,
Require: Ig. the model tasked to predict the number of D in G
Require: (()) a discrete probability distribution function defined on T,
Require: h(n), which given graph size returns probability distribution h, on the number of nodes.
1: for each graph G; = (X, E,) E G: do
2: n |X
3:
n² ~ (n)
4:
U~Uniform(1,7
5:
m.x, m. marginal distribution computed on the node and edges of G
6:
S" = (O..... .,......) 0} vector storing the insertion times of the nodes X E. (X.) Qx (E)) 4/0
7:
8:
A n" n
9:
Sample U = {uy} timesteps, with j [0...|47| - 1}, where uj Ç
Remove from U the elements greater than
11:
for each My E U do
12:
if A > 0 then Sample new node mx, with insertion time uj, and add it to X;
Insert
Sample x' from the distribution 'Q and add it X;" Sample from m., for each node already in the graph, an inward and outward edge with the newly added æ;
15:
16:
Sample the edges e's final state e' from Q and add them to Ex
17:
Append My at the end of S
18:
else A < ( then
>
Delete
19:
Sample random node among the ones in G, that set to state DEL or DEL
20:
if u then
21:
æ DEL
22:
Set all incoming and outgoing edges of DEL.
23:
else
Remove æ from X
25:
Remove all incoming and outgoing edges of æ from E
Remove the entry in S* associated to the node
27:
end if end
29:
end for
G x E
31:
{(x.0 00 E
S 1+ (
32:
loss; Ax CE CB x=0) CE EE")) + Cs s-))
33:
nogL = number of nodes set to DEL* G
34:
Remove all nodes set to DEL" in G well the associated edges
35:
loss = C (Gg((t)t) PDEL-)
end for
14

---

Algorithm 2 Pseudocode of the denoising algorithm
Require: T, the number of denoising timesteps,
Require: n² the number of nodes in the noisy graph G²,
Require: ps. the main model,
Require: 9g. the model tasked to predict the number of DEL's in G
Require: x and g. the dataset's 's marginal distributions on the nodes and the edges:
1: Sample G (x" EET) with n² nodes from an the marginal distributions m and me:
2: for each timestep t E [T..... 1]: do
3: Use (a((") to predict the number of DEL" entries to insert in the graph, and insert them.
4: Set all the edges connecting the newly inserted nodes to the rest of the graph as DEL".
5: Use pa (G)) to predict S", the insertion times for each node in the graph, and G
6: Remove the nodes æ; such that S = I, as well as their edges.
7:
Use S and G to sample, for each node ar and edge eiy G their type at step / 1:
g/ac ket "a(a" las (====))) =2)
8:
of po (2) (xx) = EzEx
pa (0 = x(xx).
(<)l,, =z)
eletylel, ")alef "leyy Ve mao(s, =e) po(e; eeet,le
o po 1-1 p = =Eeer
mar(S, ele
9:
masts; 35)
G (fa) }.(e)j11
11: end for
Each Graph Transformer layer (Figure 5b) follows the standard architecture: self-attention, dropout, residual connection, layer normalization, and feedforward block. The self-attention mechanism (Figure (c) applies scaled dot-product attention to the augmented node features X. The resulting
attention weights are modulated by the edge tensor E via a FiL. layer (Perez et al. (018). normalized with softmax, and used re- re-weight X. The output is flattened and combined with through another FiLM layer before linear projection to prediction logits.
Separately, X and E are passed through independent PNA layers [Corso et al. 2020 and their outputs are summed with a linearly projected u to form the final graph representation. In GRIDDD, the final output u is employed by our auxiliary model to predict the amount of DEL* nodes that must be reintroduced at the current timestep. The final Graph Transformer Layer's 's output designed to predict the matrix X' is passed two Linear layers, instead of just one. The first is used, just like before, to predict the final node matrix. The second one is used instead to predict the activation timestep s of the various nodes.
B.3 Baselines
we compared our model against DiGress (Vignac al. 2023a], Graph Diffusion Model capable of performing conditional graph generation through a classífier guidance, and FreeGress [Ninmiri et al. 2024], a Graph Diffusion Model which employs classifier-free guidance system instead. For the task of property optimization, discussed Section [.2] we employ a baseline of three models. In particular, we employ the
For the task of property targeting, discussed in Section 4.1
Junction- Tree Variational Autoencoder (JT-VAE, Jin al. [2018a)) D, an Autoencoder that works on the molecular sub-structural level rather than the atomic level, the Graph Convolutional Policy Network (GCPN, You al. (2018a)), which employs Reinforcement Learning to generate molecules atom-by-atom, and the Constrained Graph Variational Autoencoders (CG- CAF, [Liu et al. (2018). a graph-based Autoencoder whose peculiarity is that applies chemical validity constraints during the generative process to increase its performance in molecular tasks.
B.4
Considerations on the independence between nodes and edges in the forward process
When node is switched to DEL", are the edges associated with it. This effectively creates a dependence between the two, while standard Diffusion Models assume a mutual independence between them. This is similar to what is done with Masked Diffusion (Kong et al. 2023), where the edges associated with masked nodes are masked as well. In our case, what we effectively do replacing the forward process p(ef, (e with p(e/z |e x xf), and p(e'z leig et,) with p(et "el,, e,, æ/, x.). However, it should be kept in mind how the latter equation is, in all practical
15

---

X
Leyre Nurm
FIE Loooer
PUM
Linoe
Lauour
Linear
Groph Trantfoemee Layer
Lio. Dropout
PNA
PMM
Lax
Drspor
Sclamu
Gruph Trunslemnr Layer
Norm
Dropout
Concac
Concat
Coneat
Sell Attention
Linear
Linear
Libeae
X
AT
V
(b)
(c)
Figure 5: (a) High-level architecture of GRIDDD. (b) Detail of the graph transformer layer. V represents the triple (x, E, u). (c) The self attention-layer within the graph transformer layer. In the pictures, orange boxes indicate inputs, blue boxes indicate outputs, and green boxes indicate layers/operations.
scenarios, identical to "ley. ely)), as the only situations where æ/,; could influence the value of e when the former is labeled as DEL* and the latter to DEL, in which case e should switch to DEL* as well. However, this scenario not possible in practice, neither the nodes nor the edges can appear explicitly with label DEL during the reverse process. We leave to future research the study of independent edge processes that still allow for the deletion the associated nodes.
C Additional computational considerations
C.1 Runtime
On average, one training epoch of GRIDDD takes 309.1 seconds on two nVidia A100 GPUs. FreeGress, on the same problem (hence, without insert and delete operations), takes 243 seconds. This means that GRIDDD is approximately 27% slower than its counterpart. This is similar to the 33.3% increase have found on QM9, where GRIDDD takes 18 seconds average complete training epoch on single A100, while FreeGress takes 13.5.
C.2 Sampling speed
Interestingly enough, GRIDDD can be faster at sampling than standard Diffusion Models. When tasked with generating a variegated batch of graphs with different sizes, standard Diffusion Model would first sample the graph sizes from the dataset's node distribution. This means that, in practice, dimensions equal the highest
the sizes of the node and edge matrices will have graph size sampled. In turn, this means that every graph generated will require a processing power proportional such size.
On the other hand, GRIDDD capable of generating a graph of any shape, even when starting from different size. In this scenario, could start the sampling process with very small latents (even zero nodes). Since the tensors representing the sample batch would be very small, the computational power required process these tensors will be small well. This means that the early phases of the sampling process, before most nodes are inserted, will likely be faster than standard diffusion (although this is highly dependent {())).
To test this hypothesis, we have sampled 128 molecules on ZINC-250K, conditioning on the LogP value, with FreeGress and GRIDDD. For the latter, we have performed three experiments. In the first,
16

---

GRIDDD starts from latents with size of two nodes. In the second, we have used an initial size of 24 (the most frequent graph size in the training set), and then we have sampled the initial graph size using the training set's node distribution, as is done in FreeGress. The results, summarized in Table 5 show how GRIDDD is capable of performing as well as FreeGress while halving the sampling time in the best case scenario, and even in the worst case scenario it requires as little as five percent more time to perform as well as FreeGress.
Table 5: Sampling speed comparison between GrIDDD and FreeGress when generating 128 samples on ZINC-250k conditioned on the LogP value, using different initial latent sizes.
Model
Validity
MAE
Sampling time
FreeGress
89.8%
0.31
152.05s
GRIDDD (initial size=2)
91.4%
0.23
68.09s
GRIDDD (initial size=24)
90.6%
0.31
120.19s
GRIDDD (initial size ~ p(n))
88.2%
0.26
159.858
D Additional results
D.1
Comparison with EDM-SyCO
We compared to the 3D Diffusion Model by Ketata et al. 2025 called EDM-SyCO, since it performs property optimization similarly GRIDDD. Notably, EDM-SyCO also inputs 3D coordinates to the diffusion process. To ensure a fair comparison, we slightly adapted our original setup to the less restrictive one used in the related paper. In particular, they attempt optimize each molecule 100 times, rather than 20 as in our previous experiment. Then, among the ones within the similarity threshold, they select the 10 ones with the best improvement, duplicate each one of them ten times, and run the optimization process a second time. The process is repeated four times (for total of 400 optimization rounds), after which the molecule with the best improvement within the similarity threshold is selected. All the successful molecules obtained in the process (that is, within the similarity threshold) are considered when computing the diversity score. cases where the number of successful experiments is less than two, the diversity score is set to zero. Results of the comparison are reported in Table 6 showing competitive or superior performances by GRIDDD, even though it does not benefit from using 3D information as input. We have compared our results against EDM-SyCo on property targeting as well. Specifically, we have trained a model on ZINC-250K with 1000 denoising timesteps (as is done by our baseline) that is conditioned on the molecular weight. We have obtained a MAE of 2.13 + 0.19 and a validity of 86.2%, while EDM-SyCo records a MAE of 3.86 + 0.08 and validity of 88%.
Table 6: Comparison with EDM-SyCo on property optimization.
LogP (sim
0.4)
LogP (sim 0.6)
QED (sim
0.4)
DRD2 (sim
0.4)
Method
Improv.
Div.
Improv. f
Div.t
Succ.
Div.
Suce.
Div.
EDM-SYCo
3.11 + 1.27
0.555
1.51
1.10
0.360
46.4 %
0.163
27.3 %
0.083
GRIDDD
3.27 + 0.91
0.511
1.59 +: 0.54
0.359
64.1 %
0.269
19.7 %
0.058
D.2
Hyperparameter sensitivity analysis
We show in Table 7 an analysis of GRIDDD's sensitivity of the hyperparameters regulating the scheduler (D and w) and the node counts distribution (Pmin and Pmax). One interesting insight that can be gathered from the table the fact that the number of split molecules generated significantly increases with smaller Ds. From a practical perspective, small values for this hyperparameter imply that, during the denoising process, the model can insert nodes relatively late during the reverse process. We conjecture that split molecules are mostly caused by nodes inserted too late during such a process, as they cannot be re-attached in time to the main graph since the noise scheduler does not allow for too many changes in the graph's 's structure during the last phases of the process. We
17

---

have also noted that this phenomenon is sensibly reduced when GRIDDD is given in input, as extra features, different powers the adjacency matrix.
Table 7: Study of GrIDDD's s sensitivity to hyperparameter changes when sampling 100 molecules after training the QM9 dataset. The hyperparameters tested are the delete scheduler's parameters D, w. and the node count distribution's 's parameters Pmin and Pmax- Val: validity, Avg NC: average number of connected components, Max NC: maximum number of connected components, NSC: number of graphs sampled with a single connected component.
Val f
Avg NC
Max NC
NSC
Val f
Avg NC
Max NC
NSC
0.25
0.93
1.15
4
88
0.025
0.94
1
100
0.50
0.93
1.02
2
0.050
1.02
0.75
0.95
1.01
2
99
0.075
0.92
1.03
97
Pmin
Val f
Avg NC +
Max NC
NSC 1
Puax
Val f
Avg NC
Max NC L
NSC 1
0.2
0.93
1.02
2
0.5
0.86
1.02
0.4
0.91
1
100
0.1
0.93
1.02
98
0.6
0.96
1.02
3
99
0.01
0.89
1.01
99
0.8
0.94
1.02
2
98
0.001
0.94
1.02
98
D.3
Generated molecules
Figure Sshows 6 (3 for the QED task, for the LogP task) randomly selected molecules optimized by GRIDDD on the ZINC-250K dataset. For each molecule, we show different successful optimizations. Figure shows random molecules generated without conditioning by GRIDDD on QM9.
D.4
Failure Analysis
While analyzing the situations where GRIDDD fails in conditional generation the most, we have noticed how, unlike FreeGress, the experiments targeting μ tend fail most frequently when targeting values close to zero. The few molecules generated tend to be small, split graphs. We have investigated the phenomenon and found out that the molecule with SMILES string CC (ethane) is known have dipole moment equal to zero. We believe that it is likely that GRIDDD tries to generate ethane or similar small molecules to minimize 14, with higher failure rates than usual since these small molecules are under-represented in the dataset. Remarkably, DiGress and FreeGress are unable to pursue this minimization, since they almost inevitably start the reverse process from latents with larger graph size and cannot adapt to produce smaller molecules later on. Overall, this is an interesting behavior which shows that GRIDDD can successfully learn the properties of molecules with infrequent sizes well.
18

---

LogP: -1.4483 Similarity: 0.6393
LogP: -1.9506 Similarity: 0.6949
LogP: -1.9536 Similarity: 0.6032
LogP: -1.6673
Original LogP. -3.3677
Similarity: 0.6250
LogP: -2.1297 Similarity: 0.8000
LogP: -1.8929 Similarity: 0.6657
LogP: -1.9788 Similarity: 0.7167
LogP: -2.0308
Original -3g3:-3.3100 -3.3100
Similarity: 0.6875
LogP: -1.9749 Similarity: 0.6364
LogP: -1.9749 Similarity: 0.6087
LogP: -18188 Similarity: 0.6087
LogP: -199749
Original LogP: -3,2214
Similarity: 0.6087
QED: 0.9044 Similarity: 0.5195
QED: 0.9158 Similarityy 0.4286
OED: 0.9027 Similarity: 0.4304
QED: 0.9054 Similarity: 0.4051
Original QED: 0.7246
QED: 0.9407 Similarity: 0.5161
QED: 0.9056 Similarity: 0.6102
QED: 0.9117 Similarity: 0.4754
QED: 0.9200 Similarity 0.5323
Original QED: 0.7401
QED: 0.9106 Similarity: 0.4861
QED: 0.9068 Similarity: 0.4675
QED: 0.9106 Similarity: 0.4861
QED: 0.9172 Similarity 0.6935
Original QED: 0.7351
Figure 6: Randomly selected optimization experiments on ZINC-250k. The first element in each row is the original molecule, while the other four are results of different optimization experiments. We report, for each experiment, the original value and the resulting one.
19

---

HO
Hy
Figure 7: 30 non-curated samples generated without conditioning on QM9.
20

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have introduced new model called GRIDDD, which can perform insert and delete operations. The Results section shows how it match and even surpass the baseline used despite being trained on an arguably more difficult problem.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have discussed the limitations our model the Conclusions. have been transparent possible.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
21

---

Answer: [NA]
Justification: Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have been as complete as possible. Our experiments employ the same architecture of previously published models with the same hyperparameters (albeit with minor modifications discussed in the paper). Regardless, we have discussed the architecture in detail the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
22

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We include the source code, well instructions how up Conda environment to run it, both in the supplemental material and the GitHub repository referenced in the main document.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ctipss.c//ublic//uiie gwwiii/gguee//eeeeeoeooi.icsssee CodeeSummeee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: we have discussed the dataset splits in the supplemental material. As stated in the main paper, most of our hyperparameters the same used by FreeGress, and have only listed the differences with our solution. Our source code contains the various configurations files in yaml format.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We reported the same metrics used by the bascline.
Guidelines:
The answer NA means that the paper does not include experiments.
23

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We discussed the type of GPU employed. In the appendix, we discussed the run time compared to the baseline. The runtime is similar for all experiments, so we did not find it necessary to report every single run time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform. in every respect, with the NeurIPS Code of Ethics httppp////arrpp..cc/paaaa//eeeea
Answer: [Yes]
Justification:
We believe our work is conforming to the Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We do not believe that our work will have any socictal impact.

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have credited the paper on which we based our own code. We have cited both ZINC-250k and QM9. We are not aware of any license terms of use concerning these assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
25

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperssii paperswi thcode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We did not release any new asset.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing and we did not perform research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsourcing and we did not perform research with human subjects.
Guidelines:
26

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy for what should or should not be described.
27


Reviews:
[{"review_id": "x6uh3njQck", "reviewer": "Reviewer_kSBz", "summary": "The manuscript presents a method, GRIDDD, that extends the DIGRESS method for discrete graph diffusion by monotonic node insertion or removal. Original DIGRESS method shows excellent performance in molecular generation tasks, but number of nodes of the graph have to be specified beforehand. For some applications, e.g., drug design, one aims to generate a molecule with specific properties and the size of the molecule might not be known beforehand. The GRIDDD method allows to change the number of atoms during the diffusion process; in combination with classifier guidance this makes it possible to optimize the generated molecule towards a certain property. For this, the manuscript introduces new transition states for deleting nodes, a scheduling function that determines when and how likely a deletion/insertion happens as well as an auxiliary neural network that predicts how many nodes to delete at a specific time step.", "questions": "As stated above, I see the main advantage of this method in the theoretical possibility to insert nodes/edges at a very late timestep. Otherwise, I could just use the original DIGRESS method (either by inserting \"DEL\" nodes in the beginning, or by simply adding nodes in the middle of the diffusion process. But the latter will probably give me worse results as there is not enough time to properly denoise them). I do not really understand the experimental setup of the \"property optimization\" section. Do you start from a input molecule and only add a low/middle amount of noise to guarantee that the generated molecule is close to the input molecule? Or do you generate new molecules from random latent and just keep the ones that are close to the input molecule? I think an evaluation would be interesting where the input molecule is noised by a lower amount of timesteps (e.g. t=100) and then denoised with a classifier-guidance.\n\nSecond, as I stated above it's relatively trivial to implement node insertions/deletions in DIGRESS, although this naive implementation is probably not as good as a proper implementation. I would still be interested in how well such a baseline method performs in your evaluations.\n\nFinally, I just do not understand why you need two different DEL states. Can you explain this in more detail please?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "First of all, the limitation that graph size has to be specified beforehand is not as critical as it seems. One can simply start from multiple graph sizes and sample all of them, deciding for the best graph afterwards. This is more time consuming than changing graph size during denoising, though. Another possibility is to learn a predictor that estimates an optimal graph size for the given problem. This is done by FREEGRESS and the evaluation shows that the GRIDDD method performs more or less the same for most problems. Only for molecular weight, the GRIDDD method outperforms FREEGRESS clearly. However, this is a property that completely depends on graph size and might be somewhat arbitrary and, in practice, rather uninteresting.\n\nI would argue that the most interesting problems for generative models are sampling of similar molecules that optimize certain properties (e.g., given an already known drug, find another one that is highly similar but has better water solubility). For such tasks the size of the optimized/generated molecule is often close to the input molecule anyways. The interesting question here would be rather: can I start from the input molecule and only add a little bit of noise to ensure that the generated molecule is close. Here, a method like GRIDDD could be advantageous, as it allows, in theory, to add new nodes rather late in the diffusion process. I'm not sure if the \"Property optimization\" section goes in this direction, as the description does not mention if they start from a random latent or from a input molecule.\n\nThe decision to use monotonic insertions and deletions make the underlying markov chain time irreversible. My feeling is that this might introduces a lot of problems and it also makes the underlying math very complicated. I wonder if there is any advantage in the way GRIDDD introduces insertions and deletions. The original DIGRESS model keeps the number of nodes fixed, but it already allows for arbitrary insertions and deletions of edges. The way it's done is that edges just have a special \"delete\" state which is treated similarly to their \"single bond\", \"double bond\" and so on states. During diffusion, edges can be inserted and deleted. The same would also be possible with atoms. One would still have to decide for a \"maximum number of atom\" constraint at the beginning, but besides that no changes of the method are necessary to allows node insertions/deletions. Clearly, there are a lot of disadvantages of such a method. For example, edges could connect deleted atoms. But it is already possible that more than 4 edges connect to a single carbon atom and the model just learns to prevent this. In my opinion the GRIDDD method should evaluate against this very simple baseline.\n\nFinally, I found several small errors in the manuscript\n- the negative sign in the probability density function zeta has to be removed, otherwise, all probabilities are negative\n- the scaling parameter w has to be inverted, i.e. the manuscript states to use w=0.05 but they probably mean w=1/0.05. Otherwise, the exponential becomes extremely large and the zeta function very steep.\n- equation 2 is missing an absolute, otherwise the probability is maximum when all atoms are deleted and minimum if all atoms are inserted. But the maximum should be at zero insertions/deletions.", "was_revised": true, "final_justification": "The underlying method feels very complicated, but the authors could demonstrate that the method outperforms simpler solutions such as adding DEL nodes in DIGRESS.", "rebuttal": "We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.\n\n- First of all, the limitation that graph size has to be specified beforehand is not as critical as it seems. One can simply start from multiple graph sizes and sample all of them, deciding for the best graph afterwards. This is more time consuming than changing graph size during denoising, though. Another possibility is to learn a predictor that estimates an optimal graph size for the given problem. This is done by FREEGRESS and the evaluation shows that the GRIDDD method performs more or less the same for most problems.\n\nThe reviewer is correct to point out that existing methods resort to _pre-hoc_ (auxiliary models, estimation) or _post-hoc_ (multiple sampling) heuristics to accommodate for different graph sizes. In contrast, GrIDDD is an _ad-hoc_ strategy that directly learns the graph size according to the conditional generative task, while being as effective as, or better, than the baselines. This also brings advantages in terms of its broader applicability: in fact, while GrIDDD can be used for molecular optimization, FREEGRESS cannot (since it cannot adapt the graph size dynamically to optimize the property). \n\n- Only for molecular weight, the GRIDDD method outperforms FREEGRESS clearly. However, this is a property that completely depends on graph size and might be somewhat arbitrary and, in practice, rather uninteresting.\n\nWe agree with the reviewer that molecular weight directly depends on graph size. We, however, respectfully disagree on the fact that it is not critical, somewhat arbitrary, and not interesting. A relevant number of molecular properties are directly relatable to molecular weight, and thus, to the graph size. For example, among the 5 Lipinski rules for drug development [1], the first one is indeed a constraint on molecular weight, and the others (number of rotatable bonds, number of hydrogen donors/acceptors, LogP to a minor extent) are related to the graph size. Other key properties that relate to graph size include polar surface or synthetic accessibility, which are crucial in drug design. Therefore, we argue that methods to adapt the graph size during the generative process are worth studying, researching, and improving.\n\n[1] Lipinski et al. (1997). Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews, 23(1–3), 3–25.\n\n- I would argue that the most interesting problems for generative models are sampling of similar molecules that optimize certain properties (...). For such tasks the size of the optimized/generated molecule is often close to the input molecule anyways.\n\nAgain, the reviewer’s statement is correct. Nonetheless, the size of the molecule is likely to change during the optimization process (e.g., consider removing a toxicophore group from a compound to improve its safety, or adding carbon chains to improve hydrophobicity). Contrary to existing methods, GrIDDD is capable by design of adapting to different graph sizes without workarounds such as training larger adjacency matrices to accommodate the out-of-distribution graph sizes. \n\n- The interesting question here would be rather: can I start from the input molecule and only add a little bit of noise to ensure that the generated molecule is close. Here, a method like GRIDDD could be advantageous, as it allows, in theory, to add new nodes rather late in the diffusion process. I'm not sure if the \"Property optimization\" section goes in this direction, as the description does not mention if they start from a random latent or from a input molecule\n\nWe thank the reviewer for spotting this shortcoming and for giving us the opportunity to clarify this point. We described this exact setup in Section 4.3 (Out-of-distribution sampling) in the paper, but we forgot to add it to the experimental setup of Section 4.2 (Property optimization) as well. Indeed, as guessed by the reviewer, for property optimization, we start from an input molecule, we corrupt it for $t=100$ timesteps, and then perform the denoising process as usual. The example suggested by the reviewer (adding new nodes late in the diffusion process) is exactly why GrIDDD is so advantageous and effective in the molecular optimization task.\n\n- The decision to use monotonic insertions and deletions make the underlying markov chain time irreversible. My feeling is that this might introduces a lot of problems and it also makes the underlying math very complicated.\n\nIf the reviewer is referring to our use of absorbing states (DEL), there is already literature discussing similar strategies [2], which shows that diffusion models work regardless of whether the underlying Markov chain is irreversible or not. At the same time, we point out how our reverse process only deals with the reversible portion of the Markov chain, as it only deals with DEL* nodes which are not absorbing nodes (rather, a gateway between proper classes and the absorbing DEL state, but still reversible). The irreversible part of the process is dealt implicitly by just predicting when one or more DEL nodes should switch back to DEL*. Therefore, we do not consider this an issue or a limitation. \n\n[2] Austin et al. (2021). Structured Denoising Diffusion Models in Discrete State-Spaces. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).\n\n- I wonder if there is any advantage in the way GRIDDD introduces insertions and deletions. The original DIGRESS model keeps the number of nodes fixed, but it already allows for arbitrary insertions and deletions of edges. (...) In my opinion the GRIDDD method should evaluate against this very simple baseline.\n\nAs the reviewer already noticed, fixing the graph size to a large number of nodes to account for out-of-distribution graph sizes would be extremely resource-inefficient, with slower training and sampling times especially for datasets with larger molecules such as ZINC250k.\nWith that being said, we have implemented two baselines as requested by the reviewer and tested them on QM9. One is essentially DIGRESS with an additional DEL atom type, while the other adds a further edge type EDEL to mark any bond whose endpoint is of type DEL.\nThe results of the comparison are shown below.\n\n- val: validity (higher=better)\n- avg. nc: average number of isolated connected components (lower=better)\n- nc max: maximum number of isolated connected components (lower=better)\n- n conn: number of valid connected graphs generated (higher=better)\n- TV: total variation between generated distribution and the dataset’s (lower=better)\n- CE: Cross-Entropy (lower=better)\n\n$$\n\\begin{array}{lcccccccc}\nModel & Val & Avg. NC & \\\\text{NC Max} & \\\\text{N Conn} & \\\\text{Val X CE} & \\\\text{Val E CE} & Atom TV & Edge TV \\\\\\\\\n\\\\hline\nGrIDDD & 0.97 & 1 & 1 & 100 & 0.44 & 0.32 & 0.016 & 0.01 \\\\\\\\\nDIGRESS+DEL & 1 & 1.56 & 8 & 69 & 0.47 & 0.37 & 0.047 & 0.10 \\\\\\\\\nDIGRESS+DEL+EDEL & 1 & 1.2 & 4 & 84 & 0.48 & 0.38 & 0.05 & 0.09 \\\\\\\n\\end{array}\n$$\n\nAs can be seen, both baselines perform poorly when compared to GrIDDD, especially in terms of connected components, indicating that the model has difficulties in generating molecules with less than nine nodes. The validity rates of the baselines appear higher, but a close inspection of the generated molecules shows that they are inflated by the presence of isolated carbon atoms which were not correctly attached to the main component.\n\n- Finally, I found several small errors in the manuscript (...)\n\nWe thank the reviewer for pointing the small errors out. We confirm that we used $w$=0.05. The slope is not too steep for our needs (unfortunately, we can not post pictures). Equation 1 is indeed the derivative (with respect to $t$) of $\\zeta(t)=1/(1+exp(-(D - t) / w))$, the function used for the transition matrices, but we compute the absolute value of the gradient to turn $\\zeta’(t)$ into a bell-shaped function (and then normalize). All errors will be revised whenever we will be allowed to edit the original submission.\n\n- (...) I do not really understand the experimental setup of the \"property optimization\" section \n\nWe believe we have answered to this concern above. Nonetheless, we are open to further discussion or clarification, if needed.\n\n- (...) it's relatively trivial to implement node insertions/deletions in DIGRESS (...) I would still be interested in how well such a baseline method performs in your evaluations.\n\nWe believe we have answered to this concern above. Nonetheless, we are open to further discussion or clarification, if needed.\n\n- Finally, I just do not understand why you need two different DEL states. Can you explain this in more detail please?\n\nThe DEL* state is the first state that a deleted node enters during the forward process. Within the GrIDDD formulation, it is necessary since the model is taught to learn the exact timestep where a node gets deleted. This is because, when denoising the sample at step t, we force a newly inserted node to switch immediately back to a \"proper\" node category at step $t-1$. The most intuitive category to assign at re-insertion would be DEL. However, the semantics of the DEL state is two-fold: it may either turn back into a proper node category, or it could stay in the absorbing state DEL (in the reverse process). Therefore, we determined that the best course of action is to treat the first timestep after a forward deletion as its own class (which we called DEL*). By doing so, the reverse process will always switch to a \"proper\" class at step $t-1$. We hope that our answer resolved the reviewer’s doubts, and we remain open to discussing further clarifications."}, {"review_id": "gkw8DgJ5Wo", "reviewer": "Reviewer_uVPd", "summary": "The paper proposes a method that allows to vary the number of nodes in graph discrete diffusion models. In particular, this method allows to insert and delete nodes throughout both the forward and reverse processes. This is relevant for problems such as property optimisation, where the objective is to modify a molecule, parsed to a graph, such that given properties of the molecule are optimised, while still remaining structurally close to the initial molecule. The newly introduced diffusion process is combined with pre-existing classifier-free conditional guidance to target the tasks of property targeting, property optimisation, and out-of-distribution sampling.", "questions": "1. The removal of the edges connected to deleted nodes causes discontinuities in the edge diffusion processes as well. Did the authors analysed this component? Any relevant observation?\n2. \"One problem of using a single deletion type is that nodes will loop in the absorbing state indefinitely once taken the DEL type, while the reverse process is supposed to revert all deletions into normal atom types\" (l. 150-152). This is not true for the marginal noise model employed in the diffusion process as, contrarily to masking discrete diffusion processes, it allows transition between every state at any timestep. Therefore, it seems to me that the motivation for the definition of the state Del* is incorrect. While I still see the importance of using the Del* state to signal to the model at sampling time to also predict for that node, the motivation of such formulation has to be modified. Please clarify.\n3. The node and edge classes assigned at insertion time by sampling from the marginal distribution of nodes and edges, respectively, can be further motivated by the fact that the usual marginal noise model preserves marginal distributions across the forward process. Without this motivation, seems an ad-hoc trick. Did you try anything different?\n4. An important baseline to incorporate consists of a DiGress implementation with, as similar to edges, a \"non-existing\" node class (equivalent to DEL) and let the process unroll. Edges connected to non-existing nodes should be treated consistently with how edges connected to DEL/DEL* nodes are handled.\n This process also allows to vary the number of nodes throughout the diffusion process without incurring into extra computational costs neither the more intricate diffusion process formulation of this paper. A superior performance to this baseline would clearly fundament the utilisation of the proposed diffusion process, with the additional model.", "limitations": "Yes, the authors explicitly address the limitations of the paper and potential future directions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n1. The writing of the paper is clear. \n2. The targeted problems (property targeting, property optimisation, out-of-distribution sampling) are relevant. The proposed method outperforms existing alternatives for property optimisation.\n3. While the idea of varying the dimensionality of the generated instance by the diffusion model is not novel in general [1], neither for discrete state-spaces in particular [2], this work is the first, to the best of my knowledge, employing successfully such idea for graph diffusion models. \n4. *Empirical results*: GrIDDD clearly outperforms existing alternatives for property optimisation setting. This method can also be used to ensure better out-of-distribution graph generation in terms of number of nodes, as its training allow the model to learn some representations over graphs that have more or less nodes than the ones in the training set (contrarily to vanilla discrete diffusion processes). \n5. *Reproducibility*: the authors provide the code.\n\n**Weaknesses**:\n1. Empirical performance for property targeting is not remarkable for property targeting, as it only dominates existing methods (FreeGress) in 2 out of 5 different settings.  \n2. The paper does not compare/discuss to alternative graph continuous diffusion models (that model graph diffusion over continuous state-spaces), and their alternatives for varying dimensionality across diffusion process [1].  \n3. As mentioned by the authors in their limitations, this more intricate diffusion process incurs computational overhead. An objective analysis of this aspect of the contribution is of utmost importance for and correct evaluation of the contribution, e.g., to better understand the scalability of the method to larger molecular datasets.\n4 While the utility of the method for property optimization is clear, I have some concerns regarding the pertinence of the proposed method for the other experimental settings.\n\n\n**Minor weaknesses**:\n1. The formatting of Table 1 and Figure 3 should be improved. \n2. Still regarding reproducibility, it would benefit the paper to include a discussion on hyperparameter selection, particularly for the newly introduced hyperparameters. While the remaining ones follow prior work, a description of how the new parameters were chosen would strengthen the reproducibility and transparency of the experimental setup.\n\n\n[1] - Trans-Dimensional Generative Modeling via Jump Diffusion Models, Campbell et al., NeurIPS 2023\n\n[2] - Beyond In-Place Corruption: Insertion and Deletion in Denoising Probabilistic Models, Johnson et al., ICML 2021 Workshops", "was_revised": true, "final_justification": "**Final Justification**\n\nFollowing the rebuttal and discussion, several of my initial concerns have been addressed:\n\n- The discontinuity in the edge diffusion process was acknowledged, justified with precedent from prior work, and proposed as future work.  \n- The broader applicability of the method beyond property optimization was clarified and convincingly argued. A particularly important shift in my perspective came from recognizing its potential for property targeting, especially for size-dependent properties, not only in molecular domains but across general graph settings. The flexibility to dynamically adjust the number of nodes during denoising is a distinctive and valuable capability. The results in Tables 1 and 2 support this: while GrIDDD does not outperform baselines when property–size dependence is minimal, it clearly excels when this dependency is stronger. The authors were encouraged to make this connection more explicit in the final version.  \n- Additional details will be included in the final version regarding: continuous diffusion models and dimension-varying approaches, a clear explanation of the sampling time overhead, improved formatting of Table 1 and Figure 3, justification of the newly introduced hyperparameters and their selection procedure, clearer motivation for sampling from the marginal distribution of nodes and edges, and comparison with DiGress baselines that include DEL / DEL* node and edge types.  \n\nWhile the motivation for the DEL* node type is now clearer, its necessity remains debatable (see discussion in rebuttal). It adds complexity and may limit potential benefits from remasking/self-correction.  \n\n**Weighting:** The clear articulation of broader utility and the supporting results significantly improved my assessment, outweighing remaining reservations. However, the extent of the score increase is limited by the amount of revisions still needed and the questionable necessity of the DEL* node type. My overall positive view is sustained by the fact that, to date, this is the only graph diffusion model that allows varying dimension (number of nodes).", "rebuttal": "We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.\n\n- Empirical performance for property targeting is not remarkable\n\nWe strongly believe that our model specifically addresses a known limitation of all current discrete denoising diffusion models for graphs (including FREEGRESS) in a principled and elegant way. To our knowledge, this is the first method that can learn the graph size and adapt it to the generative task without resorting to estimates, heuristics, or auxiliary models. At the same time, it does so while performing on par or better than the existing baselines, meaning that its capability to reach and even surpass the same performance is actually a signal of better generalization capabilities than the former method. Moreover, our method is more widely applicable than  both FREEGRESS and DIGRESS (which lack the ability to adapt the graph size needed for optimization tasks). \n\n- The paper does not compare to graph continuous diffusion models\n\nWe thank the reviewer for the reference. From our understanding, the main difference between jump diffusion and our solution is that the former can only reduce the number of entries in the data point during the forward process (in the case of molecules, these would be the atoms), while our model can also increase them. This implies that graph continuous diffusion cannot be adapted for property optimization tasks, since it lacks the capability of deleting additional nodes during the reverse process besides the ones deleted during the forward process. Overall, jump diffusion is better suited to be applied to continuous data such as videos and, more in general, to data with positional dependencies. Nonetheless, we will certainly add the reference to the revised paper and extend the related works section to discuss the differences between graph continuous diffusion models and GrIDDD.\n\n- An analysis of the computational overhead is required\n\nWe have discussed the overall estimated overhead in the appendix, in section C.4. Clearly, it is unavoidable that a model that does more than a standard discrete diffusion will feature some sort of overhead. However, we take the opportunity to point out that GrIDDD can, under some specific conditions, be faster than the alternatives. In particular, since our model is capable of starting the sampling process from a latent featuring any number of nodes, the reverse process can be much faster than DIGRESS if the initial latent is very small. As an example, GrIDDD takes approx. 42 seconds to sample 128 molecules on ZINC250k starting from latents with 2 nodes each, and approx. 64 when starting with 24 nodes (the most frequent size in the dataset). In contrast, DIGRESS, which needs to be initialized with a node matrix of size close to 36 to accommodate for the largest graph size in the dataset, requires approx. 75 seconds to run the same sampling task.\n\n- Concerns regarding the pertinence of GrIDDD for property targeting\n\nWe believe our solution is also well suited for property targeting. Clearly, the distinctive trait of GrIDDD is that it can flexibly adapt the graph size during the forward and reverse processes. Therefore, its best usage is to target properties that directly relate to graph size. However, in drug design, several properties (such as molecular weight, polar surface, and synthetic accessibility, and, to a minor extent, solubility) significantly correlate with the number of atoms. At the same time, although GrIDDD has been applied only on molecular datasets due to the abundance of available baselines and datasets, it can target or optimize arbitrary graphs, meaning that it can be employed for a variety of tasks beyond molecular generation. \n\n- Formatting of Table 1 and Figure 3 \n\nWe thank the reviewer for spotting these mistakes. We will fix them in the revised version of the paper.\n\n- Discussion on hyperparameter selection\n\nWe thank the reviewer for the opportunity to clarify this point further. Overall, the choice of the hyperparameters was the result of a series of preliminary tests. We have chosen the values for $w$ as 0.05 and $D$ as 0.5 because they gave us some of the best results in terms of validity and number of connected components using a validation set. Setting $D$ any lower causes the model to generate many split molecules. The $\\lambda$ associated with the insertion time of the nodes does not seem to affect training much, although setting it higher than 0.5 causes the model to underperform. Same thing for $p_{min}$  and $p_{max}$  (although it seems like setting $p_{min}$  between 0.2 and 0.4 bears slightly better results). We will add this discussion in the revised version of the paper to improve clarity and reproducibility.\nWe believe this answer should resolve the reviewer’s concern; however, we remain open to provide the reviewer with more in-depth comparative results if needed.\n\n- Possible discontinuities in the edge diffusion processes. Any relevant observations?\n\nRemoving edges during the forward process is not an issue, as each node and edge is corrupted independently without considering the rest of the graph. We have no relevant observations in this merit; we remain, however, curious to read whether the reviewer has any hypothesis in this sense that we could test.\n\n- Clarification on the DEL* node type (l. 150-152)\n\nWe apologize for the lack of clarity, and we will correct the sentence in the final version of our paper. Specifically, we were referring to the reverse process. During the reverse step, in a setup with only one DEL type, a node with such a category can either switch to a \"proper\" class (the expected behavior) or remain a DEL. As a result, we defined the DEL* type, which is assigned to nodes marked for deletion. With this addition, we can guarantee that the nodes re-inserted during the reverse process will immediately switch back to a proper class, as the transition matrices do not allow for switching from DEL to DEL* during the forward process (and, hence, from DEL* to DEL while denoising, or DEL* to DEL* for that matter).\n\n- Motivations behind sampling from the marginal distribution of nodes and edges\n\nThe reviewer is correct, this is the exact motivation behind this design choice. During our preliminary studies, we tried different options for completeness, such as always inserting the least frequent node/edge class. However, using the sample’s marginal distribution consistently gave us better performance, aligning to our expectations. We will update the revised text to include this motivation to better clarify our intent.\n\n- Comparison against DIGRESS baselines with DEL/DEL* node/edge types\n\nIf we understand the reviewer’s intent correctly, we argue that it is not possible to treat the edges connected to non-existing nodes exactly as we treat the ones connected to DEL* nodes. In fact, GrIDDD removes edges altogether while the suggested variant has fixed graph size, and the edges would have to switch back and forth between DEL and “proper” types. As a result, DEL nodes could appear in the final latent $X^T$, and the associated edges would have to be set as DEL as well. This, in turn, would imply that the noisy graph $X^T$ is not sampled from an actual stationary distribution. GrIDDD solves this problem with a forward process that does not allow the presence of DEL nodes at the final corruption timestep $T$. Another minor issue would be caused by mismatches between the node and edge types (due to, for example, an edge to which non-DEL nodes are connected switching independently to DEL). A naive solution to these shortcomings would be to simply lift the constraint proposed by the reviewer everywhere but at timestep 0. By doing so, we can train the model to predict, at time step zero, that the edges associated to a DEL node have to be set to a specific category (be it a DEL or a “proper” edge class). We have run a comparison experiment to answer the reviewer’s inquiry, by implementing two baseline variants on QM9. One is essentially DIGRESS with an additional DEL atom type, while the other adds a further edge type EDEL to mark any bond whose endpoint is of type DEL.\nThe results of the comparison are shown below.\n\n- val: validity (higher=better)\n- avg. nc: average number of isolated connected components (lower=better)\n- nc max: maximum number of isolated connected components (lower=better)\n- n conn: number of valid connected graphs generated (higher=better)\n- TV: total variation between generated distribution and the dataset’s (lower=better)\n- CE: Cross-Entropy (lower=better)\n\n$$\n\\begin{array}{lcccccccc}\nModel & Val & Avg. NC & \\\\text{NC Max} & \\\\text{N Conn} & \\\\text{Val X CE} & \\\\text{Val E CE} & Atom TV & Edge TV \\\\\\\\\n\\\\hline\nGrIDDD & 0.97 & 1 & 1 & 100 & 0.44 & 0.32 & 0.016 & 0.01 \\\\\\\\\nDIGRESS+DEL & 1 & 1.56 & 8 & 69 & 0.47 & 0.37 & 0.047 & 0.10 \\\\\\\\\nDIGRESS+DEL+EDEL & 1 & 1.2 & 4 & 84 & 0.48 & 0.38 & 0.05 & 0.09 \\\\\\\n\\end{array}\n$$\n\nAs can be seen, both baselines perform poorly when compared to GrIDDD, especially in terms of connected components, indicating that the model has difficulties in generating molecules with less than nine nodes. The validity rates appear higher, but a close inspection of the generated molecules shows that they are inflated by the presence of isolated carbon atoms which were not correctly attached to the main component."}, {"review_id": "gaFvaDUEIR", "reviewer": "Reviewer_ckHS", "summary": "GRIDDD extends discrete DDPMs for graphs by introducing mechanisms for node insertion and deletion within the forward (noising) and reverse (denoising) diffusion processes. It enables step-wise monotonic node insertions and removals during diffusion, and is specifically designed to change the graph size dynamically throughout the generative process, providing the necessary flexibility to incorporate conditional information more effectively.", "questions": "Failure Analysis: For cases where GRIDDD shows slightly lower validity (e.g., QM9, μ target), what are the common types of invalid molecules generated? Understanding these failure modes could guide future improvements.", "limitations": "Sensitivity to Hyperparameters: How sensitive is GRIDDD's performance to the choice of hyperparameters, particularly pmin, pmax, D, w, and the λ weights in the loss function? Are there general guidelines for tuning these for new datasets?", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros\nAddresses a Critical Limitation: Directly tackles the inability of prior graph DDPMs to change graph size, which is vital for many real-world molecular design tasks (e.g., property-driven design where size correlates with properties).\n\nImproved Performance in Property Targeting: GRIDDD matches or exceeds the performance of state-of-the-art graph diffusion models (DiGress, FreeGress) on molecular property targeting benchmarks (QM9 and ZINC-250k) in terms of Mean Absolute Error (MAE) and chemical validity.\n\nCons\nAssumption of Mutual Independence: The reverse process factorizes probabilities assuming mutual independence of nodes and edges, which might be a simplification given the highly interconnected nature of molecules.", "was_revised": true, "final_justification": "Thank you for the detailed rebuttal and additional analyses. The clarification on the mutual independence assumption is helpful and addresses my concern.\n\nThe failure analysis for QM9 μ≈0 is insightful, and I appreciate the concrete example and explanation of how graph size adaptation may lead to higher invalid rates in rare-size molecules. This adds useful context to interpret the slightly lower validity in those cases.\n\nThe sensitivity experiments on pmin, pmax, D, w, and λ provide practical guidance for tuning on new datasets and indicate reasonable robustness in typical ranges, which is valuable for practitioners.\n\nOverall, the rebuttal addresses my questions well, improves the clarity and completeness of the paper, and does not raise new concerns. I will maintain my borderline accept rating, as the paper is technically solid with meaningful contributions, though there is still some room to further discuss practical tuning guidelines in the final version.", "rebuttal": "We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.\n\n- Assumption of Mutual Independence: The reverse process factorizes probabilities assuming mutual independence of nodes and edges, which might be a simplification given the highly interconnected nature of molecules.\n\nThe reviewer is correct in principle. However, all discrete graph diffusion models make the same assumption of mutual independence. As far as we know, there are no discrete diffusion models for graphs that do not make such assumption, as it significantly simplifies the underlying math. At the same time, we point out that the neural network tasked to compute the posterior $q(x_i^0|G^t)$ does not make use of that assumption, as it receives the entire graph in input to predict the categories of every node (and edge) in the graph. To summarize, we agree with the reviewer that this is an important limitation, although it is shared by all known discrete denoising diffusion models for graphs. We will extend the limitations section accordingly.\n\n- Failure Analysis: For cases where GRIDDD shows slightly lower validity (e.g., QM9,  $\\mu$ target), what are the common types of invalid molecules generated? Understanding these failure modes could guide future improvements.\n\nOne very interesting insight we have gathered is that, unlike FreeGress, the experiments targeting $\\mu$ tend to fail most frequently when targeting μs close to zero. The few molecules generated tend to be small, split graphs. We have investigated the phenomenon and found out that the molecule with SMILES string \"CC\" (ethane) is known to have a dipole moment equal to zero. We hypothesize that GrIDDD tries to generate ethane or similar small molecules to minimize the dipole moment, with higher failure rates than usual since these small molecules are under-represented in the data. Remarkably, DiGress and FreeGress are unable to pursue this minimization, since they almost inevitably start the reverse process from latents with a larger graph size and cannot adapt it to produce smaller molecules later on. Overall, this is an interesting behaviour which shows that GrIDDD successfully can learn the properties of molecules with infrequent sizes as well. We will extend the revised version to include a discussion of a failure analysis. \n\n- Sensitivity to Hyperparameters: How sensitive is GRIDDD's performance to the choice of hyperparameters, particularly $p_{min}$, $p_{max}$, $D$, $w$, and the $\\lambda$ weights in the loss function? Are there general guidelines for tuning these for new datasets?\n\nWe have performed some experiments, tweaking the values of $p_{min}$ (we observed how $p_{max}$ can be kept equal to one, and that editing $p_{min}$ is enough), $D$, $w$, and the $\\lambda$ associated with the insertion time's loss. We have trained several GrIDDD variants on QM9 while monitoring the behaviour of the training and validation losses, sampling  100 molecules from the trained models. The results are summarized in the table below. Overall, we observed that setting $D$ too small (which corresponds to starting the operations of insertion and deletion too early in the forward process) causes the model to generate higher rates of split molecules, since the model has little time to connect them back into the main graph while denoising. For values of $D$ higher than 0.5, we did not observe meaningful differences. We observed a similar behavior when setting large $w$ values, which was expected since this makes the \"slope\" of the $\\zeta$ function less steep and, as a result, allows late reinsertions during the reverse process. Finally, we find the model to be relatively robust to different values of $p_{min}$, as well as the $\\lambda$ associated with the insertion time loss (unless it is too high).\n\n- val: validity (higher=better)\n- avg. nc: average number of isolated connected components (lower=better)\n- nc max: maximum number of isolated connected components (lower=better)\n- n conn: number of valid connected graphs generated (higher=better)\n\n$$\n\\begin{array}{ccccc}\nD & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.25 & 0.93 & 1.15 & 4 & 88 \\\\\\\\\n0.50 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.75 & 0.95 & 1.01 & 2 & 99 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\nw & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.025 & 0.94 & 1 & 1 & 100 \\\\\\\\\n0.050 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.075 & 0.92 & 1.03 & 2 & 97 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\np_{min} & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.2 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.4 & 0.91 & 1 & 1 & 100 \\\\\\\\\n0.6 & 0.96 & 1.02 & 3 & 99 \\\\\\\\\n0.8 & 0.94 & 1.02 & 2 & 98 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\nlamb & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.5 & 0.86 & 1.02 & 2 & 98 \\\\\\\\\n0.1 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.01 & 0.89 & 1.01 & 2 & 99 \\\\\\\\\n0.001 & 0.94 & 1.02 & 2 & 98 \\\\\\\\\n\\end{array}\n$$"}, {"review_id": "yboxpvbXV5", "reviewer": "Reviewer_kZuT", "summary": "The paper introduces GRIDDD, a novel discrete Denoising Diffusion Probabilistic Model (DDPM) for graph generation, which supports size-adaptive graph generation through monotonic insertions and deletions of nodes (atoms). Existing graph diffusion models have a fixed graph size throughout the generation process, limiting their application in tasks like property targeting and optimization, where graph size (number of atoms) is crucial. GRIDDD addresses this limitation by dynamically adjusting graph size during generation. It is evaluated on two tasks: property targeting (e.g., molecular weight, dipole moment) and property optimization (improving existing molecules). GRIDDD demonstrates superior performance on benchmark datasets (QM9 and ZINC-250k), particularly excelling in property targeting and optimization tasks.", "questions": "1. **Split Molecules Issue:** Could you elaborate on the causes behind split molecule generation? Have you considered additional constraints to mitigate this?\n\n2. **Computational Overhead:** Have you explored optimization techniques (e.g., sparse updates, gradient checkpointing) to address computational overhead?\n\n3. **Larger Graphs Generalization:** How might GRIDDD be adapted to effectively handle larger molecules beyond current limitations?\n\n4. **Impact of Hyperparameters:** Could you provide sensitivity analyses regarding hyperparameters such as noise schedules and insertion/deletion probability distributions?", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. **Originality:** GRIDDD introduces a novel extension to DDPMs for graph generation by allowing node insertion and deletion during diffusion, significantly advancing beyond fixed-size models and enabling practical applications in molecular property targeting.\n\n2. **Technical Contribution:** GRIDDD’s dynamic graph size adjustment during generation provides a substantial improvement in molecular generation tasks, effectively maintaining valid chemical structures throughout the process.\n\n3. **Empirical Results:** The comprehensive experiments demonstrate GRIDDD’s superior performance on widely-used datasets (QM9 and ZINC-250k), achieving notable results in both property targeting and optimization, especially in enhancing LogP and QED scores.\n\n4. **Clarity of Methodology:** The detailed technical explanations clearly illustrate how node insertions and deletions integrate into the diffusion process, including transition matrices and insertion-time predictions.\n\nWeaknesses:\n\n1. **Computational Overhead:** GRIDDD's insertion and deletion operations introduce significant computational overhead. The paper lacks concrete optimizations (e.g., sparse graph updates, mixed-precision training) or quantification of additional computational costs (training time, memory usage, FLOPs).\n\n2. **Split Molecules:** GRIDDD occasionally generates more split molecules due to late-stage node insertions. Further exploration to address this issue through additional constraints or algorithmic modifications is needed.\n\n3. **Limited Generalization to Larger Graphs:** GRIDDD’s effectiveness decreases significantly when generating molecules larger than its training distribution. Enhanced methods for improving generalization capabilities to larger graph sizes would be valuable.", "was_revised": true, "final_justification": "Based on the feedback, the three weaknesses I mentioned are indeed esixt and cannot be fully addressed at this stage. Therefore, although I consider this a good paper overall, I do not think it is appropriate to raise my score from weak accept to accept.", "rebuttal": "We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.\n\n- Computational Overhead: GRIDDD's insertion and deletion operations introduce significant computational overhead. The paper lacks concrete optimizations (e.g., sparse graph updates, mixed-precision training) or quantification of additional computational costs (training time, memory usage, FLOPs). Have you explored optimization techniques (e.g., sparse updates, gradient checkpointing) to address computational overhead?\n\nWe have discussed the model's training time in the appendix (Section C.4), where we estimate that GrIDDD to be approx. 33% slower to train with respect to DiGress, denoting a typical tradeoff between expressivity/flexibility and computational overhead. Interestingly enough, we also observed that GrIDDD can, under some specific conditions, be more computationally efficient to sample than the alternatives. In particular, since our model is capable of starting the sampling process from a latent featuring any number of nodes, the reverse process can be much faster than DIGRESS if the initial latent is very small. As an example, GrIDDD takes approx. 42 seconds to sample 128 molecules on ZINC250k starting from latents with 2 nodes each, and approx. 64 when starting with 24 nodes (the most frequent size in the dataset). In contrast, DIGRESS, which needs to be initialized with a node matrix of size close to 36 to accommodate for the largest graph size in the dataset, requires approx. 75 seconds to run the same sampling task.\nAs regards the number of parameters, GrIDDD uses a slightly larger but still comparable number with respect to FREEGRESS and DIGRESS (5M vs. 4.6M on QM9, 15.3 vs. 13.5 on ZINC250k). Overall, the tradeoff could be considered positive, since GrIDDD is more flexible than FREGRESS and DIGRESS and can be used for molecular optimization tasks. We commit to complement Section C.4 of the revised paper with these numbers and observations.\nAs suggested by the reviewer, the slightly superior computational burden of GrIDDD could be mitigated by using appropriate optimization techniques. These would ultimately affirm GrIDDD as an comparable alternative to DIGRESS and FREEGRESS even in lower resource scenarios. However, we did not specifically employ any at this stage of development.\n\n- Split Molecules: GRIDDD occasionally generates more split molecules due to late-stage node insertions. Further exploration to address this issue through additional constraints or algorithmic modifications is needed. Could you elaborate on the causes behind split molecule generation? Have you considered additional constraints to mitigate this?\n\nEarlier in the development of GrIDDD, we analyzed where, in the reverse process, split molecules appeared, plotting the number of occurrences as a function of the denoising timestep. Overall, the statistics perfectly matched the shape of the $\\zeta’(t)$ function which modulates the deletion schedule. As a result, we conjecture that split molecules are mostly caused by nodes inserted too late during the reverse process, which cannot be re-attached in time to the main graph, as the noise scheduler does not allow for too many changes in the graph’s structure during the last phases of the process.\nAs regards implementing additional constraints, during our preliminary tests, we have found that GrIDDD generates less split molecules when given, as extra features, different powers of the adjacency matrix. This can be interpreted as a sufficiently accurate approximation of the number of connected components in the graph (as opposed to the computationally demanding task of computing the eigenvalues of the adjacency matrix's Laplacian). However, we have ultimately decided to proceed without this feature to ensure a fair comparison with the baselines. We will clarify these additional aspects regarding the issue of split molecules in the revised version of the paper.\n\n- Limited Generalization to Larger Graphs: GRIDDD’s effectiveness decreases significantly when generating molecules larger than its training distribution. Enhanced methods for improving generalization capabilities to larger graph sizes would be valuable. How might GRIDDD be adapted to effectively handle larger molecules beyond current limitations?\n\nFirst of all, we would like to state that according to our results, GrIDDD already takes a convincing step forward towards a more reliable out-of-distribution sampling with respect to existing approaches, as shown in Section 5.2 of the paper (which shows an improved capacity of generating valid molecules out-of-distribution with respect to the baselines). Clearly, we acknowledge that the problem is not completely solved and more research is needed, as generative models are trained to imitate the dataset distribution and not something they did not see during training. One possible direction to take in this regard is to parameterize both insertions and deletions (i.e., letting the neural network decide which kind of node to add, rather than relying on sampling from the marginal distribution of the nodetypes). In this sense, GrIDDD presents excellent opportunities to be extended and improved, and we plan to address the out-of-distribution sampling problem with innovative approaches in future studies. We will improve the conclusions section with this discussion.\n\n- Impact of Hyperparameters: Could you provide sensitivity analyses regarding hyperparameters such as noise schedules and insertion/deletion probability distributions?\n\nWe have performed some experiments, tweaking the values of $p_{min}$ (we observed how $p_{max}$ can be kept equal to one, and that editing $p_{min}$ is enough), $D$, $w$, and the $\\lambda$ associated with the insertion time's loss. We have trained several GrIDDD variants on QM9 while monitoring the behaviour of the training and validation losses, sampling 100 molecules from the trained models. The results are summarized in the table below. Overall, we observed that setting $D$ too small (which corresponds to starting the operations of insertion and deletion too early in the forward process) causes the model to generate higher rates of split molecules, since the model has little time to connect them back into the main graph while denoising. For values of $D$ higher than 0.5, we did not observe meaningful differences. We observed a similar behavior when setting large $w$ values, which was expected since this makes the \"slope\" of the $\\zeta$ function less steep and, as a result, allows late reinsertions during the reverse process. Finally, we find the model to be relatively robust to different values of $p_{min}$, as well as the $\\lambda$ associated with the insertion time loss (unless it is too high).\n\n- val: validity (higher=better)\n- avg. nc: average number of isolated connected components (lower=better)\n- nc max: maximum number of isolated connected components (lower=better)\n- n conn: number of valid connected graphs generated (higher=better)\n\n$$\n\\begin{array}{ccccc}\nD & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.25 & 0.93 & 1.15 & 4 & 88 \\\\\\\\\n0.50 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.75 & 0.95 & 1.01 & 2 & 99 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\nw & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.025 & 0.94 & 1 & 1 & 100 \\\\\\\\\n0.050 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.075 & 0.92 & 1.03 & 2 & 97 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\np_{min} & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.2 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.4 & 0.91 & 1 & 1 & 100 \\\\\\\\\n0.6 & 0.96 & 1.02 & 3 & 99 \\\\\\\\\n0.8 & 0.94 & 1.02 & 2 & 98 \\\\\\\\\n\\end{array}\n$$\n\n$$\n\\begin{array}{ccccc}\nlamb & val & \\text{avg. nc} & \\text{nc max} & \\text{n conn} \\\\\\\\\n\\\\hline\n0.5 & 0.86 & 1.02 & 2 & 98 \\\\\\\\\n0.1 & 0.93 & 1.02 & 2 & 98 \\\\\\\\\n0.01 & 0.89 & 1.01 & 2 & 99 \\\\\\\\\n0.001 & 0.94 & 1.02 & 2 & 98 \\\\\\\\\n\\end{array}\n$$"}]

Author rebuttal:

Meta-review: This paper provides an important advance in graph diffusion models with a focus on molecule generation. In particular, the authors provide an approach that allows the number of nodes in the graph/molecule to vary, whereas previous work required the number of nodes to be predetermined. This is an extremely important property for many applications in molecule generation. 

While some of the reviewers were a little dissatisfied with some of the specifics of the approach, the authors' method does work well at what it was designed to do with no current competitors. Based on my reading of the reviews and the responses, I believe that the authors addressed the reviewers' concerns well.
Decision: Accept (poster)