Lift Your Molecules: Molecular Graph Generation in Latent Euclidean Space

## Abstract

Abstract We introduce a new framework for molecular graph generation with 3D molecular generative models. Our Synthetic Coordinate Embedding ( SyCo ) framework maps molecular graphs to Euclidean point clouds via synthetic conformer coordinates and learns the inverse map using an E( n n )-Equivariant Graph Neural Network (EGNN). The induced point cloud-structured latent space is well-suited to apply existing 3D molecular generative models. This approach simplifies the graph generation problem – without relying on molecular fragments nor autoregressive decoding – into a point cloud generation problem followed by node and edge classification tasks. Further, we propose a novel similarity-constrained optimization scheme for 3D diffusion models based on inpainting and guidance. As a concrete implementation of our framework, we develop EDM-SyCo based on the E(3) Equivariant Diffusion Model (EDM). EDM-SyCo achieves state-of-the-art performance in distribution learning of molecular graphs, outperforming the best non-autoregressive methods by more than 30% on ZINC250K and 16% on the large-scale GuacaMol dataset while improving conditional generation by up to 3.9 times.

## 1 Introduction

Machine learning-based drug discovery has recently shown great potential to accelerate the drug development process while reducing the costs associated with clinical trials (Jiménez-Luna et al., ; Kim et al., ). Among the steps of this process, generating novel molecules with relevant properties such as drug-likeness and synthesizability, or optimizing existing ones are of crucial importance. Molecular graph generation is an active area of research that aims to solve these tasks (Gómez-Bombarelli et al., ; Stokes et al., ; Bilodeau et al., ).

Molecular graph generation is challenging due to the discrete and sparse nature of graphs as well as to the presence of symmetric substructures such as rings. Many methods have been proposed to tackle this problem, offering different tradeoffs (Jin et al., ; Shi et al., ). A common approach within these methods is to train a Variational Autoencoder (VAE) to encode the graph into a continuous fixed-size latent space, then decode it back – oftentimes autoregressively – node-by-node (Liu et al., ) or fragment-by-fragment (Jin et al., , ; Maziarz et al., ; Wollschläger et al., ). While such methods achieve good generation and optimization performance, they have inherent limitations.
First, using a fixed-size latent space does not account for the variable size of molecular graphs, which can go from a few atoms for small molecules to thousands for macromolecules.
Second, autoregressive decoding requires choosing a concrete generation order (Schneider et al., ), which prevents learning permutation-invariant graph distributions. In practice, Maziarz et al. () found that different tasks, like generation from scratch or starting from a scaffold, require training models with different generation orders to achieve optimal performance.
Fueled by recent developments in diffusion models (Ho et al., ; Austin et al., ), new approaches to molecular graph generation have emerged that overcome these limitations (Jo et al., ). Notably, DiGress (Vignac et al., ) defines a discrete diffusion process directly on the node and edge attributes and learns to reverse this process to generate graphs all at once, i.e., not autoregressively. However, it struggles to accurately estimate the joint distribution of nodes and edges, and discrete optimization remains a challenge.

In this work, we develop a method that combines the benefits of all-at-once generation with the benefits of a continuous latent space for optimization tasks while avoiding the information bottleneck of fixed-size latent spaces.
Inspired by recent developments in the adjacent field of 3D molecule generation, we propose to embed molecular graphs as latent 3D point clouds that implicitly encode the discrete graph structure in synthetic Euclidean coordinates and use point cloud generative models to learn their distribution. We call this framework Synthetic Coordinate Embedding (SyCo).

When combined with EDM  (Hoogeboom et al., ), our framework gives rise to an atom-based, all-at-once, and permutation-invariant generative model for molecular graphs, which we call EDM-SyCo, depicted in Figure 1. Compared to DiGress, EDM-SyCo improves upon its distribution learning performance by 30% and its conditional generation performance by up to 15.6 times on ZINC250K, highlighting the benefit of our continuous latent space on generation and optimization.

In addition, we propose a new similarity-constrained optimization procedure for diffusion models on point clouds without retraining or specialized architectures, visualized in Figure 3. Notably, our proposed algorithm enables the addition of new atoms during the reverse diffusion process, addressing a well-known limitation of these models, namely the fixed number of nodes.

Figure: Figure 1: Overview of EDM-SyCo. *(Training)* First, the autoencoder is trained to map between molecular graphs and latent Euclidean point clouds. Then, the diffusion model is trained on the fixed latent space. *(Sampling)* Starting with a Gaussian sample, the diffusion model denoises it for $T$ steps to predict the clean point cloud, which is mapped to a molecular graph using the decoder.
Refer to caption: x1.png

## 2 Related Work

#### Molecular Graph Generation

Early work on molecule generation (Gómez-Bombarelli et al., ; Segler et al., ) used language models to generate SMILES (Weininger, ), a text-based representation of molecules.
However, this type of representation cannot capture the inherent structure of molecules well, which are more accurately depicted as graphs. For instance, small changes in the graph structure may correspond to large differences in the SMILES string (Jin et al., ).
Therefore, recent methods have focused on graph-based approaches that can be broadly classified into two categories. All-at-once generation methods generate nodes and edges in parallel, though possibly in a multi-step process such as diffusion models (Niu et al., ; Jo et al., ; Vignac et al., ). Autoregressive methods specify a generation order and generate molecules either atom-by-atom (Li et al., ; Shi et al., ) or fragment-by-fragment based on a pre-computed fragment vocabulary (Jin et al., , ; Maziarz et al., ; Kong et al., ; Geng et al., ). EDM-SyCo is an all-at-once molecular graph generative model that operates on the atom level.

#### Molecule Generation in 3D

Another line of work aims to generate molecules in 3D Euclidean space and solve a point cloud generation problem (Gebauer et al., ; Luo & Ji, ; Garcia Satorras et al., ).
For instance, EDM (Hoogeboom et al., ) and GeoLDM (Xu et al., ) are diffusion-based approaches for 3D molecule generation that jointly generate atomic features and coordinates.
While tangentially related, these methods solve a different problem from molecular graph generation and require molecules with 3D information for training.
Our proposed SyCo framework is an attempt to connect these two lines of work by enabling the training of 3D generative models on graph datasets through synthetic coordinates, formally introducing new graph generation methods.
While we are the first to explore synthetic coordinates for generative models, Gasteiger et al. () demonstrated that synthetic coordinates improve molecular property prediction tasks.

#### Latent Generative Models

The idea of defining the generative model on a space different from the original data space is reminiscent of latent generative models (Dai & Wipf, ; Vahdat et al., ).
Recently, this approach has become increasingly popular with latent diffusion models. Stable Diffusion models (Rombach et al., ) achieve impressive results on text-guided image generation, and GeoLDM (Xu et al., ) extends this to molecular point cloud generative models.
While EDM-SyCo operates on a similar latent space to GeoLDM, their fundamental difference lies in the original data space. Our model maps discrete molecular graphs to continuous point clouds and can be seen as a cross-modality latent generative model. In contrast, GeoLDM embeds point clouds as new point clouds with a reduced feature dimension. Further, as explained in the previous paragraph, GeoLDM is a 3D molecule generative model and is not directly applicable to generating graphs.

## 3 Synthetic Coordinate Embedding ( SyCo ) Framework

Our goal is to model molecular graph distributions through distributions of Euclidean point clouds that implicitly encode the graph structure into their coordinates. For this, we propose a novel autoencoder architecture that maps between molecular graphs and 3D point cloud representations.
By training it using a reconstruction objective, we can reformulate the problem of generating discrete molecular graphs to an equivalent problem of generating 3D point clouds defined on the induced latent space.

#### Note on equivariance

When dealing with physical objects like molecules, it is important to consider equivariance. Formally, a function $f:A\to B$ is equivariant to the action of a group $G$ iff $f(T^{A}_{g}({\bm{z}}))=T^{B}_{g}(f({\bm{z}}))$ for all $g\in G,$ where $T^{A}_{g}$ and $T^{B}_{g}$ are the representations of the group element $g$ in $A$ and $B$ respectively (Serre et al., ). As a special case, it is invariant iff $f(T^{A}_{g}({\bm{z}}))=f({\bm{z}})$ with $T^{B}_{g}$ being the identity map.
In the context of generative modeling, we aim to learn distributions invariant to the data’s symmetries, namely graph permutations in our case. In practice, this has the benefit of not relying on any generation order and enabling efficient likelihood estimation since an invariant likelihood is identical for all permutations of the same graph.
We leverage equivariant architectures such as EGNNs to learn such distributions, removing the need for data augmentation during training, thus further increasing efficiency.
EGNNs  (Satorras et al., ) are a special type of graph neural networks that learn functions equivariant to rotations, translations, reflections, and permutations of their input point cloud. More details on this architecture are given in Appendix D.

#### Notation

We introduce helpful notation for the two molecule representations used in this work. Let $N$ denote the number of atoms of a given molecule.
On the one hand, the molecular graph ${\mathcal{G}}=({{\bm{h}}},{{\bm{A}}})\in{\mathbb{G}}$ consists of atoms as nodes and chemical bonds as edges. Each atom has one of $a$ atom types and one of $c$ formal charges, while each bond has one of $b$ bond types.
We represent atoms as stacked one-hot encodings ${{\bm{h}}}\in\{0,1\}^{N\times(a+c)}$ and chemical bonds as one-hot vectors in an adjacency matrix ${{\bm{A}}}\in\{0,1\}^{N\times N\times b}$.
Note that the molecular graph is sometimes called a "2D molecule" as it can be typically drawn as a planar graph.
On the other hand, the molecular point cloud ${\mathcal{P}}=({{\bm{h}}},{{\bm{x}}})\in{\mathbb{P}}$ is a 3D point cloud, where ${{\bm{x}}}\in{\mathbb{R}}^{N\times 3}$ represents the coordinates of atoms’ nuclei in Euclidean space. Note that the point cloud encodes the bond information implicitly in the atomic coordinates.
We now present our autoencoder model, whose overview is shown in Figure 2.

#### Encoding

The encoder ${\mathcal{E}}_{\phi}:{\mathbb{G}}\to\mathbb{R}^{N\times(d+3)}$ with parameters $\phi$ maps a molecule’s graph representation ${\mathcal{G}}=({{\bm{h}}},{{\bm{A}}})$ to its point cloud representation ${\mathcal{P}}=({{\bm{h}}},{{\bm{x}}})$, then further compresses it into a node-structured latent representation ${\bm{z}}=({\bm{z}}^{(h)},{\bm{z}}^{(x)})$, where ${\bm{z}}^{(h)}\in{\mathbb{R}}^{N\times d}$ and ${\bm{z}}^{(x)}\in{\mathbb{R}}^{N\times 3}$ are continuous embeddings for ${{\bm{h}}}$ and ${{\bm{x}}}$ with $d<a+c$ and ${\bm{z}}\in{\mathbb{R}}^{N\times{(d+3)}}$ is their concatenation.

In the first encoding step, the coordinates ${{\bm{x}}}$ are computed using the ETKDG algorithm (Riniker & Landrum, ) available in RDKit (Landrum et al., ), which uses distance geometry and torsion angle preferences to generate 3D conformations of a molecular graph. These coordinates encode the bond information of the molecular graph and ideally define an injective mapping from ${\mathbb{G}}$ to ${\mathbb{P}}$.
Note that this step is not learned and, thus, not required to be differentiable. In practice, we generate conformers for the whole training dataset in a preprocessing step and reuse them throughout training.

Figure: Figure 2: Autoencoder architecture. The encoder maps a molecular graph to a point cloud, and the decoder learns the inverse. Both are trained jointly to minimize the reconstruction loss.
Refer to caption: x2.png

Then, we apply an EGNN that maps the intermediate representation ${\mathcal{P}}=({{\bm{h}}},{{\bm{x}}})$ to the latent representation ${\bm{z}}=({\bm{z}}^{(h)},{\bm{z}}^{(x)})$, defining the space of the generative model.
The main benefit of this is to incorporate the neighborhood information into each atom’s embeddings and coordinates while mapping the discrete features ${{\bm{h}}}$ into a continuous and lower-dimensional embedding ${\bm{z}}^{(h)}$.

#### Decoding

The decoder $\mathcal{D}_{\xi}:\mathbb{R}^{N\times(d+3)}\to{\mathbb{G}}$ with parameters $\xi$ aims to approximate the inverse function ${\mathcal{E}}_{\phi}^{-1}$ and maps the latent point cloud ${\bm{z}}$ of a molecule back to its graph representation ${\mathcal{G}}$. It consists of an atom and a bond classifier. For each node $i$, the decoder applies a 2-layer Multi-Layer Perceptron (MLP) on its latent embedding to predict the logits for the atom type and the formal charge as $\hat{{h}}_{i}=\text{MLP}_{node}({z}^{(h)}_{i}).$
For each pair of nodes $i$ and $j$, the bond type is inferred by running a different 2-layer MLP on their edge features to obtain the corresponding logits as follows:

$$ $\hat{{A}}_{ij}=\frac{1}{2}\left(\text{MLP}_{edge}\left(\begin{bmatrix}{z}^{(h)}_{i}\\ {z}^{(h)}_{j}\\ d_{ij}^{2}\end{bmatrix}\right)+\text{MLP}_{edge}\left(\begin{bmatrix}{z}^{(h)}_{j}\\ {z}^{(h)}_{i}\\ d_{ij}^{2}\end{bmatrix}\right)\right),$ $$

where $d_{ij}=\|{z}^{(x)}_{i}-{z}^{(x)}_{j}\|_{2}$ and we use the simple averaging trick to ensure that $\hat{{{\bm{A}}}}$ is symmetric.

#### Autoencoder Training

The encoder and decoder are jointly optimized in a variational autoencoder (VAE) framework.
We define probabilistic encoding and decoding processes as $q_{\phi}({\bm{z}}|{\mathcal{G}})={\mathcal{N}}({\mathcal{E}}_{\phi}({\mathcal{G}}),\sigma_{0}^{2}{\bm{I}})$ and $p_{\xi}({\mathcal{G}}|{\bm{z}})=\prod_{i=1}^{N}p_{\xi}({h}_{i}|{\bm{z}})\prod_{j=1}^{N}p_{\xi}({A}_{ij}|{\bm{z}})$, respectively, where $\sigma_{0}\in{\mathbb{R}}$ controls the variance in the latent space. The VAE is trained by minimizing the loss function

$$ $\displaystyle\mathcal{L}_{VAE}=-\mathbb{E}_{q({\mathcal{G}})q_{\phi}({\bm{z}}|{\mathcal{G}})}\left[p_{\xi}({\mathcal{G}}|{\bm{z}})\right],$ (1) $$

which, in practice, is computed as the cross-entropy loss. The decoder performs three classification tasks to reconstruct a molecular graph: atom types, formal charges, and bond types. The total loss is the unweighted sum of the corresponding three cross-entropy terms.

**Table 1: Graph reconstruction accuracy on ZINC250K.**
| Method | Accuracy |
| --- | --- |
| EGNN + MLP | 99.93% |
| MLP | 11.77% |
| Rule-based | 11.78% |

Before continuing with our approach, we present an ablation study on our autoencoder architecture. We fix the conformer generation part and replace the encoder’s EGNN and the decoder with (i) only the decoder and (ii) a rule-based system akin to Hoogeboom et al. (). More details on the experimental setup are in Appendix G.1. Results on ZINC250K are shown in Table 1, and similar results are observed for GuacaMol.
Our autoencoder model achieves an almost perfect reconstruction accuracy, highlighting the importance of the EGNN in our architecture.
This experiment supports our assumptions (i) that the used mapping from ${\mathbb{G}}$ to ${\mathbb{P}}$ is injective and (ii) that the decoder can accurately approximate its inverse.

## 4 EDM-SyCo

Our SyCo framework lifts the graph generation problem to a point cloud generation problem in 3D.
Concretely, we are interested in learning the distribution of latent point clouds ${\bm{z}}\sim q_{\phi}({\bm{z}}|{\mathcal{G}})q({\mathcal{G}})$ defined by the underlying molecular graph distribution and the trained encoder by approximating it with a parametric distribution $p_{\theta}$.
While this generation problem can, in principle, be approached by any 3D molecular generative method, this work adapts EDM under the SyCo framework due to its simplicity and empirical performance in 3D molecule generation tasks (Hoogeboom et al., ) .

### 4.1 Equivariant Diffusion on Latent Point Clouds

A diffusion model generates samples by reversing a diffusion process, the process of adding noise to data (Sohl-Dickstein et al., ; Ho et al., ).
For ease of notation, we flatten the latent representation of the point clouds ${\bm{z}}$, namely their latent coordinates and features, into a one-dimensional vector denoted as ${\bm{z}}_{0}\in{\mathbb{R}}^{d^{\prime}}$ with $d^{\prime}=N(d+3)$.
The diffusion or forward process is a fixed Markov chain of Gaussian updates that transform ${\bm{z}}_{0}$ into latent variables ${\bm{z}}_{1},\dots,{\bm{z}}_{T}$ of the same dimension as ${\bm{z}}_{0}$, defined as

$$ $q({\bm{z}}_{1:T}|{\bm{z}}_{0})\coloneqq\prod_{t=1}^{T}q({\bm{z}}_{t}|{\bm{z}}_{t-1}),\text{ with }q({\bm{z}}_{t}|{\bm{z}}_{t-1})\coloneqq{\mathcal{N}}({\bm{z}}_{t};\sqrt{1-\beta_{t}}{\bm{z}}_{t-1},\beta_{t}{\bm{I}}),$ (2) $$

where $\beta_{t}$ is typically chosen such that $q({\bm{z}}_{T})$ converges to ${\mathcal{N}}({\bm{z}}_{T};{\bm{0}},{\bm{I}})$.
By defining $\alpha_{t}\coloneqq 1-\beta_{t}$ and $\bar{\alpha}_{t}\coloneqq\prod_{i=1}^{t}\alpha_{i}$, we can also map ${\bm{z}}_{0}$ directly to ${\bm{z}}_{t}$ through

$$ $\displaystyle q({\bm{z}}_{t}|{\bm{z}}_{0})={\mathcal{N}}({\bm{z}}_{t};\sqrt{\bar{\alpha}_{t}}{\bm{z}}_{0},(1-\bar{\alpha}_{t}){\bm{I}}).$ (3) $$

To reverse the above process, we start from a Gaussian noise sample ${\bm{z}}_{T}$ and iteratively sample from the posterior distributions $q({\bm{z}}_{t-1}|{\bm{z}}_{t})$, which is also Gaussian if $\beta_{t}$ is small (Sohl-Dickstein et al., ). However, estimating them requires using the entire dataset. Therefore, diffusion models learn a reverse process $p_{\theta}({\bm{z}}_{0:T})\coloneqq p({\bm{z}}_{T})\prod_{t=1}^{T}p_{\theta}({\bm{z}}_{t-1}|{\bm{z}}_{t})$, with $p({\bm{z}}_{T})={\mathcal{N}}({\bm{z}}_{T};{\bm{0}},{\bm{I}})$ and

$$ $p_{\theta}({\bm{z}}_{t-1}|{\bm{z}}_{t})\coloneqq{\mathcal{N}}({\bm{z}}_{t-1};\bm{\mu}_{\theta}({\bm{z}}_{t},t),\sigma_{t}^{2}{\bm{I}}),$ (4) $$

where the mean $\bm{\mu}_{\theta}({\bm{z}}_{t},t)\in\mathbb{R}^{d^{\prime}}$ is parametrized by a neural network.

We follow Ho et al. () and, instead of directly predicting the mean, predict the Gaussian noise added to the latent point cloud representation during the diffusion process. The mean then becomes $\bm{\mu}_{\theta}({\bm{z}}_{t},t)=\frac{1}{\sqrt{\alpha_{t}}}\left({\bm{z}}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)\right)$, where $\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)\in\mathbb{R}^{d^{\prime}}$ is the output of the neural network. Following Hoogeboom et al. (), we parametrize $\bm{\epsilon}_{\theta}$ via an EGNN (Satorras et al., ). Additionally, the reverse process ensures equivariance to translations by defining the learned distribution $p_{\theta}({\bm{z}}_{0})$ on the zero center of gravity subspace.

This network can then be trained by optimizing the $L_{2}$ loss between the sampled Gaussian noise at each step and the network’s output,

$$ $\displaystyle\mathcal{L}_{DM}=\mathbb{E}_{{\bm{z}}_{0},\bm{\epsilon},t}\left[\|\bm{\epsilon}-\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)\|_{2}^{2}\right].$ (5) $$

During sampling, we start from a Gaussian sample ${\bm{z}}_{T}\sim p({\bm{z}}_{T})$ and iteratively denoise it by sampling $\bm{\epsilon}_{t}\sim{\mathcal{N}}({\bm{0}},{\bm{I}})$ and transforming $z_{t}$ to $z_{t-1}$:

$$ ${\bm{z}}_{t-1}=\underbrace{\frac{1}{\sqrt{\alpha_{t}}}({\bm{z}}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bm{z}}_{t},t))}_{\bm{\mu}_{\theta}({\bm{z}}_{t},t)}+\underbrace{\frac{\sqrt{1-\bar{\alpha}_{t-1}}}{\sqrt{1-\bar{\alpha}_{t}}}\sqrt{\beta_{t}}}_{\sigma_{t}}\bm{\epsilon}_{t}.$ (6) $$

### 4.2 Training and Sampling from EDM-SyCo

After training the autoencoder as described in Section 3, EDM is trained in a second stage by optimizing the loss function defined in Equation 5. We describe the full training procedure of both parts in Algorithm 1.
Consistent with previous work on latent diffusion models (Rombach et al., ; Xu et al., ), we found this two-stage training procedure to perform better than jointly training the autoencoder and the diffusion model.

For sampling, we need access to the trained EDM and the decoder but not the encoder. First, starting with a sample ${\bm{z}}_{T}\sim{\mathcal{N}}({\bm{0}},{\bm{I}})$ representing a random 3D point cloud, we iteratively denoise it with the diffusion model (Equation 6) to obtain the clean sample ${\bm{z}}_{0}$. Then, we apply the decoder to get the corresponding molecular graph ${\bm{h}},{\bm{A}}=\mathcal{D}_{\xi}({\bm{z}}_{0})$. Algorithm 2 formally describes this procedure, and Figure 1 further illustrates it.
This sampling procedure defines a permutation-invariant molecular graph distribution.
The following proposition, which we prove in Appendix C.2, formalizes this:

###### Proposition 4.1 .

The marginal distribution of molecular graphs $p_{\theta,\xi}({\mathcal{G}})=\mathbb{E}_{p_{\theta}({\bm{z}}_{0})}\left[p_{\xi}({\mathcal{G}}|{\bm{z}}_{0})\right]$ defined by the EDM and the decoder, is an $S_{N}$-invariant distribution, i.e. for any molecular graph ${\mathcal{G}}=({\bm{h}},{\bm{A}})$, $p_{\theta,\xi}({\bm{h}},{\bm{A}})=p_{\theta,\xi}(P{\bm{h}},P{\bm{A}}P)$ for any permutation matrix $P\in S_{N}$.

So far, we have assumed that the number of atoms $N$ is fixed across all molecules. In practice, to generate molecules with different sizes, we follow previous work (Hoogeboom et al., ) and estimate the empirical distribution of the number of atoms in the training set $p(N)$ and use it to sample different numbers of atoms before running the sampling procedure described above.
In Appendix F.3, we describe the use of conditional distributions to sample $N$ in a constrained generation or optimization.

###### Proposition 4.1 .

## 5 Controllable Generation and Similarity-constrained Optimization

So far, we have introduced our approach to distribution learning on molecular graphs.
However, many tasks in drug discovery require generating molecules with specific conditions such as chemical properties, similarity constraints, or the presence of certain desirable substructures. In this section, we describe how we can adapt the sampling procedure of EDM-SyCo (or other models combined with SyCo) – trained in an unconditional setting – to different conditioning modalities.
While previous works on 3D molecule generative models approached conditional generation (Hoogeboom et al., ; Bao et al., ) and scaffolding (Schneuing et al., ), the common graph generation task of similarity-constrained optimization (Jin et al., ) was absent. We fill this gap by introducing a novel sampling algorithm for diffusion models to perform similarity-constrained optimization.

#### Property-Conditional Generation via Diffusion Guidance

In a conditional generation setting, we want to generate a molecule with a specific property $c$ by sampling from the conditional distribution $q({\bm{z}}|c)$. To achieve this, we adapt the classifier guidance algorithm (Dhariwal & Nichol, ), initially proposed to guide the generation of an image diffusion model using a classifier.
Instead, we use a property regressor to guide the generation towards molecules with specific continuous properties
similar to Bao et al. ().
One can show that the denoising network’s output relates to the score function of the data distribution via $\nabla_{{\bm{z}}_{t}}\log q({\bm{z}}_{t})\approx-\frac{1}{\sqrt{1-\bar{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)$ (Dhariwal & Nichol, ) and the sampling procedure of diffusion models (Equation 6) is equivalent to running Langevin dynamics using the score function $\nabla_{{\bm{z}}_{t}}\log q({\bm{z}}_{t})$ (Welling & Teh, ; Song & Ermon, ). This provides a way to sample from the distribution $q$ only through its score function.
For the case of $q({\bm{z}}_{t}|c)$, its score function can be written using Bayes’ rule and approximated as:

$$ $\displaystyle\nabla_{{\bm{z}}_{t}}\log q({\bm{z}}_{t}|c)=\nabla_{{\bm{z}}_{t}}\log q({\bm{z}}_{t})+\nabla_{{\bm{z}}_{t}}\log q(c|{\bm{z}}_{t})$ $\displaystyle\approx-\frac{1}{\sqrt{1-\bar{\alpha}_{t}}}(\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)-\sqrt{1-\bar{\alpha}_{t}}\nabla_{{\bm{z}}_{t}}\log p_{\eta}(c|{\bm{z}}_{t})),$ $$

where $p_{\eta}(c|{\bm{z}}_{t})\sim\exp(-s(g_{\eta}({\bm{z}}_{t})-c)^{2})$ is a Gaussian distribution approximating the probability of molecule ${\bm{z}}_{t}$ having property $c$, $g_{\eta}$ is a property regressor trained to predict the property $c$ given a noisy molecule ${\bm{z}}_{t}$, and $s$ is a scale factor that controls the skewness of the distribution. With this, we can rewrite the conditional score function as $\nabla_{{\bm{z}}_{t}}\log q({\bm{z}}_{t}|c)\approx-\frac{1}{\sqrt{1-\bar{\alpha}_{t}}}(\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)+\sqrt{1-\bar{\alpha}_{t}}s\nabla_{{\bm{z}}_{t}}(g_{\eta}({\bm{z}}_{t})-c)^{2})$.
To sample from this distribution, we use the same sampling algorithm defined by Equation 6 and replace $\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)$ by

$$ $\displaystyle\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)+\sqrt{1-\bar{\alpha}_{t}}s\nabla_{{\bm{z}}_{t}}(g_{\eta}({\bm{z}}_{t})-c)^{2}.$ (7) $$

Intuitively, this can be seen as minimizing the loss $(g_{\eta}({\bm{z}}_{t})-c)^{2}$.
Appendix E.2 provides additional details and discussions about the equivariance of the regressor.

#### Unconstrained Property Optimization via Diffusion Guidance

With the formulation introduced above, we can sample molecules with high property values by simply using the denoising step

$$ $\displaystyle\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)-\sqrt{1-\bar{\alpha}_{t}}s\nabla_{{\bm{z}}_{t}}g_{\eta}({\bm{z}}_{t}).$ (8) $$

Notice that we replaced the loss with the negative gradient of the regressor.

#### Scaffold-Constrained Generation via Inpainting

It is often desirable to generate molecules that contain a specific substructure, usually called a scaffold (Schuffenhauer et al., ; Maziarz et al., ), for tasks such as structure-based drug design. This problem can be viewed as a completion task and is reminiscent of inpainting, which aims to complete missing parts of an image (Song et al., ; Lugmayr et al., ). Lugmayr et al., propose RePaint, a simple modification to the sampling procedure of denoising diffusion models, enabling them to perform inpainting. This approach has already been successfully applied to molecule generation in 3D (Schneuing et al., ).

The core idea of RePaint is to start the sampling procedure from a random sample ${\bm{z}}_{T}$ and, at each step, enforce the part that we want to be present in the final sample. To illustrate this for the case of molecules, let $\tilde{\bm{z}}_{0}$ denote the point cloud of a molecule containing the desired scaffold. Let $m$ be a binary mask that indicates which nodes from $\tilde{\bm{z}}_{0}$ belong to the scaffold such that $m\odot\tilde{\bm{z}}_{0}$ denotes the scaffold point cloud and $(1-m)\odot\tilde{\bm{z}}_{0}$ the unknown part. We aim to sample a new point cloud ${\bm{z}}$ containing this scaffold while extending it to get a harmonized point cloud. Formally, we require that $m\odot{\bm{z}}_{0}=m\odot\tilde{\bm{z}}_{0}$.
To achieve this, we iteratively apply the following modified sampling step starting from $t=T$:

$$ $\displaystyle{\bm{z}}_{t-1}^{\text{scaffold}}$ $\displaystyle\sim{\mathcal{N}}(\sqrt{\bar{\alpha}_{t-1}}\tilde{\bm{z}}_{0},(1-\bar{\alpha}_{t-1}){\bm{I}})$ (9) $\displaystyle{\bm{z}}_{t-1}^{\text{unknown}}$ $\displaystyle\sim{\mathcal{N}}(\bm{\mu}_{\theta}({\bm{z}}_{t},t),\sigma_{t}^{2}{\bm{I}})\quad(\text{as in Equation~\ref{sampling_eq}})$ (10) $\displaystyle{\bm{z}}_{t-1}$ $\displaystyle=m\odot{\bm{z}}_{t-1}^{\text{scaffold}}+(1-m)\odot{\bm{z}}_{t-1}^{\text{unknown}}$ (11) $$

We refer to Appendix E.3 for more implementation details.

Figure: Figure 3: Overview of our constrained optimization procedure. Based on a noising/denoising approach, we run the first steps of the reverse diffusion process using the inpainting algorithm to add new atoms and the remaining steps using the guidance algorithm to increase the target property. The depicted molecules have QED values of 0.79 (initial) and 0.91 (optimized), with a 53% similarity.
Refer to caption: x3.png

#### Similarity-Constrained Optimization

In this task, we aim to improve the target properties of a given molecule while satisfying a similarity constraint.
We propose a novel approach to this task by combining the regressor guidance algorithm and the inpainting algorithm discussed above. The key idea is to add noise to the initial molecule and then denoise it with the regressor guidance algorithm to improve the target property.
Formally, starting from the initial point cloud ${\bm{z}}_{0}$ with $N$ atoms, we sample a noised ${\bm{z}}_{t}$ following Equation 3 for $t\in(0,T)$. $t$ is a hyperparameter that controls the optimization quality. On the one hand, increasing $t$ results in more reverse diffusion steps with the guidance algorithm, thus increasing the chance of improving the target property. On the other hand, it also results in losing information about the initial molecule due to noise.
Intuitively, the best $t$ yields the highest property improvement while satisfying the similarity constraints.

However, since the diffusion model operates on $N$ points, the denoised molecule has $N$ atoms. This limits the optimization of a molecule’s property as it prevents adding new atoms.
To overcome this limitation, we sample a Gaussian point cloud ${\bm{z}}_{T}$ with $N^{\prime}$ atoms, where $N^{\prime}>N$, and run the inpainting algorithm from $T$ to $t$ with ${\bm{z}}_{t}$ as the scaffold, meaning that ${\bm{z}}_{t}$ will replace $m\odot\tilde{\bm{z}}_{0}$ in the inpainting algorithm described above. This results in a new intermediate representation ${\bm{z}}_{t}^{\prime}$ whose first $N$ points correspond to ${\bm{z}}_{t}$, and the rest are the newly added atoms, as desired. Finally, ${\bm{z}}_{t}^{\prime}$ is denoised from $t$ to 0 using the guidance algorithm with maximization objective (Equation 8) to get the point cloud ${\bm{z}}_{0}^{\prime}$ of the optimized molecule, which is then decoded to the graph using the decoder ${\mathcal{D}}_{\xi}$. An overview of this procedure and an example test molecule are shown in Figure 3.

## Limitations

EDM-SyCo can be seen as an extension to EDM, which carries the limitations of diffusion models like high inference cost.
To mitigate this issue in future work, we can incorporate more efficient sampling strategies  (Karras et al., , ), or use more efficient models such as recent conditional flow matching approaches  (Tong et al., ; Song et al., ).
In addition, our sampling approach requires specifying a priori the number of nodes. While that works well empirically, we leave tackling the fundamental issue of generating variable-sized graphs or point clouds to future work.
Lastly, our constrained optimization procedure cannot remove atoms during inference. Our experiments found that the decoder tends to predict disconnected subgraphs, which we treat as removing atoms, but a rigorous solution may improve performance.

## 6 Experiments

In our experiments, we compare the performance of EDM-SyCo to several autoregressive baselines and to all-at-once diffusion-based model DiGress (Vignac et al., ) on de novo and conditional generation tasks as well as on a constrained optimization task.
Specifically, comparing the two diffusion-based models, EDM-SyCo and DiGress, highlights the effect of our SyCo framework on generation and optimization performance.
More experiments are discussed in Appendix G.
We use two datasets of different sizes and complexities: ZINC250K (Irwin et al., ) containing 250K molecules with up to 40 atoms, and GuacaMol (Brown et al., ) containing $\approx$1.5M drug-like molecules with up to 88 atoms.
We train EDM-SyCo as described in Sections 3 and 4 on these two datasets. Training details and hyperparameters are given in Appendix F.

**Table 2: FCD and KL scores on ZINC250K. We split the methods into autoregressive and all-at-once. The best scores within each category are in bold, and the best overall are underlined. All models are trained from scratch, and we report mean and standard deviation using 3 random seeds.**
|  | Autoregressive Methods | All-at-once Methods |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | GraphAF | PS-VAE | HierVAE | MiCaM | JT-VAE | MAGNet | MoLeR | DiGress | EDM-SyCo |
|  | () | () | () | () | () | () | () | () | (Ours) |
| FCD ($\uparrow$) | 0.05 $\pm$ 0.00 | 0.28 $\pm$ 0.01 | 0.50 $\pm$ 0.14 | 0.63 $\pm$ 0.02 | 0.75 $\pm$ 0.01 | 0.76 $\pm$ 0.00 | 0.83 $\pm$ 0.00 | 0.65 $\pm$ 0.00 | 0.85 $\pm$ 0.01 |
| KL ($\uparrow$) | 0.67 $\pm$ 0.01 | 0.84 $\pm$ 0.00 | 0.92 $\pm$ 0.00 | 0.94 $\pm$ 0.00 | 0.93 $\pm$ 0.00 | 0.95 $\pm$ 0.00 | 0.97 $\pm$ 0.00 | 0.91 $\pm$ 0.00 | 0.96 $\pm$ 0.00 |

**Table 3: FCD and KL scores on GuacaMol. * means numbers are from (Maziarz et al., ), and DiGress is from Vignac et al. (). For our model, we additionally report standard deviation across 3 validations of the same model.**
|  |  | FCD ($\uparrow$) | KL ($\uparrow$) |
| --- | --- | --- | --- |
| Autoreg. | CGVAE* () | 0.26 | - |
| HierVAE* () | 0.62 | - |  |
| JT-VAE* () | 0.73 | - |  |
| MoLeR* () | 0.81 | - |  |
| AAO | DiGress () | 0.68 | 0.93 |
| EDM-SyCo (Ours) | 0.79 $\pm$ 0.002 | 0.93 $\pm$ 0.006 |  |

#### De novo Generation

We leverage the GuacaMol benchmark (Brown et al., ), an evaluation framework for de novo molecular graph generation. We consider two metrics that measure the similarity between the generated molecules and the training set. The Fréchet ChemNet Distance (FCD) compares the hidden representations of ChemNet, a neural network capturing chemical and biological features of molecules (Preuer et al., ), and the KL divergence (KL) compares the probability distributions of a variety of physiochemical descriptors.

Tables 2 and 3 show the FCD and KL scores of EDM-SyCo and the baselines on the ZINC250K and GuacaMol datasets, respectively.
Notably, EDM-SyCo outperforms DiGress on the FCD metric by 30% on ZINC250K and 16% on GuacaMol. This shows the capacity of our model to more accurately capture the joint distribution of nodes and edges encoded in the latent features and coordinates. EDM-SyCo sets the new state-of-the-art FCD score on ZINC250K, outperforming all autoregressive and fragment-based baselines, which incorporate more domain knowledge.
Note that we use the same hyperparameters from ZINC250K to train our model on GuacaMol, while DiGress and MoLeR have been tuned for both datasets.
We do not report novelty and uniqueness scores because all methods achieve near-perfect results on these metrics.
Regarding validity, all autoregressive baselines achieve 100% validity scores due to valency checks after each intermediate step. DiGress achieves 85%, and EDM-SyCo achieves 88%.
We show some sample molecules generated by our model in Figures 4 and 5 for the two datasets.

**Table 4: Conditional generation results.**
| Method | LogP | MW |  |  |
| --- | --- | --- | --- | --- |
| MAE ($\downarrow$) | Valid ($\uparrow$) | MAE ($\downarrow$) | Valid ($\uparrow$) |  |
| DiGress | 0.49 $\pm$ 0.05 | 65% | 60.30 $\pm$ 6.10 | 73% |
| FreeGress | 0.15 $\pm$ 0.02 | 85% | 15.18 $\pm$ 2.71 | 75% |
| EDM-SyCo | 0.18 $\pm$ 0.01 | 76% | 03.86 $\pm$ 0.08 | 88% |

#### Property-Conditioned Generation

To evaluate the conditional generation performance of our approach, we follow Ninniri et al. () and condition the generation of 1000 molecules on LogP and molecular weight (MW) values sampled from ZINC250K. We report the mean absolute error (MAE) between the target and estimated values for the generated molecules (using RDKit) and the validity rate. More details on the experimental setup are in Appendix F.7.
We run the regressor guidance algorithm, discussed in Section 5, by using EDM-SyCo from the previous experiment and a single regressor trained to predict LogP and MW values. As baselines, we compare to DiGress, which uses a discrete guidance scheme, and FreeGress Ninniri et al. (), which proposes a regressor-free guidane mechanism.
Table 4 shows the results. While EDM-SyCo achieves a comparable score to FreeGress on LogP (within standard deviation), it outperforms FreeGress by a factor of 3.9 on MW. Compared to DiGress, it improves on LogP and MW by factors 2.7 ad 15.6, respectively. This experiment highlights the benefit of our continuous latent space on conditional generation compared to operating directly on the discrete graph space. It also shows that graph-level properties are represented sufficiently well in this space. We include additional experiments in Appendix G.4 and Appendix G.5.

**Table 5: Similarity-constrained optimization results from (Jin et al., ). We split all learning-based baslines into translation and optimization approaches.**
|  | Method | LogP (sim $\geq$ 0.4) | LogP (sim $\geq$ 0.6) | QED (sim $\geq$ 0.4) | DRD2 (sim $\geq$ 0.4) |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Improv. | Div. | Improv. | Div. | Succ. | Div. | Succ. | Div. |  |
| Transl. | Seq2Seq | 3.37 $\pm$ 1.75 | 0.471 | 2.33 $\pm$ 1.17 | 0.331 | 58.5% | 0.331 | 75.9% | 0.176 |
| JTNN | 3.55 $\pm$ 1.67 | 0.480 | 2.33 $\pm$ 1.24 | 0.333 | 59.9% | 0.373 | 77.8% | 0.156 |  |
| AtomG2G | 3.98 $\pm$ 1.54 | 0.563 | 2.41 $\pm$ 1.19 | 0.379 | 73.6% | 0.421 | 75.8% | 0.128 |  |
| HierG2G | 3.98 $\pm$ 1.46 | 0.564 | 2.49 $\pm$ 1.09 | 0.381 | 76.9% | 0.477 | 85.9% | 0.192 |  |
| Optim. | JT-VAE | 1.03 $\pm$ 1.39 | - | 0.28 $\pm$ 0.79 | - | 8.8% | - | 3.4% | - |
| CG-VAE | 0.61 $\pm$ 1.09 | - | 0.25 $\pm$ 0.74 | - | 4.8% | - | 2.3% | - |  |
| GCPN | 2.49 $\pm$ 1.30 | - | 0.79 $\pm$ 0.63 | - | 9.4% | 0.216 | 4.4% | 0.152 |  |
| EDM-SyCo | 3.11 $\pm$ 1.27 | 0.555 | 1.51 $\pm$ 1.10 | 0.360 | 46.4% | 0.163 | 27.3% | 0.083 |  |

#### Similarity-Constrained Optimization

We follow Jin et al. () and evaluate our optimization procedure from Section 5 with EDM-SyCo on their constrained optimization tasks: LogP, QED, and DRD2 on ZINC250K. These tasks consist of improving the properties of a set of 800 test molecules under the constraint that the Tanimoto similarity with Morgan fingerprints (Rogers & Hahn, ) between the optimized and initial molecule is above a given threshold.
We report the average improvement for the LogP tasks and success rates for the binary tasks of QED and DRD2 consisting of translating molecules from a low range into a higher range.
Additionally, we report the average diversity among the optimized molecules. More details are given in Appendix F.8.

We split the learning-based baselines from (Jin et al., ) into optimization and translation approaches. Translation approaches are specifically designed for such tasks and are trained on pairs of molecules exhibiting the improvement and similarity constraints, giving them an advantage over optimization methods. Since we leverage the same generative model and regressor from previous experiments without further training on this task, optimization approaches are the main point of comparison.
Table 5 shows that EDM-SyCo outperforms all optimization baselines by an average factor of 3.5 across all tasks while achieving comparable performance to the translation approaches. These results further support the effectiveness of our SyCo framework and our novel constrained optimization algorithm.
The relatively low diversity scores indicate that EDM-SyCo tends to find more similar molecules for a given starting molecule, which might be an artifact of our noising/denoising procedure.
Figures 6 and 7 illustrate some of EDM-SyCo’s optimized molecules.
Thanks to our atom-based approach, the optimized molecules frequently only change in a few atoms from the starting molecule, which would be harder to achieve for fragment-based models.

## 7 Conclusion

Despite the common practice of using fragment-based autoregressive models for molecular graph generation, we demonstrated with SyCo that a mapping between molecular graphs and latent Euclidean point clouds enables atom-based all-at-once approaches to be competitive with special-purpose molecular graph generators.
Further, we introduced a similarity-constrained optimization procedure for 3D diffusion models based on guidance and inpainting.
Based on this framework, we developed EDM-SyCo, which sets a new state-of-the-art FCD score on ZINC250K.
In our conditional generation and optimization experiments, we found our guidance-based approach to accurately guide the generation process, outperforming all baselines.
With these results, we conclude that latent Euclidean generative models hold significant promise for advancing molecular graph generation and accelerating drug discovery. We discuss our work’s broader impact in Appendix B.

## Acknowledgments

This project is supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research.
Further, it is funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Länder.
This project is also supported by the Bavarian Ministry of Economic Affairs, Regional Development and Energy with funds from the Hightech Agenda Bayern

## Appendix A Visualizations for generated and optimized molecules by EDM-SyCo

We visualize some of the molecules generated by EDM-SyCo.
For de novo generation, we show molecules of the model trained on ZINC250K in Figure 4, and of the one trained on GuacaMol in Figure 5.
For the similarity-constrained optimization task, we show some molecules found by our optimization approach in the LogP task in Figure 6, and in the QED task in Figure 7

Figure: Figure 4: Sample molecules generated by EDM-SyCo trained on ZINC250K.
Refer to caption: x4.png

Figure: Figure 5: Sample molecules generated by EDM-SyCo trained on GuacaMol.
Refer to caption: x5.png

Figure: Figure 6: Molecules with the highest improvement in the LogP-constrained optimization task. Each row corresponds to one test molecule and 4 successful optimization results. We show the LogP value of the initial molecules, and for the optimized molecules, the achieved improvement (imp.) and the Tanimoto similarity to the initial molecule (sim.).
Refer to caption: x6.png

Figure: Figure 7: Molecules with highest improvement in the QED constrained optimization task. Each row corresponds to one test molecule and 4 successful optimization results. We show the QED value of the initial molecules, and for the optimized molecules, the achieved improvement (imp.) and the Tanimoto similarity to the initial molecule (sim.).
Refer to caption: x7.png

## Appendix B Broader Impact

This paper advances the field of computational drug discovery by proposing new efficient methods for generative models.
While there may be unintended consequences, such as the misuse of chemical weapons, we firmly believe that their benefits significantly outweigh the minuscule chance of misuse.

## Appendix C Invariance and Equivariance

In this work, we are dealing with two types of symmetries arising from how we represent the data and/or physical symmetries. Specifically, graphs are invariant to permuting their nodes, meaning that a single graph of $N$ nodes can be represented by $N!$ different representations corresponding to $N!$ different orderings of its nodes. We refer to this transformation as the elements of the symmetric group $S_{N}$. Similarly, a point cloud representing a molecule in 3D can be arbitrarily rotated, reflected, and/or translated and still refer to the same molecule. This corresponds to an infinite number of different representations for the same object. We refer to this transformation as the action of the Euclidean group $E(3)$.

### C.1 S N S_{N} - and E ​ ( 3 ) E(3) -Invariant Training Process

To efficiently learn from such data, we need to develop training methods invariant to these symmetries without needing data augmentation techniques. Concretely, we must ensure that the gradient updates used to train our model do not change if the molecular graph is permuted and/or the latent point cloud representation is rotated/reflected/translated. We require two ingredients to achieve this: an equivariant architecture and an invariant loss (Vignac et al., ). Based on the model architecture and the loss function described in the main text, we can show that EDM-SyCo satisfies both requirements and that its training process is, thus, invariant to the action of the permutation group $S_{N}$ and the Euclidean group $E(3)$.

As our ultimate goal is to generate molecular graphs, we provide more details on the permutation invariance properties of the distribution of molecular graphs learned by EDM-SyCo in the next section.

### C.2 Proof of Proposition 4.1 ( S N S_{N} -Invariant Marginal Distribution)

In this section, we provide a formal proof for the statement from the main text that the molecular graph distribution defined by EDM-SyCo is invariant to permutations. For convenience, we provide the proposition again:

###### Proposition C.1 .

The marginal distribution of molecular graphs $p_{\theta,\xi}({\mathcal{G}})=\mathbb{E}_{p_{\theta}({\bm{z}}_{0})}\left[p_{\xi}({\mathcal{G}}|{\bm{z}}_{0})\right]$ defined by the EDM and the decoder, is an $S_{N}$-invariant distribution, i.e. for any molecular graph ${\mathcal{G}}=({\bm{h}},{\bm{A}})$, $p_{\theta,\xi}({\bm{h}},{\bm{A}})=p_{\theta,\xi}(P{\bm{h}},P{\bm{A}}P)$ for any permutation matrix $P\in S_{N}$.

Note: This invariance property is required for efficient likelihood computation, as the likelihood of a graph is the sum of the likelihoods of its $N!$ permutations, which is intractable to compute. However, when ensuring that all these permutations are assigned equal likelihood, it suffices to compute the likelihood of an arbitrary permutation.

###### Proof.

The idea of the proof is when the initial distribution of the diffusion model $p_{\theta}({\bm{z}}_{T})$ is invariant and the transition distributions $p_{\theta}({\bm{z}}_{t-1}|{\bm{z}}_{t})$ are equivariant, then all marginal distributions at all diffusion time steps $p_{\theta}({\bm{z}}_{t})$ are invariant, including $p_{\theta}({\bm{z}}_{0})$ (Xu et al., , ; Hoogeboom et al., ). With the same logic and with the equivariant decoder taking the place of the transition distribution, the graph distribution $p_{\theta,\xi}({\mathcal{G}})$ will be invariant.

We prove this result by induction:

Base case: $p_{\theta}({\bm{z}}_{T})={\mathcal{N}}({\bm{z}}_{T};{\bm{0}},{\bm{I}})$ is permutation-invariant, i.e. $p_{\theta}({\bm{z}}_{T})=p_{\theta}(P{\bm{z}}_{T})$ for all permutation matrices $P\in\{0,1\}^{N\times N}$ acting on ${\bm{z}}_{T}\in{\mathbb{R}}^{N\times(d+3)}$ by permuting its rows. This holds because all rows are i.i.d., as $p_{\theta}({\bm{z}}_{T})$ has a diagonal covariance matrix.

Induction step: Assume $p_{\theta}({\bm{z}}_{t})$ is permutation-invariant. We show that if $p_{\theta}({\bm{z}}_{t-1}|{\bm{z}}_{t})$ is permutation-equivariant, i.e. $p_{\theta}({\bm{z}}_{t-1}|{\bm{z}}_{t})=p_{\theta}(P{\bm{z}}_{t-1}|P{\bm{z}}_{t})$ (which holds when using a permutation-equivariant architecture $\bm{\epsilon}_{\theta}$), then $p_{\theta}({\bm{z}}_{t-1})$ will be permutation-invariant.

$$ $\displaystyle p_{\theta}(P{\bm{z}}_{t-1})$ $\displaystyle=\int_{{\bm{z}}_{t}}p_{\theta}(P{\bm{z}}_{t-1}|{\bm{z}}_{t})p_{\theta}({\bm{z}}_{t})\qquad(\text{Chain rule of probability})$ $\displaystyle=\int_{{\bm{z}}_{t}}p_{\theta}(P{\bm{z}}_{t-1}|PP^{-1}{\bm{z}}_{t})p_{\theta}(PP^{-1}{\bm{z}}_{t})\qquad(PP^{-1}={\bm{I}})$ $\displaystyle=\int_{{\bm{z}}_{t}}p_{\theta}({\bm{z}}_{t-1}|P^{-1}{\bm{z}}_{t})p_{\theta}(P^{-1}{\bm{z}}_{t})\qquad(\text{Equivariance and invariance})$ $\displaystyle=\int_{{\bm{u}}}p_{\theta}({\bm{z}}_{t-1}|{\bm{u}})p_{\theta}({\bm{u}})\underbrace{|\det P|}_{=1}\qquad(\text{Change of variables }{\bm{u}}=P^{-1}{\bm{z}}_{t}\text{ and }P\text{ is orhtogonal, so}\det P=\pm 1)$ $\displaystyle=p_{\theta}({\bm{z}}_{t-1})$ $$

By induction, $p_{\theta}({\bm{z}}_{T}),\dots,p_{\theta}({\bm{z}}_{0})$ are all permutation-invariant.

Finally, since the decoder $p_{\xi}({\mathcal{G}}|{\bm{z}}_{0})$ is permutation-equivariant (as it applies the same operation on each node and each pair of nodes, thus not depending on the nodes ordering), with the same derivation, we also conclude that the final induced distribution $p_{\theta,\xi}({\mathcal{G}})$ is permutation-invariant.
∎

Note: A similar derivation can also be applied to show the invariance of $p_{\theta}({\bm{z}}_{0})$ to rotations and reflections of the coordinates, as the only requirement on $P$ for the derivation to hold is to be orthogonal, which is the case for rotation and reflection matrices $R$.

###### Proposition C.1 .

###### Proof.

## Appendix D E ​ ( n ) E(n) Equivariant Graph Neural Networks (EGNNs)

To perform equivariant updates in our latent space, we use the $E(n)$ Equivariant Graph Neural Network (EGNN) architecture (Satorras et al., ).

An EGNN operates on a point cloud of size $N$, where each point has features ${h}_{i}\in\mathbb{R}^{d}$ and coordinates ${x}_{i}\in\mathbb{R}^{3}$ associated with it. Let ${\bm{h}}\in\mathbb{R}^{N\times d}$ and ${\bm{x}}\in\mathbb{R}^{N\times 3}$ be the stacked features and coordinates, respectively, of all points. An EGNN is composed of Equivariant Graph Convolutional Layers (EGCLs) ${\bm{h}}^{l+1},{\bm{x}}^{l+1}=\text{EGCL}({\bm{h}}^{l},{\bm{x}}^{l})$. A single EGCL layer is defined as

$$ $\displaystyle{m}_{ij}$ $\displaystyle=\phi_{e}({h}_{i}^{l},{h}_{j}^{l},d_{ij}^{2},a_{ij})$ (12) $\displaystyle{h}_{i}^{l+1}$ $\displaystyle=\phi_{h}({h}_{i}^{l},\sum_{j\neq i}\tilde{e}_{ij}{m}_{ij})$ (13) $\displaystyle{x}_{i}^{l+1}$ $\displaystyle={x}_{i}^{l}+\sum_{j\neq i}\frac{{x}_{i}^{l}-{x}_{j}^{l}}{d_{ij}+1}\phi_{x}({h}_{i}^{l},{h}_{j}^{l},d_{ij}^{2},a_{ij}),$ (14) $$

where $l$ denotes the layer index, $d_{ij}=\|{x}_{i}^{l}-{x}_{j}^{l}\|_{2}$ is the Euclidean distance between points $i$ and $j$, and $a_{ij}$ are optional edge attributes which we set to $\|{x}_{i}^{0}-{x}_{j}^{0}\|_{2}^{2}$. $\tilde{e}_{ij}$ serves as an attention mechanism that infers a soft estimation of the edges $\tilde{e}_{ij}=\phi_{inf}({m}_{ij})$. Note that to update each point’s features and coordinates, the EGCL layers consider all the other nodes, effectively treating the point cloud as a fully connected graph. All learnable components ($\phi_{e},\phi_{h},\phi_{x},$ and $\phi_{inf}$) are fully connected neural networks. An EGNN architecture is then composed of $L$ EGCL layers, denoted as ${\bm{h}}^{L},{\bm{x}}^{L}=\text{EGNN}({\bm{h}}^{0},{\bm{x}}^{0})$ with ${\bm{h}}^{0}={\bm{h}}$ and ${\bm{x}}^{0}={\bm{x}}$. The main hyperparameters of the EGNN architectures are the number of layers $L$ and a feature dimension $n_{f}$ used to control the width of the fully connected neural networks.

## Appendix E Additional Method Details

In this section, we provide additional details for our method introduced in Sections 3, 4, and 5.

### E.1 Training and Sampling

We start by describing in more detail the training and sampling algorithms for EDM-SyCo in Algorithms 1 and 2.

Figure: Algorithm 1 Training Algorithm

Figure: Algorithm 2 Sampling Algorithm

### E.2 Conditional Generation

Here, we detail the regressor guidance algorithm we developed in Section 5. We showed in the main text that we can sample from the conditional distribution $q({\bm{z}}_{t}|c)$ by replacing the predicted noise of the denoising neural network by Equation 7. By putting this back into the sampling equation, Equation 6, we get the following modified denoising step

$$ $\displaystyle{\bm{z}}_{t-1}=\bm{\mu}_{\theta}({\bm{z}}_{t},t)-\frac{1-\alpha_{t}}{\sqrt{\alpha_{t}}}s\nabla_{{\bm{z}}_{t}}(g_{\eta}({\bm{z}}_{t})-c)^{2}+\sigma_{t}\bm{\epsilon}_{t}.$ (15) $$

A more general formulation for the new guidance term is given by $-s(t)\nabla_{{\bm{z}}_{t}}l({\bm{z}}_{t},c)$, where $s(t)$ is a time-dependent scaling function and $l$ can be any invariant loss function. In the specific case that we derived, $s(t)=\frac{1-\alpha_{t}}{\sqrt{\alpha_{t}}}s$ and $l({\bm{z}}_{t},c)=(g_{\eta}({\bm{z}}_{t})-c)^{2}$. $l$ can be replaced by any other loss function that we want to optimize, or in the case of property maximization, it can be the prediction of the regressor directly (Equation 8). In our experiments, we found a simple linear schedule $s(t)=at$ to perform well, where $a\in\mathbb{R}$ is a hyperparameter that we tune for each task.

To ensure the equivariance of the distribution even with guidance, the new term $s(t)\nabla_{{\bm{z}}_{t}}l({\bm{z}}_{t},c)$ needs to be $E(3)$-equivariant. This is done by subtracting its center of mass at each sampling step and by choosing an invariant loss function $l$, as the differential operator of an invariant function yields an equivariant function (Chmiela et al., ; Schütt et al., ).

### E.3 Scaffold-constrained Generation

Here, we provide additional details for the inpainting algorithm described in Section 5, where we described an approach that allows to generate a point cloud that contains a chosen scaffold.

Lugmayr et al. () observed that the version of their algorithm described in the main text leads to inconsistent completions and proposes to run each sampling step multiple times to allow the model to harmonize its generations.
Concretely, each denoising step is repeated $r$ times to improve the generations’ quality and allow the model to harmonize the unknown part to the scaffold. This is achieved by applying the forward diffusion model (Equation 2) after each denoising step.
Lugmayr et al. () also propose to go more than one forward step with the diffusion model to allow the model to better harmonize its generations. This choice is fixed by the jump length $j$. Note that this procedure increases the computation cost of running the model $r$ times, which might be a limiting factor. For this reason, we combine this approach with the approach described in Appendix G.9 that allows to reduce the number of reverse diffusion steps to run.

The procedure described in the main text gives a way to condition the diffusion model, which operates on the latent point cloud space. However, we are ultimately interested in generating a molecular graph containing a scaffold defined by a subset of atoms and the bonds between them. We map the molecular scaffold to its point cloud representation using our encoder to get $m\odot{\bm{z}}_{0}$. This enables running the inpainting algorithm described above. Then, we decode the sampled 3D point cloud to a molecular graph. Since our decoder model operates on each node and edge separately, the final molecular graph is guaranteed to contain the original scaffold as long as the scaffold in isolation can be correctly reconstructed with our autoencoder.

## Appendix F Implementation Details

### F.1 Source Code

Source code for our method and all experiments in this paper will be made publicly available upon publication.

### F.2 Licenses for used code, models, and datasets

We start by listing the assets we used in our work and their respective licenses. We use the following tools, codebases, and datasets:

- •
RDKit (Landrum et al., ) (BSD 3-Clause License)
- •
PyTorch (Paszke et al., ) (BSD 3-Clause License)
- •
EDM (Hoogeboom et al., ) (MIT license)
- •
GeoLDM (Xu et al., ) (MIT license)
- •
ZINC250K dataset (Irwin et al., ) (Custom license)
- •
GuacaMol dataset (Brown et al., ) (MIT license)

### F.3 Architecture Details

We train two sets of models with the same architectures on the two used datasets, ZINC250K and GuacaMol. The EDM, together with the autoencoder, has 9.2M total parameters, while the regressor has 4.2M parameters.

Encoder. The first part of the encoder ${\mathcal{E}}$, introduced in Section 3, is the conformer generation method, while the second part is an EGNN with 1 layer and 128 hidden features (See Appendix D). We found that having at least 1 EGNN layer is crucial to achieve high reconstruction accuracy for the autoencoder, and as this allows us to reach more than 99% accuracy, we did not use more layers. This EGNN takes as input a point cloud with the one-hot encodings of the atom types and formal charges as features ${\bm{h}}$ and the synthetic coordinates computed by the conformer generation part as coordinates ${\bm{x}}$. The output is a processed point cloud with continuous node embeddings of dimension $d=2$ and updated coordinates.

Decoder. The decoder ${\mathcal{D}}$, also introduced in Section 3, consists of 2 fully connected networks $\text{MLP}_{node}$ and $\text{MLP}_{edge}$, both having 256 hidden dimensions.

EDM. The denoising network of EDM is parametrized with an EGNN with 9 layers and 256 hidden features. The EDM has $T=1000$ diffusion steps.

Regressor. The regressor used for the conditional generation and optimization experiments (see Section 6) is also an EGNN with 4 layers and 256 hidden features, followed by a sum pooling operation over the node features and then a fully connected network with 2 layers and a hidden dimension of 256 that maps the pooled graph embedding to the target values of the properties.

### F.4 Training Details

We use the original train/validation/test splits of the used datasets (Irwin et al., ; Brown et al., ).
The autoencoder is trained in the first stage to minimize the cross-entropy loss between the ground truth and predicted graphs for a maximum of 100 epochs, with early stopping if the validation accuracy does not improve for 10 epochs.
To deal with the class imbalance problem caused by the sparsity of the graph adjacency matrices and the dominance of some atom types and formal charge values, we scale each term of the cross-entropy loss with a class-specific value based on the statistics of the training set (King & Zeng, ).
In the second training stage, the EDM model is trained for 1000 epochs on ZINC250 and approximately 300 epochs on GuacaMol.

The regressor is trained with the $L_{1}$ loss to predict the molecular properties on the noisy latent point cloud representations of the molecules by applying the same noise schedule of the EDM. The regressor is trained for 500 epochs, with early stopping if the MAE on the validation set does not improve after 50 epochs.

All models are trained with a batch size of 64 and using the recent Prodigy optimizer (Mishchenko & Defazio, ) with $d_{coef}=0.1$, which we found to be a very important hyperparameter for the stability of training.

We train all models on ZINC250K on a single Nvidia A100 GPU, and on GuacaMol, we use multi-GPU training on 4 Nvidia A100 GPUs.

### F.5 Dataset Details

We use two datasets in our experiments: ZINC250K (Irwin et al., ) and GuacaMol (Brown et al., ). For both datasets, we represent a molecule as a graph. The nodes are atoms with the one-hot encoding of their atom type and their formal charge as features. In addition to modeling all heavy atoms, we also model explicit H atoms, which we found to increase the validity of the generated molecules as it reduces some kekulization errors produced by rdkit (^1^11https://github.com/rdkit/rdkit/wiki/FrequentlyAskedQuestions#cant-kekulize-mol). The edges are chemical bonds with the one-hot encoding of the bond type as features. We model the following bond types: no-bond, single, double, triple, and aromatic. We model the absence of a bond as a separate bond type to allow the autoencoder to reconstruct the full bond information. These graphs present very unbalanced scales for the atom types, formal charges, and bond types. We use class-specific weights to train the autoencoder, as outlined in Section 3.

ZINC250K has a total of 250K molecules and 220,011 training molecules. It has 10 atom types (including H) and 3 formal charge values (-1, 0, 1), and the largest molecule has 40 atoms.

GuacaMol has a total of $\approx$1.5M molecules, of which we use 1,273,104 for training. It has 13 atom types (including H) and 5 formal charge values (-1, 0, 1, 2, 3), and the largest molecule has 88 atoms.

We compute the synthetic coordinates needed for our model in a preprocessing step for both datasets and use the computed coordinates throughout the training.

### F.6 Sampling the number of atoms

In this work, the diffusion generative model operates on a fixed-size point cloud $N$. During sampling, the first step is to sample this number $N$. To sample molecules of similar sizes to those in the training set, we approximate the distribution of the number of nodes using a categorical distribution.

To this end, we count the number of occurrences of each molecule size in the training set and use the categorical distribution parameterized by the normalized counts to sample the number of atoms of generated molecules. This simple procedure works well for the task of unconditional generation. However, in the property-conditioned generation, we must consider that some properties are highly correlated to molecule size, such as molecular weight (MW). Therefore, for this task, we start by dividing the range of property values from the training set into 100 equal-sized bins. We define a different categorical distribution for each bin that counts the sizes of molecules falling in that bin. Given the conditioning value, we first specify the bin it falls into and use the corresponding categorical distribution to sample the number of nodes. Finally, for molecule optimization, we curate a dataset of molecule pairs that exhibit the desired property improvement and similarity constraints and create different conditional categorical distributions, one for each size of the initial molecule.

Note that these different procedures are needed due to the inherent limitation of diffusion models on graphs that require the number of nodes to be known and fixed in advance. Fundamentally, solving this issue would be a valuable contribution to the field, and we leave it for future work.

### F.7 Property-Conditional Generation Experiment

Following Ninniri et al. (), for each task, we sample 100 molecules from the test set and condition on their property values to generate 10 valid molecules each and report the mean absolute error between the target and the estimated values for the generated molecules measured by RDKit.
We experimented with different values of the guidance scale $s$ and with using the $l_{1}$ and $l_{2}$ losses to compute the guidance loss but found the $l_{2}$ loss with scale values $s=2.5$ and $s=1.0$ for LogP and MW, respectively, to work best.

### F.8 Constrained Molecule Optimization

Here, we provide more details on the constrained optimization tasks described in Section 6.
Given a molecule ${\mathcal{G}}$, these tasks aim to find a different molecule ${\mathcal{G}}^{\prime}$ with higher property values satisfying the similarity constraint $sim({\mathcal{G}},{\mathcal{G}}^{\prime})\geq 0.4\text{ (or }0.6$), where $sim$ computes the Tanimoto similarity with Morgan fingerprints (Rogers & Hahn, ). For the LogP task, the new molecule ${\mathcal{G}}^{\prime}$ needs to have a higher LogP value and we report the average improvement over 800 molecules from the test set with the lowest LogP values. For the QED task, the goal is to optimize another 800 molecules from the test with QED values in the low range $[0.7,0.8]$ into the high range $[0.9,1.0]$, and we report the success rate over these molecules.
For DRD2, the goal is to translate inactive compounds $(p\leq 0.05)$ into active ones $(p\geq 0.5)$, and we similarly report the success rate over these molecules.
In addition, for all tasks, we report the diversity as the average pairwise distance between the successfully optimized molecules for each initial test molecule. The distance is defined as $dist({\mathcal{G}},{\mathcal{G}}^{\prime})=1-sim({\mathcal{G}},{\mathcal{G}}^{\prime})$.

Choosing $t$. As described in Section 5, the intermediate time step $t$ controls the trade-off between the similarity to the initial molecule and the capacity to improve its target properties. In principle, for each test molecule, there is an optimal $t$ yielding the highest improvement under the similarity constraints. However, finding the optimal $t$ for each single molecule is computationally expensive. Therefore, we use a subset of 20 molecules and run our optimization algorithm for different values of $t$ on these molecules to find the value of $t$ that yields the highest average improvement under the considered similarity constraints. We use this value for all test molecules. Our experiments found that a value of $t$ in the range $[500,700]$ usually works best with the total $T=1000$. Figure 8 shows the effect of the time step $t$ on the similarity and property value for one molecule from the test set. Table 6 lists the exact value of $t$ used for each task.

Figure: Figure 8: Effect of using different time steps $t$ on the similarity and improvement values in the similarity-constrained optimization task of PLogP. We visualize 4 different molecules from the test set.
Refer to caption: x8.png

**Table 6: Values of the time step $t$ used in all optimization tasks.**
|  | Time step $t$ |
| --- | --- |
| PLogP (sim $\geq$ 0.4) | 600 |
| PLogP (sim $\geq$ 0.6) | 500 |
| QED | 650 |
| DRD2 | 650 |

## Appendix G Additional Experiments

In this section, we provide additional experiments to showcase the performance of our model and analyze some of its properties.

### G.1 Bond Type Prediction from Synthetic Coordinates

A crucial part of successfully applying our method is to be able to accurately reconstruct the molecular graph from the Euclidean point cloud. This section provides an ablation study to compare our approach to other methods.

Formally, we are given a point cloud with features ${\bm{h}}\in\{0,1\}^{N\times(a+c)}$ containing the atom type and formal charge of each atom and coordinates ${\bm{x}}\in\mathbb{R}^{N\times 3}$ computed using the ETKDG conformer generation algorithm (Riniker & Landrum, ). The goal is to reconstruct the bond types stored in the adjacency matrix ${\bm{A}}\in\{0,1\}^{N\times N\times b}$.
We test the following approaches:

- •
Rule-based: This approach is widely used by generative models for molecules in 3D (Hoogeboom et al., ; Xu et al., ) and consists of using typical bond type lengths to infer the bond types between two atoms based on their features and the inter-atomic distances between them.(^2^22https://www.wiredchemist.com/chemistry/data/bond_energies_lengths.html) It uses a set of if-else statements and predicts a specific bond type if the length falls within a specific range that takes the features of the two atoms into account.
- •
MLP: We develop this baseline to generalize the lookup table approach and replace the set of if-else statements by an MLP. Formally, we construct edge features between atoms $i$ and $j$ as $[{h}_{i},{h}_{j},d_{ij}]$, where $d_{ij}$ is the Euclidean distance between the two atoms. This is also the same information that the lookup table method above uses to infer the bond types. Instead, we train an MLP to predict the bond type from this edge feature in a classification setting. Note that this MLP is defined on the edge level, so a molecule with $N$ atoms provides $N^{2}$ training examples for the MLP to learn from. Also note that we do not need to have an extra MLP for the atom type as they are directly available in this case since we do not run an EGNN in the first step.
- •
EGNN + MLP: The major limitation of the above approaches is that to predict the bond type between two atoms they only consider the distance between them and their respective features, without considering other atoms. However, not all molecules exhibit typical bond lengths between their atoms (Hoogeboom et al., ). To overcome this limitation, we combine the MLP approach with an EGNN that first computes updated coordinates and atom features considering the whole graph before running the local MLP to predict the bond type. Here, we add an extra MLP to predict the node labels (type and formal charge). This is the same architecture presented in the main text.

We evaluate these three approaches on ZINC250K using the synthetic coordinates computed as outlined in Section 3. We report the molecule reconstruction accuracy on the validation set, where a molecule is considered correctly reconstructed if all of its atom features (atom types and formal charges) and bond types are correctly reconstructed. Table 1 shows that the EGNN approach achieves an almost perfect reconstruction accuracy. This shows that (i) the synthetic coordinates contain all the needed information to reconstruct the bond types, which might not be directly clear if we are limited to the other methods, and (ii) incorporating the global graph information into the prediction task is crucial to accurately reconstruct the bonds. We adopt the EGNN approach into our model.

### G.2 Conformer Generation Comparison

To analyze the effect of different conformer generation algorithms on the performance of our model, we perform an ablation study using 2 different algorithms/packages from RDKit to compute the synthetic coordinates: the ETKDG algorithm and the Pharm3D package. We perform two training runs using the same setup except for the conformer generation method. For these two runs, we use smaller models (we halve the number of layers) than those reported in the main text and train for 500 epochs, so they are not directly comparable. Results are shown in Table 7.

**Table 7: Model performance on ZINC using different conformer generation methods from RDKit.**
|  | ETKDG | Pharm3D |
| --- | --- | --- |
| VAE Reconstruction Accuracy | 99.92% | 99.90% |
| KL | 0.94 | 0.93 |
| FCD | 0.75 | 0.70 |

### G.3 Complete Benchmark Results

We provide complete benchmark results for the GuacaMol benchmark (Brown et al., ) and the MOSES benchmark (Polykovskiy et al., ) in Tables 8 and 9, respectively. In addition to the baselines reported in the main text, we add two SMILES-based baselines and we add different flavours of MoLeR with different vocabulary sizes, denoted as MoLeR-$V$, where $V$ denotes the vocabulary size.

**Table 8: GuacaMol benchmark metrics on the ZINC250K dataset. We report mean and standard deviations across 3 random training runs for all methods and metrics.**
| Metric | FCD $(\uparrow)$ | KL $(\uparrow)$ | Novelty $(\uparrow)$ | Uniqueness $(\uparrow)$ | Validity $(\uparrow)$ |
| --- | --- | --- | --- | --- | --- |
| CharVAE | 0.17 $\pm$ 0.08 | 0.78 $\pm$ 0.04 | 0.99 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 0.09 $\pm$ 0.01 |
| SMILES-LSTM | 0.93 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 0.98 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 0.96 $\pm$ 0.01 |
| GraphAF | 0.05 $\pm$ 0.00 | 0.67 $\pm$ 0.01 | 0.91 $\pm$ 0.01 | 0.91 $\pm$ 0.01 | 1.00 $\pm$ 0.00 |
| HierVAE | 0.50 $\pm$ 0.14 | 0.92 $\pm$ 0.00 | 0.96 $\pm$ 0.01 | 0.96 $\pm$ 0.01 | 1.00 $\pm$ 0.00 |
| MiCaM | 0.63 $\pm$ 0.02 | 0.94 $\pm$ 0.00 | 0.98 $\pm$ 0.00 | 0.98 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| JTVAE | 0.75 $\pm$ 0.01 | 0.93 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| PSVAE | 0.28 $\pm$ 0.01 | 0.84 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| MAGNet | 0.76 $\pm$ 0.00 | 0.95 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| MoLeR-5 | 0.40 $\pm$ 0.01 | 0.93 $\pm$ 0.01 | 0.99 $\pm$ 0.00 | 0.97 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| MoLeR-350 | 0.80 $\pm$ 0.01 | 0.98 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| MoLeR-2000 | 0.83 $\pm$ 0.00 | 0.97 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| DiGress | 0.65 $\pm$ 0.00 | 0.91 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 0.99 $\pm$ 0.00 | 0.85 $\pm$ 0.01 |
| EDM-SyCo | 0.85 $\pm$ 0.01 | 0.96 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 1.00 $\pm$ 0.00 | 0.88 $\pm$ 0.01 |

**Table 9: MOSES benchmark metrics on the ZINC250K dataset. We report mean and standard deviations across 3 random training runs for all methods and metrics.**
| Metric | Filters $(\downarrow)$ | IntDiv $(\uparrow)$ | IntDiv2 $(\uparrow)$ | QED $(\downarrow)$ | SA $(\downarrow)$ | logP $(\downarrow)$ | weight $(\downarrow)$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| CharVAE | 0.43 $\pm$ 0.02 | 0.88 $\pm$ 0.01 | 0.87 $\pm$ 0.01 | 0.06 $\pm$ 0.03 | 0.48 $\pm$ 0.13 | 0.87 $\pm$ 0.14 | 34.23 $\pm$ 07.05 |
| SMILES-LSTM | 0.59 $\pm$ 0.01 | 0.87 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.00 $\pm$ 0.00 | 0.04 $\pm$ 0.02 | 0.12 $\pm$ 0.01 | 02.62 $\pm$ 00.16 |
| GraphAF | 0.47 $\pm$ 0.03 | 0.93 $\pm$ 0.00 | 0.91 $\pm$ 0.00 | 0.22 $\pm$ 0.01 | 0.88 $\pm$ 0.10 | 0.41 $\pm$ 0.02 | 96.93 $\pm$ 04.99 |
| HierVAE | 0.58 $\pm$ 0.05 | 0.87 $\pm$ 0.01 | 0.86 $\pm$ 0.01 | 0.03 $\pm$ 0.00 | 0.19 $\pm$ 0.17 | 0.35 $\pm$ 0.21 | 18.97 $\pm$ 10.43 |
| MiCaM | 0.54 $\pm$ 0.02 | 0.87 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.08 $\pm$ 0.00 | 0.51 $\pm$ 0.03 | 0.20 $\pm$ 0.05 | 51.19 $\pm$ 06.58 |
| JTVAE | 0.57 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.01 $\pm$ 0.00 | 0.35 $\pm$ 0.01 | 0.29 $\pm$ 0.01 | 02.93 $\pm$ 00.07 |
| PSVAE | 0.85 $\pm$ 0.01 | 0.89 $\pm$ 0.00 | 0.88 $\pm$ 0.00 | 0.05 $\pm$ 0.00 | 1.16 $\pm$ 0.06 | 0.34 $\pm$ 0.01 | 38.26 $\pm$ 02.74 |
| MAGNet | 0.70 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.01 $\pm$ 0.00 | 0.13 $\pm$ 0.00 | 0.23 $\pm$ 0.01 | 14.52 $\pm$ 00.40 |
| MoLeR-5 | 0.57 $\pm$ 0.01 | 0.86 $\pm$ 0.00 | 0.85 $\pm$ 0.00 | 0.01 $\pm$ 0.00 | 0.11 $\pm$ 0.04 | 0.17 $\pm$ 0.10 | 10.67 $\pm$ 04.13 |
| MoLeR-350 | 0.56 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.01 $\pm$ 0.01 | 0.06 $\pm$ 0.01 | 0.13 $\pm$ 0.02 | 05.96 $\pm$ 01.16 |
| MoLeR-2000 | 0.55 $\pm$ 0.01 | 0.86 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.02 $\pm$ 0.00 | 0.08 $\pm$ 0.02 | 0.11 $\pm$ 0.00 | 07.62 $\pm$ 01.80 |
| DiGress | 0.77 $\pm$ 0.01 | 0.86 $\pm$ 0.00 | 0.86 $\pm$ 0.00 | 0.05 $\pm$ 0.00 | 0.09 $\pm$ 0.01 | 0.61 $\pm$ 0.02 | 32.51 $\pm$ 00.20 |
| EDM-SyCo | 0.59 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 0.003 $\pm$ 0.00 | 0.19 $\pm$ 0.01 | 0.06 $\pm$ 0.02 | 04.49 $\pm$ 00.22 |

### G.4 Conditional vs Unconditional Generation

In addition to the conditional generation experiment from the main text, we perform an extra experiment to evaluate our approach and compare it to the unconditional model, which does not consider the conditioning value. We consider four properties: (1) the penalized LogP score (PLogP), a compound’s solubility and synthetic accessibility; (2) QED, a compound’s drug-likeness; (3) DRD2, an estimate of the biological activity against dopamine type 2 receptors; (4) TPSA, a drug’s ability to permeate cells.

We generate 10,000 valid molecules for each property and report the mean absolute error between the targets and the estimated values for the generated molecules measured by RDKit. The results are shown in Table 10. Our guidance mechanism achieves an average improvement factor of more than 5.5 times compared to the unconditional model across all properties.

**Table 10: Mean absolute error between the targets and the property values of 10,000 generated molecules using regressor guidance, and the improvement factor compared to unconditional generation. We report standard deviation across 3 runs with different random seeds.**
|  | PLogP | QED | DRD2 | TPSA |
| --- | --- | --- | --- | --- |
| Unconditional | 2.2440 $\pm$ 0.0018 | 0.1530 $\pm$ 0.0002 | 0.0148 $\pm$ 0.0000 | 25.9017 $\pm$ 0.0206 |
| Guidance | 0.3557 $\pm$ 0.0042 | 0.0350 $\pm$ 0.0004 | 0.0051 $\pm$ 0.0001 | 03.0382 $\pm$ 0.0164 |
| Imp. Factor ($\uparrow$) | 6.3087 | 4.3714 | 2.9020 | 08.5253 |

### G.5 Conditional Generation with Conditional Diffusion Models and Regressor-Free Guidance

In this section, we benchmark additional approaches for conditional generation presented in Section 6. In the main text, we reported results with the regressor guidance algorithm on top of an unconditional diffusion model inspired by the classifier guidance algorithm (Dhariwal & Nichol, ). Here, we additionally test a conditional diffusion model, which is a simple extension to the unconditional model, obtained by adding the target property to the node features and training a new model (Hoogeboom et al., ), referred to as Conditional Diffusion in Table 11. Inspired by the classifier-free guidance algorithm (Ho & Salimans, ), we train another variant of the conditional model by dropping the target property 15% of the time during training. This allows us to sample from the same model in a conditional or unconditional way, and by combining the two modes, we can guide the generation process towards the target properties, similar to the regressor-guidance case. We refer to this case as Regressor-Free Guidance in Table 11. We additionally test guiding the conditional model with an external regressor model (Regressor Guidance Conditional). All results for these models are shown in Table 11. Note that we experimented with smaller models than the one reported in the previous experiment due to the computational costs associated with training and running these models. Therefore, these numbers are not directly comparable to the numbers from the previous experiment. However, note that the best model presented here, the Regressor Guidance Conditional, has an MAE of 0.36, which is very close to the number from the previous experiment (0.3581), even though the model here is smaller. We expect that guiding the conditional model performs better if compared properly. However, it has the drawback of training a new diffusion model for each target property, compared to using the same diffusion model and one regressor that predicts all properties of interest.

**Table 11: MAE between condition value $c$ and the property value computed on the generated molecules. The considered property in PLogP. $\bar{s}=\sqrt{1-\bar{\alpha}_{t}}s.$**
| Method | Score Function | MAE |
| --- | --- | --- |
| Unconditional | - | 2.24 |
| Conditional Diffusion | $\bm{\epsilon}_{\theta}({\bm{z}}_{t},t,c)$ | 0.60 |
| Regressor-Free Guidance | $(s+1)\bm{\epsilon}_{\theta}({\bm{z}}_{t},t,c)+s\bm{\epsilon}_{\theta}({\bm{z}}_{t},t)$ | 0.48 |
| Regressor Guidance Conditional | $\bm{\epsilon}_{\theta}({\bm{z}}_{t},t,c)+\bar{s}\nabla_{{\bm{z}}_{t}}(g_{\eta}({\bm{z}}_{t},t)-c)^{2}$ | 0.36 |

### G.6 Unconstrained Molecule Optimization

This task aims to generate molecules from scratch with as high property values as possible without any other constraints. We use this task to demonstrate the performance of the proposed guidance mechanism for property maximization, described in Equation 8. Similarly to the conditional generation task, we start from Gaussian noise and run the modified sampling algorithm to generate molecules with high properties. Table 12 shows the top-3 scores of our model’s generated molecules and for all baselines from (Kong et al., ). Note that we achieved the best scores compared to all these baselines, even though we only generated 1,000 molecules compared to 10,000 molecules generated by these baselines. This illustrates the high performance of our method and motivates its use for the harder task of constrained optimization.

**Table 12: Top-3 property values for molecules generated in the unconstrained optimization task. Results for all baselines are taken from (Kong et al., ). All baselines generate 10,000 molecules, while we only generate 1,000 for this task.**
| Method | PLogP |  | QED |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1st | 2nd | 3rd |  | 1st | 2nd | 3rd |  |
| JT-VAE | 5.30 | 4.93 | 4.49 |  | 0.925 | 0.911 | 0.910 |
| GCPN | 7.98 | 7.85 | 7.80 |  | 0.948 | 0.947 | 0.946 |
| MRNN | 8.63 | 6.08 | 4.73 |  | 0.844 | 0.796 | 0.736 |
| GraphAF | 12.23 | 11.29 | 11.05 |  | 0.948 | 0.948 | 0.947 |
| GraphDF | 13.70 | 13.18 | 13.17 |  | 0.948 | 0.948 | 0.948 |
| GA | 12.25 | 12.22 | 12.20 |  | 0.946 | 0.944 | 0.932 |
| MARS | 7.24 | 6.44 | 6.43 |  | 0.944 | 0.943 | 0.942 |
| FREED | 6.74 | 6.65 | 6.42 |  | 0.920 | 0.919 | 0.908 |
| H-VAE | 11.41 | 9.67 | 9.31 |  | 0.947 | 0.946 | 0.946 |
| F-VAE | 13.50 | 12.62 | 12.40 |  | 0.948 | 0.948 | 0.947 |
| PS-VAE | 13.95 | 13.83 | 13.65 |  | 0.948 | 0.948 | 0.948 |
| EDM-SyCo | 14.73 | 14.31 | 14.21 |  | 0.948 | 0.948 | 0.948 |

### G.7 Fragment Occurences

In this section, we test our method’s ability to generate molecules containing fragments of different occurrence frequencies in the training set and compare it with several baselines. Table 13 shows the results of this experiment. Even though our model is not motif-based, it can match the distribution of fragments very well.

**Table 13: Frequency of occurrence of different fragments computed on the generated molecules from different methods. Methods are split into SMILES-based, graph-based motif-level, and graph-based atom-level. The method that achieves the closest percentage to that in the training set is highlighted. SyCo can successfully reproduce the percentages of different fragments in its generated molecules without relying on motifs.**
| Fragment |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| training | 76.47% | 14.15% | 9.37% | 1.19% | 1.09% | 0.97% | 0.39% | 0.27% |
| CHARVAE | 52.98% | 4.06% | 1.09% | 0.13% | 0.40% | 2.31% | 1.21% | 0.00% |
| SM-LSTM | 75.73% | 14.02% | 9.73% | 1.01% | 1.09% | 1.10% | 0.34% | 0.24% |
| PS-VAE | 71.31% | 13.08% | 5.58% | 2.84% | 1.07% | 2.61% | 0.27% | 0.12% |
| HierVAE | 75.78% | 12.80% | 7.92% | 0.71% | 0.45% | 0.56% | 0.03% | 0.00% |
| MiCaM | 79.47% | 11.37% | 5.74% | 0.79% | 0.94% | 2.16% | 0.37% | 0.06% |
| JT-VAE | 80.15% | 13.84% | 9.70% | 1.99% | 1.07% | 1.48% | 0.60% | 0.41% |
| MAGNet | 69.01% | 11.54% | 7.58% | 0.99% | 1.15% | 0.87% | 0.23% | 0.19% |
| MoLeR | 78.47% | 15.12% | 7.85% | 1.26% | 0.64% | 1.24% | 0.25% | 0.21% |
| GCPN | 53.37% | 2.03% | 0.33% | 0.44% | 3.53% | 0.38% | 0.89% | 0.13% |
| GraphAF | 34.16% | 1.27% | 2.29% | 0.11% | 0.29% | 0.25% | 0.03% | 0.00% |
| DiGress | 83.96% | 11.63% | 8.03% | 1.18% | 0.52% | 1.49% | 0.38% | 0.10% |
| EDM-SyCo | 75.57% | 12.84% | 9.17% | 1.02% | 0.56% | 1.89% | 0.33% | 0.16% |

### G.8 Model Runtime

To understand the computational cost associated with our model, we report the average runtime for training and inference in Table 14. On the ZINC250K dataset, our model is trained for 1000 epochs, which takes approximately 10 days (considering intermediate evaluations every 10 epochs), with an average of 740 seconds for training per epoch. During sampling, our model greatly benefits from running batch-wise and takes, on average, 86 seconds to generate a batch of 100 molecules, which corresponds to an amortized runtime of 0.86 seconds per single molecule. Generating molecules conditioned on target values further increases the runtime because it requires running one forward and one backward pass of the regressor model at each diffusion step, increasing the average runtime to 1.8 seconds per molecule. All numbers were computed using a single Nvidia A100 GPU.

**Table 14: Average runtime of EDM-SyCo. Numbers are on the ZINC250K dataset, the training set contains 220K molecules, and the sampling times are obtained using a batch size of 100.**
|  | Runtime |
| --- | --- |
| Training (Epoch) | 740 second / epoch |
| Training (Total) | $\sim$ 10 days |
| Unconditional Sampling | 0.86 second / molecule |
| Conditional Sampling (Guidance) | 1.8 second / molecule |

### G.9 Sampling Speed versus Generation Quality Trade-off

Our diffusion model is trained with $T=1000$ diffusion steps, which can be a limiting factor in some practical applications due to the resulting slow sampling process. One simple way to accelerate this process is to run a strided sampling schedule Nichol & Dhariwal (). The key idea is that to reduce the sampling steps from $T$ to $S$, we run the denoising network (Equation 6) every $\lceil T/S\rceil$ steps and update the noise schedule accordingly Nichol & Dhariwal (). In this experiment, we evaluate the quality of the molecules generated by running the model, initially trained with $T=1000$ steps, using only $500$, $250$, $200$, and $100$ steps evenly spaced between 1 and $T$ (without any additional training).
Figure 9 shows that the model performs well even with only 200 steps, which offers 5x speedup (if we neglect the cost of running the decoder, which is less than 1 step of reverse diffusion).

Figure: Figure 9: Effect of varying the number of reverse diffusion steps on the performance, using the same model trained for 1000 steps.
Refer to caption: x17.png

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: What is the KL divergence references at the end of page 8? The description "a variety of physiochemical descriptors" is too vague. Sorry if I missed a more precise definition elsewhere.
Is there a reason the results for "Similarity-Constrained Optimization" are not presented in more detail in the main paper? This is by far the most real-world useful application, and the one which has the highest probability of showing signal on your hypothesis (in my opinion).
soundness: 3
strengths: The model and experiments are clearly described. The paper acknowledges the importance of generation tasks beyond vanilla distribution matching on the space of 2D molecular graphs. The experiments are chosen to attempt to shine light on the effectiveness of the 3D latent inductive bias hypothesis.

For guided generation, the comparable results with "Translation" methods are impressive.
summary: The paper introduces a novel 3D latent embedding scheme for 2D molecular graph generation. It presents a set of experiments which try to show that this inductive bias is helpful for a set of 2D molecule generation tasks.
weaknesses: The experiments are ambiguous on whether this inductive bias is, indeed, helpful. 

It appears from table 3 that results on the ZINC250K dataset are saturated. The FCD and KL values reported in Table 4 are interesting, but presented without much interpretation. Without prior expertise it is difficult to evaluate FCD scores. Furthermore, de novo generation is not a terribly useful task in practice.

The most compelling experiment presented in the paper is "Similarity-Constrained Optimization". I could see how a 3D latent would improve performance in this area, but the data presented for this section is scant.

Overall, none of the experiments prove or disprove the inductive bias hypothesis of the architecture.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: - How does your model handle chirality, since the diffusion model is E(3)-equivariant? This could motivate the use of SE(3)-equivariant GNNs instead.
- Section 6.2: Please report how the normalization of the FCD and KL divergence metrics was performed for clarity.
- How is the novelty metric so high? This is very different from the 3D molecular graph generation setting (Appendix C of https://arxiv.org/abs/2203.17003) where the novelty usually reduces as training progresses. (This might be because the datasets for 3D molecular generation such as QM9 are the exact enumeration of all compounds with a certain property. However, it is surprising to see a novelty score of 100% in any case, indicating that none of the training molecules were sampled.)
- Some experiments on generalization would be very nice to see; can you train only on smaller graphs but generate much larger ones?
- Line 503 has a typo: “... factors 2.7 ad 15.6”
- Analysis of invalid molecules would be good, especially to motivate future work.
- Comparison of training and sampling time to other models would be nice, to understand the tradeoff between compute and performance better.
soundness: 3
strengths: - The paper is very well-written and full of details which are hard to find in most papers on this topic.
- The overall idea is definitely interesting and novel; it matches up with geometric intuition that molecules naturally live in 3D space.
- The authors have definitely improved results for non-autoregressive models on ZINC250K.
- Experiments on inpainting and conditional generation are creative, and quite comprehensive.
summary: This paper proposes a new method for 2D molecular graph generation using 3D molecular diffusion models, using the following steps:
- Train an autoencoder on paired (2D molecular graph, latent 3D molecular graphs).
  - The encoder consists of two parts:
       - First, 3D coordinates for all atoms are obtained using ETKDG, an well-known 3D conformation generation algorithm. No learning is required here.
       - Then, an EGNN is used to map the 3D molecular graph to a latent 3D graph, with updated node features and coordinates. The idea here is to incorporate neighborhood information for easier decoding of atom and bond types (as seen in Appendix G.1).
   - The decoder consists of two MLPs: one to predict atom types, and another to predict the bonding between the atoms.
The encoder and decoder are trained in the standard variational (VAE) framework.
* Train a diffusion model to generate latent 3D molecular graphs.
Here, the authors use the well-known EDM model (Hoogeboom, et al. 2022).
While the task is similar to that of standard 3D molecular graph generation, here the latent 3D molecular graphs do not necessarily correspond to anything physical)
* To sample:
    - Run the diffusion model, starting from noise, to generate latent 3D molecular graphs.
    - Run the decoder on the latent 3D molecular graphs to get 2D molecular graphs.

First, the authors compare their framework (EDM-SyCO) to GeoLDM, a 3D molecular graph generation algorithm combined with a simple bond prediction algorithm to create 2D molecular graphs. They show that the performance drops significantly. However, it is unclear whether the performance drop is due to the inaccurate bond prediction, or the use of standard 3D molecular graphs instead of the author’s latent 3D molecular graphs.

Next, the authors compare their model to autoregressive baselines on the well-established ZINC250K and GuacaMol datasets. They find that their model outperforms existing all-at-once 2D graph generative models. However, the performance improvement over state-of-the art autoregressive models is unclear, especially on the larger GuacaMol dataset.

Overall though, this is an interesting idea for 2D molecular graph generation. My only real concern is that the utility of the EGNN latent space is not conclusively shown; I have mentioned two experiments in the first bullet point of my Weaknesses section that I would love to see performed.

EDIT: I am happy with the author's response and additional experiments, and have increased my score in response.
weaknesses: - If the motivation for having an EGNN in the encoder is to incorporate neighborhood information for predicting atom and bond types for decoding, I would have liked a comparison to a setup where the EGNN is used as a decoder (over the simple MLPs). To summarize this scheme:
    - The encoder would just output the ETKDG-generated coordinates.
    - The decoder would be an EGNN + MLP to predict atom and bond types.
    - A diffusion model could then generate (as standard) 3D molecular graphs. In fact, any pretrained diffusion model could be used, in contrast to the author’s scheme here where the diffusion model must target the latent space of the EGNN encoder.

- In Section 6.1, I would have liked to see GeoLDM paired with a more sophisticated bond-determining algorithm such as RDKit's rdDetermineBonds (an implementation of xyz2mol) or OpenBabel. For example, as shown in Table 1 of https://openreview.net/pdf?id=MIEnYtlGyv, there can be significant variation in the results from each of these bond-determining algorithms. This could help prove the utility of the EGNN latent space for the diffusion model to target.

I would be willing to increase my score if the above experiments are performed convincingly.

- Training time seems high (~10 days on a single A100 GPU) relative to other methods; this is probably related to why the authors did not tune hyperparameters on the GuacaMol dataset. I would like a confirmation from the authors on this point.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please address the weakness mentioned above.
soundness: 3
strengths: - As molecules have 3D structures, using the 3D information to generate 2D molecular graphs seems intuitive and reasonable.

- The reconstruction accuracy ablation study on autoencoder shows the justification of using EGNN architecture.

- The framework can also be used for conditional generation/optimization of molecules in diverse settings, such as scaffold-constrained generation and similarity-constrained optimization.

- The experimental results show the effectiveness of this approach.
summary: This work presents a 2D molecule graph generation method that uses 3D molecule generative models by mapping 2D molecule graphs to 3D point clouds via synthetic coordinates and learning the inverse mapping using EGNN. Based on EDM, the proposed method shows superior generation quality.
weaknesses: Although using latent Euclidean point clouds instead of a simple 2D graph structure seems to be a reasonable and effective way of modeling molecular graphs, it is likely that this approach is not scalable with respect to the molecule size. Working with latent point clouds would likely require far higher computational costs and memory compared to previous diffusion models. Especially, it seems that learning the 3D information of large molecule structures would be challenging compared to learning the 2D graph distribution. Is there any experiments or explanations on this issue?

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - The authors should clarify what the error bars in Table 3 represent. Standard deviations across 3 runs of a model, as in Table 4?
soundness: 3
strengths: - SyCO's simplicity and versatility should encourage more development in the intersection of 2D and 3D graph generation models.
- SyCO's autoencoding reconstruction performance for bond type prediction is compelling.
- The authors provide a succinct and well-motivated case study of how to add atoms to a molecule during molecular optimization via diffusion generation.
- The authors' classifier guidance is well explained and motivated for property-specific molecule generation and optimization.
- The authors' code is well documented.
summary: **Summary:** The authors present SyCO as a straightforward method for generating 2D graphs using 3D generative models.

**Recommendation:** Given the notable strides yet remaining work to be clarified, discussed, or explored in this manuscript, I am currently recommending a weak reject.

**Rationale behind Recommendation:** If the authors can discuss some of the empirical benchmarking shortcomings of their model (e.g., what future steps might look like) and also include a small sample of experiments using a geometric neural network architecture other than EGNNs (to verify that the authors' ideas generalize beyond this architecture), I will consider raising my score.
weaknesses: - SyCO borrows most of its diffusion methodology from previous works [1, 2], to the best of my knowledge with adding nodes during inpainting being the only notable example to the contrary.
- Although it is simple to start with, the EDM architecture is no longer state-of-the-art for 3D molecule generation [3], suggesting the authors' results could be improved with a different choice of e.g., geometric GNN architecture. The authors seem to have a taken a wide approach to benchmarking rather than a deeper approach to seeing which geometric GNNs can best encode 2D graphs as latent 3D point clouds. This latter area of investigation could strengthen this work notably.
- The authors' conditional generation results for LogP in Table 5 and diversity throughout Table 6 seem a bit underwhelming. Besides attributing this as a flaw of molecule denoising networks, it would be helpful for the authors to expound how they think these results might be improved over time.
- The authors should discuss the pros and cons of relying on RDKit for conformer generation. For example, are there inherent biases in its distance-based coordinates estimation algorithm that might limit or impact the behavior of molecule generative models trained on its coordinate embeddings?

**References:**
1. Hoogeboom, Emiel, et al. "Equivariant diffusion for molecule generation in 3d." International conference on machine learning. PMLR, 2022.
2. Du, Yuanqi, et al. "Machine learning-aided generative molecular design." Nature Machine Intelligence (2024): 1-16.
3. Morehead, Alex, and Jianlin Cheng. "Geometry-complete diffusion for 3D molecule generation and optimization." Communications Chemistry 7.1 (2024): 150.

Meta-review: **Summary**

This paper introduces SyCO, a novel method for 2D molecular graph generation that leverages 3D latent embeddings. The approach involves mapping 2D molecular graphs to 3D point clouds using synthetic coordinates generated via the ETKDG algorithm, and then learning a latent 3D representation using EGNN. It aims to incorporate 3D inductive biases into the generation process, potentially improving performance on various molecule generation tasks.

**Strengths and Weaknesses**

The strengths of the paper include its novel and intuitive approach of using 3D latent embeddings for 2D molecular graph generation, leveraging the natural geometric properties of molecules. The method is well-presented, with clear descriptions and comprehensive experiments that demonstrate its effectiveness in improving results over existing non-autoregressive models on datasets like ZINC250K. 
However, there are some small issues regarding to the experiments, such as the empirical verification of the 3D inductive bias,  scalability  for larger molecules and performance improvement. 

**Final Decision**

Based on the reviewers' assessments and the strengths of the paper, I recommend the acceptance of this paper.

Decision: Accept (Poster)