Title: Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation
Venue: NeurIPS 2024
Accepted: yes
Keywords: molecular conformer generation

diffusion models

hierarchical models
Abstract: How can diffusion models process 3D geometries in a coarse-to-fine manner, akin to our multiscale view of the world?
In this paper, we address the question by focusing on a fundamental biochemical problem of generating 3D molecular conformers conditioned on molecular graphs in a multiscale manner. 
Our approach consists of two hierarchical stages: i) generation of coarse-grained fragment-level 3D structure from the molecular graph, and ii) generation of fine atomic details from the coarse-grained approximated structure while allowing the latter to be adjusted simultaneously.
For the challenging second stage, which demands preserving coarse-grained information while ensuring SE(3) equivariance, we introduce a novel generative model termed Equivariant Blurring Diffusion (EBD), which defines a forward process that moves towards the fragment-level coarse-grained structure by blurring the fine atomic details of conformers, and a reverse process that performs the opposite operation using equivariant networks.
We demonstrate the effectiveness of EBD by geometric and chemical comparison to state-of-the-art denoising diffusion models on a benchmark of drug-like molecules.
Ablation studies draw insights on the design of EBD by thoroughly analyzing its architecture, which includes the design of the loss function and the data corruption process.
Codes are released at https://github.com/Shen-Lab/EBD.

Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation

Abstract

How can diffusion models process 3D geometries in a coarse-to-fine manner, akin to our multiscale view of the world? In this paper, we address the question by focusing on a fundamental biochemical problem of generating 3D molecular conformers conditioned on molecular graphs in a multiscale manner. Our approach consists of two hierarchical stages: i) generation of coarse-grained fragment-level 3D structure from the molecular graph, and ii) generation of fine atomic details from the coarse-grained approximated structure while allowing the latter to be adjusted simultaneously. For the challenging second stage, which demands preserving coarse-grained information while ensuring SE(3) equivariance, we introduce a novel generative model termed Equivariant Blurring Diffusion (EBD), which defines a forward process that moves towards the fragment-level coarse-grained structure by blurring the fine atomic details of conformers, and a reverse process that performs the opposite operation using equivariant networks. We demonstrate the effectiveness of EBD by geometric and chemical comparison to state-of-theart denoising diffusion models on a benchmark of drug-like molecules. Ablation studies draw insights on the design of EBD by thoroughly analyzing its architecture, which includes the design of the loss function and the data corruption process. Codes are released at https://github.com/Shen-Lab/EBD .

1 Introduction

The advancement of generative models to understand the multiscale properties of objects facilitates their application across a range of granularity levels, transcending individual scales. To enable generative models in processing multiscale structures, there has been a surge in hierarchical design methodologies across multiple domains, spanning from images [35, 40] to speech [20, 17]. These methods initially capture coarse-grained structures and subsequently generate finer details.

In the field of computer vision, recent efforts [16, 42, 2, 7] have yielded successful designs of coarse-to-fine generative frameworks for 2D pixels of images, leveraging denoising diffusion models that corrupt and restore the data by adding and removing noises [49, 50, 52, 15, 27]. Notably,

Dog

Coarse

Fine-grained

Condition

structure

structure

[42] generates images from blurred prior distributions (average of pixel intensities) motivated from the heat equation of partial differential equations as Fig. 1.

In the field of biochemistry and drug discovery, however, denoising diffusion models for 3D conformers of stable molecular structures have not yet taken advantage of coarse-to-fine multiscale frameworks. Current methods either disregard the scale hierarchy [46, 59, 19, 23, 60] or consider that in very limited ways [38, 41]. For instance, within the recent hierarchical method of unconditional conformer generation [38], a denoising diffusion model [19] is solely applied to generation of coarse-grained structure, without extending to coarse-to-fine generation.

The primary bottleneck in extending denoising diffusion models for molecular conformers to hierarchical designs is that random noise corrupts not only fine atomic details but also structural information of coarse-grained structures indiscriminately. To tackle this challenging problem, we exploit fragments that are frequently occurring substructures or functional groups in 2D molecular graphs. These fragments can be a promising candidate for the coarse-grained structural information in 3D geometry. Introducing fragments divides the generation process into two stages: i) generating coarse-grained structures represented by fragments, and ii) restoring fine atomic details from fragment structures. In the first stage of generating fragment coordinates from molecular graphs, we efficiently generate approximations of fragment structures comprising the center of mass and attributes of each fragment from a cheminformatics tool.

For the challenging second step of coarse-to-fine generation, we propose a novel diffusion model, Equivariant Blurring Diffusion (EBD) detailed as follows. Motivated from the blurring corruption of the heat equation [42], we design EBD to generate 3D molecular conformers from coarse-grained fragment approximated structures as Fig. 1, rather than from random noise. In our design of EBD, the forward process moves atom positions of conformers towards the center of mass of their respective fragments, while the reverse process restores full-atom details from the prior distribution of the 3D fragment structure. The blurring schedule we designed for EBD allows the diffusion model to focus on restoring fine atomic details while retaining coarse-grained information throughout the entire generative process. We validated our coarse-to-fine EBD model using a benchmark of drug-like molecules. We obtained superior results in conformer generation compared to the denoising diffusion model, even with 100 times fewer diffusion time steps.

The major contribution of this paper can be summarized as follows:

We design EBD which generates atomic details from coarse-grained estimation of fragment structures using equivariant networks, motivated by the blurring corruption of heat equation.

We propose a novel blurring scheduler and a revised loss function that significantly impacts performance, instead of directly applying those of existing image blurring diffusion model.

We conduct a thorough analysis of the effects of fragment granularity, data corruption methods, and loss reformulation. We obtained geometrically and chemically more plausible conformers compared to state-of-the-art denoising diffusion models.

2 Background

2.1 Blurring diffusion

The denoising diffusion models [49, 50, 52, 15, 27], which corrupt data by adding random noise and generate data through denoising, have significantly advanced across diverse domains [55, 54, 44]. Recently, a few works [2, 42, 7, 18] have introduced data corruption into the design space of diffusion models [26], going beyond random noise corruption in the vision domain.

Inverse Heat Dissipation Model (IHDM) [42] proposed a coarse-to-fine generation in the pixel space. Their forward process follows a partial differential equation of heat dissipation on grids:

where x represents the data on the grid and ∆ is the Laplacian operator. IHDM derived the solution of this equation at time step t , x t , using eigendecomposition of ∆ as:

where V T and Λ are discrete cosine transform and a diagonal matrix whose elements are eigenvalues of ∆ , respectively. As t → T , the eigenbasis of eigenvalue 0 only remains and this leads to the

convergence of pixel intensities to their average value. Based upon this blurring process, IHDM defined a forward process as:

which means that the state at t is equal to the data blurred until t with small amount of noise. Note that the function of data corruption B t was defined at a spectral space of eigenvalues Λ . Then, the reverse generative process was defined to deblur each state:

where the mean at t -1 is the result of a deblurring network µ θ and δ is the small amount of standard deviation for noise. As t approaches 0 , µ θ gradually restores fine details from coarse-grained information about pixel intensities by effectively deblurring state values. The loss was defined to minimize the distance between the result of deblurring network and less blurred state at randomly sampled t as:

IHDM was evaluated on image generation task using FID score [14], but its performance lagged behind that of denoising diffusion models. For instance, IHDM achieves an FID score of 18 . 96 while DDPM [15] have 3 . 17 on CIFAR-10 [30].

2.2 Equivariance

In this work, we consider the SE(3) group to address the roto-translational equivariance of molecular conformers [28, 19, 59]. A function f is equivariant to a group G if T g ( f ( x )) = f ( S g ( x )) holds for all g ∈ G , where T g , S g are transformations of the group element g . In our coarse-to-fine generative framework, the invariant prior distribution of coarse-grained structure represents the coordinates of fragments. Therefore, the design of the transition distribution and the loss function in our diffusion model need to ensure that the generated likelihood is invariant, so that the generated conformers are not affected by rotation or translation.

3 Methods

3.1 Problem definition

Suppose that we have a molecular graph G whose nodes V are n atoms with SE(3)-invariant features h a ∈ R n × d and edges E are inter-atomic bonds. Our objective is to generate an ensemble of 3D molecular conformers x a ∈ R n × 3 given G . Our hierarchical approach is in two stages. i) p ( x f | G ) : generating a coarse-grained 3D structure of fragment coordinates x f ∈ R m × 3 from G which was decomposed into m fragments, and ii) p ( x a | x f , G ) : the diffusion model generating fine atomic details x a ∈ R n × 3 conditioned on the generated fragment structure x f . To map between atoms and their respective fragments, we defined a mapping matrix M ∈ R n × m with M ik = 1 if the i -th atom belongs to the k -th fragment and 0 otherwise. Mx f makes each atom located at its respective fragment. On the other hand, M † x a results in fragment coordinates being the average of the coordinates of its constituent atoms, where M † is a pseudoinverse matrix of M ( M † M = I ).

3.2 Fragmentation and 3D fragment structures

We decompose a molecule G = ( V , E ) into m non-overlapping fragments { S k } m k =1 , where S k = ( V k , E k ) and V = ⋃ m k =1 V k , E = ⋃ m k =1 E k using Principal Subgraph (PS) [29]. Starting from all unique atoms in the fragment vocabulary S , PS iteratively merges neighboring fragments. The most frequent fragment among the newly merged fragments was added to the vocabulary at each iteration, which was repeated until the desired size of the vocabulary was reached. The smaller the size of fragment vocabulary, the finer fragments and detailed coarse-grained structures can be obtained. After fragmentation was finished, from the relation between {V k } m k =1 and V , the mapping matrix M can be constructed.

To generate the initial coordinates of fragments, we utilize RDKit distance geometry [5, 31], an efficient cheminformatics tool, instead of training an additional deep generative model. After generating initial atom coordinates ˆ x a ∼ p RDKit ( x a ) , we define the initial fragment coordinates x f as averages of their constituent atom coordinates, M † ˆ x a . Since the atom coordinates generated by

/

O

O

O

FragmentationofG

Fragments coordinatesf

O

O

O

Pe(xa-1 1xa,xf,G)

x =Mxf

xa

xa

x = Mxf

q(xg /x%, x)

x = (1-)x+≠Mxf.

Graphical model

Blurringschedule

RDKit are approximations of the ground truth conformer, the resulting fragment coordinates are also an approximation (thus need to be adjusted in the next stage in Sec. 3.3), which we denote as ˆ x f ∼ p RDKit ( x f ) . For fragment features h f ∈ R m × 3 , we define a 3 -dimensional vector as a frequency histogram of its constituent atom types based on their chemical properties, including hydrophobicity, hydrogen bond center, and negative charge center following [38].

The processes of fragment structure generation are illustrated in the step 1 of Fig. 2. Note that every step in this subsection does not harm the efficiency of our framework, as they can be completed before training our diffusion model and the process itself is efficient. To generate 5 distinct fragment coordinates ˆ x f for each of the 45 , 000 molecules in the training and validation set of the GEOM-Drug benchmark [1], it took 38 hours, averaging 3 . 04 seconds per molecule. The details of fragmentation and fragment vocabulary are demonstrated in Appendix C.1.

3.3 Equivariant blurring diffusion

In this subsection, we elaborate on the design of our diffusion model, Equivariant Blurring Diffusion (EBD), drawing inspiration from the principles of the heat equation. This model is designed to generate fine details of conformers x a , starting from a coarse-grained, approximate structure ˆ x f and a molecular graph G . We introduce a forward process and a data corruption function of blurring process in Sec. 3.3.1, a reverse process and a deblurring network to reach an SE(3)-invariant likelihood in Sec. 3.3.2, and a definition and reparameterization of an SE(3)-invariant loss function in Sec. 3.3.3. The overall scheme of EBD is illustrated in the step 2 of Fig. 2.

3.3.1 Forward process and blurring schedule

We define the data corruption of the forward process as a blurring operation that gradually shifts ground truth atom positions x a 0 ∼ q ( x a 0 ) to their corresponding fragment coordinates:

where f B is a deterministic blurring operator. Consequently, every atom will be positioned according to its fragment coordinates M ˆ x f in the prior fragment structure distribution.

When defining f B for the forward process, we cannot directly adopt the spectral blurring operator B t = V exp( -Λ t ) V T of IHDM [42] in Eq. (2) for the two reasons: i) For a single molecule, we need to calculate and decompose the fragment graph Laplacian { V k Λ k V T k } m k =1 for each fragment S k = ( V k , E k ) , unlike a single Laplacian operator per image in IHDM. Given the varying sizes and structures across fragments, it becomes challenging to uniformly adjust the movement of atoms across all fragments using a function of { Λ k } m k =1 in spectral space. ii) As t → T , the ground truth atom coordinates x a 0 will converge to the ground truth scaffold structure x f = B T x a 0 by spectral

graph theory [4]. However, there exists a mismatch between the ground truth coordinates x f and the approximation coordinates ˆ x f from RDKit in the generative processes. This distributional shift of the fragment structure can potentially harm the performance during the inference.

To circumvent these issues, we transition the space of the blurring operator from spectral domain to spatial domain while retaining the essence of the blurring process. We define f B as a linear interpolation between M ˆ x f and x a 0 in Euclidean space:

As t progresses from 0 to T , the atom coordinates x a t will gradually converge to the fragment structure M ˆ x f , allowing for uniform adjustment of atom movement. Additionally, we can mitigate the need for excessive eigendecomposition of the fragment graph Laplacian. The example of our blurring schedule on a single fragment is depicted in the step 2 of Fig. 2.

3.3.2 Reverse process and deblurring networks

The aim of the reverse process is to generate fine details at the atom-level from a prior distribution of 3D fragment structure p ( x a T ) = N ( x a T | M ˆ x f , δ 2 I ) that is roto-translational invariant to the group. Drawing upon proofs regarding the conditions for an invariant likelihood [28, 59], we develop the deblurring process on the zero center-of-mass subspace using equivariant transition distributions:

where µ θ is a parameterized mean function consisting of a deblurring network. To ensure equivariance in the transition distribution, we devise µ θ inspired by equivariant networks [45]. Our equivariant deblurring network updates invariant features of fragments and atoms h f , h a , (Eqs. (9, 10)), and the equivariant coordinates of atoms x a (Eq. (11)) by leveraging the hierarchical relationship between atoms and fragments. Let the i -th atom x a i belongs to the k -th fragment x f k , then the l -th layer of equivariant deblurring networks for fragment- and atom-level message passing and feature updates is defined as follows:

where x f k is the k -th row of M † x a t , ϕ are multilayer perceptrons, e a ij are inter-atomic bond types, and d a ,l ij = ∥ x a ,l i -x a ,l j ∥ are inter-atomic distances. We consider a complete graph for fragment-level interactions and expand the edge set by incorporating multi-hop and radius neighbors for atom-level interactions. The details of the deblurring networks are provided in the Appendix A.

3.3.3 Training

Following Eq. (5) of IHDM [42], our loss of previous deblurred state estimation can be defined as:

where ρ is the Kabsch algorithm [25] to obtain the optimal rotation matrix for alignment. Through alignment ρ between the prediction from µ θ and less blurred state f B ( x a 0 , ˆ x f , t -1) after translating both terms to the zero center-of-mass subspace, the loss function becomes invariant to the SE(3)transformation of the prediction.

However, we empirically observed that this previous state estimator generates unsatisfactory conformers, similar to the unsatisfactory FID scores observed in image generation of IHDM [42]. We conjectured the reason as the model limited to learn the locally small steps towards the ground truth distribution at each time step [7]. Thus, we reparameterize µ θ ( x a t , ˆ x f , G, t ) as

(1 -t -1 T ) f θ ( x a t , G, t ) + t -1 T M ˆ x f to make the deblurring network estimates the ground truth state x a 0 instead of the previous less blurred state via neural networks f θ :

The derivation of the new loss is detailed in Appendix B. By loss reparameterization, ρ aligns the prediction to the ground truth state. At time step t of the sampling process, after estimating ground truth ˜ x a 0 from x a t , the next state x a t -1 is computed from a deterministic blurring function f B using ˜ x a 0 . The training and sampling processes are provided in Algorithms 1, 2.

4 Related work

Hierarchical generation. A hierarchical design of generative models is evident across multiple domains, including image generation [35, 40, 16, 42] and speech synthesis [20, 17], aimed at enhancing the interpretability and quality of samples derived from coarse-grained information. In the field of computational biology, recent studies on molecular graph generation [21, 22, 12], backmapping of protein structure [61] and conformer generation [56] conditioned on the given ground truth coarse-grained information have reported the effectiveness of the hierarchical design. In recent unconditional conformer generation [38], a denoising diffusion model [19] was exclusively used in the fragment structure generation step and not designed for coarse-to-fine generation.

Data corruption in diffusion models. The choice of data corruption can be considered a crucial aspect of the design space of diffusion models [26], depending on the characteristics of the data domain and the specific problem definition. Recently, several studies on diffusion models have revealed that the choice of data corruption can be extended beyond random noise [49, 15, 50, 36, 43] to methods such as masking [2, 7, 8], blurring [42, 2, 7, 18], and varying data dimension [3]. We designed the data corruption process as a blurring operation in Euclidean space, transitioning from atom-level fine details to fragment-level coarse structures. This approach is more effective for multiscale frameworks compared to random noise, which corrupts both fragment and atom geometries.

5 Experiments

In this section, we evaluate our hierarchical molecular conformer generation framework via Equivariant Blurring Diffusion (EBD) on molecular conformer generation task. We conducted experiments to answer the following questions: i) Ablation studies (Sec. 5.2): What are the effects of granularity of the fragment vocabulary, loss reparameterization, and data corruption processes of diffusion models? ii) Geometric evaluation (Sec. 5.3): Can EBD generate more diverse and accurate molecular conformers in Euclidean space than previous deep generative approaches? iii) Property prediction (Sec. 5.4): Can EBD generate low-energy, stable conformers?

5.1 Experiment setup

Dataset. We use GEOM-QM9 (QM9) [39] and GEOM-Drugs (Drugs) [1] which are small molecules and drug-like molecules, respectively. Each dataset comprises 40,000 molecules for the training set and 5,000 molecules for the validation set, with each molecule containing 5 conformers following data split of [46]. For the test set, we selected 200 molecules for each dataset, resulting in 22,408 and 14,324 conformers existing in QM9 and Drugs, respectively. The details of dataset were demonstrated in Appendix C.1.

1.2

PS 1000

1.7

xg-1 est.

1.7

DecompDiff

1.125

PS200

1.5

x est.

1.5

EBD

PS 50

1.05

1.3

1.3

0.975

1.1

1.1

0.9

0.9

0.9

0.825

0.7

0.7

0.75

0.5

0.5

MAT-R

MAT-RMAT-P

MAT-P

MAT-R

RMAT-RMAT-P

MAT-P

MAT-R

MAT-R

MAT-P

MAT-P

mean

median

mean

median

mean

median

mean

median

mean

median

mean

median

(a) What level of granularity?

(b) What to estimate?

(c) How to corrupt?

Metrics. To measure the accuracy and diversity of the generated conformer set C , we adopted metrics proposed by [11]. The metrics are based on root-mean-square deviation (RMSD), which is a normalized Frobenius norm between two atomic coordinate matrices aligned using the Kabsch algorithm [25]. Given the ground truth conformer set C ∗ and the generated sample set C , four metrics that follow precision and recall are defined as:

where COV and MAT are coverage metric and matching metric [57], respectively. COV quantifies the proportion of one set covered by another, with 'covered' indicating RMSD values are within a threshold δ . MAT measures the average of RMSD values of one conformer set with its closest conformer in another set. If C and C ∗ are exchanged in Eqs. (15, 16), then metrics become COV-P (Precision) and MAT-P (Precision). The recall metric is focused on the diversity, while the precision metric measures the quality. The threshold δ is set to 0 . 5 Å for QM9 and 1 . 25 Å for Drugs. For each molecule, we generated conformers C that are twice the size of the ground truth conformers C ∗ .

Baselines. We compare EBD to existing deep generative models for molecular conformer generation. The performance of RDKit [31] that was used to generate the fragment structure of our model was measured as a baseline. Besides RDKit, machine learning models including CVGAE [34], GraphDG [47], CGCF [57], ConfVAE [58], GeoMol [11], ConfGF [46], and GeoDiff [59] were compared to our model. Among these, GeoDiff is the denoising diffusion model restoring the target distribution from random noise in atom coordinates. We adhered to their settings by configuring the maximum time step T to 5,000. For EBD, we use the T = 50 , a noise scale of 0 . 01 for the forward process ( σ in Eq. 6) and 0 . 0125 for the reverse process ( δ in Eq. 8) in every experiments. The implementation details were reported in Appendix C.2.

5.2 Ablation studies

We conducted ablation studies to validate our model design, encompassing the size of the fragment vocabulary, the reparameterization of loss, and the blurring data corruption. For each ablation study, we calculated the mean and median of matching scores MAT-R and MAT-P on Drugs test set. Note that lower values of MAT-R and MAT-P indicate better results.

Effects of fragment granularity. Weassessed the performance variation as fragment structure became more detailed and informative by measuring the generation performances across different fragment vocabulary sizes |S| ∈ { 50 , 200 , 1000 } . Since PS [29], the fragmentation method we used, initializes the vocabulary from unique single atoms, reducing the size |S| results in obtaining finer fragments ˆ x f . We reported the statistics of the average number of fragments per graph

( # frags /G ) and atoms per fragment ( # atoms / frag) in Table 1 and the generation results in Fig. 3 (a).

Thanks to the increased level of detail in fragments, |S| = 50 can obtain better performance compared to other vocabulary sizes. This is because more specific fragment structures decrease the amount of

atomic-level detail that needs to be generated. From this observation, we use |S| = 50 in all subsequent experiments. We also conducted fine-to-fine generation to observe the ability of EBD. We measured the performance when the prior is ˆ x a ∼ p RDKit ( x a ) . Table 2 presents coarse-to-fine (C2F) result when |S| = 50 and fine-to-fine (F2F) result. As expected, F2F shows more accurate results compared to C2F as it includes more detail in the prior distribution.

Effects of loss reparameterization. We presented the performance comparison between the less blurred previous state estimator in Eq. (12) and ground truth estimator in Eq. (14) after loss reprarameterization in Fig. 3 (b). From the previous state estimator, we acquired degenerated conformers with relatively high matching scores, which align with low FID score of IHDM [42] in image generation. On the other hand, we observed distinct advantages in introducing the ground truth estimator across all metrics. We speculate that the ground truth estimator facilitates the diffusion model in learning beyond locally blurring distributions towards the target distribution.

Effects of data corruptions. We provide the same initial fragment structures ˆ x f to both EBD and DecompDiff [13] so that the data corruption method becomes the primary distinction to examine. DecompDiff is a better candidate than GeoDiff [59] to compare for this purpose because, unlike GeoDiff that generates conformers from a single prior distribution, DecompDiff denoises multiple prior distributions, where each mean corresponds to the coordinates of each fragment ˆ x f . The generation results and sampling trajectories are compared between the two models ( T = 50 for both) in Fig. 3 (c) and Fig. 4. At first, we observed that the conformers generated from DecompDiff exhibit lower diversity scores compared to EBD. This is because the results of DecompDiff tend to adhere closely to the approximate fragment structure ˆ x f , whereas EBD attempts to transition towards the ground

DecompDiff

t=50

40

30

20

10

0

B

D

truth fragment structure x f . We speculate that our blurring schedule, which entails a linear interpolation between M ˆ x f and x a 0 , facilitates the learning process for the diffusion model compared to a stochastic trajectory between prior and target distributions. As empirical evidence, we observed that DecompDiff primarily focuses on denoising the fragment structure throughout most of the sampling process in Fig. 4. On the other hand, EBD focuses on the entirety of the sampling process to generate fine details, resulting in better quality.

We conducted further analysis to observe how accurately the fragment coordinates generated by RDKit were corrected toward the ground truth by EBD. For 200 molecules in Drugs test set, we measured RMSD ( x f RDKit , x f gt ) and RMSD ( x f EBD , x f gt ) . If RMSD ( x f EBD , x f gt ) is lower than RMSD ( x f RDKit , x f gt ) for a molecule, then it indicates that the model has corrected the fragment coordinates. In Fig. 5, the points below the red line represent cases where the model corrected the coordinates accurately. We observed that the greater the RMSD ( x f RDKit , x f gt ) (points further to the right on the x-axis), the larger the reduction in RMSD ( x f EBD , x f gt ) . In other words, the lower the quality of the coarse-grained prior, the more accurately the model tends to make corrections.

RMSD(XEBD,X

3

1

RMSD(xRDKitxt)

2

3

4

S

b

Additional visualizations of sampling results on Drugs and QM9, as well as the sampling processes, are illustrated in the Appendix E.

5.3 Geometric evaluation

We compared our hierarchical framework to the baseline RDKit and machine learning models for molecular conformer generation on Drugs, and the results are reported in Table 3. EBD achieves superior performance across all metrics by generating diverse and accurate molecular conformers. In comparison to RDKit, which was used to generate fragment structures ˆ x f , EBD achieved a significant improvement in the generation of diverse fine atomic details, as evidenced by higher COV-R and MAT-R scores. We also observed that, due to the informative fragment structure prior distribution and the proposed blurring schedule, EBD produces more diverse and higher-quality conformers with statistical significance (see Appendix D), even with 100 times fewer T compared to GeoDiff. We also reported EBD's better performance on QM9 with statistical significance in Appendix D.

GeoDiff

EBD

10-

Power spectral density

Power spectral density

Forward

Reverse

Forward

Reverse

10-1

10-1

102

10-3

10-3

10-1

100

10-1

100

Eigenvalue

Eigenvalue

We take an additional step to delve into the rationale behind the ability of EBD to achieve better performance with smaller T by analyzing the spectral domain in Fig 6. Motivated from the analysis of [42], we calculated the power spectral density, PSD ( x a t ) i = 1 3 ∑ c ∈{ x,y,z } | V T i x a ,c t | 2 , where V i is the i -th smallest eigenvector of graph Laplacian ∆ of the molecular graph G , and x a ,c t is one of the { x, y, z } coordinate vectors of atomic coordinate matrix x a t . The smaller eigenvalues correspond to coarser-grained structures, while higher eigenvalues correspond to finer details [4]. Therefore, by measuring PSD ( x a t ) i during the forward processes { x a t } T t =0 of EBD and GeoDiff, we can ascertain which structural components are corrupted during the forward process and will be restored during the reverse process, respectively. For EBD, there is less significant perturbation across lower frequency parts of the PSD that corresponds to the coarser-grained discrepancy between ground truth x f and approximate ˆ x f fragment positions. Thus, EBD primarily focuses on restoring perturbed fine-grained structures in the higher frequency parts throughout the entire generative process. This explains why EBD does not require excessive T . In contrast, GeoDiff requires relatively more T because random noise corrupts the overall structural information, and the amount of perturbations is also significant.

5.4 Chemical evaluation

In addition to geometric evaluation, we assessed the quality of generated conformers by their chemical properties. After training EBD on QM9, following [46], we generated 50 samples for each of the 30 molecules, which constitute a subset of QM9. Using PSI4 [48], we calculated properties of each conformer including the energy E , the lowest energy E min, HOMO-LUMO gap ϵ , the average gap ∆ ϵ , the minimum gap ∆ ϵ min and the maximum gap ∆ ϵ max. Then, we

measured the mean absolute errors between the properties of generated and ground truth (Table. 4). We observed that EBD can generate the most stable conformers compared to other methods, as evidenced by lower energy and HOMO-LUMO gap.

6 Conclusion

We introduced a novel hierarchical generative model for molecular conformers via Equivariant Blurring Diffusion (EBD), a diffusion model designed for coarse-to-fine generative scheme. After generating the initial distribution of fragment coordinates from a cheminformatics tool, EBD generated fine atomic details from coarse-grained structures through equivariant networks. We also proposed a simple and effective linear blurring scheduler and ground truth state estimator to enhance the model's ability to produce diverse and accurate conformers. Through extensive analysis of the proposed model and comparison between competitive denoising diffusion models, we substantiated the validity of the model design.

As a coarse-to-fine generative scheme, EBD can be extended for larger and more complex molecular structures, because hierarchical systems similar to those utilized in EBD exists widely across molecular systems (ranging from proteins as linear polymers of amino acids to materials as lattices of molecules). We discussed a few limitations of our model in the Appendix F.

Acknowledgments and Disclosure of Funding

We thank Yuning You and Hoseok Do for valuable feedback on the manuscript. This work was partially funded by the U.S. National Science Foundation under grant CCF-1943008. Portions of this research were conducted with the advanced computing resources provided by Texas A&M High Performance Research Computing.

References

Axelrod, Simon and Gomez-Bombarelli, Rafael. Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data , 9(1):185, 2022.

Bansal, Arpit, Borgnia, Eitan, Chu, Hong-Min, Li, Jie, Kazemi, Hamid, Huang, Furong, Goldblum, Micah, Geiping, Jonas, and Goldstein, Tom. Cold diffusion: Inverting arbitrary image transforms without noise. Advances in Neural Information Processing Systems , 36, 2023.

Campbell, Andrew, Harvey, William, Weilbach, Christian, De Bortoli, Valentin, Rainforth, Thomas, and Doucet, Arnaud. Trans-dimensional generative modeling via jump diffusion models. Advances in Neural Information Processing Systems , 36, 2023.

Chung, Fan RK. Spectral graph theory , volume 92. American Mathematical Soc., 1997.

Crippen, Gordon M, Havel, Timothy F, et al. Distance geometry and molecular conformation , volume 74. Research Studies Press Taunton, 1988.

Crouse, David F. On implementing 2d rectangular assignment algorithms. IEEE Transactions on Aerospace and Electronic Systems , 52(4):1679-1696, 2016.

Daras, Giannis, Delbracio, Mauricio, Talebi, Hossein, Dimakis, Alexandros, and Milanfar, Peyman. Soft diffusion: Score matching with general corruptions. Transactions on Machine Learning Research , 2023.

Daras, Giannis, Shah, Kulin, Dagan, Yuval, Gollakota, Aravind, Dimakis, Alex, and Klivans, Adam. Ambient diffusion: Learning clean distributions from corrupted data. Advances in Neural Information Processing Systems , 36, 2023.

Degen, Jorg, Wegscheid-Gerlach, Christof, Zaliani, Andrea, and Rarey, Matthias. On the art of compiling and using'drug-like'chemical fragment spaces. ChemMedChem , 3(10):1503, 2008.

Du, Weitao, Zhang, He, Du, Yuanqi, Meng, Qi, Chen, Wei, Zheng, Nanning, Shao, Bin, and Liu, Tie-Yan. Se (3) equivariant graph neural networks with complete local frames. In International Conference on Machine Learning , pp. 5583-5608. PMLR, 2022.

Ganea, Octavian, Pattanaik, Lagnajit, Coley, Connor, Barzilay, Regina, Jensen, Klavs, Green, William, and Jaakkola, Tommi. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems , 34, 2021.

Geng, Zijie, Xie, Shufang, Xia, Yingce, Wu, Lijun, Qin, Tao, Wang, Jie, Zhang, Yongdong, Wu, Feng, and Liu, Tie-Yan. De novo molecular generation via connection-aware motif mining. In The Eleventh International Conference on Learning Representations , 2023.

Guan, Jiaqi, Zhou, Xiangxin, Yang, Yuwei, Bao, Yu, Peng, Jian, Ma, Jianzhu, Liu, Qiang, Wang, Liang, and Gu, Quanquan. Decompdiff: Diffusion models with decomposed priors for structurebased drug design. In International Conference on Machine Learning , pp. 11827-11846. PMLR, 2023.

Heusel, Martin, Ramsauer, Hubert, Unterthiner, Thomas, Nessler, Bernhard, and Hochreiter, Sepp. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems , 30, 2017.

Ho, Jonathan, Jain, Ajay, and Abbeel, Pieter. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33, 2020.

Ho, Jonathan, Saharia, Chitwan, Chan, William, Fleet, David J, Norouzi, Mohammad, and Salimans, Tim. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research , 23(47):1-33, 2022.

Hono, Yukiya, Tsuboi, Kazuna, Sawada, Kei, Hashimoto, Kei, Oura, Keiichiro, Nankaku, Yoshihiko, and Tokuda, Keiichi. Hierarchical multi-grained generative model for expressive speech synthesis. Interspeech 2020 , 2020.

Hoogeboom, Emiel and Salimans, Tim. Blurring diffusion models. In The Eleventh International Conference on Learning Representations , 2023.

Hoogeboom, Emiel, Satorras, Vıctor Garcia, Vignac, Clément, and Welling, Max. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning , pp. 8867-8887. PMLR, 2022.

Hsu, Wei-Ning, Zhang, Yu, Weiss, Ron, Zen, Heiga, Wu, Yonghui, Cao, Yuan, and Wang, Yuxuan. Hierarchical generative modeling for controllable speech synthesis. In International Conference on Learning Representations , 2019.

Jin, Wengong, Barzilay, Regina, and Jaakkola, Tommi. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning , pp. 2323-2332. PMLR, 2018.

Jin, Wengong, Barzilay, Regina, and Jaakkola, Tommi. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learning , pp. 48394848. PMLR, 2020.

Jing, Bowen, Corso, Gabriele, Chang, Jeffrey, Barzilay, Regina, and Jaakkola, Tommi. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems , 35, 2022.

Joshi, Chaitanya K, Bodnar, Cristian, Mathis, Simon V, Cohen, Taco, and Lio, Pietro. On the expressive power of geometric graph neural networks. In International Conference on Machine Learning , pp. 15330-15355. PMLR, 2023.

Kabsch, Wolfgang. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography , 32 (5):922-923, 1976.

Karras, Tero, Aittala, Miika, Aila, Timo, and Laine, Samuli. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems , 35, 2022.

Kingma, Diederik, Salimans, Tim, Poole, Ben, and Ho, Jonathan. Variational diffusion models. Advances in Neural Information Processing Systems , 34, 2021.

Köhler, Jonas, Klein, Leon, and Noé, Frank. Equivariant flows: exact likelihood generative learning for symmetric densities. In International Conference on Machine Learning , pp. 53615370. PMLR, 2020.

Kong, Xiangzhe, Huang, Wenbing, Tan, Zhixing, and Liu, Yang. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems , 35, 2022.

Krizhevsky, Alex, Hinton, Geoffrey, et al. Learning multiple layers of features from tiny images. 2009.

Landrum, Greg et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum , 8(31.10):5281, 2013.

Lewell, Xiao Qing, Judd, Duncan B, Watson, Stephen P, and Hann, Michael M. Recap retrosynthetic combinatorial analysis procedure: a powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of chemical information and computer sciences , 38(3):511-522, 1998.

Loshchilov, Ilya and Hutter, Frank. Decoupled weight decay regularization. In International Conference on Learning Representations , 2019.

Mansimov, Elman, Mahmood, Omar, Kang, Seokho, and Cho, Kyunghyun. Molecular geometry prediction using a deep generative graph neural network. Scientific reports , 9(1):20381, 2019.

Menick, Jacob and Kalchbrenner, Nal. Generating high fidelity images with subscale pixel networks and multidimensional upscaling. In International Conference on Learning Representations , 2019.

Nichol, Alexander Quinn and Dhariwal, Prafulla. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning , pp. 8162-8171. PMLR, 2021.

Paszke, Adam, Gross, Sam, Chintala, Soumith, Chanan, Gregory, Yang, Edward, DeVito, Zachary, Lin, Zeming, Desmaison, Alban, Antiga, Luca, and Lerer, Adam. Automatic differentiation in pytorch. 2017.

Qiang, Bo, Song, Yuxuan, Xu, Minkai, Gong, Jingjing, Gao, Bowen, Zhou, Hao, Ma, Wei-Ying, and Lan, Yanyan. Coarse-to-fine: a hierarchical diffusion model for molecule generation in 3d. In International Conference on Machine Learning , pp. 28277-28299. PMLR, 2023.

Ramakrishnan, Raghunathan, Dral, Pavlo O, Rupp, Matthias, and Von Lilienfeld, O Anatole. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data , 1(1):1-7, 2014.

Razavi, Ali, Van den Oord, Aaron, and Vinyals, Oriol. Generating diverse high-fidelity images with vq-vae-2. Advances in Neural Information Processing Systems , 32, 2019.

Reidenbach, Danny and Krishnapriyan, Aditi. Coarsenconf: Equivariant coarsening with aggregated attention for molecular conformer generation. In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop , 2023.

Rissanen, Severi, Heinonen, Markus, and Solin, Arno. Generative modelling with inverse heat dissipation. In The Eleventh International Conference on Learning Representations , 2023.

Rombach, Robin, Blattmann, Andreas, Lorenz, Dominik, Esser, Patrick, and Ommer, Björn. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 10684-10695, 2022.

Saharia, Chitwan, Chan, William, Saxena, Saurabh, Li, Lala, Whang, Jay, Denton, Emily L, Ghasemipour, Kamyar, Gontijo Lopes, Raphael, Karagol Ayan, Burcu, Salimans, Tim, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems , 35, 2022.

Satorras, Vıctor Garcia, Hoogeboom, Emiel, and Welling, Max. E (n) equivariant graph neural networks. In International Conference on Machine Learning , pp. 9323-9332. PMLR, 2021.

Shi, Chence, Luo, Shitong, Xu, Minkai, and Tang, Jian. Learning gradient fields for molecular conformation generation. In International Conference on Machine Learning , pp. 9558-9568. PMLR, 2021.

Simm, Gregor and Hernandez-Lobato, Jose Miguel. A generative model for molecular distance geometry. In International Conference on Machine Learning , pp. 8949-8958. PMLR, 2020.

Smith, Daniel GA, Burns, Lori A, Simmonett, Andrew C, Parrish, Robert M, Schieber, Matthew C, Galvelis, Raimondas, Kraus, Peter, Kruse, Holger, Di Remigio, Roberto, Alenaizan, Asem, et al. Psi4 1.4: Open-source software for high-throughput quantum chemistry. The Journal of chemical physics , 152(18), 2020.

Sohl-Dickstein, Jascha, Weiss, Eric, Maheswaranathan, Niru, and Ganguli, Surya. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning , pp. 2256-2265. PMLR, 2015.

Song, Jiaming, Meng, Chenlin, and Ermon, Stefano. Denoising diffusion implicit models. In International Conference on Learning Representations , 2020.

Song, Yang and Ermon, Stefano. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems , 32, 2019.

Song, Yang, Sohl-Dickstein, Jascha, Kingma, Diederik P, Kumar, Abhishek, Ermon, Stefano, and Poole, Ben. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations , 2020.

Thomas, Nathaniel, Smidt, Tess, Kearnes, Steven, Yang, Lusann, Li, Li, Kohlhoff, Kai, and Riley, Patrick. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219 , 2018.

Vahdat, Arash, Kreis, Karsten, and Kautz, Jan. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems , 34, 2021.

Vahdat, Arash, Williams, Francis, Gojcic, Zan, Litany, Or, Fidler, Sanja, Kreis, Karsten, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems , 35:10021-10039, 2022.

Wang, Wujie, Xu, Minkai, Cai, Chen, Miller, Benjamin K, Smidt, Tess, Wang, Yusu, Tang, Jian, and Gomez-Bombarelli, Rafael. Generative coarse-graining of molecular conformations. In International Conference on Machine Learning , pp. 23213-23236. PMLR, 2022.

Xu, Minkai, Luo, Shitong, Bengio, Yoshua, Peng, Jian, and Tang, Jian. Learning neural generative dynamics for molecular conformation generation. In International Conference on Learning Representations , 2021.

Xu, Minkai, Wang, Wujie, Luo, Shitong, Shi, Chence, Bengio, Yoshua, Gomez-Bombarelli, Rafael, and Tang, Jian. An end-to-end framework for molecular conformation generation via bilevel programming. In International Conference on Machine Learning , pp. 11537-11547. PMLR, 2021.

Xu, Minkai, Yu, Lantao, Song, Yang, Shi, Chence, Ermon, Stefano, and Tang, Jian. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations , 2022.

Xu, Minkai, Powers, Alexander S, Dror, Ron O, Ermon, Stefano, and Leskovec, Jure. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning , pp. 38592-38610. PMLR, 2023.

Yang, Soojung and Gómez-Bombarelli, Rafael. Chemically transferable generative backmapping of coarse-grained proteins. In International Conference on Machine Learning , pp. 3927739298, 2023.

Zhu, Jinhua, Xia, Yingce, Liu, Chang, Wu, Lijun, Xie, Shufang, Wang, Yusong, Wang, Tong, Qin, Tao, Zhou, Wengang, Li, Houqiang, et al. Direct molecular conformation generation. Transactions on Machine Learning Research , 2022.

A Deblurring network architectures

In SE(3)-equivariant deblurring networks, there are update functions of SE(3)-invariant fragment and atom features h f , h a , as well as an update function of SE(3)-equivariant atom coordinates x a motivated from equivariant graph neural networks [45]. For the fragments, we constructed a complete graph to account for dense interactions among them. In the case of atoms, we expanded the neighbor set of each atom by including multi-hop neighbors derived from the powers of the adjacency matrix and a radius graph, which includes atoms within a specified cutoff distance. The benefits of dense interactions for accurate conformers estimation have been confirmed in several studies [47, 59, 19].

The architecture of the SE(3)-invariant message passing and feature update functions at the fragmentand atom-level is as follows:

i

where m ij ∈ R d is the message for each interactions, and h ∈ R d is the feature vector from the aggregated messages and features from different hierarchy level. For every invariant update functions ϕ f m , ϕ f h , ϕ a m , ϕ a h , we used multilayer perceptrons. For initial features h f , 0 i of fragments, we defined a 3 -dimensional vector as a frequency histogram of its constituent atom types based on their chemical properties, including hydrophobicity, hydrogen bond center, and negative charge center following [38]. The detailed definition of the initial fragment features is in Table 5. For initial atom features h a , 0 i ∈ R d and bond features e a ij ∈ R d , we used embeddings from atom types and bond types, respectively.

For the i -th atom x a i belongs to the k -th fragment x f k , the architecture of the equivariant atom coordinate update function is as follows:

where x f k is the k -th row of M † x a t , and d a ,l ij = ∥ x a ,l i -x a ,l j ∥ are inter-atomic distances. For every equivariant update functions ϕ a x , ϕ f x , we used multilayer perceptrons. For three terms in right-hand side of Eq. (A.3), the first term is the coordinate from the previous layer, the second term is an equivariant update function that accounts for atom-level interactions, and the third term is an equivariant update function that considers the deviation of the current atom coordinate from its respective fragment's coordinate.

B Derivation of loss function

In this section, we explain the derivation of the loss function for the ground truth state estimator from the previous state estimator. The loss function of previous state estimation is defined as:

where ρ is the Kabsch algorithm [25] to obtain the optimal rotation matrix for alignment. Through alignment ρ of the prediction from µ θ to the less blurred state f B ( x a 0 , ˆ x f , t -1) after translating both terms to the zero center-of-mass subspace, the loss function becomes invariant to the translation and rotation of the prediction.

However, this previous state estimator generates unsatisfactory conformers as empirically observed in Sec. 5.2. We conjectured the reason as the model limited to learn the locally small steps towards the ground truth distribution at each time step [7]. Thus, we reparameterize µ θ ( x a t , ˆ x f , G, t ) as (1 -t -1 T ) f θ ( x a t , G, t ) + t -1 T M ˆ x f to make the deblurring network estimates the ground truth state x a 0 instead of the previous less blurred state via neural networks f θ . We first start with the non-invariant previous state estimation, which is without the alignment ρ :

In the last stage from Eq. (A.9) to Eq. (A.10), we simplified the loss function by discarding the time-dependent weight as [15]. Finally, we make the loss function for ground truth estimation invariant by aligning the prediction from f θ to the ground truth state using Kabsch alignment ρ [25].

C Implementation details

C.1 Datasets

We used GEOM-QM9 (QM9) [39] and GEOM-Drugs (Drugs) [1] for analysis and comparison between molecular conformer generation models. Each dataset comprises 40,000 molecules for the training set and 5,000 molecules for the validation set, with each molecule containing 5 conformers following data split of [46]. We obtained the raw data, the pre-processed data and the data split at https://github.com/DeepGraphLearning/ConfGF . For the test set, we selected 200 molecules for each dataset, resulting in 22,408 and 14,324 conformers existing in QM9 and Drugs, respectively.

For fragmentation of the molecular graphs G = ( V , E ) in Drugs and QM9, we used Principal Subgraph (PS) [29] ( https://github.com/THUNLP-MT/PS-VAE ) which can construct a fragment vocabulary S whose elements are the largest and frequent repetitive subgraphs of molecules. Starting from all unique atoms in S at initial stage, PS iteratively merges neighboring fragments. The most frequent fragment among the newly merged fragments was added to the vocabulary at each iteration, and this operation was repeated until the desired size of the vocabulary was reached. Thus, the smaller the fragment vocabulary, the finer fragments can be obtained. One of the advantages of PS compared to existing fragmentation methods such as RECAP [32], BRICS [9], junction tree decomposition [21] is the ability to control the vocabulary size, allowing us to observe how performance varies with fragment granularity. We constructed S for each dataset with three fragment vocabulary sizes |S| ∈ { 50 , 200 , 1000 } . The average numbers of fragments per graph ( # frags /G ) and atoms per fragment ( # atoms / frag) of Drugs and QM9 were reported in Table 6.

Additionally, the frequency depends on the size of fragments (number of constituent atoms) in Drugs and QM9 was reported in Fig. 7. For each |S| , the frequency distribution across fragment sizes is smooth and not biased toward certain sizes.

In the training and validation sets of the Drugs and QM9 datasets, there are 5 different ground truth conformers for each molecule. Thus, we generated 5 different conformers from RDKit to compute the fragment coordinates ˆ x f for each molecule in train and validation sets. Following [23], we computed the optimal matching between 5 RDKit generated conformers and 5 different ground truth conformers

80

PS 1000

PS 1000

80

70

PS200

PS200

09

PS 50

70

PS50

60

Frequency

50

Frequency

30

30

20

20

10

10

0

5

10

15

20

25

30

35

10

15

20

25

30

Sizeoffragments

Sizeoffragments

(a) Drugs

(b) QM9

for a single molecule. After computing the cost matrix whose the ( i, j ) -th element means RMSD between the i -th RDKit generated conformers and the j -th ground truth conformer, we assigned optimal RDKit conformer to each ground truth conformer using linear sum assignment problem [6]. After finding the optimal matching, we aligned each ground truth conformer to its assigned RDKit conformer using the Kabsch algorithm [25]. The aligned ground truth conformers were then used in the blurring schedule (Eq. (7)) and loss function (Eq. (14)) of the training process.

C.2 Training and time

We used a single NVIDIA A100 GPU for every training and generation tasks. For training, we used a learning rate 10 -4 with the AdamW optimizer [33]. The training time for both Drugs and QM9 was required around 3 . 8 days. For sampling, Drugs required 145 minutes for 14,324 conformers in 200 molecules, and QM9 required 71 minutes for 22,408 conformers in 200 molecules. We reported hyperparameters of EBD training including the maximum time step T , number of layers ( # l ) and number of features ( # d ) in the deblurring networks, number of multi-hops ( # of hops) and cutoff value for the expansion of atom interactions, batch size, and number of iteration in Table 7.

C.3 Performance of compared methods

For the results of compared methods in geometric evaluation of Drugs (Table 3) and QM9 (Table 8), COV-R and MAT-R scores of CVGAE [34], GraphDG [47], CGCF [57], and ConfGF [46] were borrowed from [46]. The performance of GeoMol and ConfVAE were borrowed from [62] and [59], respectively. In a case of RDKit [31], we reported the performance from the generated conformers from RDKit that we utilized to compute the approximate fragment coordinates ˆ x f ∼ p RDKit ( x f ) . For GeoDiff [59], we downloaded their implementation code from https://github. com/MinkaiXu/GeoDiff/tree/main and trained GeoDiff model for our experiments. We reported the performance of GeoDiff after sampling conformers using Langevin dynamics [51], as they did in their implementation.

For the method in the ablation study, DecompDiff [13] is a denoising diffusion model conditioned on coarse-grained structures, where the number of prior distributions corresponds to the number of fragments, and the mean of each prior is the respective fragment coordinates. By comparing the proposed method with DecompDiff in a controlled manner, we aimed to isolate the effect of the proposed blurring scheduler and random noise injection on learning in the coarse-to-fine molecular conformer generation task. Our use of DecompDiff was not to demonstrate its suitability for the molecular conformer generation task but rather to show that the stochastic trajectory from random

noise corruption is more challenging for the coarse-to-fine generation task than the proposed blurring schedule, even when the prior distributions are conditioned on the coarse-grained structures.

C.4 Pseudo-code

In this subsection, we provide the Pytorch-style [37] pseudo-codes. The RDKit conformer generator to obtain the approximate fragment structure, linear interpolation blurring schedule, training process, and sampling process were given in Pseudo-codes 1, 2, 3, and 4, respectively.

D Further results on geometric evaluation

GEOM-QM9. We compared our EBD to the baseline RDKit and machine learning models on small molecules GEOM-QM9, and the results are reported in Table 8. Compared to the most of machine learning models, EBD achieved superior performances especially on the precision score. We observed that RDKit, the distance geometry-based conformer generator, outperformed in coverage metrics for small molecules. However, as the size of molecules increases and the tasks become more challenging, RDKit suffers a significant performance drop, as shown in Table 3.

Statistical significance. We report the statistical significance of our model's improvements in geometric evaluation (COV-P, COV-R, MAT-P, and MAT-R scores). We measured p-value from

one-sided Wilcoxon signed-rank test (a non-parametric version of paired t-test) over those scores of EBD and GeoDiff [59] on Drugs and QM9, and the results are reported in Fig. 8. Except for the COV-R score on QM9, our EBD achieved statistically significant improvement in generating more diverse and more accurate conformers for every score on either dataset, as evidenced by the p-value.

COV-P,p-value=7.99e-07

COV-R,p-value=2.65e-04

COV-P,p-value=2.94e-11

COV-R,p-value=8.12e-02

Frequency

EBD

GeoDiff

Frequency

15

GeoDiff

EBD

F08

40

10-

50

5

20

o1

0

0

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Percentage

Percentage

Percentage

Percentage

MAT-P, p-value=7.13e-24

MAT-R,p-value=7.07e-31

MAT-P,p-value=3.68e-34

MAT-R, p-value=1.26e-32

Frequency

Frequency

15

20

Frequency

20

Frequency

5

5

0.5

1.0

1.5

2.0

0.5

1.0

1.5

0.0

0.2

0.4

0.6

0.8

0

0.1

0.2

0.3

0.4

(a) Drugs

(b) QM9

E Visualizations

We provide additional samples and sampling processes of EBD for the test set of Drugs in Figs. 9, 11 and the test set of QM9 in Figs. 10, 12.

Molecular

graphs

t=50

t=0

.·

01018

O

08

Molecular

graphs

t=50

t=0

00

HO

H2N

HO

NH.

Molecular

Ground

EBD

graphs

truth

samples

Molecular

Ground

EBD

graphs

truth

samples

OH

关L

OH

NH,

OH

HO

F Limitations

Although our Equivariant Blurring Diffusion achieves significant performance on coarse-to-fine generative problems in a hierarchical molecular conformer generation scheme, there are still several limitations.

Due to the change of the estimation target from the previous state (Eq. (12)) to the ground truth state (Eq. (14)) during sampling (Algorithm 2), the next step x a t -1 cannot be directly computed from the current state x a t and requires an additional step of the deterministic blurring function f B . This additional step in the sampling process can make the entire process slower compared to the previous state estimator.

As the size of molecule increases, the discrepancy between the ground truth x f and the approximate ˆ x f ∼ p RDKit ( x f ) of fragment structures becomes more severe. This increased discrepancy can make it more challenging for the model to learn the trajectory from the coarse fragment structures to the fine atomic details. To circumvent this issue, increasing the time step T to more than 50 can be applied. Also, for a more accurate deblurring network than equivariant graph neural networks [45] we used, more powerful geometric graph neural networks [24] can be applied such as local complete frames [10] and higher-order tensors from spherical harmonics [53].

G Broader impacts

We presented a deep generative model for the coarse-to-fine generation of molecular conformers. Our proposed model can be applied to problems in fragment-based drug discovery, such as scaffold hopping and linker generation, to achieve improved performance. In drug discovery applications, potential negative societal impacts may arise if the training set is contaminated and includes toxins. In such cases, the generated samples could potentially be harmful to humans. From a more general perspective, the generative models to which our model belongs can be misused to create false information that appears authentic. Therefore, users must be aware of the potential risks associated with generative models before using them.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See Sections 3 and 5.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: No theoretical assumption was made.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See Section 5.1 and Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in this paper conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We used the pre-processed data of GEOM [1] from [46] ( https:// github.com/DeepGraphLearning/ConfGF ) which was released under MIT license. For experiments, we used the codes of EDM [19] ( https://github.com/ehoogeboom/ e3_diffusion_for_molecules ), GeoDiff [59] ( https://github.com/MinkaiXu/ GeoDiff ), GeoMol [11] ( https://github.com/PattanaikL/GeoMol ), Torsional Diffusion [23] ( https://github.com/gcorso/torsional-diffusion ), and RDKit [31] ( http://www.rdkit.org/docs/index.html ). EDM, GeoDiff, GeoMol, and Torsional Diffusion were released under MIT license. RDKit is licensed under Creative Commons Attribution-ShareAlike 4.0 License.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors introduce an hierarchical diffusion model for molecular conformer generation. The framework starts with fragment positions initialized by RDKit and designs a diffusion process that generates atomic positions from substructure positions. The reverse diffusion process is modeled by an equivariant neural network. Experiments on standard molecular conformer generation benchmarks: GEOM-QM9 and GEOM-DRUGS show better performance than some previous baselines. The authors also include ablation studies that validate some important design choices in the model.\n\nSTRENGTHS:\n1. The model investigates a popular yet important problem: molecular conformer generation with deep generative model. \n2. The proposed model is benchmarked on standard dataset to demonstrate its performance.\n\nWEAKNESSES:\n1. The model is introduced in a way that it is derivated from blurring diffusion. However, the final formulation doesn't seem to strongly relate the concept. \n2. The work misses comparison with some strong baselines in molecular conformer generation. \n3. Though there are ablation studies to validate some design choices, there are still some important framework designs that are not well discussed. \n\nMore details can be found in the following Questions section.\n\nQUESTIONS:\n1. Following #1 in weakness, the final blurring operator (eq. 7) is basically a linear interpolant in Euclidean space between the substructure coordinate space and atom coordinate space. Unlike usual linear interpolant that starts from random Gaussian, here it starts from substructure coordinate space. I feel it is a bit confusing to introduce the framework as a variant of blurring diffusion which can obscure the actual contributions of this paper. \n2. Following # 2 in weakness, there are recent deep generative models for molecular conformation generation [1,2] that achieve state-of-the-art but are not included in the comparison to proposed method. The authors are strongly recommended to include the comparison to better validation the performance. \n3. Following # 3 in weakness, the work relies on principal subgraph (PS) to obtain molecular fragments. I wonder if the authors have tried other cheminformatic methods like BRICS [3]. \n4. Also, the vocabulary size |S| is set to 50, which means there are quite some isolated atoms. I wonder what are the ratios of isolated atoms. \n5. The work designs a diffusion process from substructure space to atom space, I wonder how it compare with a standard diffusion model that conditioned on substructure coordinates. Have the authors by any chance investigated similar settings?\n5. How is $\\delta^2$ in Eq. 8 determined in diffusion process?\n\nReferences:\n\n[1] Torsional Diffusion for Molecular Conformer Generation: https://arxiv.org/abs/2206.01729\n\n[2] Swallowing the Bitter Pill: Simplified Scalable Conformer Generation: https://arxiv.org/abs/2311.17932\n\n[3] On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces: https://chemistry-europe.onlinelibrary.wiley.com/doi/abs/10.1002/cmdc.200800178\n\nLIMITATIONS:\nThe authors adequately addressed the limitations in the work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel diffusion method for molecular conformers based on blurring diffusion. The method utilizes RDKit to predict the 3D structure of small molecule fragments and trains a diffusion model to generate the full-atomistic molecule from the RDKit prior, leveraging hierarchical modeling. The method is evaluated based on the GEOM dataset, testing both the quality of the sampled geometries and physical properties of the conformers, as is standard in the field.\n\nSTRENGTHS:\n1. The authors apply blurred diffusion developed for image applications to the domain of molecular conformer generation to leverage hierarchical modeling in molecular settings.\n2. The problem of coarse-to-fine prediction is a core problem in coarse-grained molecular modeling (referred to as backmapping in the respective literature) and the proposed approach might be applicable these (large-scale) problems as well.\n3. The proposed method demonstrates good performance compared to other diffusion-based approaches with models of comparable size in the literature.\n\nWEAKNESSES:\n1. The authors did not consider the molecular conformer fields (MCF) paper (https://arxiv.org/pdf/2311.17932), which is the current state-of-the-art approach to molecular conformer generation. The MCF method is more performant than the proposed approach.\n2. The approach depends strongly on coarse-grained fragment generation via RDKit, which could become a problem for larger systems of practical relevance. \n3. The authors should consider also reporting their performance metrics on the stricter threshold on GEOM-Drugs (delta = 0.75 A).\n\nQUESTIONS:\n1. How many parameters does the model use?\n2. Did the authors try to build an end-to-end pipeline, where a generative model predicts the coarse-grained coordinates instead of RDKit?\n\nLIMITATIONS:\nThe authors discuss the limitation of RDKit in application to larger molecular structure and the additional cost of the deblurring function.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper addresses the question by focusing on a fundamental biochemical problem: generating 3D molecular conformers based on molecular graphs in a multiscale manner. It consists of two stages:

1. Generating a coarse-grained fragment-level 3D structure from the molecular graph.
2. Generating fine atomic details from the coarse-grained approximated structure while allowing simultaneous adjustments to the latter.

STRENGTHS:
1. The paper proposed the EDB, which can generate atomic details from a coarse-to-grained estimation of fragment structures using equivariant networks
2. The paper proposed a novel blurring scheduler and a revised loss function that significantly impacts performance instead of directly applying those of the existing image blurring diffusion model.
3. The experiments and analysis demonstrate more plausible conformers compared to SOTA denoising diffusion models.

WEAKNESSES:
None

QUESTIONS:
None

LIMITATIONS:
None
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- This paper presents a model for small-molecule 3D structure generation, conditioning on its 2D molecular graphs. \n\n- The authors proposed a two-step process to address the problem: 1) first, using an off-the-shelf bioinformatics tool, RDKit, to generate a template scaffold structure; 2) then focusing on training a diffusion model to generate the fine-grained atom positions given the scaffold.\n\n- In essence, the model in the second step should learn to: 1) generate fine-grained atom positions given coarse fragment centers; 2) correct potential biases from RDKit-generated fragments.\n\n- To achieve this, they proposed a diffusion-like deblurring process inspired by heat diffusion (IHDM) but over a linear trajectory in Euclidean space, from fragment-averaged coordinates to predicted atomic coordinates. This design allows efficient training and sampling for the targeted problem.\n\n- Experiments on two small-molecule benchmarks show the model's superior performance compared to other generative model baselines.\n\nSTRENGTHS:\n- The proposed model comprises a novel combination of rational design choices:\n    1. A scaffold-atomic two-step generation process that defers the first step to well-established tools, converting the problem to generate atomic details and correcting prior distribution.\n    2. Borrowing the idea from heat diffusion, it uses constant noise instead of varying noise levels as in regular diffusion models.\n    3. It uses updated trajectory matching objectives by matching $x_0$ instead of the next step $x_{t−1}$.\n- Empirical analysis shows these design choices bring noticeable improvement in sampling coverage and accuracy over previous diffusion models. They also present several ablation studies and analyses to understand the performance and pinpoint some design factors: 1) fragment size; 2) diffusion trajectory and noise schedules; 3) loss reparameterization.\n- The manuscript is presented in a clear manner and is easy to follow.\n- Overall, this paper demonstrates that certain design choices can lead to improved performance and can be valuable for further research in small-molecule generation tasks.\n\nWEAKNESSES:\n- While the authors demonstrated better empirical performance and conducted ablation studies to verify selected design factors, some questions still remain on why and how some of the factors are critical, particularly:\n  - The effect of using RDKit as prior distribution: see Q1 - Q2\n  - Experiment details on comparing constant noising schedule (proposed) to regular diffusion (DecompDiff like): see Q3\n  - The effect of choosing noising levels: see Q4\n\n\n- Minor Typos: page 16: Pseudo-code 1: label is not for training code but the RDKit conformer generator.\n\nQUESTIONS:\nQ1. The superior performance of the proposed models might be due to 1) the accurate generation of fine-grained atomic positions and/or 2) correcting the biases from RDKit-generated scaffolds. However, which component plays a more important role is not clearly addressed. The authors showed in Section 5.2 that a small fragment size of $|S|=50$ achieves the best performance due to the decreased atomic-level details needing to be learned, raising a natural question of whether the main benefit was from correcting the prior biases. For example, can the model achieve similar or better performance without fragments (i.e., set $S = \\\\{\\text{atoms}\\\\}$ and the model only learns an error correction trajectory from the RDKit-predicted structure)?\n\n\nQ2. Related to Q1, one may wonder to what extent the model's performance relies on the quality of RDKit-generated scaffolds. Despite the discussion in the limitations, can the authors provide more analysis on EBD’s performance vs. RDKit’s performance?\n\n\nQ3. As discussed in 5.2, Effects of Data Corruptions, DecompDiff is the most similar model except for the choice of the diffusion process. However, the comparison with DecompDiff was limited: 1) it was not included in the full benchmark (Table 1); 2) T=50 steps were used for DecompDiff compared to T>200 in their paper, which might lead to different performances; 3) it was not clear if the authors retrained DecompDiff following the same setup, given the original DecompDiff was proposed for a different task (pocket-conditioned ligand generation). Can the authors provide clarification on above concerns?\n\n\nQ4. Sampling noise ($\\sigma$, $\\delta$) is a key hyperparameter in the proposed diffusion process. Can the authors provide theoretical or empirical analysis on the choices of these two parameters? Are the results sensitive to their choices?\n\nLIMITATIONS:\nThe authors have included discussion on the limitation in Appendix F.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Equivariant Blurring Diffusion (EBD), a unique generative model for hierarchical molecular conformer creation. A coarse-to-fine production process is presented by the model, with an emphasis on producing fragment-level structures first and then honing them down to atomic details. The method guarantees equivariance of SE(3), which is necessary for molecular structures. Comparisons with the most recent models on drug-like chemicals reveal that EBD performs better in geometric and chemical evaluations, demonstrating its effectiveness.\n\nSTRENGTHS:\nAn important step forward is the two-step process of creating fragment-level structures and then honing in on atomic details. This hierarchical method fits in nicely with molecular structures' multiscale structure.\n In order to preserve the geometrical and physical integrity of molecular conformers throughout the generation process, the model guarantees SE(3) equivariance.\nThe experimental results show that EBD can generate accurate and diversified molecular conformers with fewer diffusion steps than state-of-the-art models.\nComprehensive ablation investigations and in-depth comparisons with current models are included in the study, which offers a comprehensive understanding of the design decisions and how they affect performance.\nThe examination of the chemical properties, which includes HOMO-LUMO gaps and energy estimates, provides a great deal of value by demonstrating that EBD can produce stable and chemically realistic conformers.\n\nWEAKNESSES:\n1. The implementation and computational resource requirements may become more complex due to the hierarchical structure and requirement for fragmentation. This might make the model less useful and accessible for wider applications.\n2. RDKit is used extensively in the first generation of fragment coordinates. The quality of the initial fragment structures that RDKit provides could limit the model's performance.\n3. The concept works well for drug-like molecules, but it hasn't been fully investigated how well it scales to larger and more complex molecular structures. The claims would be strengthened by additional validation using larger datasets or more complicated compounds.\n4. The geometric (RMSD) and chemical properties are the main metrics used for evaluation. An evaluation of the model's capabilities that is more thorough might be obtained by incorporating further metrics pertaining to the novelty and variety of the generated conformers.\n\nQUESTIONS:\n1. Have you looked into any other options except RDKit for creating initial fragment coordinates? To what extent does the quality of these initial coordinates affect EBD performance?\n2. Could you elaborate on how EBD scales up to more complex molecules? Have you used these datasets for any preliminary experiments?\n3. Although the conformers developed exhibit chemical plausibility, what is their performance in real-world scenarios like docking simulations or property prediction? Is it planned to validate the conformers that are generated in these kinds of real-world situations?\n4. There is a brief mention of generation times and training. Could you elaborate on the amount of computing power needed to train EBD and produce conformers? In what way does this differ from the resources required for other cutting-edge models?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-15"}