Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation
Authors: Jiwoong Park
Yang Shen
Venue: NeurIPS 2024 poster (2024)
Keywords: molecular conformer generation
diffusion models
hierarchical models
Primary area: generative_models

Abstract: How can diffusion models process 3D geometries in a coarse-to-fine manner, akin to our multiscale view of the world?
In this paper, we address the question by focusing on a fundamental biochemical problem of generating 3D molecular conformers conditioned on molecular graphs in a multiscale manner. 
Our approach consists of two hierarchical stages: i) generation of coarse-grained fragment-level 3D structure from the molecular graph, and ii) generation of fine atomic details from the coarse-grained approximated structure while allowing the latter to be adjusted simultaneously.
For the challenging second stage, which demands preserving coarse-grained information while ensuring SE(3) equivariance, we introduce a novel generative model termed Equivariant Blurring Diffusion (EBD), which defines a forward process that moves towards the fragment-level coarse-grained structure by blurring the fine atomic details of conformers, and a reverse process that performs the opposite operation using equivariant networks.
We demonstrate the effectiveness of EBD by geometric and chemical comparison to state-of-the-art denoising diffusion models on a benchmark of drug-like molecules.
Ablation studies draw insights on the design of EBD by thoroughly analyzing its architecture, which includes the design of the loss function and the data corruption process.
Codes are released at https://github.com/Shen-Lab/EBD.

Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation
Jiwoong Park, Yang Shen
Department of Electrical and Computer Engineering
Texas A&M University
ptywoongggmail.com com, yshen@tamu.edt edu
Abstract
How can diffusion models process 3D geometries in coarse-to-fine manner, akin to our multiscale view of the world? In this paper, we address the question by focusing on a fundamental biochemical problem of generating 3D molecular conforners conditioned molecular graphs in multiscale manner. Our approach consists two hierarchical stages: i) generation of coarse- soure--rained fragment-level 3D structure from the molecular graph, and ii) generation of fine atomic details from the coarse-graines approximated structure while allowing the latter to be adjusted simultaneously. For the challenging second stage, which demands preserving coarse-grained information while ensuring SE(3) equivariance, we introduce a novel generative model termed Equivariant Blurring Diffusion (EBD), which defines a forward process that moves towards the fragment-level coarse-grained structure by blurring the fine atomic details of conformers, and reverse process that performs the opposite operation using equivariant networks. We demonstrate the effectiivness of EBD by geometric and chemical comparison to state-of-the- art denoising diffusion models on benchmark of drug-like like molecules. Ablation studies draw insights on the design EBD by thoroughly analyzing its architecture, which includes the design of the loss function and the data corruption process. Codes are released at h/tp::h////thbb com/ Shen- Labn--La/BBBB EBD
Introduction
The advancement of generative models to under- stand the multiscale properties of objects facili-
tates their application across range of granularity levels, transcending individual scales. To enable generative models in processing multiscale strue- tures, there has been surge in hierarchical design methodologies across multiple domains, spanning from images [35] 40 to speech (20 [7]. These
Dog
methods initially capture coarse-grasd goasnee structures and subsequently generate finer details.
In the field of computer vision, recent efforts have yielded successful designs of coarse-to-fine generative frameworks for 2D pix- els of images, leveraging denoising diffusion mod- els that corrupt and restore the data by adding and
16 42
Condition
Coarse structure
Fine-grained structure
Figure 1: Blurring diffusion generative processes
image 42 and molecular conformer.
removing noises [49 500 52
15 27]. Notably,
42 generates images from blurred prior distributions (average of pixel intensities) motivated from the heat equation of partial differential equations as Fig. [
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In the field of biochemistry and drug discovery, however, denoising diffusion models for 3D con- formers of stable molecular structures have not yet taken advantage of coarse-to to--io--inn -ine multiscale frameworks. Current methods either disregard the scale hierarchy [46 59
60 or consider that very limited ways [8 41 For instance, within the recent hierarchical method of uncondi-
19
23
tional conformer generation 38 denoising diffusion model [19] is solely applied to generation of
coarse-grained structure, without extending to coarse-to-fine generation.
The primary bottleneck in extending denoising diffusion models for molecular conformers to hi- erarchical designs is that random noise corrupts not only fine atomic details but also structural information of coarse- coarse-grained structures indiscriminately. To tackle this challenging problem, we fragments that are frequently occurring substructures or functional groups 2D molecular graphs. These fragments can be promising candidate for the coarse- sourse-rraied structural information 3D geometry. Introducing fragments divides the generation process into two stages: i) generating coarse-graie graseegg structures represented by fragments, and ii) restoring fine atomic details from fragment structures. In the first stage of generating fragment coordinates from molecular graphs, we efficiently generate approximations of fragment structures comprising the center of mass and attributes of each fragment from a cheminformatics tool.
For the challenging second step of coarse-to-fine generation, we propose a novel diffusion model, Equivariant Blurring Diffusion (EBD) detailed as follows. Motivated from the blurring corruption of the heat equation 42) we design EBD to generate 3D molecular conformers from coarse- graiined fragment approximated structures as Fig. _l rather than from random noise. In our design of EBD, the forward process moves atom positions of conformers towards the center of mass of their respective fragments, while the reverse process restores full-atom details from the prior distribution of the 31) fragment structure. The blurring schedule we designed for EBD allows the diffusion model to focus on restoring fine atomic details while retaining gparse-grained information throughout the entire generative process. We validated our coarse-to-fine EBD model using benchmark of drug-like molecules. We obtained superior results in conformer generation compared to the denoising diffusion model, even with 100 times fewer diffusion time steps.
The major contribution of this paper can be summarized as follows:
We design EBD which generates atomic details from coarse grasse-raaied estimation of fragment structures using equivariant networks, motivated by the blurring corruption of heat equation.
We propose novel blurring scheduler and a revised loss function that significantly impacts performance, instead of directly applying those of existing image blurring diffusion model.
We conduct a thorough analysis of the effects fragment granularity, data corruption methods, and loss reformulation. We obtained geometrically and chemically more plausible conformers compared to state-of-the-ee denoising diffusion models.
2 Background
2.1 Blurring diffusion
The denoising diffusion models [49 50, 52, 15 277, which corrupt data by adding random noise and generate data through denoising. have significantly advanced across diverse domains [55] |54, 44). Recently, a few works 2 42 18 have introduced data corruption into the design space of diffusion models (26), going beyond random noise corruption in the vision domain.
Inverse Heat Dissipation Model (IHDM) 42 proposed a coarse-to generation in the pixel space. Their forward process follows partial differential equation of heat dissipation on grids:
x = Axx
where x represents the data on the grid and A is the Laplacian operator. IHDM derived the solution of this equation at time step t, X,, using eigendecooppssiiin of A as:
= B,x V exp(-At)VT Xp.
where VT and A are discrete cosine transform and a diagonal matrix whose elements are eigenvalues of A, respectively. As t + T. the eigenbasis of eigenvalue 0 only remains and this leads to the

---

convergence of pixel intensities to their average value. Based upon this blurring process, IHDM defined forward process as:
(xx|/x0)= = (xx, B,xo. T²),,
which means that the state at t is equal to the data blurred until with small amount of noise. Note that the function of data corruption B, was defined at spectral space of eigenvalues A. Then, the reverse generative process was defined to deblur each state:
po( x [x,) =
12),
where the mean at t - 1 is the result of a deblurring network μi and å is the small amount of standard deviation for noise. As t approaches 0, μo gradually restores fine details from baase-grainee information about pixel intensities by effectively deblurring state values. The loss was defined to minimize the distance between the result of deblurring network and less blurred state at randomly sampled as:
レ x,, B - 1 14 (xt, 0)
IHDM was evaluated on image generation task using FID score [14], but its performance lagged behind that of denoising diffusion models. For instance, IHDM achieves an FID score of 18.96 while have 3.17 CIFAR-10 33
DDPM L5
2.2 Equivariance
In this work, we consider the SE(3) group to address the into-translational equivariance of molecular conformers [28] 19 [59]. A function f is equivariant to group G if T. (f())) fx)) holds for all E G, where Tg Sg are transformations of the group element g. In our coarse-to-fine generative framework, the invariant prior distribution of coarse-grained structure represents the coordinates of fragments. Therefore, the design of the transition distribution and the loss function in our diffusion model need ensure that the generated likelihood is invariant, so that the generated conformers not affected by rotation or translation.
3 Methods
3.1 Problem definition
Suppose that we have molecular graph G whose nodes V are atoms with SE(3)-invariant features h E gaxd and edges E are inter-atomic bonds. Our objective to generate an ensemble of 3D molecular conformers x" E _2xx given G. Our hicrarchical approach is in two stages. i) p>x (G): generating a coarse-grained 3D structure fragment coordinates xf E gmm from G which was decomposed into m fragments, and ii) pxx"xx G): the diffusion model generating fine atomic details x" E R conditioned on the generated fragment structure xf. To map between atoms and their respective fragments, belongs to the k-th fragment and otherwise. Mxf makes each atom located at its respective fragment. On the other hand, M results in fragment coordinates being the average of the coordinates of its constituent atoms, where M is a pseudoinverse matrix of M (M'M I).
defined mapping matrix M Ra with M 1 if the i-th atom
3.2 Fragmentation and 3D fragment structures
We decompose molecule G (V. E) into m non-overlapping fragments (Sx) where S₃- (Vr. and V = Uh V4. E = U & using Principal Subgraph (PS) 22] Starting from all unique atoms in the fragment vocabulary S, PS iteratively merges neighboring fragments. The most frequent fragment among the newly merged fragments was added to the vocabulary at each iteration, which was repeated until the desired size of the vocabulary was reached. The smaller the size of fragment vocabulary, the finer fragments and detailed coarse-grained structures can be obtained. After fragmentation was finished, from the relation between (VA { and V. the mapping matrix M can be constructed.
To generate the initial coordinates of fragments, utilize RDKit distance geometry 5 31.
efficient cheminformatics tool, instead of training an additional deep generative model. After generating initial atom coordinates x (xDXa(x)) define the initial fragment coordinates xf averages of their constituent atom coordinates, M Since the atom coordinates generated by
3

---

Step 1: Fragment structure generation
HCI
Fragmentation of G
Fragments coordinates x
Step 2: Equivariant Blurring Diffusion
Pd
X
x7 MR
x²
x MX
g(x] %
x = (1- (1 MR
Graphical model
Blurring schedule
Figure 2: Our hierarchical molecular conformer generation framework. We first decompose molecular graph G into fragments and generate fragment coordinates x/. Then, conditioned on x and G. Equivariant Blurring Diffusion generates atom-level fine details using the blurring schedule.
RDKit are approximations of the ground truth conformer, the resulting fragment coordinates are also an approximation (thus need to be adjusted in the next stage in Sec. S.c. which we denote as x ~ PRDKU (x"). For fragment features h' E Rmx3,, we define a 3-dimensional vector as a frequency histogram of its constituent atom types based on their chemical properties, including hydrophobicity, hydrogen bond center, and negative charge center following (38).
The processes of fragment structure generation are illustrated in the step of Fig. 2 Note that every step in this subsection does not harm the efficiency of our framework, as they can be completed before training our diffusion model and the process itself is efficient. To generate 5 distinct fragment coordinates x for each of the 45, 000 molecules in the training and validation set of the GEOM-Drug benchmark [1] it took 38 hours, averaging 3.04 seconds molecule. The details of fragmentation and fragment vocabulary are demonstrated in Appendix C.I
3.3 Equivariant blurring diffusion
In this subsection, we elaborate on the design of our diffusion model, Equivariant Blurring Diffusson (EBD), drawing inspiration from the principles of the heat equation. This model is designed to generate fine details of conformers x", starting from a coarse-grained, approximate structure x and molecular graph G. We introduce a forward process and data corruption function of blurring process in Sec. 3 reverse process and a deblurring network to reach an SE(3)- invariant likelihood in Sec. 3.3.2, and definition and reparameterization of an 3E(3)-invariant loss function in Sec. 3.3.3 The overall scheme of EBD is illustrated in the step 2 of Fig. 2
3.3.1 Forward process and blurring schedule
We define the data corruption of the forward process blurring operation that gradually shifts ground truth atom positions x (xx])) to their corresponding fragment coordinates:
(xx) x xf) (x]) & o1))
(6)
where fa a deterministic blurring operator. Consequently, every atom will be positioned according its fragment coordinates M& in the prior fragment structure distribution.
When defining fB for the forward process, we cannot directly adopt the spectral blurring operator B, V exp(=At) VT of IHDM [2) in Eq. (2) for the two reasons: i) For single molecule, need to calculate and decompose the fragment graph Laplacian { AR V tm J for each fragment S. (Vx, Ex), unlike single Laplacian operator per image in IHDM. Given the varying sizes and structures across fragments, becomes challenging to uniformly adjust the movement of atoms across all fragments using a function of (A+) mm in spectral space. ii) As t - T. the ground truth atom coordinates x will converge the ground truth scaffold structure xf B7x; by spectral

---

graph theory 44. However, there exists a mismatch between the ground truth coordinates x' and the approximation coordinates x from RDKit in the generative processes. This distributional shift of the fragment structure can potentially harm the performance during the inference.
To circumvent these issues, transition the space of the blurring operator from spectral domain to spatial domain while retaining the essence of the blurring process. We define fB as a linear interpolation between Mx and xij in Euclidean space:
(1x.,,,)) - -1- 1 +1+ Mx!.
(7)
As progresses from 0 to T, the atom coordinates x² will gradually converge to the fragment structure M allowing for uniform adjustment of atom movement. Additionally, we can mitigate the need for excessive eigendecomposition of the fragment graph Laplacian. The example of our blurring schedule on single fragment is depicted in the step 2 of Fig. 2
3.3.2 Reverse process and deblurring networks
The aim of the reverse process is to generate fine details at the atom-level from a prior distribution
of 3D fragment structure p(x+)) . (x+ |M& 1))1) that is roto-translational invariant to the group. Drawing upon proofs regarding the conditions for an invariant likelihood [28 [9], we develop the deblurring process the center-of- mass subspace using equivariant transition distributions:
pe pe(x"_1|xx, G) NNxxt_lluxxx;, x G,t),
(8)
where Hi is a parameterized mean function consisting of deblurring network. To ensure equivariance in the transition distribution, devise j4g inspired by equivariant networks [45]. Our equivariant deblurring network updates invariant features of fragments and atoms h', hh, (Bqs. ( 100, and the equivariant coordinates of atoms x (Eq. (IT) by leveraging the hierarchical relationship between atoms and fragments. Let the i-th atom x; belongs to the k-th fragment x then the 1-th layer of equivariant deblurring networks for fragment- and atom-level message passing and feature updates defined as follows:
m!, &' (hh4, h ||x| xxfl),
h[1+1 0% (b.' Σ JEN(SE) mlg: had),
(9)
mig = 2h (h hal ||x - x5 (),
h h2.l1 = 0% (h, Σ jEN())) m h1(t1)
(10)
a.
a.
xi
al+1
xi + Σ
xj
22 (h
(h .h." 1 mmi. eij)
dij
+1
JEN(x;)
(11)
x a.l
xf
xf|l +1 6 (h.
.n
61+1
||xx
x
x
where x is the k-th row of xx, o are multilayer perceptrons, eig are inter-atomic bond types, and dyj = |xx - x1 are inter-atomic distances. We consider complete graph for fragment-level interactions and expand the edge set by incorporating multi-hop and radius neighbors for atom-level interactions. The details of the deblurring networks are provided the Appendix A
3.3.3 Training
Following Eq. 5 of IHDM IH
42 our loss of previous deblurred state estimation can be defined as:
T--1 F, x œ [L/ (x] & I. 1) - p(uo x G,)
(12)
where is the Kabsch algorithm [25] to obtain the optimal rotation matrix for alignment. Through alignment p between the prediction from (4 and less blurred state fa fB(x), x -- 1) after translating both terms to the zero center-of-ne mass subspace, the loss function becomes invariant to the SE(3)-
-ass transformation of the prediction.
However, we empirically observed that this previous state estimator generates unsatisfactory conformers, similar to the unsatisfactory FID scores observed in image generation of IHDM 42.. We conjectured the reason as the model limited to learn the locally small steps towards the ground truth distribution at each time step 77 Thus, we reparameterize pe pe x! G,t) as
5

---

Algorithm 1 Training
Algorithm 2 Generation
Sample x PRIKII (x))
Sample x "RDDK X7
Sample xi ~ {(x))
MX for t in fT.....1. 1] do
Sample 24[1.]] T7
Sample 6 ~ NN,
Sample E ~ (0,0
x < [B (x.. x ,1, +
x 5 So(x) G.G.) x JB x
Minimize |x ((f 1
end for
(1 f) G.t)+ MM make the deblurring network estimates the ground truth state xi instead of the previous less blurred state via neural networks so:
1 T
T--1 = R xx
[B xi x - -((1 T
No No(x;, G,1)
Mx) [P]]
(13)
R Bx [|x] (x) G,
(14)
The derivation of the new loss is detailed in Appendix By loss reparamcterization,, p aligns the prediction to the ground truth state. At time step of the sampling process, after estimating ground truth x from xj. the next state x is computed from deterministic blurring function fB using Xi. The training and sampling processes are provided in Algorithms
Related work
Hierarchical generation. A hierarchical design of generative models is evident across multiple
domains, including image generation [35 40, 16 42] and speech synthesis [20] [7], aimed at
enhancing the interpretability and quality of samples derived from coarse-grainet information. In the field computational biology, recent studies on molecular graph generation [2T] 22] [2],
backmapping of protein structure [61] and conformer generation [56] conditioned on the given ground truth coarse-gras coarse-grained grained information have reported the effectiveness of the hierarchical design. In recent unconditional conformer generation [8]. denoising diffusion model [19] exclusively used in the fragment structure generation step and not designed for coarse-to- to-fine generation.
Data corruption in diffusion models. The choice of data corruption can be considered a crucial aspect of the design space of diffusion models [26), depending on the characteristics of the data domain and the specific problem definition. Recently, several studies diffusion models have revealed that the choice of data corruption can be extended beyond random noise [49 [5 50 36 43) to methods such masking 22] 7 81, blurring [42 2 [8], and varying data dimension [3] We designed the data corruption process as a blurring operation in Euclidean space, transitioning from atom-level fine details to fragment- level coarse structures. This approach is more effective for multi- scale frameworks compared to random noise, which corrupts both fragment and atom geometries.
Experiments
In this section, evaluate our hierarchical molecular conformer generation framework via Equivari- ant Blurring Diffusion (EBD) on molecular conformer generation task. We conducted experiments to answer the following questions: i) Ablation studies (Sec. 5.2): What are the effects of granularity of the fragment vocabulary, loss reparameterization, and data corruption processes of diffusion models? ii) Geometric evaluation (Sec. 5.3): Can EBD generate more diverse and accurate molecular conformers in Euclidean space than previous deep generative approaches? iii) Property prediction (Sec. 5.4 Can EBD generate low-energy, stable conformers?
5.1 Experiment setup
Dataset. We use GEOM- -M- (QM9) 39 and GEOM-Drugh (Drugs) I which are small molecules and drug-like molecules, respectively. Each dataset comprises 40,000 molecules for the training set and 5,000 molecules for the validation set, with each molecule containing conformers following data split of [46]. For the test set, we selected 200 molecules for each dataset, resulting in 22,408 and 14,324 conformers existing QM9 and Drugs, respectively. The details of dataset were demonstrated in Appendix C.I

---

PS 1608
estest.
DecompDif1
1.225
PS 200
est.
1.5
EBD
50
1.05
1.3
1.3
0.975
1.1
1.1
8.825
0.7
0.75
0.5
0.5
MAT-R iccon
MAT-R
MAT-P
MAT-P
MAT-R moan
MAT-R
MAT-P
NAT-P median
MAT-R moon
MAT-8
MAT-P
MAT-P mossan
(a) What level of granularity?
(b) What estimate?
(c) How to corrupt?
Figure 3: Ablation studies on the motivation and design choice of EBD. (a) Fragment vocabulary
granularity; (b) Target of state estimator; (c) Choice of data corruption processes.
Metrics. To measure the accuracy and diversity of the generated conformer metrics proposed by [I]. The metrics are based on root-mean-square deviation (RMSD), which is normalized Frobenius norm between
we adopted
atomic coordinate matrices aligned using the Kabsch algorithm 25 Given the ground truth conformer set C* and the generated sample set C, four metrics that follow precision and recall are defined as:
COV-R (Recall)
{C- E C""RRMD(C",C)
(15)
C
MAT-R (Recall)
min RMSD(C",,
(16)
C
CEC
C-
where COV and MAT are coverage metric and matching metric [57], respectively. COV quantifies the proportion one set covered by another, with "covered" indicating RMSD values are within threshold 5. MAT measures the average of RMSD values of one conformer set with its closest conformer in another set. If and C are exchanged (Precision) and MAT-P (Precision). The recall metric is focused
Eqs. [
[6). then metrics become COV-P the diversity, while the precision
metric measures the quality. The threshold 8 is set to 0.5A for QM9 and 1.25A for Drugs. For each molecule, we generated conformers that are twice the size of the ground truth conformers C*.
We compare EBD to existing deep generative models for molecular conformer generation. The performance of RDKit (3T] that used generate the fragment structure of model was measured as baseline. Besides RDKit, machine learning models including CVGAE [34], GraphDG 47], CGCF (57]. ConfVAE (58). GeoMol (II), ConfGF (46), and GeoDiff (59) were compared to our model. Among these, GeoDiff is the denoising diffusion model restoring the target distribution from random noise in atom coordinates. We adhered to their settings by configuring the maximum time step T to 5,000. For EBD, we use the T = 50, noise scale of 0.01 for the forward process (a Eq. 6) and 0.0125 for the reverse process (8 in Eq. g every experiments. The implementation details were reported in Appendix C.2
Baselines.
5.2 Ablation studies
We conducted ablation studies validate model design, encompassing the size of the fragment vocabulary, the reparameterization of loss, and the blurring data corruption. For each ablation study, calculated the mean and median of matching scores MAT-R and MAT-P Drugs test set. Note that lower values of MAT-R and MAT-P indicate better results.
Effects of fragment granularity. We assessed the performance variation as fragment structure became more detailed and in- formative by measuring the generation performances across different fragment vocabulary sizes |S] < (50, 200, 1000). Since PS [29]. the fragmentation method used, initial izes the vocabulary from unique single atoms, reducing the
Table 1: Statistics of fragment vo- cabulary S in Drugs
[S]
#frags/G
#atoms fatoms/fragg
11.77
4.02
200
7.60
6.34
size [8] results in obtaining finer fragments x!. We reported the statistics of the average number of fragments per graph
1000
5.26
9.25
(H frags/G) and atoms per fragment (#atoms/ /fraa) frag) in Table and the generation results in Fig.
3 (a).

---

Thanks to the increased level of detail in fragments, |S] 50 can obtain better per- formance compared to other vocabulary sizes. This is because more specific frag- ment structures decrease the amount of atomic-level detail that needs to generated. From this observation, we use |S 50 all
Table 2: Fine-to-fine generation on Drugs.
(%)
MAT-R(A)
COV.P (%)
MAT.
Meon
Med
Monn
Med
Mcan
Med
Mean
Mei
C2F
92.60
98.73
0.8216
0.8299
66.24
68.39
1,1237
1.0916
89.44
98.73
0.7986
0.7710
76.62
88.64
1.0900
0.9397
subsequent experiments. We also conducted fine-to-fine generation to observe the ability of EBD. We measured the performance when the prior is ~ (RDKK((")) Table 2 presents coarse-to-fine (C2F) result when |S| 50 and fine-to-fing (F2F) result. As expected, F2F shows more accurate results compared to C2F as it includes more detail in the prior distribution.
Effects of loss reparameterization. We presented the performance comparison between the less blurred previous state estimator Eq. (2) and ground truth estimator in Eq. (14) after loss reprarameterization in Fig. 3 (b). From the previous state estimator, we acquired degenerated
conformers with relatively high matching scores, which align with low FID score of IHDM [42 in image generation. On the other hand, we observed distinct advantages in introducing the ground truth estimator across all metrics. We speculate that the ground truth estimator facilitates the diffusion model in learning beyond locally blurring distributions towards the target distribution.
Effects of data corruptions. We provide the same initial fragment structures x to both EBD
DecompDiff
and DecompDiff [33 so that the data corruption method becomes the primary distinction to ex-
amine. DecompDiff a better candidate than GeoDiff (59] to compare for this purpose be- cause, unlike GeoDiff that generates conformers from a single prior distribution, DecompDiff' de- noises multiple prior distributions, where each mean corresponds to the coordinates of each fragment x The generation results and sam- pling trajectories
t=50
40
30
20
10
coofc
compared between the two models (T = 50 for both) in Fig. (c) and Fig.
EBD
4 At first, we observed that the conformers gen- erated from DecompDiff exhibit lower diversity scores compared to EBD. This is because the results of DecompDiff tend to adhere closely to the approximate fragment structure X whereas EBD attempts to transition towards the ground truth fragment structure xf. We speculate that our blurring schedule, which entails a linear interpo- lation between MX' and x)) facilitates the learning process for the diffusion model compared to stochastic trajectory between prior and target distributions. DecompDiff primarily focuses on denoising the fragment structure throughout most of the sampling process in Fig. On the other hand, EBD focuses on the entirety of the sampling process to generate fine details, resulting in better quality.
Figure 4: Sampling processes of two conformers depending on data corruptions.
empirical evidence, we observed that
We conducted further analysis observe how accu- rately the fragment coordinates generated by RD- Kit were corrected toward the ground truth by EBD. For 200 molecules in Drugs test set, we mea- sured RMMS(xX<Dai x and RMSD(xEBD: x If R.SD(xXFRD) xf) is lower than R.MSD(x (ASD(X<DKH> xf. for molecule, then it indicates that the model has corrected the fragment coordinates. points below the red line represent model corrected the coordinates accurately. We ob-
RMSDW xx)
Fig. []] the
where the
served that the greater the RMSD(X x (points further to the right on the x-axis), the larger the reduc- tion RMSD( (XEDD ) In other words, the lower the quality of the coarse-grained prior, the more ac- curately the model tends to make corrections.
RMSDDX X
Figure 5: Correction on fragment coordinates

---

Table 3: Geometric evaluation on Drugs (Ó = 1.25A).
COV-R (%) 1
MAT-R(A) Á) +
COV-P (%) f
MAT-P (A)
Models
Mean
Med
Mean
Med
Mean
Med
Mean
Med
RDKit
45.74
31.75
1.5376
1.4004
54.78
59.48
1.3341
1.1996
CVGAE
0.00
0.00
3.0702
2.9937
GraphDG
8.27
0.00
1.9722
1.9845
2.08
0.00
2.4340
2.4100
CGCF
53.96
57.06
1.2487
1.2247
21.68
13.72
1.8571
1.8066
ConfVAE
55.20
59.43
1.2380
1.1417
22.96
14.05
1.8287
1.8159
GeoMol
67.16
71.71
1.0875
1.0586
ConfGF
62.15
70.93
1.1629
1.1596
23,42
15.52
1.7219
1,6863
GeoDiff (T 5000)
89.40
96.86
0.8571
0.8495
61,28
65.00
1.1642
1.1272
EBD (T 50)
92.60
98.73
0.8216
0.8279
66.24
68.39
1.1237
1.0916
Additional visualizations of sampling results on Drugs and QM9, as well as the sampling processes, are illustrated in the Appendix E
5.3 Geometric evaluation
We compared our hierarchical framework to the baseline RDKit and machine learning models for molecular conformer generation Drugs, and the results reported in Table 3 EBD achieves superior performance across all metrics by generating diverse and accurate molecular conformers. In comparison to RDKit, which was used to generate fragment structures x EBD achieved significant improvement in the generation of diverse fine atomic details, as evidenced by higher COV-R and MAT-R scores. We also observed that, due the informative fragment structure prior distribution and the proposed blurring schedule, EBD produces more diverse and higher-quality conformers with statistical significance (see Appendix D), even with 100 times fewer T compared to GeoDiff. We also reported EBD's 's better performance on QM9 with statistical significance in Appendix D
GeoDiff
EBD
Poreer spectall denalty
Power speccrra secotty
torvan
Eigenvalue
Eigenvalue
Figure 6: Power spectral density analysis on forward processes of GeoDiff and EBD. The darkest line is the PSD coefficient of x and the lines become lighter as t + T.
We take an additional step delve into the rationale behind the ability of EBD to achieve better performance with smaller I' by analyzing the spectral domain in Fig Motivated from the analysis of [42], we calculated the power spectral density, PSD(x"): = Eesta.y.>) VV x 2.00 where V₃ is the i-th smallest eigenvector of graph Laplacian of the molecular graph G. and x is one of the {z, 2 coordinate vectors of atomic coordinate matrix x The smaller eigenvalues correspond to coarser-grainet structures, while higher cigenvalues correspond to finer details [4]. Therefore, by measuring Px"() during the forward processes {x:)²=> of EBD and GeoDiff, we can ascertain which structural components are corrupted during the forward process and will be restored during the reverse process, respectively. For EBD, there is less significant perturbation across lower frequency parts of the PSD that corresponds to the coarser- -eeeind discrepancy between ground truth x' and approximate & fragment positions. Thus, EBD primarily focuses on restoring perturbed fine-grained structures in the higher frequency parts throughout the entire generative process. This explains why EBD does not require excessive T. contrast, GeoDiff requires relatively more because random noise corrupts the overall structural information, and the amount of perturbations is also significant.

---

5.4 Chemical evaluation
In addition to geometric evaluation, we assessed the quality of generated conformers by their chemical properties. After training EBD on QM9, following [46]. we generated 50 samples for each of the 30 molecules, which constitute subset of QM9. Using PSI4 [48), calculated properties of each conformer including the en- ergy E, the lowest energy Emin. HOMO-LUMO gap e, the average gap Σc, the minimum gap memin and the maximum gap Emaax. Then, we measured the mean absolute errors between the properties of generated and ground truth (Table.
Table 4: Mean absolute errors between generated and ground truth ensemble properties eV.
Models
E
Enin
4min
RDKit
0.9233
0.6585
0.3698
0.8021
0.2359
GraphDXi
9.1027
0.8882
1.7973
4.1743
0.4776
CGCF
28,9661
2.8410
2.8356
10,6361
0.5954
COnIVAR
8,2080
0.6100
1,6080
3.9111
0,2429
ConfGF
2.7886
0.1765
0.4688
2.1843
0.1433
GeoDiff
0.2597
0.1551
0.3091
0.7033
0.1909
EBD
0.1812
0.1214
0.1253
0.5306
0.2153
We observed that EBD can generate the most stable conformers compared to other methods, as
evidenced by lower energy and HOMO-LUMO gap.
Conclusion
We introduced novel hierarchical generative model for molecular conformers via Equivariant Blurring Diffusion (EBD), a diffusion model designed for coarse-to-fine generative scheme. After generating the initial distribution of fragment coordinates from cheminformaties tool, EBD gen- erated fine atomic details from coarse- gaarse-grainddd structures through equivariant networks. We also proposed simple and effective linear blurring scheduler and ground truth state estimator to enhance the model's s ability to produce diverse and accurate conformers. Through extensive analysis of the proposed model and comparison between competitive denoising diffusion models, we substantiated the validity of the model design.
As coarse-to-fine generative scheme, EBD can be extended for larger and more complex molec- ular structures, because hierarchical systems similar to those utilized in EBD exists widely across molecular systems (ranging from proteins as linear polymers of amino acids materials as lattices of molecules). We discussed few limitations of our model in the Appendix
Acknowledgments and Disclosure of Funding
We thank Yuning You and Hoseok Do for valuable feedback on the manuscript. This work was partially funded by the U.S. National Science Foundation under grant CCF-1943008. Portions of this research were conducted with the advanced computing resources provided by Texas A&M High Performance Research Computing.
References
[1] Axelrod, Simon and Klome--oomarelle Rafael. Geom, tions for property prediction and molecular generation. Scientific Data, 911:1885, 2022.
molecular conforma-
[2] Bansal, Arpit. Borgnia, Eitan, Chu, Hong-Min. Li, Jie, Kazemi, Hamid, Huang, Furong, Goldblum, Micah, Geiping, Jonas, and Goldstein, Tom. Cold diffuusion: Inverting arbitrary image transforms without noise. Advances Neural Information Processing Systems, 36, 2023.
[3] Campbell, Andrew, Harvey, William, Weilbach, Christian, De Bortoli, Valentin, Rainforth, Thomas, and Doucet, Arnaud. Trans-dimensional generative modeling via jump diffusion models, Advances in Neural Information Processing Systems, 36, 2023.
[4] Chung, Fan RK. Spectral graph theory, volume 92. American Mathematical Soc., 1997.
[5] Crippen, Gordon M, Havel, Timothy F, et al. Distance geometry and molecular conformation, volume 74. Research Studies Press Taunton, 1988.
[6] Crouse, David F. On implementing 2d rectangular assignment algorithms. IEFE Transactions on Aerospace and Electronic Systems, 1274):1679-1666 2016.
10

---

[7] Daras, Giannis, Delbracio, Mauricio, Talebi, Hossein, Dimakis, Alexandros, and Milanfar, Peyman. Soft diffusion: Score matching with general corruptions. Transactions on Machine Learning Research, 2023.
[8] Daras, Giannis, Shah, Kulin, Dagan, Yuval, Gollakota, Aravind, Dimakis, Alex, and Klivans, Adam. Ambient diffusion: Learning clean distributions from corrupted data. Advances in Neural Information Processing Systems, 36, 2023.
[9] Degen, Jorg, Wegscheid- Gerlaa,, Christof, Zaliani, Andrea, and Rarey, Matthias. On the art of compiling and using drug-likee chemical fragment spaces. ChemMedChem, 3(10):1503, 2008
[10] Du, Weitao, Zhang, He, Du, Yuanqi. Meng, Qi, Chen, Wei, Zheng, Nanning, Shao, Bin, and Liu, Tie- Yan. Se (3) equivariant graph neural networks with complete local frames. International Conference Machine Learning, pp. 5583-5608. PMLR, 2022.
[11] Ganea, Octavian, Pattanaik, Lagnajit, Coley, Connor, Barzilay, Regina, Jensen, Klavs, Green, William, and Jaakkola, Tommi. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems, 34, 2021.
[12] Geng, Zijie, Xie, Shufang, Xia, Yingce, Wu, Lijun, Qin, Tao, Wang, Jie, Zhang, Yongdong, Wu, Feng, and Liu, Yan. De novo molecular generation via connection-awart motif mining. In The Eleventh International Conference on Learning Representations, 2023.
[13] Guan, Jiaqi. Zhou, Xiangxin, Yang, Yuwei, Bao, Yu, Peng, Jian, Ma, Jianzhu, Liu, Qiang, Wang, Liang, and Gu, Quanquan. Decompdiff: Diffusion models with decomposed priors for structure- based drug design. In International Conference on Machine Learning, pp. 11827-11844 PMLR, 2023.
[14] Heusel, Martin, Ramsauer, Hubert, Unterthiner, Thomas, Nessler, Bernhard, and Hochreiter, Sepp. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, 30, 2017.
[15] Ho, Jonathan, Jain, Ajay, and Abbeel, Pieter. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 2020.
[16] Ho, Jonathan, Saharia, Chitwan, Chan, William, Fleet, David J. Norouzi, Mohammad, and Salimans, Tim. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1-33, 2022.
[17] Hono, Yukiya, Tsuboi, Kazuna, Sawada, Kei, Hashimoto, Kei, Oura, Keiichiro, Nankaku, Yoshihiko, and Tokuda, Keiichi. Hierarchical multi-grained generative model for expressive speech synthesis. Interspeech 2020, 2020.
[18] Hoogeboom, Emiel and Salimans, Tim. Blurring diffusion models. In The Eleventh International Conference on Learning Representations, 2023.
[19] Hoogehoom, Emiel, Satorras, Victor Garcia, Vignac, Clément, and Welling, Max. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pp. 8887. PMLR, 2022.
[20] Hsu, Wei-Ning, Zhang, Yu, Weiss, Ron, Zen, Heiga, Wu, Yonghui, Cao, Yuan, and Wang. Yuxuan. Hierarchical generative modeling for controllable speech synthesis. In International Conference on Learning Representations, 2019.
[21] Jin, Wengong, Barzilay, Regina, and Jaakkola, Tommi. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pp. 2323-2332.
PMLR, 2018.
[22] Jin, Wengong, Barzilay, Regina, and Jaakkola, Tommi. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learning, pp. 4839- 4848. PMLR, 2020.
[23] Jing, Bowen, Corso, Gabriele, Chang, Jeffrey, Barzilay, Regina, and Jaakkola, Tommi. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems, 35, 2022.

---

[24] Joshi, Chaitanya K, Bodnar, Cristian, Mathis, Simon V, Cohen, Taco, and Lio, Pietro. On the expressive power of geometric graph neural networks. International Conference on Machine Learning, pp. 13330-13355, PMLR, 2023.
[25] Kabsch, Wolfgang. A solution for the best rotation to relate two sets of vectors. Acta Crystallo- graphica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32 (5):922-992) 923, 1976.
[26] Karras, Tero, Aittala, Miika, Aila, Timo, and Laine, Samuli. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35, 2022.
[27] Kingma, Diederik, Salimans, Tim, Poole, Ben, and Ho, Jonathan. Variational diffusion models. Advances in Neural Information Processing Systems, 34, 2021.
[28] Köhler, Jonas, Klein, Leon, and Noé, Frank. Equivariant flows: exact likelihood generative learning for symmetric densities. In International Conference on Machine Learning, pp. 5361- 5370. PMLR, 2020.
[29] Kong, Xiangzhe, Huang, Wenbing, Tan, Zhixing, and Liu, Yang. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems, 35, 2022.
[30] Krizhevsky, Alex, Hinton, Geoffiry, et al. Learning multiple layers of features from tiny images. 2009.
[31] Landrum, Greg et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8(31.10):5281, 2013.
[32] Lewell, Xiao Qing, Judd, Duncan B, Watson, Stephen P, and Hann, Michael M. Recap retrosynthetic combinatorial analysis procedure: a powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of chemical information and computer sciences, 38(3):111-52,, 1998.
[33] Loshchilov, Ilya and Hutter, Frank. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
[34] Mansimov, Elman, Mahmood, Omar, Kang, Seokho, and Cho, Kyunghyun. Molecular geometry prediction using a deep generative graph neural network. Scientific reports, 9(1):20381, 2019.
[35] Menick, Jacob and Kalchbrenner, Nal. Generating high fidelity images with subscale pixel networks and multidimensional upscaling. In International Conference on Learning Represen- tations, 2019.
[36] Nichol, Alexander Quinn and Dhariwal, Prafulla. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pp. 8162-8171, PMLR, 2021.
[37] Paszke, Adam, Gross, Sam, Chintala, Soumith, Chanan, Gregory, Yang, Edward, Detioo, Zachary, Lin, Zeming, Desmaison, Alban, Antiga, Luca, and Lerer, Adam. Automatic differen- tiation in pytorch. 2017.
[38] Qiang, Bo, Song, Yuxuan, Xu, Minkai, Gong, Jingjing, Gao, Bowen, Zhou, Hao, Ma, Wei- Ying, and Lan, Yanyan. Coarse-to fine: a hierarchical diffusion model for molecule generation in 3d. In International Conference on Machine Learning, pp. 28277-28299. PMIR, 2023.
[39] Ramakrishnan, Raghunathan, Dral, Pavlo O, Rupp, Matthias, and Von Lilienfeld, O Anatole. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
[40] Razavi, Ali, Van den Oord, Aaron, and Vinyals, Oriol. Generating diverse high-fidelity images with vg-vae-2. Advances in Neural Information Processing Systems, 32, 2019.
[41] Reidenbach, Danny and Krishnapriyan, Aditi. Coarsenconf: Equivariant coarsening with aggregated attention for molecular conformer generation. NeurlPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
12

---

[42] Rissanen, Severi, Heinonen, Markus, and Solin, Amo. Generative modelling with inverse heat dissipation. The Eleventh International Conference on Learning Representations. 2023.
[43] Rombach, Robin, Blattmann, Andreas, Lorenz, Dominik, Esser, Patrick, and Ommer, Bjöm. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
[44] Saharia, Chitwan, Chan, William, Saxena, Saurabh, Li, Lala, Whang, Jay, Denton, Emily L, Ghasemipour, Kamyar, Gontijo Lopes, Raphael, Karagol Ayan, Burcu, Salimans, Tim, et al. Photorealistic text-to-image diffusion models with deep language understanding, Advances in Neural Information Processing Systems, 35, 2022.
[45] Satorras, Victor Garcia, Hoogeboom, Emiel, and Welling, Max. E (n) equivariant graph neural networks. International Conference on Machine Learning, pp. 9323-9332. PMLR, 2021.
[46] Shi, Chence, Luo, Shitong, Xu, Minkai, and Tang, Jian. Learning gradient fields for molecular conformation generation. In International Conference on Machine Learning, pp. 9558-556.. PMLR, 2021.
[47] Simm, Gregor and Hermandez-Lobato, Jose Miguel. A generative model for molecular distance geometry. International Conference on Machine Learning. pp. 8949-8958. PMLR, 2020.
[48] Smith, Daniel GA, Burns, Lori A, Simmonett, Andrew C, Parrish, Robert M, Schieber, Matthw C. Galvelis, Raimondas, Kraus, Peter, Kruse, Holger, Di Remigio, Roberto, Ale- naizan, Asem, et Psi4 1.4: Open-source software for high-throughput quantum chemistry. The Journal of chemical physics, 152(18), 2020.
[49] Sohl- Sohl-Dickstein, Jascha, Weiss, Eric, Maheswaranathan, Niru, and Ganguli, Surya. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
[50] Song, Jiaming, Meng, Chenlin, and Ermon, Stefano. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
[51] Song, Yang and Ermon, Stefano. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
[52] Song, Yang, Dohl-Dicksttin, Jascha, Kingma, Diederik P, Kumar, Abhishek, Ermon, Stefano, and Poole, Ben. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
[53] Thomas, Nathaniel, Smidt, Tess, Kearnes, Steven, Yang, Lusann, Li, Li, Kohlhoff, Kai, and Riley, Patrick. Tensor field networks: Rotation-and translation-equivarian neural networks for 3d point clouds. arXiv preprint arXiv: 1002:::80000000 08219, 2018.
[54] Vahdat, Arash, Kreis, Karsten, and Kautz, Jan. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 34, 2021.
[55] Vahdat, Arash, Williams, Francis, Gojeic, Zan, Litany, Or, Fidler, Sanja, Kreis, Karsten, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems, 15:10021-1003, 2022.
[56) Wang. Wujie, Xu, Minkai, Cai, Chen, Miller, Benjamin K, Smidt, Tess, Wang, Yusu, Tang, Jian, and Gomez-Bombarelli, Rafael. Generative coarse-graining of molecular conformations. In International Conference on Machine Learning, pp. 23213-23236 PMLR, 2022.
[57] Xu, Minkai, Luo, Shitong, Bengio, Yoshua, Peng, Jian, and Tang, Jian. Learning neural generative dynamics for molecular conformation generation. In International Conference on Learning Representations, 2021.
[58] Xu, Minkai, Wang, Wujie, Luo, Shitong, Shi, Chence, Bengio, Yoshua, Gomez-Bombarelli, Rafael, and Tang, Jian. An end-to-end framework for molecular conformation generation via bilevel programming. International Conference on Machine Learning, pp. 11537-11547. PMLR, 2021.
13

---

[59] Xu, Minkai, Yu, Lantao, Song, Yang. Shi, Chence, Ermon, Stefano, and Tang, Jian. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Reprexentations, 2022.
[60] Xu, Minkai, Powers, Alexander S, Dror, Ron O, Ermon, Stefano, and Leskovec, Jure. Geometric latent diffusion models for molecule generation. In International Conference on Machine Learning, pp. 38592-386100 PMLR, 2023.
[61] Yang, Soojung and ping of coarse-grained proteins. In International Conference on Machine Learming, pp. 39277- 39298, 2023.
Rafael. Chemically transferable generative backmap-
[62] Zhu, Jinhua, Xia, Yingce, Liu, Chang, Wu, Lijun, Xie, Shufang, Wang, Yusong, Wang, Tong, Qin, Tao, Zhou, Wengang, Li, Hougiang, et al. Direct molecular conformation generation. Transactions on Machine Learning Research, 2022.
14

---

A Deblurring network architectures
In eequivarra deblurring networks, there are update functions of SE(3)-invarian fragment and atom features h', h", as well as an update function of SE(3)-equivarian atom coordinates x² motivated from equivariant graph neural networks [45]. For the fragments, we constructed complete graph to account for dense interactions among them. In the case of atoms, we expanded the neighbor set of each atom by including multi-hop neighbors derived from the powers of the adjacency matrix and a radius graph, which includes atoms within specified cutoff distance. The benefits of dense interactions for accurate conformers estimation have been confirmed in several studies [47 59 19]
The architecture of the SB(3)-invrrint message passing and feature update functions at the fragment- and atom-level is follows:
m!, = o (h h ||ж x5I1),
h. hitt1 = 0% (h! E JEN()) m h²d),,
(A.1)
mig 6m ad , hy' ||x] 11,
h 1alt1 = 0% (h Σ SEN(E) mfg. b(*t),
(A.2)
where m,j E Rd is the message for each interactions, and h E I2 is the feature vector from the aggregated messages and features from different hierarchy level. For every invariant update functions o'm o o of, we used multilayer perceptrons. For initial features h," of fragments, we defined a 3 3-dimensional vector frequency histogram its constituent atom types based on their chemical properties, including hydrophobicity, hydrogen bond center, and negative charge center following
38
The detailed definition of the initial fragment features is in Table For initial atom features
h E Rd and bond features efg E Rdd we used embeddings from atom types and bond types, respectively.
Table 5: Initial fragment feature based chemical properties
Properties
Details
Types
Hydrophobicity
Frequency of C element
Integer
Hydrogen bond center
Frequency of O, N. S, P elements
Integer
Negative charge center
Frequency of F, CI, Br, elements
Integer
For the i-th atom x belongs to the k-th fragment xf, the architecture of the equivariant atom coordinate update function is as follows:
0.1
2.1
Σ
xj
ha. mi.. eig) 2 (h
.n.
dig
+1
(EN(x))
(A.3)
x
ad
xx
(h
hy 1.+1
.|xx
x ll)
|
x([
+ 1
where xf, is the k-th row of M and d:j = ||x - - are inter-atomic distances. For every equivariant update functions Φe of we used multilayer perceptrons. For three terms in right-hand side of Eq. (A.3). the first term is the coordinate from the previous layer, the second term is an equivariant update function that accounts for atom-level interactions, and the third term is an equivariant update function that considers the deviation of the current atom coordinate from its respective fragment's 's coordinate.
B Derivation of loss function
In this section, we explain the derivation of the loss function for the ground truth state estimator from the previous state estimator. The loss function of previous state estimation is defined as:
L-1 Ex,,, x s [Il (x x 1) p(uo ( (x), x I)
(A.4)
where p is the Kabsch algorithm (25 to obtain the optimal rotation matrix for alignment. Through alignment p of the prediction from μg to the less blurred state fn (x]] x (-(--1 1) after translating both
terms to the zero center-of-ne mass subspace, the loss function becomes invariant to the translation and rotation of the prediction.
15

---

However. this previous state estimator generates unsatisfactory conformers as empirically observed in Sec. 5.2 We conjectured the reason as the model limited learn the locally small steps towards the ground truth distribution at each time step 7 Thus, we reparameterize no no(x); x G. as (1 - Ja (X,, G.I)+ + 7 Mxx make the deblurring network estimates the ground truth state x6 instead of the previous less blurred state via neural networks fa. We first start with the non-invariant previous state estimation, which is without the alignment p:
1 = xx,,, xg [1l (x); x L - 1) - px x G, G,
(A.5)
xx,x [IlfB( (x): X
- -1- G,t) - 7 - M
(A.6)
= E xt xx. <[](1 '7') )x + ' ' - (1 [66(x G,t) - - 1
(A.7)
E, xx se [1] *72 fo(x), G,t)) + 77' (Mx' -M
(A.8)
= xx 21 211x (x(,,,,1)11
(A.9)
~
x xx,, [||x] - (() G,
(A.10)
In the last stage from Eq. A.9 to Eq. (A.10), we simplified the loss function by discarding the
time-dependent weight 15). Finally, make the loss function for ground truth estimation
invariant by aligning the prediction from fo to the ground truth state using Kabsch alignment [25]
C Implementation details
C.1
Datasets
We used GEOM 1EOM-QM9 (QM9) 39 and GEOM-Drugs (Drugs) M for analysis and comparison between molecular conformer generation models. Each dataset comprises 40,000 molecules for the training set and 5,000 molecules for the validation set, with each molecule containing 5 conformers following data split of [46). We obtained the raw data, the pre-processed data and the data split at https://g thub. com/ for each dataset, resulting in 22,408 and 14.324 conformers existing in QM9 and Drugs, respectively.
Conf For the test set, we selected 200 molecules
For fragmentation of the molecular graphs G (V.E) in Drugs and QM9, we used Principal Subgraph (PS) 229] (https:/ //t::////tttbbbb com/THUNLP PWA-PP-MMPPPPPPP VAE) which can construct fragment vocabulary S whose elements are the largest and frequent repetitive subgraphs of molecules. Starting from all unique atoms in S initial stage, PS iteratively merges neighboring fragments. The most frequent fragment among the newly merged fragments was added to the vocabulary at each iteration, and this operation was repeated until the desired size of the vocabulary was reached. Thus, the smaller the fragment vocabulary, the finer fragments can be obtained. One of the advantages of PS compared to existing fragmentation methods such as RECAP (3). BRICS [9. junction tree decomposition (21 is the ability to control the vocabulary size, allowing us to observe how performance varies with fragment granularity. We constructed S for each dataset with three fragment vocabulary sizes |S] E (50, 200, 1000}. The average numbers of fragments per graph (#frags/ G) and atoms per fragment (#Hatoms/frag /frag) of Drugs and QM9 were reported in T
Table 6: Statistics of fragment vocabulary S.
Drugs
QM9
[S]
#trags/G
#atoms /from///raa////
#frags/G
#atoms/frag
50
11.77
4.02
5.17
3.91
200
7.60
6.34
3.70
5.45
1000
5.26
9.25
2.91
6.98
Additionally, the frequency depends the size of fragments (number of constituent atoms) in Drugs and QM9 was reported in Fig. 7 For each [S], the frequency distribution across fragment sizes is smooth and not biased toward certain sizes.
In the training and validation sets of the Drugs and QM9 datasets, there are 5 different ground truth conformers for each molecule. Thus, we generated 5 different conformers from RDKit to compute the fragment coordinates & for each molecule in train and validation sets. Following (23, we computed the optimal matching between 5 RDKit generated conformers and 5 different ground truth conformers
16

---

00
PS 1000
PS 1000
70
PS 200
PS 200
60
PS 50
70-
PS 50
60
Frequency
Frequency
50-
40
30-
20
20
10
10-
5
30
15
20
25
30
35
10
15
20
30
Size fragments
Size fragments
(a) Drugs
(b) QM9
Figure 7: Frequency depends the size of fragments in the fragment vocabulary of Drugs and QM9.
for single molecule. After computing the matrix whose the (i, element means RMSD between the i-th RDKit generated conformers and the j-th ground truth conformer, we assigned optimal RDKit conformer to each ground truth conformer using linear sum assignment problem [6]. After finding the optimal matching, we aligned each ground truth conformer to its assigned RDKit conformer using the Kabsch algorithm [25]. The aligned ground truth conformers were then used in the blurring schedule (Eq. (7) and loss function (Eq. (14) of the training process.
C.2 Training and time
We used single NVIDIA A100 GPU for every training and generation tasks. For training, we used learning rate 10-4 with the AdamW optimizer 33 The training time for both Drugs and QM9 was required around 3.8 days. For sampling, Drugs required 145 minutes for 14,324 conformers in 200 molecules, and QM9 required 71 minutes for 22,408 conformers in 200 molecules. We reported hyperparameters of EBD training including the maximum time step T, number layers (# I) and number of features (# d) in the deblurring networks, number of multi-hops (## of hops) and cutoff value for the expansion atom interactions, batch size, and number iteration in Table 7
Table 7: Hyperparameters of EBD.
Dataset
T
#
# d
# of hops
cutoff
batch size
training iter.
Drugs QM9
50
6
128
10 Á
32
650k
50
6
128
10 Á
64
650k
C.3
Performance of compared methods
For the results of compared methods in geometric evaluation of Drugs (Table 3 and QM9 (Table B). COV-R and MAT-R scores of CVGAE [34), GraphDG [7). CGCF 57 and ConfGF [66
were borrowed from 46). The performance of GeoMol and ConfVAE were borrowed from
and [59], respectively. In case of RDKit (3], we reported the performance from the generated conformers from RDKit that we utilized to compute the approximate fragment coordinates x PROKa (x'). For GeoDiff [59], we downloaded their implementation code from https://tiiii thub thub Wwwiiiiaiii/GeDDiiiiii teriffffttreemmiie and trained GcoDiff model for our experiments. We reported the performance of GeoDiff after sampling conformers using Langevin dynamics [51], they did in their implementation.
For the method in the ablation study, DecompDiff on coarse-gres -oause- structures. where the number of prior distributions corresponds to the number of fragments, and the mean of each prior is the respective fragment coordinates. By comparing the proposed method with DecompDiff in a controlled manner, we aimed isolate the effect of proposed blurring scheduler and random noise injection on learning in the coarse-to-fine molecular conformer generation task. Our use of DecompDiff was not to demonstrate its suitability for the molecular conformer generation task but rather to show that the stochastic trajectory from random
13 is denoising diffusion model conditioned
17

---

noise corruption is more challenging for the coarse-to carse-to-fine fine generation task than the proposed blurring schedule, even when the prior distributions are conditioned on the coarse gaasee-rained structures.
C.4 Pseudo-code
In this subsection, we provide the Pytorch-style [37 pseudo-codes, The RDKit conformer generator obtain the approximate fragment structure, linear interpolation blurring schedule, training process, and sampling process were given in Pseudo-codes and respectively.
import torch
import copy
from rdkit Chem import A1lChem
def
get mol - copy. dwooeeooooopyyoolclllll mol ( ps ALIChem.ETDG()
(molecule, num_conf).
pa mt..axtteratiinss 5000
10
raa.radooSSed 2023
ps useBaaickoooleeeeee = False
ps www.aepporroiiionngaeee.........
False
13
ps. useRandonCoords
False
14
ids = Al1Chem. Bag.heeeenbballlllpeeeeoorrrssse (mol, num_conf, ps)
15
if -1 in ids or mol. Ce...e/e/ != num_conf:
print (Use DG random coords.")
17
ps. useRandonCoords
True
18
ids AllChem. Rnbedhaltippeeonnfs (mol, num_conf, ps)
19
confs []
for cid in range (num_conf):
confs append (torch. tensor (mol GetConformers (cid). GetPositions ()
22
23
return confs
Pseudo-code 1: Initial atom coordinate generation from RDKit.
import torch
def blurring(t, x_a_gt, x_f_rdkit, mapping.matrix).
# prior distribution
e_f_rddii_exttend capping.matris x_f_rdkit
7
move positions to zero center of mass subspace wemove_mean(ee...eee r_f_rdxit_extend remove mean fx.__fxdkiteeeteee
-ass
x_a_gt remove mean (x__gx_))))
10
11
linear interpolation
12
blurred_pos torch. lerp (x_a_gt, z_f_ref_ext_ee t)
return blurred_pos
Pscudo-code 2: Blurring schedule in Eq.
import torch
def (x_agg_,_ x_f_rdkit, mapping.matrix, sigma, T): sample time torch.randint(h. т, (1.)) T
blurred aton position from blurring schedule
blurred_pos blurring(t, x_a_g,, rdkit, mapping.matrix)
10
add noise
noise torch. randn_ like (blurred_pos) noise remove mean (noise)
18

---

blurred_pos blurred_pos + noise sigma
15
estimate ground truth state from blurred aton position x.a.gt_estimated deblur_network (blurred.pp. aapping.mttriz, t)
16
17
translate the zero center-or mass subspace
z_a-gt remove mean (x_a_gt) =_a_gt_est remove_meat (x_a_gt_estinnge estimated)
20
21
22
optimal rotation matrix from Kabsch algorithm rot_matrix Kabsch_aligamant (z.a-gt_est, x_a_gt)
24
25
mean squared error
26
loss mean ((x_a_gt rot_matria 0 x_a_gtt_stt ** 2)
return loss
Pseudo-code Training process in Algorithm
import torch import copy
def cample(x_f_rde
mapping_matrix, delta, T): # initial atom position located at fragment position
x_a_init mapping.matrix x_f_rdkit
x_a_init remove_mean (x____iinix__ x_a copy. deepcopy(xx._.iii)
for i in range (T-1, 0, -1):
i/T
12
13
# add noise noise = rarch_ noise remove_ (noise)
16
x_a x.a noise delta
17
18
# estimate ground truth state from blurred atom position (x_a, matrix, t)
x_a_gt_est
21
# translate to the zero of subspace
22
x_a_gt_est semove_mean (x_a_gt_est)
23
# optimal rotation matrix from Kabsch algorithm rot_matrix Kabsch (x.a.tt_stt x_a_init)
25
26
27
# next step from estimated ground truth and initial positions x_a_gt_est = rot matria @ x_a_gt_est
I_a blurring((i11))) x_a_gt_eat, x_a_init, mapping_matrx)
30
31
return x_a
Pseudo-code 4: Sampling process in Algorithm 2
D Further results on geometric evaluation
GEOM-QM9. We compared our EBD to the baseline RDKit and machine learning models on small molecules GEOM-QM9, and the results are reported in Table Compared to the most of machine learning models, EBD achieved superior performances especially on the precision score. We observed that RDKit, the distance geometry-basee conformer generator, outperformed in coverage metrics for small molecules. However, as the size molecules increases and the tasks become more challenging, RDKit suffers significant performance drop, shown in Table 3
Statistical significance. We report the statistical significance our model's improvements in geometric evaluation (COV-P, COV-R, MAT-P, and MAT-R scores). We measured p p-value from
19

---

Table 8: Geometric evaluation GEOM-QM9 benchmark (6 0.5A).
COV-R (%)
MAT-R(Á)
COV-P
(%) f
MAT-P
(A)
Models
Mean
Med
Mean
Med
Mean
Med
Mean
Med
RDKit
88.34
95.08
0.3544
0.2974
83.42
88.17
0.3747
0.3692
CVGAE
0.09
0.00
1.6713
1.6088
GraphDG
73.33
84.21
0.4245
0.3973
43,90
35.33
0.5809
0.5823
CGCF
78.05
82.48
0.4219
0.3900
36.49
33.57
0.6615
0.6427
ConfVAE
77.84
88.20
0.4154
0.3739
38.02
34.67
0.6215
0.6091
GeoMol
71.26
72.00
0.3731
0.3731
ConfGF
88.49
94.31
0.2673
0,2685
46.43
43,41
0.5224
0.5124
GeoDiff (T 5000)
88.02
92.33
0.2199
0,2116
53.72
52.36
0,4362
0,4259
EBD 50)
89.37
93.21
0.2374
0.1903
61.31
60.46
0.3622
0.3517
one-sided Wilcoxon signed-rank test (a non-parametric version of paired t-test) over those scores of EBD and GeoDiff [59] on Drugs and QM9, and the results are reported in Fig. g Except for the COV-R score on QM9, our EBD achieved statistically significant improvement in generating more diverse and more accurate conformers for every score either dataset, as evidenced by the p-value.
COWPP. p-velue 7, 1.99-07
COV-R, p.valuem2 65e-04
COV-P, p-value22 94e-11
COV-R, p-value-8.11
Frequency
Dua
100-
Trequency
Frequenccy
50
20-
0.00
0.25
0.50
0.75
1.50
0.00
0.2
0.50
0.75
1.50
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Parcentage
Parcentage
Parcentage
Percentage
pivalue-71188
p-velue-
p.voue-=1.
frequercy
Frequency
Frequency
Frequency
5
0.5
1.0
1.5
2.0
0.5
1.0
1.3
0.0
0.4
0.6
0.5
0.1
0.2
0.3
Angstrom
Angstram
Angstrom
Angstrom
(a) Drugs
(b) QM9
Figure 8: p-vulue of COV-P, COV-R, MAT-P, and MAT-R Drugs and QM9.
20

---

E Visualizations
We provide additional samples and sampling processes of EBD for the of Drugs in Figs. and the test set of QM9 in Figs. TO T2
Mo Malecular graphs
t=50
t=0
Figure 9: Sampling processes of EBD Drugs
Mo Molecula lar g graphs
t=50
t=0
OH
HO
Figure 10: Sampling processes of EBD on QM9.
21

---

Molecular graphs
Ground truth
EBD samp les
Figure 11: Visualization of molecular graphs, ground truth conformers, and samples of EBD on Drugs
22

---

Molecular graphs
Ground truth
EBD samp les
OH
2H OH
Figure 12: Visualization of molecular graphs, ground truth conformers, and samples of EBD on QM9.
23

---

Limitations
Although our Equivariant Blurring Diffusion achieves significant performance on coarse-to- tinfine generative problems in a hierarchical molecular conformer generation scheme, there are still several limitations.
Due to the change of the estimation target from the previous state (Eq. (12 ) to the ground truth state (Eq. (14)) during sampling (Algorithm 2), the next step x²_ cannot be directly computed from the current state x; and requires an additional step of the deterministic blurring function /B- This additional step in the sampling process can make the entire process slower compared to the previous state estimator.
As the size of molecule increases, the discrepancy between the ground truth x' and the approximate & ~ (xD) of fragment structures becomes more severe. This increased discrepancy can make more challenging for the model to learn the trajectory from the coarse fragment structures to the fine atomic details. To circumvent this issue, increasing the time step T to more than 50 can be applied. Also, for more accurate deblurring network than equivariant graph neural networks [45] we used, more powerful geometric graph neural networks [24] can be applied such as local complete frames 10 and higher-order tensors from spherical harmonies 5
G Broader impacts
We presented deep generative model for the coarse-to-fing generation of molecular conformers. Our proposed model can applied problems fragment- based drug discovery, such as scaffold hopping and linker generation, to achieve improved performance. In drug discovery applications, potential negative societal impacts may arise if the training set is contaminated and includes toxins. In such cases, the generated samples could potentially be harmful to humans. From a more general perspective, the generative models to which our model belongs can be misused to create false information that appears authentic. Therefore, users must be aware of the potential risks associated with generative models before using them.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See Sections and
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
No theoretical assumption was made.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 5.1 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
See Appendix
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cpippi.ccppulli//uuuii wwwtii/gguie///eeeeceoocc CoodeSubeiiie ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
27

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ftctps//neuurpp::: http://.....c..lccccccccccceeeaa
Question:
Answer: [Yes]
Justification: Ethics.
The research conducted this paper conform with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We used the pre-processed data of GBOM from 46
(https://
github. wwtp://Deeepappeeeeninnggggegee ConffFF which was released under MIT license. For
experiments, used the codes of EDM [19 hhttps://githut: com ehoogeboom/
e3 23_diffusion__oo for__fffoooooooeso.o_____ GeoDiff [59] (https://githut) cow/MinkaiXu/
GeoDiff), Geo.ol [[]] (https://githul: com com/Paataaniii//GeMMolll Torsional Dif-
fusion (23] /https://github. com http://coooooooossoooooaaaffffia
and RDKit [31 (http:// htttp://www.rkii.n org/ perrg//css//e tcocs/inded htteee.ttl).............mmmmmmim EDM, GeoDiff, GeoMol, and Torsional
Diffusion were released under MIT license. RDKit is licensed under Creative Commons
StributioonSSharAAlkke 4.0 License.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
29

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
30

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "xedrUFBmOf", "reviewer": "Reviewer_UJjj", "summary": "The paper introduces Equivariant Blurring Diffusion (EBD), a unique generative model for hierarchical molecular conformer creation. A coarse-to-fine production process is presented by the model, with an emphasis on producing fragment-level structures first and then honing them down to atomic details. The method guarantees equivariance of SE(3), which is necessary for molecular structures. Comparisons with the most recent models on drug-like chemicals reveal that EBD performs better in geometric and chemical evaluations, demonstrating its effectiveness.", "questions": "1. Have you looked into any other options except RDKit for creating initial fragment coordinates? To what extent does the quality of these initial coordinates affect EBD performance?\n2. Could you elaborate on how EBD scales up to more complex molecules? Have you used these datasets for any preliminary experiments?\n3. Although the conformers developed exhibit chemical plausibility, what is their performance in real-world scenarios like docking simulations or property prediction? Is it planned to validate the conformers that are generated in these kinds of real-world situations?\n4. There is a brief mention of generation times and training. Could you elaborate on the amount of computing power needed to train EBD and produce conformers? In what way does this differ from the resources required for other cutting-edge models?", "limitations": "NA", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "An important step forward is the two-step process of creating fragment-level structures and then honing in on atomic details. This hierarchical method fits in nicely with molecular structures' multiscale structure.\n In order to preserve the geometrical and physical integrity of molecular conformers throughout the generation process, the model guarantees SE(3) equivariance.\nThe experimental results show that EBD can generate accurate and diversified molecular conformers with fewer diffusion steps than state-of-the-art models.\nComprehensive ablation investigations and in-depth comparisons with current models are included in the study, which offers a comprehensive understanding of the design decisions and how they affect performance.\nThe examination of the chemical properties, which includes HOMO-LUMO gaps and energy estimates, provides a great deal of value by demonstrating that EBD can produce stable and chemically realistic conformers.", "weaknesses": "1. The implementation and computational resource requirements may become more complex due to the hierarchical structure and requirement for fragmentation. This might make the model less useful and accessible for wider applications.\n2. RDKit is used extensively in the first generation of fragment coordinates. The quality of the initial fragment structures that RDKit provides could limit the model's performance.\n3. The concept works well for drug-like molecules, but it hasn't been fully investigated how well it scales to larger and more complex molecular structures. The claims would be strengthened by additional validation using larger datasets or more complicated compounds.\n4. The geometric (RMSD) and chemical properties are the main metrics used for evaluation. An evaluation of the model's capabilities that is more thorough might be obtained by incorporating further metrics pertaining to the novelty and variety of the generated conformers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1 (W2).** RDKit is used extensively in the first generation of fragment coordinates. The quality of the initial fragment structures that RDKit provides could limit the model's performance. Have you looked into any other options except RDKit for creating initial fragment coordinates? To what extent does the quality of these initial coordinates affect EBD performance?\n\n**Answer.**  \n\n* Thank you for your feedback. Regarding the analysis on the relationship between the quality of RDKit fragment coordinates and the performance of the proposed model, please check the general response of **G-Q1**.\n\n* Multi-scale generative models consist of two stages: i) generating a coarse-grained structure from random noise, and ii) generating a fine-grained structure from the coarse-grained structure. We have prioritized developing the coarse-to-fine generation stage. This is because the development of 3D molecular conformer models for coarse-to-fine generative processes, such as designing data corruption that preserves the coarse-grained structure, has not been sufficiently explored. In contrast, generating a coarse-grained structure from random noise can be leveraged by various existing successful denoising diffusion models or off-the-shelf tools like RDKit.\n\n* As the reviewer mentioned, existing denoising diffusion models can also be applied to generate coarse-grained structures. However, this approach requires first training and generating coarse-grained structures, and then training and generating fine-grained structures, which may result in higher accuracy and diversity compared to using RDKit but will likely take more training time.\n\n\n\n**Q2 (W3).** The concept works well for drug-like molecules, but it hasn't been fully investigated how well it scales to larger and more complex molecular structures. The claims would be strengthened by additional validation using larger datasets or more complicated compounds. Could you elaborate on how EBD scales up to more complex molecules? Have you used these datasets for any preliminary experiments?\n\n**Answer.** \n* We sincerely appreciate your constructive suggestions. We believe that hierarchy utilized in EBD exists widely across molecular systems, ranging from proteins as linear polymers of amino acids to materials as lattices of molecules.  As reviewer pba2 mentioned, we believe that our proposed model could also be applied to backmapping problems of proteins, in addition to drug-like molecules. The goal of the protein backmapping problem is predicting the coordinates of side chain atoms given the protein backbone structure. Compared to drug-like molecules, proteins have repeating linear structures and larger sizes. While we do not yet have results from preliminary experiments, we anticipate that our proposed model, with modifications such as the addition of internal coordinate loss, can achieve promising results.\n\n\n**Q3.** Although the conformers developed exhibit chemical plausibility, what is their performance in real-world scenarios like docking simulations or property prediction? Is it planned to validate the conformers that are generated in these kinds of real-world situations?\n\n**Answer.** \n* Thank you for your suggestions regarding the future extensions of our proposed method. While our primary target task is generating molecular conformers through an unconditional coarse-to-fine generative model for 3D structures, we believe our approach can be extended to the (conditional) docking problem through maintaining SE(3) equivariance to the conditioning protein structures or pockets. When decomposing the ligand compound, we could use a decomposition method optimized for the docking problem instead of the principal subgraph approach we used. For example, DecompDiff decomposes the ligand into arms that interact with the pocket and a scaffold that connects these arms. By taking the averaged coordinates of these decomposed arms and scaffold as the coarse-grained structure of the prior distribution, and conditioning the deblurring networks on the protein pockets, we could apply our method to the docking problem. \n\n\n\n**Q4 (W1).** The implementation and computational resource requirements may become more complex due to the hierarchical structure and requirement for fragmentation. This might make the model less useful and accessible for wider applications. There is a brief mention of generation times and training. Could you elaborate on the amount of computing power needed to train EBD and produce conformers? In what way does this differ from the resources required for other cutting-edge models?\n\n**Answer.** \n* Given a molecular graph, performing decomposition and calculating the prior distribution of fragment coordinates before training the generative model is a key difference in resource requirements compared to other cutting-edge models. This preprocessing step does not require GPU usage. For GEOM-Drugs, calculating the coarse-grained prior distribution took 38 hours on 16 Intel Xeon 8352Y CPUs, averaging 3 seconds per molecule.\n\n* The training of the proposed model requires similar resources as other cutting-edge models do. We trained our model on a single A100 GPU for 3.8 days. The comparison model, GeoDiff, also required a similar training time. The primary factor influencing training time is the number of parameters in the deblurring networks. We used a 6-layer, 128-feature-dimension deblurring network with 2,457,356 parameters which was 3 times larger than the encoder of GeoDiff (803,858 parameters).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0ZXHzrFIqB", "reviewer": "Reviewer_78eu", "summary": "- This paper presents a model for small-molecule 3D structure generation, conditioning on its 2D molecular graphs. \n\n- The authors proposed a two-step process to address the problem: 1) first, using an off-the-shelf bioinformatics tool, RDKit, to generate a template scaffold structure; 2) then focusing on training a diffusion model to generate the fine-grained atom positions given the scaffold.\n\n- In essence, the model in the second step should learn to: 1) generate fine-grained atom positions given coarse fragment centers; 2) correct potential biases from RDKit-generated fragments.\n\n- To achieve this, they proposed a diffusion-like deblurring process inspired by heat diffusion (IHDM) but over a linear trajectory in Euclidean space, from fragment-averaged coordinates to predicted atomic coordinates. This design allows efficient training and sampling for the targeted problem.\n\n- Experiments on two small-molecule benchmarks show the model's superior performance compared to other generative model baselines.", "questions": "Q1. The superior performance of the proposed models might be due to 1) the accurate generation of fine-grained atomic positions and/or 2) correcting the biases from RDKit-generated scaffolds. However, which component plays a more important role is not clearly addressed. The authors showed in Section 5.2 that a small fragment size of $|S|=50$ achieves the best performance due to the decreased atomic-level details needing to be learned, raising a natural question of whether the main benefit was from correcting the prior biases. For example, can the model achieve similar or better performance without fragments (i.e., set $S = \\\\{\\text{atoms}\\\\}$ and the model only learns an error correction trajectory from the RDKit-predicted structure)?\n\n\nQ2. Related to Q1, one may wonder to what extent the model's performance relies on the quality of RDKit-generated scaffolds. Despite the discussion in the limitations, can the authors provide more analysis on EBD’s performance vs. RDKit’s performance?\n\n\nQ3. As discussed in 5.2, Effects of Data Corruptions, DecompDiff is the most similar model except for the choice of the diffusion process. However, the comparison with DecompDiff was limited: 1) it was not included in the full benchmark (Table 1); 2) T=50 steps were used for DecompDiff compared to T>200 in their paper, which might lead to different performances; 3) it was not clear if the authors retrained DecompDiff following the same setup, given the original DecompDiff was proposed for a different task (pocket-conditioned ligand generation). Can the authors provide clarification on above concerns?\n\n\nQ4. Sampling noise ($\\sigma$, $\\delta$) is a key hyperparameter in the proposed diffusion process. Can the authors provide theoretical or empirical analysis on the choices of these two parameters? Are the results sensitive to their choices?", "limitations": "The authors have included discussion on the limitation in Appendix F.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed model comprises a novel combination of rational design choices:\n    1. A scaffold-atomic two-step generation process that defers the first step to well-established tools, converting the problem to generate atomic details and correcting prior distribution.\n    2. Borrowing the idea from heat diffusion, it uses constant noise instead of varying noise levels as in regular diffusion models.\n    3. It uses updated trajectory matching objectives by matching $x_0$ instead of the next step $x_{t−1}$.\n- Empirical analysis shows these design choices bring noticeable improvement in sampling coverage and accuracy over previous diffusion models. They also present several ablation studies and analyses to understand the performance and pinpoint some design factors: 1) fragment size; 2) diffusion trajectory and noise schedules; 3) loss reparameterization.\n- The manuscript is presented in a clear manner and is easy to follow.\n- Overall, this paper demonstrates that certain design choices can lead to improved performance and can be valuable for further research in small-molecule generation tasks.", "weaknesses": "- While the authors demonstrated better empirical performance and conducted ablation studies to verify selected design factors, some questions still remain on why and how some of the factors are critical, particularly:\n  - The effect of using RDKit as prior distribution: see Q1 - Q2\n  - Experiment details on comparing constant noising schedule (proposed) to regular diffusion (DecompDiff like): see Q3\n  - The effect of choosing noising levels: see Q4\n\n\n- Minor Typos: page 16: Pseudo-code 1: label is not for training code but the RDKit conformer generator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1-1.** The superior performance of the proposed models might be due to 1) the accurate generation of fine-grained atomic positions and/or 2) correcting the biases from RDKit-generated scaffolds. However, which component plays a more important role is not clearly addressed. \n\n**Q2.** Related to Q1, one may wonder to what extent the model's performance relies on the quality of RDKit-generated scaffolds. Despite the discussion in the limitations, can the authors provide more analysis on EBD’s performance vs. RDKit’s performance?\n\n**Answer.** \n* Regarding the analysis on the relationship between the quality of RDKit fragment coordinates and the performance of the proposed model, please check the general response of **G-Q1**.\n\n\n**Q1-2.** The authors showed in Section 5.2 that a small fragment size of  achieves the best performance due to the decreased atomic-level details needing to be learned, raising a natural question of whether the main benefit was from correcting the prior biases. For example, can the model achieve similar or better performance without fragments (i.e., set  and the model only learns an error correction trajectory from the RDKit-predicted structure)?\n\n**Answer.** \n* Thank you for your interesting suggestion. While learning the trajectory from the atom coordinates generated by RDKit to the ground truth atom coordinates can be similarly achieved in EBD, it would deviate from the primary goal of this paper, which is to develop a coarse-to-fine generative model for multi-scale learning.\n\n\n**Q3.** As discussed in 5.2, Effects of Data Corruptions, DecompDiff is the most similar model except for the choice of the diffusion process. However, the comparison with DecompDiff was limited: 1) it was not included in the full benchmark (Table  1  – actually Table 2?); 2) T=50 steps were used for DecompDiff compared to T>200 in their paper, which might lead to different performances; 3) it was not clear if the authors retrained DecompDiff following the same setup, given the original DecompDiff was proposed for a different task (pocket-conditioned ligand generation). Can the authors provide clarification on above concerns?\n\n\n**Answer.**\n*  Regarding the comparison with DecompDiff in the ablation study, please check the general response of **G-Q4**.\n\n\n**Q4.** Sampling noise ($\\sigma, \\delta$) is a key hyperparameter in the proposed diffusion process. Can the authors provide theoretical or empirical analysis on the choices of these two parameters? Are the results sensitive to their choices?\n\n\n**Answer.**\n* Regarding the choice of noise scales in forward and reverse processes, please check the general response of **G-Q2**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5hTVIqHB8P", "reviewer": "Reviewer_8MLu", "summary": "The paper addresses the question by focusing on a fundamental biochemical problem: generating 3D molecular conformers based on molecular graphs in a multiscale manner. It consists of two stages:\n\n1. Generating a coarse-grained fragment-level 3D structure from the molecular graph.\n2. Generating fine atomic details from the coarse-grained approximated structure while allowing simultaneous adjustments to the latter.", "questions": "None", "limitations": "None", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposed the EDB, which can generate atomic details from a coarse-to-grained estimation of fragment structures using equivariant networks\n2. The paper proposed a novel blurring scheduler and a revised loss function that significantly impacts performance instead of directly applying those of the existing image blurring diffusion model.\n3. The experiments and analysis demonstrate more plausible conformers compared to SOTA denoising diffusion models.", "weaknesses": "None", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for appreciating our work and for providing a great summary.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JxZ7zNRhuJ", "reviewer": "Reviewer_pba2", "summary": "The paper proposes a novel diffusion method for molecular conformers based on blurring diffusion. The method utilizes RDKit to predict the 3D structure of small molecule fragments and trains a diffusion model to generate the full-atomistic molecule from the RDKit prior, leveraging hierarchical modeling. The method is evaluated based on the GEOM dataset, testing both the quality of the sampled geometries and physical properties of the conformers, as is standard in the field.", "questions": "1. How many parameters does the model use?\n2. Did the authors try to build an end-to-end pipeline, where a generative model predicts the coarse-grained coordinates instead of RDKit?", "limitations": "The authors discuss the limitation of RDKit in application to larger molecular structure and the additional cost of the deblurring function.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors apply blurred diffusion developed for image applications to the domain of molecular conformer generation to leverage hierarchical modeling in molecular settings.\n2. The problem of coarse-to-fine prediction is a core problem in coarse-grained molecular modeling (referred to as backmapping in the respective literature) and the proposed approach might be applicable these (large-scale) problems as well.\n3. The proposed method demonstrates good performance compared to other diffusion-based approaches with models of comparable size in the literature.", "weaknesses": "1. The authors did not consider the molecular conformer fields (MCF) paper (https://arxiv.org/pdf/2311.17932), which is the current state-of-the-art approach to molecular conformer generation. The MCF method is more performant than the proposed approach.\n2. The approach depends strongly on coarse-grained fragment generation via RDKit, which could become a problem for larger systems of practical relevance. \n3. The authors should consider also reporting their performance metrics on the stricter threshold on GEOM-Drugs (delta = 0.75 A).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.** The authors did not consider the molecular conformer fields (MCF) paper, which is the current state-of-the-art approach to molecular conformer generation. The MCF method is more performant than the proposed approach.\n\n**Answer.** \n* Regarding the comparison with strong baseline, please check the general response of **G-Q3**.\n\n\n\n**W2.** The approach depends strongly on coarse-grained fragment generation via RDKit, which could become a problem for larger systems of practical relevance.\n\n**Answer.** \n* Regarding the analysis on the relationship between the quality of RDKit fragment coordinates and the performance of the proposed model, please check the general response of **G-Q1**.\n\n\n**W3.** The authors should consider also reporting their performance metrics on the stricter threshold on GEOM-Drugs (delta = 0.75 A).\n\n**Answer.**  \n|          | COV-R mean | COV-R med | COV-P mean | COV-P med |\n|----------|:----------:|:---------:|:----------:|:---------:|\n| RDKit DG |    12.29   |    2.5    |    7.25    |    1.04   |\n| GeoDiff  |    38.29   |   32.82   |    20.8    | **14.38** |\n| EBD      |  **42.07** |  **35.5** |  **21.73** |    13.3   |\n* We measured the coverage scores for RDKit DG, GeoDiff, and the proposed method on GEOM-Drugs when delta is 0.75 A.\n\n\n\n**Q1.** How many parameters does the model use?\n\n**Answer.**  \n* Our equivariant deblurring network has 2,457,356 parameters when the number of layers $l$ (line 531) is 6 and the feature dimension $d$ (line 533) is 128. Each layer consists of an invariant fragment feature update function, an invariant atom feature update function, and an equivariant atom coordinate function.\n\n\n**Q2.** Did the authors try to build an end-to-end pipeline, where a generative model predicts the coarse-grained coordinates instead of RDKit?\n\n**Answer.** \n* Thank you for your insightful suggestion. End-to-end multi-scale learning is indeed our ultimate goal. Among the two stages—generating coarse-grained structures from random noise and generating fine-grained structures from coarse-grained structures—we have focused more on developing the coarse-to-fine generation stage. This is because generating coarse-grained structures from random noise can be leveraged by many existing, successful denoising diffusion models or off-the-shelf tools like RDKit, whereas methods for coarse-to-fine generation have not been sufficiently explored.\n* We believe that the proposed method can be effectively utilized in the coarse-to-fine generation stage, and we plan to explore combining these two stages into a single end-to-end model. An end-to-end generative model needs to generate m fragments and then n atoms, which presents the challenge of changing the dimension of the state value as the time steps increase or decrease. Exploring methods to handle dimension changes during the generation process [1] is a promising future direction.\n\n    [1] Campbell, Andrew, et al. \"Trans-dimensional generative modeling via jump diffusion models.\" Advances in Neural Information Processing Systems 36 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sP9qh8uygn", "reviewer": "Reviewer_sBnm", "summary": "In this work, the authors introduce an hierarchical diffusion model for molecular conformer generation. The framework starts with fragment positions initialized by RDKit and designs a diffusion process that generates atomic positions from substructure positions. The reverse diffusion process is modeled by an equivariant neural network. Experiments on standard molecular conformer generation benchmarks: GEOM-QM9 and GEOM-DRUGS show better performance than some previous baselines. The authors also include ablation studies that validate some important design choices in the model.", "questions": "1. Following #1 in weakness, the final blurring operator (eq. 7) is basically a linear interpolant in Euclidean space between the substructure coordinate space and atom coordinate space. Unlike usual linear interpolant that starts from random Gaussian, here it starts from substructure coordinate space. I feel it is a bit confusing to introduce the framework as a variant of blurring diffusion which can obscure the actual contributions of this paper. \n2. Following # 2 in weakness, there are recent deep generative models for molecular conformation generation [1,2] that achieve state-of-the-art but are not included in the comparison to proposed method. The authors are strongly recommended to include the comparison to better validation the performance. \n3. Following # 3 in weakness, the work relies on principal subgraph (PS) to obtain molecular fragments. I wonder if the authors have tried other cheminformatic methods like BRICS [3]. \n4. Also, the vocabulary size |S| is set to 50, which means there are quite some isolated atoms. I wonder what are the ratios of isolated atoms. \n5. The work designs a diffusion process from substructure space to atom space, I wonder how it compare with a standard diffusion model that conditioned on substructure coordinates. Have the authors by any chance investigated similar settings?\n5. How is $\\delta^2$ in Eq. 8 determined in diffusion process?\n\nReferences:\n\n[1] Torsional Diffusion for Molecular Conformer Generation: https://arxiv.org/abs/2206.01729\n\n[2] Swallowing the Bitter Pill: Simplified Scalable Conformer Generation: https://arxiv.org/abs/2311.17932\n\n[3] On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces: https://chemistry-europe.onlinelibrary.wiley.com/doi/abs/10.1002/cmdc.200800178", "limitations": "The authors adequately addressed the limitations in the work.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The model investigates a popular yet important problem: molecular conformer generation with deep generative model. \n2. The proposed model is benchmarked on standard dataset to demonstrate its performance.", "weaknesses": "1. The model is introduced in a way that it is derivated from blurring diffusion. However, the final formulation doesn't seem to strongly relate the concept. \n2. The work misses comparison with some strong baselines in molecular conformer generation. \n3. Though there are ablation studies to validate some design choices, there are still some important framework designs that are not well discussed. \n\nMore details can be found in the following Questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1.** Following #1 in weakness, the final blurring operator (eq. 7) is basically a linear interpolant in Euclidean space between the substructure coordinate space and atom coordinate space. Unlike usual linear interpolant that starts from random Gaussian, here it starts from substructure coordinate space. I feel it is a bit confusing to introduce the framework as a variant of blurring diffusion which can obscure the actual contributions of this paper.\n\n**Answer.** \n* First and foremost, we want to emphasize that the purpose of the blurring operation is to transform the atomic positions in 3-dimensional Euclidean space from a coarse-grained structure (fragment coordinates) to a fine-grained structure (atomic coordinates). \n\n* We attempted to use the blurring operator in the spectral domain (Eq. 2) to transform the atomic coordinates in the spatial domain. However, as explained starting from line 154 of the manuscript, the eigendecomposition of the graph Laplacian for each fragment requires excessive time. Additionally, a large T is required to converge the positions of atoms to the prior fragment coordinates. The problem is that fragments vary in size and structure and this makes it difficult to model uniform atomic movement for different fragments with a single T value. Lastly, there is a discrepancy between the ground truth fragment coordinates, which are the convergence result of the spectral operator, and the prior RDKit fragment coordinates.\n* Therefore, we aimed to introduce an operator that maintains the essence of blurring, meaning the gradual transition from coarse-grained to fine-grained structures, while being computationally efficient, less affected by the varying sizes and structures of fragments, and considering the discrepancy between fragment coordinate distributions. The result is a linear interpolation between the coarse-grained and fine-grained distributions in the spatial domain (Eq. 7).\n* In summary, while the operator is a linear interpolation in the spatial domain, it is derived from characteristics in the spectral domain.\n\n**Q2.** Following # 2 in weakness, there are recent deep generative models for molecular conformation generation [1,2] that achieve state-of-the-art but are not included in the comparison to proposed method. The authors are strongly recommended to include the comparison to better validation the performance.\n \n**Answer.** \n* Regarding the comparison with strong baseline, please check the general response of **G-Q3**.\n\n\n**Q3.** Following # 3 in weakness, the work relies on principal subgraph (PS) to obtain molecular fragments. I wonder if the authors have tried other cheminformatic methods like BRICS [3].\n\n**Answer.** \n* We chose Principal Subgraphs for the following reasons: \n   * There are no overlapping atoms between fragments, which prevents the case where an atom in the prior distribution is present in the coordinates of more than one fragment. \n   * We can set the size of the fragment vocabulary, allowing us to observe the impact of fragment granularity on generative performance.\n\n* We also considered using the well-known BRICS [1]  and tree decomposition [2]  methods but encountered the following issues: \n    * With BRICS, it is impossible to adjust the size of the vocabulary, preventing us from observing performance based on fragment granularity. Additionally, BRICS generates large fragments with very low frequencies, which can impact generalization performance. For GEOM-Drugs, the BRICS vocabulary contains 11,356 fragments with an average size of 19.98 atoms. Moreover, 73.3% of the fragments in the vocabulary occur fewer than ten times in the entire dataset.\n   *  Tree decomposition, as observed in the analysis of Principal Subgraphs paper, generates too fine fragments. For GEOM-Drugs, tree decomposition generates fragments of size 1 (isolated atoms) and size 2 that have a 95.56% frequency of occurrence in the entire dataset. Additionally, there are overlapping atoms between different fragments.\n\n    [1] Degen, Jorg, et al. \"On the art of compiling and using 'drug-like' chemical fragment spaces.\" ChemMedChem 3.10 (2008): 1503.\n\n    [2] Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. \"Junction tree variational autoencoder for molecular graph generation.\" International conference on machine learning. PMLR, 2018.\n\n\n**Q4.** Also, the vocabulary size |S| is set to 50, which means there are quite some isolated atoms. I wonder what are the ratios of isolated atoms.\n\n**Answer.** \n|                                                   | PS50   | PS200  | PS1000 | BRICS  | Tree   |\n|---------------------------------------------------|--------|--------|--------|--------|--------|\n| Occurrence frequency of single atom fragments | 0.4699 | 0.4478 | 0.4362 | 0.8857 | 0.6139 |\n|                                                   |        |        |        |        |        |\n\n* We measured the occurrence frequency of single atom fragments for Principal Subgraphs (|S|=50, 200, 1000), BRICS, and tree decomposition on GEOM-Drugs.\n\n* The results in the table show that PS had similar frequency values across different vocab sizes. In contrast, BRICS and tree decomposition, which exhibit significant variations in occurrence frequency based on fragment size, showed notably high frequencies for single atom fragments.\n\n\n**Q5.** The work designs a diffusion process from substructure space to atom space, I wonder how it compare with a standard diffusion model that conditioned on substructure coordinates. Have the authors by any chance investigated similar settings?\n\n**Answer.** \n* Regarding the comparison with DecompDiff in the ablation study, please check the general response of **G-Q4**. \n\n\n**Q6.** How is $\\delta^2$  in Eq. 8 determined in diffusion process?\n\n**Answer.** \n* Regarding the choice of noise scales in forward and reverse processes, please check the general response of **G-Q2**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate all the reviewers for their constructive feedback and suggestions. Below, we provided general responses to the questions raised by several reviewers.

**G-Q1.** Quality of prior vs Performance.

**Answer.**
* To observe the model's performance based on the quality of fragment coordinates, we measured how accurately the fragment coordinates generated by RDKit were corrected towards the ground truth. For 200 molecules in the GEOM-Drugs test set, we measured the RMSD between RDKit fragment coordinates and ground truth fragment coordinates (RMSD(x_{RDKit}^f, x_{gt}^f)) and the RMSD between fragment coordinates generated by our model and ground truth fragment coordinates (RMSD(x_{EBD}^f, x_{gt}^f)). If RMSD(x_{EBD}^f, x_{gt}^f) is lower than RMSD(x_{RDKit}^f, x_{gt}^f) for a molecule, it indicates that the model has accurately corrected the fragment coordinates. 
* In Figure 1 of the attached PDF, the points below the red line represent cases where the model corrected the coordinates accurately. We observed that the greater the RMSD(x_{RDKit}^f, x_{gt}^f) (points further to the right on the x-axis), the larger the reduction in RMSD towards RMSD(x_{EBD}^f, x_{gt}^f). In other words, the lower the quality of the coarse-grained prior, the more accurately the model tends to make corrections.



**G-Q2.** Choice of noise scales in forward and reverse processes.

**Answer.** 
* We apologize for not including the details about the noise scale in the manuscript. We aimed to use low noise scale values in the proposed blurring and deblurring processes. Thus, in all experiments, we used a noise scale of 0.01 for the forward process ($\sigma$ in Eq. 6) and 0.0125 for the reverse process ($\delta$ in Eq. 8) based on the noise scale analysis in IHDM (Appendix C.1 in IHDM). Referencing the analysis, we ensured that the delta/sigma noise scale ratio was slightly above 1. We observed that using a noise scale small but not too close to 0 and setting the delta/sigma ratio slightly above 1 were suitable for training the blurring diffusion model.



**G-Q3.** Strong baselines.

**Answer.**
* Thank you for bringing up these great relevant studies. We discovered that due to the different data splits used in these papers, it is challenging to directly compare their performance with our results. For GEOM-Drugs, we used the data split proposed by ConfGF: train/val/test = 40,000/5,000/200 molecules. In contrast, MCF and Torsional Diffusion used the data split proposed by GeoMol: train/val/test = 243,473/30,433/1,000 molecules. Additionally, for MCF, we kindly ask for your understanding as comparing performance is difficult without access to their implementation code. In a case of comparison between Torsional Diffusion, we are implementing experiments comparing our model with torsional diffusion. We will include some of the experiments in future revisions.


**G-Q4.** Comparison with DecompDiff in the ablation study.

**Answer.** 
* We appreciate your feedback and would like to elaborate on the motivation, experimental settings, and results of the ablation study on the effects of data corruption (line 279).
* **Motivation:** DecompDiff is a denoising diffusion model conditioned on coarse-grained structures, where the number of prior distributions corresponds to the number of fragments, and the mean of each prior is the respective fragment coordinates. By comparing the proposed method with DecompDiff in a controlled manner, we aimed to isolate the effect of the proposed blurring scheduler and random noise injection on learning in the coarse-to-fine molecular conformer generation task.

    Our use of DecompDiff was not to demonstrate its suitability for the molecular conformer generation task but rather to show that the stochastic trajectory from random noise corruption is more challenging for the coarse-to-fine generation task than the proposed blurring schedule, even when the prior distributions are conditioned on the coarse-grained structures. As the reviewer 78eu clearly pointed out, DecompDiff has shown effectiveness in generating ligand compound structures docking to the target protein. They proposed a fragment decomposition method (scaffold-arms) specialized for protein-ligand complexes, rather than Principal Subgraphs we used. Since DecompDiff was designed for protein-ligand complex problems and the fragment decomposition (Principal Subgraphs) used in our experiments is not aligned with their target task, we did not include the full report of its performance in Table 2.

* **Experimental settings:** Except for the data corruption methods, specifically the proposed blurring schedule and random noise, we used the same coarse-grained prior distribution, encoder design, ground truth estimator (please note that DecompDiff also used a ground truth state estimator (Eq. (8) of DecompDiff)), and number of time steps when comparing our method and DecompDiff. This controlled setting was taken to isolate the contribution of data corruption to the coarse-to-fine generative task fairly and clearly, without any entanglement of other factors. 

* **Results:** The Table 1 in the attached pdf is a full report of the performance of DecompDiff in the experimental settings above, and detailed results were presented in Figure 3 (c) and Figure 4 of the manuscript. We observed that the conformers generated from EBD show better diversity scores compared to the stochastic trajectory, since the proposed blurring schedule of EBD facilitates the learning process of the coarse-to-fine generative models.

Meta-review: The paper introduces Equivariant Blurring Diffusion (EBD), a novel generative model for hierarchical molecular conformer generation. The model employs a coarse-to-fine process, starting with the production of fragment-level structures and refining them down to atomic details. It ensures SE(3) equivariance, which is crucial for maintaining the geometric integrity of molecular structures.

Strengths:
+ The proposed model features a novel combination of rational design choices, including a scaffold-to-atomic two-step generation process, an innovative blurring scheduler, and a revised loss function.
+ Empirical analysis demonstrates that these design choices lead to significant improvements in sampling coverage and accuracy over previous diffusion models. 
+ The manuscript is presented in a clear manner and is easy to follow.

Weaknesses:
- The quality of these initial structures provided by RDKit could potentially limit the model's overall performance.
- The work misses comparison with some strong baselines in molecular conformer generation.

In their rebuttal, the authors provided additional experiments and explanations on how accurately the proposed method corrected the fragment coordinates generated by RDKit towards the ground truth. They also considered reporting their performance metrics using a stricter threshold on GEOM-Drugs (delta = 0.75 Å). Additional evaluations were conducted on diffusion from RDKit's all-atom positions, along with further ablations regarding DecompDiff. Three of the five reviewers were satisfied with the authors' response, and two upgraded their scores. However, during the discussion period, Reviewer pba2 continued to express concerns regarding the comparison with the MCF method and the model's reliance on RDKit. After reviewing the authors' explanations on these points, I believe they have made valuable contributions to the problem of coarse-to-fine molecular conformation generation.

Overall, this paper demonstrates that specific design choices can significantly improve performance and contribute to further research in small-molecule generation tasks. The authors are strongly suggested to carefully consider the reviewers' comments and revise their final version accordingly.
Decision: Accept (poster)