Training-free Multi-objective Diffusion Model for 3D Molecule Generation
Authors: Xu Han
Caihua Shan
Yifei Shen
Can Xu
Han Yang
Xiang Li
Dongsheng Li
Venue: ICLR 2024 poster (2024)
Keywords: Multi-objective Diffusion Model
3D Molecule
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Searching for novel and diverse molecular candidates is a critical undertaking in drug and material discovery. Existing approaches have successfully adapted the diffusion model, the most effective generative model in image generation, to create 1D SMILES strings, 2D chemical graphs, or 3D molecular conformers. However, these methods are not efficient and flexible enough to generate 3D molecules with multiple desired properties, as they require additional training for the models for each new property or even a new combination of existing properties. Moreover, some properties may potentially conflict, making it impossible to find a molecule that satisfies all of them simultaneously. To address these challenges, we present a training-free conditional 3D molecular generation algorithm based on off-the-shelf unconditional diffusion models and property prediction models. The key techniques include modeling the loss of property prediction models as energy functions, considering the property relation between multiple conditions as a probabilistic graph, and developing a stable posterior estimation for computing the conditional score function. We conducted experiments on both single-objective and multi-objective 3D molecule generation, focusing on quantum properties, and compared our approach with the trained or fine-tuned diffusion models. Our proposed model achieves superior performance in generating molecules that meet the conditions, without any additional training cost.

Published as a conference paper at ICLR 2024
TRAINING-FREE MULTI-OBJECTIVE MODEL FOR 3D MOLECULE GENERATION
DIFFUSION
Xu Han', Caihua Shan², Yifei Shen', Can Xu², Han Yang', Xiang Li", Dongsheng Li Tufts University, *Microsoft Research Asia
East China Normal University, *Microsoft Research A14Science
Xu. Han@tufts edu, leoxc1 Mexc1571@@@ 63. com, xiangl lacggli@dda 1edase. ecnu. ecnu.edu, on [caihuashan, yifeishen, hanyang, dongsheng. 1i li.emiccosooft com
ABSTRACT
Searching for novel and diverse molecular candidates critical undertaking in drug and material discovery. Existing approaches have successfully adapted the diffusion model, the most effective generative model in image generation, to cre- ate ID SMILES strings, 2D chemical graphs, 3D molecular conformers. How- ever, these methods are not efficient and flexible enough to generate 3D molecules with multiple desired properties, as they require additional training for the models for each new property or even new combination of existing properties. More- over, some properties may potentially conflict, making it impossible find a molecule that satisfies all of them simultaneously. To address these challenges, we present a training-free free conditional 3D molecular generation algorithm based off-the-shelf unconditional diffusion models and property prediction models. The key techniques include modeling the loss of property prediction models as en- engy functions, considering the property relation between multiple conditions probabilistic graph, and developing stable posterior estimation for computing the conditional score function. We conducted experiments both single-objective and multi-objective 3D molecule generation, focusing on quantum properties, and compared our approach with the trained or fine-tuned diffusion models. Our pro- posed model achieves superior performance in generating molecules that meet the conditions, without any additional training cost.
INTRODUCTION
Diffusion models have emerged as a powerful family of deep generative models across various domains, including image generation, audio synthesis, etc. Due their ability generate novel and diverse molecule structures, they help accelerate the process of drug discovery by reducing the need for expensive and time-consuming wet experiments Schwalbe-Koda & Gómez-Bonmareilij 2020).
Here we focus on 3D molecules, as 3D structures provide a more accurate representation of the spatial arrangement of atoms and molecule's geometric symmetry in comparison to 2D graph rep- resentation. This enhanced accuracy results in better generalization (Thomas et al. 2018 Fuchs al. 2020 Finzi et 2020). Moreover, 3D molecule design enables the identification of promis- ing drug candidates with a range of characteristics, such as quantum properties (Ramakrishnan et al., 2014), chirality (Adams al. 202T). binding affinity with specific proteins (Lin al. (022)) etc.
Given its practical importance, there have been some pioneering works on conditional 3D molecule generation (Hoogeboom et al. 2022; Xu al. 2023 Bao al. 2022). For instance, EDM (Hooge- boom et al. 2022) is the first proposed diffusion model for 3D molecules, which learns equiv- ariant neural network that jointly operates on both atom coordinates and atom types. Despite good performance in generating molecules with single desired property, the flexibility and efficiency are limited, First, additional training is required to incorporate conditional guidance. EDM and GEOLDM (Xu et al. 2023) directly retrain
diffusion model conditioned particular prop-
This work was done during Xu Han's internship at MSRA. Correspondence to: Caihua Shan (cai- huashan @ microsoft.com Dongsheng Li (dongsheng.li@miiroooo

---

Published as a conference paper at ICLR 2024
erty. EEGSDE (Bao et al.) 2022) trains time-dependen, energy function to perform the guidance. Substantial efforts should be paid to collect the dataset and train models, which is non-negligible. Additionally, training time-dependent energy function is difficult task, the molecules at the early stages of diffusion are noisy and even invalid. Second, to accommodate multiple conditions, all previous works need to retrain the model for every combination of conditions, resulting in un- affordable costs. Third, the given properties might conflict, making it hard to find a molecule that meets all conditions. Under such cireumstances, existing methods also lack sufficient flexibility to achieve the best balance among multiple properties, further limiting their effectiveness.
To address the above challenges, we propose MuDM, a training-free multi-objective 3D molecule generation based on off-the-shelf pre-trained diffusion model and property prediction functions. Considering the intermediate molecules 2, are noisy, we map to the expected final molecule 2g by posterior approximation, use Monte Carlo sampling to compute the gradient of the property func- tions with low variance, and map back to update 7;-1 by chain rule (Chung et al.) 2022, Song al.) 2023). To tackle the multiple conditions, we utilize the probabilistic graph to model the complex relationships of properties, and adjust the expected final molecule 2n based on the dependency of properties in the graph. The overall conditional guidance is then calculated as the weighted sum of the gradient of each property with respect to the expected revised final molecule zn- The revised 2 and gradient weights potentially identify the importance of properties and mitigate the conflict among given conditions.
We conducted the experiments both single-objective and multi-objective tasks, aiming generate 3D molecules with specific quantum properties. Compared with training-requien methods, our proposed training- free MUDM achieves superior performance in all cases for single-objective tasks. Furthermore, MUDM is capable of generating 3D molecules with any combination of properties, resulting in an average reduction of 30% in mean square error for each property.
Our contributions can be summarized as follows:
1. By incorporating posterior approximation and MC sampling, MUDM can provide accurate guidance that depends solely on off-the-shhe time-independent property functions.
2. MUDM capable of managing complex relationships among properties as a probabilistic graph, and then deriving the effective approximation for the multi-objective guidance.
3. Experimental results reveal that MUDM is an effective and efficient method to design 3D molecules with desired properties, offering great flexibility in exploring the chemical space.
2 RELATED WORK
Diffusion Models. Diffusion models (Ho et al. 2020 Song al. 2020) generate samples by mod- eling the generative process as the reverse of the noising process. Specifically, the noising process injects the noise into the ground truth data. The generative process learns the score functions, which is the gradient of the ground truth data distribution's 's log-density, to reverse the noising process.
Conditional diffusion models enable the generative process with constraints, where the score func- tion should be learned not only from the ground truth data distribution but also from the given constraints. They can be broadly categorized into two types: training-required and training-free A well-known example of training-required methods is Stable Diffusion (Rombach et al. 2022) for text-guided image generations. However, the extra training cost can be significant, particularly in scenarios that require complex control with multiple conditions. Consequently, our paper focuses on exploring training-fre -free methods for their efficiency and flexibility in handling diverse constraints.
Molecule Generation with Desired Properties. To the best of our knowledge, current approaches for conditional 3D molecule generation mostly utilize the condition diffusion models. EDM (Hooge- boom et al. 2022) and GEOLDM (Xu et al. 2023) trained a specific diffusion model for each prop- erty. EEGSDE [Bao et al. (022) is more efffciintt to train time-dependent property function instead of re- re-training a new diffusion model, and then guide the generative process. In contrast, our pro- posed method is training- free, which only requires the off-the-shhd pre-trained diffusion model and time- -ndeeeddeenndnnt prediction functions, making the most efficient option. Furthermore, the quality of generated 3D molecules is also confirmed by the experiments.
2

---

Published as a conference paper at ICLR 2024
Other traditional techniques, such as reinforcement learning (De Cao & Kipf 2018 You et al. 2018 Popova et al., 2018 Shi et al. 2020) and genetic algorithms (Jensen 2019: Ahn et al. 2020 Nigam et al. 2019), have been applied to generate molecules with specific properties. Additinnlly, recent studies have also tackled multi-ohjective molecule problems, as seen in (Jin et al. 2020 Xie et al. (02T). However, they all consider 2D) graphs rather than 3D conformers, posing challenges in directly applying their techniques to 3D molecule generation.
3 BACKGROUND
3D Molecule Representation. Let us consider molecule consisting of N atoms. For the ith atom, its coordinate can be represented as R3, denoting the three-dimensional space. Collectively, the coordinates for all atoms can be represented as x = (xi,..., xN) E Nx which encapsulates the molecule's 's conformation. Beyond the spatial coordinates, every atom also has specific features, such as atomic type. Let h, E IR symbolize the feature vector for the 7th atom. Aggregating the features for all atoms, represented as G = X hj. integrating both its three-dimensional geometry and atomic features.
obtain h (h1,-.. ,hy) E Nxd Consequently, molecule can be
Invariance. Two fundamental properties associated with 3D molecules invariance and equiv- ariance. Given a transformation Rt a distribution p(x, h) is said to be invariant to R if: p(x, h) = (lxx, h) for all x and h. Here fxx (Iix1, RxM) denotes the transformation applied each coordinate. Further, a function f(x, h) is invariant to R if: f(ix, h) f(x, h) for all x and h. Invariance implies that the transformation does not affect the outcome.
Equivariance. Consider function f with outputs ax and ahe i.c. (ax, oh) = f(x, h). This function said to be equivariant R. if: f(Rx, h) (Rox, an) for all x and h. Equivariance means that changing the order of and R does not affect the outcome.
Equivariant diffusion Model (EDM) and Geometric Latent Diffusion Models (GEOLDM). They are both diffusion models for 3D molecules, which are built upon the principles proposed by (Ho et al. 2020; Kingma et al.] 202T), but with different spaces to conduct the noising and gen- erative process. The nosing process of EDM is defined in the original molecule space G [x, h):
(91T)19) II (( (9.9)111) Nan (9r Vor9t-1,,91 (=1
where Nan(ir Va.9h-1,8.1
EDM adds the noise into atom coordinates and atom features separately, where a₁ + Bg and they both control the noise scale. Nx is the Gaussian distribution of the coordinates in the zero CoM subspace to guarantee the translational invariance. Accordingly, the reverse generative process of EDM is:
Varxe-1 B,1,,,,,)h(;;; va,h,
p(Cirl9)= = p(Gr) II 19)),, p(G,-199)) = (x (xx), ( (he (h.),
Different from EDM, GEOLDM is to capture the 3D molecule's cquivariance and invariance con- straints in the latent space. GEOLDM learns the encoder z = [zz, z,] Es h) and the decoder h = (z),)), where & and De are both equivariant graph neural networks (EGNNs, Satorras et al. (202T)) For all the rotations and translations they satisfy
Rz, + t, 2n = Eg + t, h), Rx + t, = Det.. + 2n), p(zz, kn.) = pl Razas zn). (1)
The diffusion process of GEOLDM is also defined on the latent space. Its noising process and reverse generative process are similar to EDM, where [Zz,t, 2n,]]:
q 21-) = N N(z: 0; Be1), po po(zi-1lz.) = (o
The optimization objective of GEOLDM is:
Br
p (Ze, t) = Var
(z:
(0 t)), Be ea (z t)]
V1
Inverse problems. Conditional diffusion models have shown great potential in solving inverse problems. Suppose have a measurement y E R from the forward measurement operator A(.),
3

---

Published as a conference paper at ICLR 2024
Nutteeijective nuidance
Zt
+
400
Probabilaatic Cruph
Zg Reverise Generative Procsse Oiftuston Model
7
7
Ptaple
POSTR
Figure I: Overview: MUDM generates a molecule from the time T to 0 using pre-trained diffusion model, guided by multiple property functions. The relationship of multiple properties modeled by given probabilistic graph. At some time step, the guidance is computed as the weighted sum of the gradient of each property with respect to the latent variable considering the property dependencies.
which satisfies y =-=AAff A(x) + n where represents gaussian noise. Given the observed measurement y, diffusion models can be used to retrieve x by replacing the score function Vx, log Pr (x,) with the conditional score function Vx, log (x: (y). However, the absence of an analytical formulation for log pe (x(xxe) poses a significant challenge, leading to many studies (Chung et al. 2022 Song et al, 2022) proposing various approximations to address this issue.
METHODOLOGY
In this section, we treat conditional 3D molecule generation inverse problem and develop feasible solution shown in Fig- Traditionally, molecular design has been a forward process, where researchers synthesize and test molecules observe their properties. Inverse molecular design, in contrast, reverses the process by starting with the desired properties and property functions, and working backward identify molecules that exhibit those characteristics. We will first discuss how to generate a molecule with single property, and delve into multi-objective tasks in the next section.
4.1 SINGLE-CONDITION GUIDANCE.
Suppose we have property predictor As G + R, the target property value E R, and a loss function E ER R. We define the likelihood function as
plylg D(zo)) exp(-((A,(()..)))
To recover the molecules' prior distribution starting from the tractable distribution, the generative process of diffusion models can be also expressed as a reverse SDE:
dz
V., log)((z;)) / B.dw.
Now we will discuss how to incorporate the desired property into the molecule generation process. The mapping from the property to the molecule + is one-to-many, which is an ill-posed inverse problem. From the Bayesian point, we treat p(zg) as prior, and samples from the posterior p(zolv). Based Bayes' rule, we know the fact:
Vz, log (z(l)))) Vz. pz + Vz. logle()lz.).
(4)
Then Equation 3 can be modified as:
3, z Bel (2. lzg))(zz) + Vz, log log dt + /B,dw.
dz
(5)
2
4

---

Published as a conference paper at ICLR 2024
950
800
t070
6-600
500
400
== 350
2-250
t-
t= 150
t
Phase 1.
Phase
t [1000, 400]
t E [400, 0]
chaotic stage, invalid molecules
semanntc stage, valid molecules
Figure 2: Demonstration of expected final molecules G = (()o(Zi)) at different stages
Unfortunately, the second term pr(y/ze) is intractable in practice. In the previous work for 3D molecule generation. there are two main methods to overcome this. EDM (Hoogeboom et al. 2022)
and GEOLDM [uu et al. 2023) trained denoising model jointly with condition Vz, log Pe (Zz, y).
Alternatively, EEGSDE (Bao al. 2022) trained model to predict the property directly from the noised latent variables log (. 2.). As we said before, they are not efficient and flexible
Instead, propose effective plug-and-play framework for controllable 3D molccule generation without any extra training. We demonstrate that only with a pre-trained diffusion model on the dataset Vz, 2, log (z;) and the measurement operator A(g), we can generate molecules with high- quality and the desired property. We first rewrite the second term pr (M 2,) as:
pr(y(zz) (2) = f pelylzo, 2o, 2:) (z) (11z0lzs)dd dzo = f
pe pelyizo)Pr dzo = Ezo [p(y|g D(zo))
(6)
Here p(zo zz) is still intractable. Diffusion Posterior Sampling (DPS, Chung et al. (022)) is pro- posed to approximate Equation (6) as:
1
(e(l)))) p = D(2))) where 20 = E. B2 [20]
(2 + (1 a.,) Vz, log (z:)))
Vāa
(7)
Kim & Ye, 2021 to compute the posterior mean 20. which is useful when the amount of paired data (9, y) is limited.
DPS used Tweedies' formula (Efron 2011
However, we cannot generate good molecules with the desired property due to the limitation of DPS. DPS replaces Pr(zo] 2t) with a delta distribution around 20. The work (Song et al. 2023) indicates such "point estimation" could be highly inaccurate. This issue is even more serious in the 3D molecule generation due to inaccurate prediction of 2g in the early steps. Unlike other data formats such images, molecule property is very sensitive to the atom's 's positions. We show the entire process of expected final molecules 20 in Figure 2 There are two obvious phases: the chaotic stage, where the samples are far from valid molecules, and the semantic stage, where they can be modified as valid molecules. The atoms of g = D(2)) in the chaotic stage are not reliable, and the measurement A(.) can handle such invalid molecules, leading to unreasonable guidance.
The boundary between the chaotic stage and the semantic stage is decided in two ways. Initially, we visualize the expected final molecules and find the time the molecules become widely distributed and have certain shapes. Secondly, we investigate the performance change in Appendix A.4.1 and determine that starting from 400 steps is efficient for the task. Therefore, we skip guidance in [1000, 400), which is in the chaotic stage.
In addition to stage division, we implement three strategies to further ensure the stability of guidance: Monte-Carlo sampling, property checker and time-travel. Unlike DPS using delta distribution to approximate Pr(y zz),, we choose g(zo] 2.) = (2 rī1), which is a Gaussian with mean being the MMSE estimate ùn and the convariance is a hyperparameter r,. Previous work (Song et al. 2022) also adopts MC sampling, but other problem settings are different. With this new approximation, estimate:
log Pr (ylz,) = log Ezarp(zziz)) ppyl D(zo)]
m
R3 Eag = D(zo)] = V log
Σ m
exp (-8 (D((z)) v))
5

---

Published as a conference paper at ICLR 2024
Porill
Piiale
K4
Postfl
(B)
(C)
(4)
Figure Different relationships of properties
where 2 is the i.i.d. sample from the distribution ((2)) and m is the number of samples.
The property checker is to identify whether the guidance can be applied at the current step. We could first compute the range of the targeted property [ymin, ymaa, from the training dataset. At each step, check if the estimation for the property is in this range A(D(z))) E [ymin; Ymaz] Considering the invalid molecule can cause a very large estimation and hence influence the guidance direction, we opt not to apply any guidance at this step.
We employ a "time-travel" technique (Wang et al. 2022) to enhance the estimation of Zo- This technique iteratively revisits prior steps in the denoising process, allowing for repeated refinement of the molecule generation. Specifically, after each denoising step that progresses the estimation from 2, to 1 we periodically revert to previous step (e.g., from z back to 2,) and reapply the denoising operation. This repeated denoising process serves to refine the estimation, ensuring more accurate and stable convergence towards the desired Zg-
We prove that our computed conditional score function is orthogonal equivariant if the loss function of the property is orthogonal invariant. The detailed proof is provided in Appendix A.22.
Proposition 1. Suppose the loss function of the property f E(A((())): y) invariant, i.e., f (Rzz,t, Zh,t) = f(z,,,, Zh,t), where the decoder 'D is equivariant and the property predictor A invariant. Defining Ua,t; Uh,t = V logg pt (y) Zh,t) and d'z,;: dhh,t = V log then the conditional score function is orthogonal equiavariant such that Raz,t, ah,t d'z,s dha
Pe Pe(yl RZz,, Z).)),
Then, building upon the conclusion drawn by previous work (Bao et al. 2022). we can also demon- strate that our SDE is equivariant guided SDE.
4.2 MULTI-CONDITION GUIDANCE
We start to analyze the multiple-propeen property situation depicted in Fig. Fig. which involves two desired properties, Y1 and y2, with y/2 being dependent on y1. We aim to compute
V log (z) 9/, 3/2) log (zz + 1o1 32/zz))
when only have the time-independent property functions for clean data: ply1 (n) = & (20- v1) and p 20) = (z(20 82). Thus, we need to factorize pr (1(11 32|2z) (2) to exploit property functions:
Pr(11 (311892222) zz) =
p(y1. U2. zol zolz;) dzo =
p(v1, U2 lo. z )( )do
n0
/ p(u (ullzo)p(vzlzag y1 )((o Ze dzo
(10)
p(y1. v2 2o ((z)) /z: d2o
R3 p p(n (2222 y1)
where 2n (0 [2o]. We employ the property function & (2o. v1) to compute p(y1 (2)) which the same as the single objective. But we do not have a property function &g ((20, v1); y/2
that can input zu and simultaneously.
To address this challenge, we try to combine the information in (2o. v1) to remap new distribu- tion of samples z/z, in order to transfer (v 2o, y1) into Pe(y2|z)), Specifically, compute the expectation z/j with the following proposition and the proof is provided in Appendix A.2.I
Proposition Suppose the prior distribution 20 k/y v1) ~ N( r? r²T and first-order expansion
of the loss function p(y1 (2/) = exp(-e, (20: Vi) (z) 20)
VE, (2o, D1 }) we have
E Peorpiizo/oo.v [20]
20 r? Ve (2o. v

---

Published as a conference paper at ICLR 2024
Table 1: The mean absolute error (MAE) in single-objective 3D molecule generation tasks
Method
MAEL
Method
MAE
Method
MAE
Cy mol K)
(D)
(Bohr")
U-bound
6.879:10.015
U-bound
1613310.003
U-bound
8.98.10.02
#Atoms
1.971
#Atoms
1.053
#Atoms
3.86
Cond. EDM
1.065:000100
Cond. EDM
1.123++0001
Cond. EDM
2.78+0004
Cond. GeoLDM
1.025
Cond. GeoLDM
1.108
Cond. GeoLDM
2.37
EEGSDE (s=10)
0.001
EEGSDE (s=2)
0.777=:0.007
EEGSDE (s=3)
2.50:10.02
MuDM
0.290=10.024
MuDM
0.003
MuDM
0.43=00.07
L-bound
0.040
L-bound
0.043
L-bound
0.09
As (meV)
ÊHоMO (meV)
ÉLUMO (mcV)
U-bound
1464
U-bound
645
U-bound
1457
#Atoms
866
#Atoms
426
#Atoms
813
Cond. EDM
671+55
Cond. EDM
371+22
Cond. EDM
601+-7
Cond. GeoLDM
587
Cond. GeoLDM
340
Cond. GeoLDM
522
EEGSDE (==3)
487+3
BEGSDE (s=1)
302+-2
BEGSDEE (s=3)
447+66
MuDM
85:16
MuDM
72.14
MuDM
133:111
L-bound
65
L-bound
39
L-bound
36
60.9
68.3
72.3
77.1
83.3
90.3
96.7
Figure 4: Generated molecules conditioned on the single objective (polarizability a)
In summary, we arrive at the approximation V Po (v 1/2 (.)) 2 V log (0) + V log P(y kb), where 2, Vac (ze + (1-
V. log pe (z.)) and z 2n r Ve (2g, 91).
The same procedure can be extended to analyze more than two properties or other types of property relations, when the probabilistic graph of properties is determined:
Fig. 3 b(), we have p(y1, 1/2 (z) plyi (20 (o)pp(yy2)))) when yı and y/2 are independent. Thus, we simply the gradients of the linear combination of the two property functions guide.
In Fig.Blc) we consider three properties y1, 3/2 and ys. with ys being dependent on y1 and y2. We ex- press p(v1, 42 y/3 (Z.) p(y1 20) 9/ io ) pys 22. 3/. 322. Here p(ys 20. V1, (2222) is approximated by Proposition 2 and we compute 2/ (Ua)) = 20 rf Vl, (2.. y1) + Vl2 (2o: U2) for ya- Subsequently, the final conditional score function is computed by Vh (20, y1) + V (2. V2) (25(2((1a)) us)-
In Fig.B B(d), we also have three properties /1, 3/2 and ys. with ya being dependent on y2. and y/2 being dependent /1- The term p(v V2, Va (z,)) is approximated by p(y) (20) 2n. y1 )p(ss) 20. V2)- We can apply Proposition
2 to calculate (32) and z (ya) separately, where 20(y2) 20 2
ri Ve, (2g, V1), and (v3))
r Ve 20, M2). The final conditional score function com- puted by Vl, (Žo, y1) + (2 v2) + Vlg (Ba))
Currently, we can handle any direct acyclic probabilistic graphs In addition, MC sampling used to improve multi-condition guidance. By sampling multiple 2o, we obtain a more accurate estima- tion of the gradients for each property. Moreover, we also adapt the weighted sum of gradients in practice. These weights are hyperparameters and remain fixed during the generative process.
5 EXPERIMENTS
In this section, conduct comprehensive experiments single and multiple-comeeee condiiioned molecule generation tasks to evaluate our proposed method MUDM. The pseudo-code and hyperparameters provided in Appendix A. and A.3 and the code will be published later.

---

Published as a conference paper at ICLR 2024
Table 2: The mean absolute error (MAE) in multi-objective 3D molecule generation tasks.
Multi-objective Tasks
Metries
Baselines
Property
Property 2
Correlation
MAE +
Conditional EDM
EEGSDE
MuDM
C. ( K) inol
1 (D)
0.42
MAE
1.079
0.981
1,466
MAE
1.156
0.912
0.687
MAE
683
563
554
As (meV) (D)
-0.34
MAE 2
1.130
0.866
0.578
MAE 1
2.760
2.610
1.326
(Bohr²)
(D)
-0.24
MAE
1.158
0.855
0.519
MAE
372
335
317
PHоMO (mcV)
ÉLUMO (mcV)
0.22
MAE 2
594
517
455
MAE
610
526
FLUMO (meV) (D)
-0.40
MAE 2
1.143
0.860
0.497
MAE
1097
546
361
ÉLUMO (meV)
As (meV)
0.89
MAE
712
589
228
MAE
578
567
262
Eнооо (me As (meV)
-0.24
MAE 2
655
323
489
5.1 SETUP
Dataset: We perform conditional molecule generation on QM9 (Kamakrishnan et al., 2014). a dataset of over 130K molecules and 6 corresponding quantum properties. Following previous search, we split the dataset into training, valid, and test sets, each including 100K, 18K, and 13K samples respectively. The training is further separated into equal halves, Da and Dr, avoid information leak in the training phase. The half D, is used to train the ground-truth property pre- diction network while Dg is for the training of the diffusion model.
Metrics: We use the mean absolute error (MAE) to evaluate the difference between the given con- dition and the property of generated molecules.
Baselines: In this study, we compare MUDM with several competitive previous works. The con- ditional EDM trains the diffusion model on D', given the molecule and its property ([x, A), In this case, the conditional EDM needs to train separate diffusion models for each property. Simi- larly, the conditional GEOLDM is also required to train six diffusion models. EEGSDE trains one diffusion model and six time-dependent property functions on D, for each property to guide the sampling phase. Occasionally, EEGSDE fine-tunes the diffusion model based on property functions to enhance performance. Moreover, proposed method MUDM trains one diffusion model and six time-independent property functions on D, without the need for fine-tuning.
Additionally, EDM reported "#Atoms", well the upper- bound and lower-bound baselines. The "L-bound" baseline refers to the MAE of the ground-trute property function on Db. while the "U-bound" baseline shuffles the labels and calculates the MAE Dr. The "#Atoms" baseline predicts the property based on the number of atoms in molecule.
5.2 SINGLE-CONDITIONED MOLECULE GENERATION
We examine six quantum properties in QM9 as our single objective: heat capacity C.. dipole mo- ment polarizability a, highest occupied molecular orbital energy θномо- lowest unoccupied molecular orbital energy ÉLUMO, and HOMO-LUMO gap Ae. As shown Table [ our method each quantum property compared to conditional EDM, GeoLDM and EEGSDE, This result highlights the potential of using time-independent property functions directly guide the generative process. Additionally, also confirm the effectiveness of both DPS and MC sampling in providing accurate guidance for property prediction.
MUDM achieved the best performance
In terms of efficiency, training diffusion model for new property using conditional EDM and GeoLDM takes ~ 2 days on a single 1100 GPU. Meanwhile, obtaining a specialized time-dependent property function using EEGSDE also takes several days. In contrast, method MUDM directly
8

---

Published as a conference paper at ICLR 2024
MAE 2f (Bohr))
M of A. (eV)
MAE of (/
MAE (eV)
108
80
34
Number MC Samples
Nurbee MC Samples
Timestep
Figure Impact of the number of MC samples
Figure 6: MAE during the generative process
utilized the off-the-shelf property function without requiring extra training. This approach allows for faster predictions, providing researchers in the area of drug discovery with quick and flexible way to search for new molecules with desired properties.
Finally, present visualization of generated molecules with varying as the condition in Fig. The polarizability a defined as the ratio of induced dipole moment to the local electric field. Generally, molecules with less isometric shapes have larger values. We set [60, 100) to ob- serve that the generated molecules become less isometric, which is consistent with the expectation.
5.3 MULTTPPLE-CONDITIONN MOLECULE GENERATION
We investigate seven combinations of quantum properties. We first compute the correlation between each pair on Dr. We discovered that all of these combinations were correlated, so we used the situation depicted in Fig. 3(a) to model the variables y1 and y2, with y/ being dependent on y/- Our proposed method MUDM outperformed conditional EDM and EEGSDE in most cases, which demonstrates the effectiveness of the proposed multiple-objective guidance.
As for efficiency, our multi-objective guidance essentially does not increase the computation time and cost compared to single-objective tasks. In contrast, conditional EDM and EEGSDE require retraining the diffusion model for each combination, which needs huge computation resource.
Two visualizations of generated molecules with multiple objectives are shown in Appendix A.7
5.4 ABLATION STUDY
We present two ablation studies in this section and provide more ablation studies in Appendix A.44 The first one is the impact of the number Monte Carlo (MC) samples. Fig. shows the perfor- mance of multi-objective tasks with different #samples. With the increased #samples, the perfor- mance becomes better. However, this improvement came at the cost increased time consumption. Analyzing the trend in Fig. 5 we found that #samples 10 is the balanced choice between perfor- mance and time efficiency.
We also observed the difference ly A(D(2a( (2,))) I during the generative process in Fig. 6 We found that the difference initially dramatically increases in [1000, 500] and then subseguently de- creases gradually in [500, 0). This observation aligns with the finding presented in Fig, which dicates the presence of two distinct stages. Consequently. calculating property values in the chaotic stage yields inaccurate results. We should only provide guidance during the semantic stage.
CONCLUSION
In this paper, we proposed MUDM, more flexible and efficient approach for generating 3D molecules with desired properties. MuDM is capable of handling both single and multiple property objectives for molecule generation. The experiments further validate the effectiveness of MUDM. As future direction, MUDM can be applied to more realistic properties, such as chirality and bind- ing affinity with protein targets, expanding its potential applications in drug and material discovery.

---

Published as a conference paper at ICLR 2024
REFERENCES
Keir Adams, Lagnajit Pattanaik, and Connor W Coley. Learning 3d representations of molecular chirality with invariance to bond rotations. arXiv preprint arXiv:21 2021.
Sungsoo Ahn, Junsu Kim, Hankook Lee, and Jinwoo Shin. Guiding deep molecular optimization with genetic exploration. Advances in neural information processing systems, 33: 13:0208-2221,, 2020.
Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu. Equivariant energy- guided sde for inverse molecular design. arXiv preprint arXiv:2209. (tt::2209.14000 15408, 2022.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209. 14687, 2022.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs
arXiv preprint arXiv: 18X.: 2018.
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Associa- lion, 106(496):1602-166 2011.
Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolu- tional neural networks for equivariance lie groups on arbitrary continuous data. International Conference on Machine Learning, pp. 3165-3176. PMLR, 2020.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-traassssmers:: 3d roto- translation equivariant attention networks. Advances in neural information processing systems, 33:1970-1981, 2020.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3d. In International conference machine learning, pp. 8867- 8887. PM 2022.
Jan H Jensen. A graph-based based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12)3356733572 2019..
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. International conference on machine learning, pp. 4849-4859. PMLR, 2020.
Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems, 34:864- 874, 2021.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances neural information processing systems, 34:21666-21777, 2021.
Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Li. Diffbp: Generative diffusion 3d molecules for target protein binding. arXiv preprint 21X11:2211112214. 2022.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Aspuru-Guzik. Augmenting ge- netic algorithms with deep neural networks for exploring the chemical space. arXiv preprint arXiv: 11655, 2019.
Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for de novo drug design. Science advances, 4(7)xeaap7885, 2018.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014. ISSN 2052-4463.
10

---

Published as a conference paper at ICLR 2024
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image syuthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695 2022.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural net- works. In International conference on machine learning, pp. 9323-9332. PMLR, 2021.
Daniel Schwalbe-Kodee and Rafael Gómez-Bombarelli, Generative models for automatic chemical design. Machine Learning Meets Quantum Physics, pp. 445 467, 2020.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382, 2020.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning-ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part 27, pp. 412 422. Springer, 2018.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference Learning Representations, 2022.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. 2023.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:201 1, 13456, 2020.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and and translation-equivarian neural networks for 3d point clouds. arXiv preprint arXiv:/802.082/19 2018.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint 122.:2211..00490, 2022.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint 103. 10432, 2021.
Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Ermon, and Jure Leskovee. Geometric latent diffusion models for 3d molecule generation. In International Conference Machine Learning, pp. 38592-38610. PMLR, 2023.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovee. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 PSEUDO-CODE
The algorithm below illustrates the sampling process of MuDM conditioned on the single property, and two properties.
Algorithm 1 Sampling Algorithm of MuDM for single condition
1: Input: decoder network Dg. denoising network éø. condition y, property predictor A
2: Zx,T, Fh,T ~ (0,
3: for in T, T 1,,1-.... 1,.,,, do
4: 6 (0,
[Single Condition Guided Latent Denoising Loop]
5: Subtract center of gravity from 6x in E
[x,, Fhl
6: 2/-1 VI- (2t
B1 eo (z(, ((,))))))))))))))))))) 1)) + Pr6
7: 20 = (2: + V a7tl(z).t) t))
8: Zt-1 = U2, log(H 1 Ei=1 Ei= (xp(-e(A(z))) 4)), zb ~2(20,
9: end for
10: h Px( h 12 2n.0)
(Decoding)
11: return x, h
Algorithm Sampling Algorithm of MuDM for two conditions
1: Input: decoder network Dg. denoising network εg, first condition y1, first property predictor Ai, second condition y2. second property predictor Ag
2: Zx,T's Zn.T" (0,
3: for in T. 1, do
4: <~ (0,
(Two Conditions Guided Latent Denoising Loop)
5: Subtract center of gravity from ex in e = [xx, en]
6:
z/-1= = vI- (z: Vi-a
(z / t))
+ PEE
7:
20 L(z + /1 (fea(Zi,()) ())
8:
Vl, (2o, y1) = Vn, log(# Eim exp(e (xp(-e(A((2)).1))))
~ ~( r1)
2 = 20 + 1-a; Ve, (20, y1)
9:
10:
V42 (2/ /2) Vz, log( Σ"
i== exp(-e(AA(zi)) (2)()))))2).)..). zb N( rZ1)
11: 2c-1 = 1 Vl, (2o, 41) + V82 (2), 82)
12: end for
13: x, hh ~ (x, h /2 Zn.o)
(Decoding)
14: return x, h
A.2 PROOFS
A.2.1 PROOF OF PROPOSITION
Proposition 3. Suppose the prior distribution p(2g k0g 91) N(zh, r?T) and first-order expansion
of the loss function p(yn (2)) = exp((2 (20, y1) + (z) 20) 7 Ve, (20. y1)), we have
E (eo = 20 - V
Proof. We have that
p(20 1n6. yn n1(pzb) yn (p(m))
p(z/ 22, v1)
p(y1, 20)
12

---

Published as a conference paper at ICLR 2024
where both p(y1) and p(v1 2o) normalizing constants. Then have
p(z0 = 2/ v1) x p(20 2 = k, yn (nn) 126 = k)p(z) = k)
(k
20)2
v1) ) (k =00 (200033
x exp
ri
(k 20)2 (k 2n)² Vli (2o: u1)
exp
r
As the term within exp quadratic function, the expectation is achieved at the maximum point, L..,
(k -20) (k - (k-2)) V 31) = 20 V v1)
argmax E u1.
r?
k
A.2.2 PROOF OF PROPOSITION
Proposition Suppose the loss function of the property f= E(A((())) y) is invariant such that
[Rzz,, kh.1) f(Za,t, Ah,t), where the decoder 'D is equivariant and the property predictor A is invariant. Defining Oz,t; Oh,t V log pt (.,,,hh.)) and a'z,t= d/,, log Pe R2 Zhtt), then the conditional score function is orthogonal equiavariant such that Rag,s ah = 0l2,0= 0/
Proof. For the decoder D, we have R.E, h = D(Rz,, zn). When the loss function of the property f == ((A(().)) is invariant, we have f(Rr, v) = f(x, h) We take the gradient w.r. both sides and obtain
V v=s(a,h,y) Vzo V h,y) where x' R.r.
V vef(z,h,y) = R V. f(z', 4,
where x' = Rr.
Multiplying R both sides where R.T we get
RV.f(æ h, y) = V f(x', h, b)
Thus, v.f(-) is equivariant to R. The proposition is proved.
A.3 DETAILS FOR EXPERIMENTAL SETUPS
We didn't train any new model for the 3D molecule generation task. The pre-trained model was directly from GEOLDM and we kept the same hyperparameter settings without any additional fine- tuning. For any property guidance, we used the same diffusion model (G, t), with the correspond- ing property predictor A(.) - R. This property predictor is not time-dependent.
We found is important to set reasonable guidance weights for multi-condition guidance. We computed the gradients follows:
v log Pa (yh, U2lz4) = wn log (20) + u2 log p(y2l2))
We listed the main sampling settings Table 3 r is the variance level used for me sampling (2 r2.). We fixed it during the sampling process.
Table 3: Hyperparameters for two conditions sampling
Conditioned properties
Guide from
W/
W0
MC sample size
Cp cal K), j (D) mol As (me V), j (D)
400
400
1
0.3
10
(Bohr?), ), μ (D)
400
2
0.3
10
нномо (meV). ÉLUMO (meV)
I
SLUMO (meV), J p (D)
400
1
0.3
10
13

---

Published as a conference paper at ICLR 2024
A.4
OTHER ABLATION STUDIES
Comprehensive ablation studies are included in this section to demonstrate the effectiveness of each design. We kept all settings the as the main paper and only changed one setting check the performance change.
A.4.1 INFLUENCE OF GUIDANCE STEP
Table shows how the guidance steps influence the task with two conditions. The result indicates that there is significant change when guided the diffusion model from 400 steps 1000 steps. This is because, during the chaotic stage, the property predictor cannot provide accurate predictions and thus can not provide useful information for guidance. It proves that starting from 400 steps is efficient for the guidance process and can help save inference time.
Table 4: Influence of guidance steps
Conditioned properties
Guide from
MAE 1
MAE 2
400
262
489
ÉHоMO (me V), Ae (meV)
1000
271
465
A.4.2 COMPARISON OF DPS AND MC SAMPLING
We show the result for two different sampling methods DPS and MC sampling in Table indicates
the MC sampling method has a better performance. The previous work [Sog et al. (2033 proved
that MC sampling method is more closed estimation for (xze=p(oole)) (p [p(y]C Dzzo).
Table 5: Influence of MC sampling
Conditioned properties
Sampling method
MAE
MAE 2
DPS
472
396
ÉLUMO (I, As (meV)
MC sampling
361
228
A.4.3 INFLUENCE OF CONDITION WEIGHTS
We tested how different weight settings influence the performance in Table 6 It shows that different weights have impacts on the final results. But in most cases, we can simply set all weights 1.
Table 6: Influence of condition weights
Conditioned properties
W22
MAE
MAE 2
1
361
228
ÉLUMO (meV), As (meV)
2
540
456
1
393
342
3
687
545
1
613
526
A.4.4 INDEPENDENT SAMPLING FOR DONEDTTOEEOO MOLECUL GENERATIOON
Finally, we ignore the dependency between properties and believe they independent. Thus, the multiple-objective guidance just a linear combination gradients of each property. The results shown in Table 7 We found that the performance of all the properties dramatically declined.
14

---

Published as a conference paper at ICLR 2024
It verifies the effectiveness of our proposed multiple-objective guidance, which is calculated as the weighted sum the gradient of each property with respect to the latent variable, taking into account the property dependency.
Table 7: Results of a linear combination of multi-objective guidance
Property
MAE
Desc.
Property 2
MAE 2
Desc.
Cp cal K)
mo!
1.461
0.3%
(D)
0.942
-37.1%
A2 (eV)
0.851
-53.6%
J (DD)
0.591
-2.2%
a (Bohr*)
3.012
-127.1%
J (D)
0.952
-83.4%
EHOMO (eV)
0.446
-40.7%
ÉLUMO (eV)
0.522
-14.7%
ÉLUMO (eV)
0.755
-31.3%
Jr (D)
0.930
-87.1%
A.5 TIREE-CONDITIONED MOLECULE GENERATION
In our extended analysis, we introduced an additional experiment to evaluate the performance of MUDM in scenarios involving three properties, illustrated in Table & and reflected in Figure (c). This experiment was designed to compare scenarios with potential conflicts among properties against those without such conflicts. In the first scenario (denoted by X), the properties ÊномО- ÉLUMO. and A= ELUMO HHоо were independently sampled, introducing the possibility of conflicting requirements. Consequently, no molecule could perfectly satisfy the targeted properties, yet MUDM still demonstrated commendable performance. In contrast, the second scenario (denoted by v) did not present any inherent conflict among the three properties since Ae computed from the other two properties. As expected, the performance in this conffict-free setting superior, as indicated by the lower mean absolute errors (MAEs). These results not only underscore the robustness of MUDM in handling multi-objective tasks with complex inter-property relationships but also highlight its capability to generate feasible molecular structures even in the presence of potential property confticts.
Table The mean absolute error (MAE) for three objective 3D molecule generation tasks
Conflict
Property
Property 2
Property 3
MAE
MAE 2
MAE
V
<HoMo(meV) eHоMo(meV)
(LUM) ELuMo(meV)
Aa (meV)
479
545
678
As (meV)
283
392
255
A.6 DESCRIPTION OF THE QM9 DATASET
The QM9 dataset includes 134k stable small molecules, consisting of atoms C, H. O, N and F. The statistic of the number of atoms is provided in Table D We can obtain atom coordinates and corresponding quantum properties for each molecule. These quantum properties B3LYP/6-31 (LLYP//-3IG(ddfp) level of quantum chemistry. Previously, QM9 was a benchmarking dataset to evaluate whether the machine learning-basee method can predict accurately the quantum properties based on the molecule structure. Now it is also employed in molecular generation tasks.
calculated the
Table 9: Statistic of QM9 Dataset
Mean
STD
Maximum
Number of atoms
18.0
29
Number of heavy atoms
8.8
0.51
9
A.7 VISUALIZATTON OF GENERATED MOLECULES WITH MULTIPLE OBJECTIVES
We provide two visualizations of generated molecules with multiple objectives in Fig. 7 The dipole moment j measure of the separation of positive and negative electrical charges, the molecules
15

---

Published as a conference paper at ICLR 2024
95.6
95.6
95.6 j 5.0
0.5
0.5 Fauned
0.5 Fauned
#
FIUMM 5599
Figure 7: Generated molecules conditioned on the multiple objectives
with high dipole moment and polarizability are also asymmetrical. Besides, molecules with a high energy gap between HOMO and LUMO are generally less reactive and more stable, indicating that their constituent atoms are closely connected. These trends are observed in Fig. 7
A.8 RESULTS ON NOVELTY AND ATOM STABILITY
Table 10: Other Properties
Conditioned properties
Atom stability
Novelty
C₁ cal K), f (D)
mol
0.72
0.84
As (meV). V), (D)
0.66
0.92
a (Bohr"), μ (D)
0.70
0.88
Нoоо (meV), SLUMO (meV)
0.67
0.93
SLUMO (meV), y (D)
0.68
0.90
To provide comprehensive evaluation, we include results on the metrics of novelty and atom sta- bility. Novelty, as defined by (Simonovsky & Komodakis 2018), measures the percentage of gener- ated molecules that are not present in the training dataset. Ätom stability, according (Hoogeboom jet al., 22022, assesses the percentage of atoms within the molecules that exhibit correct valency. Our approach achieves similar level of novelty compared with EEGSDE. However, observe detrimental effect on atom stability with the current guidance method. This issue is recognized as limitation and should be considered future work.
16


Reviews:
[{"review_id": "VFXHAtBJs5", "reviewer": "Reviewer_yusH", "summary": "This paper presents a training-free conditional 3D molecular generation algorithm based on off-the-shelf unconditional diffusion models to address the issues of efficient, flexible, and potential properties conflict.", "questions": "Could the authors give some insights into why the proposed method can handle the potential properties conflict?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This paper is well-organized and written. The issue of potential properties conflict and the proposed training-free conditional generation method are both interesting.", "weaknesses": "Refer to the question part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BmE7d7HVES", "reviewer": "Reviewer_LmRX", "summary": "In this paper, the authors proposed a new method for 3D molecule generation. Based on the experimental results, their method showed superiority, compared with other existing methods on several real data sets.", "questions": "(1) To make their results more convincing, they should compare their method with more latest state-of-the-art methods. \n(2) It remains unclear whether the proposed method is sensitive to the hyper-parameters and how to setup the values in general cases.\n(3) Lack of description of the details of the datasets.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The author describes the fundamental algorithm well; and they seem to give all relevant information to understand and reproduce their algorithm. \n\nThe overall writing is satisfactory. The writing is fluent and clear and the ideas are easy to follow.\n\nThe proposed method is relative better than previous methods, which is not lack of significance.", "weaknesses": "(1) To make their results more convincing, they should compare their method with more latest state-of-the-art methods. \n(2) It remains unclear whether the proposed method is sensitive to the hyper-parameters and how to setup the values in general cases.\n(3) Lack of description of the details of the datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c0F2d7WBke", "reviewer": "Reviewer_7aDk", "summary": "This work introduces a new property guidance mechanism for diffusion models that: (a) does not require to retrain a diffusion model, (b) uses time-independent property estimators and (c) works with arbitrary combinations of properties and is able to manage complex relationships between them. While I find this topic very important and the proposed guidance mechanism extremely useful (potentially more usable than other existing ones), I think that several important technical aspects of the proposed mechanism are not sufficiently studied (or explained in the text). Besides, in my opinion, the experimental part of this work can be improved. I am happy to reconsider the score if the points listed in the \"Weaknesses\" section will be addressed.", "questions": "1. I had an impression that adding another gradient to the denoising process (i.e. equation 5) in reality would lead to the high instability of the sampling process. Did you experience any issues of this kind?\n\n2. How did you select the target property values when sampling? Did you randomly sample these values from the distribution taken from the training set?\n\n**Suggestions:**\n\n1. Not sure if background about EDM is necessary in Section 3 – it seems that the work is based on the latent diffusion only.\n\n2. Following question 1 discussed in \"Weaknesses\", how does the border between chaotic and semantic stage depend on the type of noise schedule? I.e. linear, cosine, polynomial, etc.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. I find this topic very important and relevant\n2. Authors provide a solid mathematical framework for the proposed guidance mechanism\n3. The proposed method clearly outperforms other state-of-the-art models in single-objective 3D molecule generation", "weaknesses": "1. I think the question about chaotic and semantic stages is very important. Authors briefly talk about it in Section 4.1 and experimentally illustrate the importance of this issue in Figure 7. Besides, authors mention a certain \"property checker\" twice in Section 4.1, however never expand on it. I find it very critical to have a clear understanding of how the sampling stage is taken into account. I would be also interested in seeing the corresponding ablation study (e.g. if to take / not to take into account the chaotic stage).\n\n2. While one of the main contributions of this work is multi-condition guidance mechanism, in my opinion a more solid experimental validation of the proposed method is required: \n    * Combinations of properties considered in Table 2 look a bit strange. Why is dipole moment used in 4 out of 5 considered combinations as Property 2? \n    * Besides, correlation 0.22 doesn't seem too high – aren't there better combinations? \n    * Combinations $(\\Delta\\epsilon, \\epsilon_{\\text{HOMO}})$ and $(\\Delta\\epsilon, \\epsilon_{\\text{LUMO}})$ seem to be good candidates to illustrate the dependency from Figure 3(a).\n    * It would be interesting to see some experiments with combination from Figure 3(c). Again, $(\\Delta\\epsilon, \\epsilon_{\\text{HOMO}}, \\epsilon_{\\text{LUMO}})$ seems to be a good candidate for it.\n\n3. _\"We found it is important to set reasonable guidance weights for multi-condition guidance\"_  – how did authors find these weights and how much do the final results depend on these weights?\n\n4. How does the sampling time depend on the number of MC samples? I would be also interested to see the comparison of the sampling time with other methods.\n\n5. As authors discuss in Section 4.1, point estimation used in DPS is very inaccurate, especially in case of 3D molecule generation. It would be interesting to see some experiments with DPS (e.g. for single property conditioning) and comparison with the proposed MC-based method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pBS5stpkOQ", "reviewer": "Reviewer_ufcA", "summary": "This paper proposes a method, called MuDM, for generating 3d molecules with both single and multiple desired quantum properties based on specially designed score function as guidance. For the single property optimization task, they followed previous work from the images domain to utilize the Monte-Carlo (MC) sampling to estimate the conditional score function which serves as the guidance. Besides, they devised a confidence boundary based on the prediction of the property function to divide the generation process into two stages, i.e., the chaotic stage and the semantic stage, and the guidance only works in the second stage. For optimizing multiple properties, they model the relationships between the multiple properties as probabilistic graph and factorized the conditional score function according to different probabilistic graphs. Their main contribution is extending the guidance on single molecular property to multiple properties simultaneously by building a probabilistic graph to model the relationships between various properties. They conducted experiments for both single-objective and multi-objective optimization tasks to conclude that the designed guidance outperforms other methods and is more efficient. Lastly, ablation studies were carried out to study the impact of the number of MC samples and to demonstrate the existence of the two stages in generation process and the effectiveness of the probabilistic graph.", "questions": "Certain aspects of the experiments require further clarification.\n\n1、Can the authors provide more information about the property checker and explain how the boundary is determined. \n\n2、In ablation studies, the reviewer agrees that the MAE curve for HOMO demonstrate the existence of the two stage in generation process. However, for the LUMO curve, no significant increase is observed, what are the possible explanations?\n\n3、The author conducted experiments focusing on the simultaneous optimization of two interrelated properties, as illustrated in Figure 3(a). Could they extend their experiments to cover the scenarios presented in Figure 3(c) where y1 and y2 are not related?  \n\n4、In proof of proposition 4, why A(Rx, h) equals to RA(x, h) while the property predictor A is invariant? Besides, there is a typo on the numerator after the second equals sign.\n\nMinor comments:\n1、In equation 8, n indicates the number of samples, not m.\n2、A typo at the beginning of the last paragraph in related works.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Their main contribution is extending the guidance on single molecular property to multiple properties simultaneously by building a probabilistic graph to model the relationships between various properties. The proposed method is novel and interesting.", "weaknesses": "The experiments can be further improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)