Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design
Authors: Lianghong Chen
Dongkyu Eugene Kim
Mike Domaratzki
Pingzhao Hu
Venue: NeurIPS 2025 poster ()
Keywords: 3D De Novo Molecular Design
Diffusion Model
Multi-Objective Reinforcement Learning
Uncertainty Quantification
Deep Learning
Primary area: applications
TL;DR: We propose an uncertainty-aware multi-objective reinforcement learning framework to guide diffusion models for 3D molecular generation, providing high-quality candidate molecules for drug discovery.

Abstract: Designing de novo 3D molecules with desirable properties remains a fundamental challenge in drug discovery and molecular engineering. While diffusion models have demonstrated remarkable capabilities in generating high-quality 3D molecular structures, they often struggle to effectively control complex multi-objective constraints critical for real-world applications. In this study, we propose an uncertainty-aware Reinforcement Learning (RL) framework to guide the optimization of 3D molecular diffusion models toward multiple property objectives while enhancing the overall quality of the generated molecules. Our method leverages surrogate models with predictive uncertainty estimation to dynamically shape reward functions, facilitating balance across multiple optimization objectives. We comprehensively evaluate our framework across three benchmark datasets and multiple diffusion model architectures, consistently outperforming baselines for molecular quality and property optimization. Additionally, Molecular Dynamics (MD) simulations and ADMET profiling of top generated candidates indicate promising drug-like behavior and binding stability, comparable to known Epidermal Growth Factor Receptor (EGFR) inhibitors. Our results demonstrate the strong potential of RL-guided generative diffusion models for advancing automated molecular design.

Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design
Lianghong Chen', Dongkyu Eugene Kim², Mike Domaratzki', Pingzhao Hu Department of Computer Science, Western University, London, ON, Canada Department Biochemistry, Western University, London, ON, Canada 1chen7760uvo.com ca, phu49duwo.ca
Abstract
Designing de novo 3D molecules with desirable properties remains fundamental challenge in drug discovery and molecular engineering. While diffusion models have demonstrated remarkable capabilities generating high-quality 3D molecular structures, they often struggle to effectively control complex multi-objective con- straints critical for real-world applications. In this study, we propose an uncertainty- aware Reinforcement Learning (RL) framework to guide the optimization of 3D molecular diffusion models toward multiple property objectives while enhancing the overall quality of the generated molecules. Our method leverages surrogate mod- els with predictive uncertainty estimation to dynamically shape reward functions, facilitating balance across multiple optimization objectives. We comprehensively evaluate our framework across three benchmark datasets and multiple diffusion model architectures, consistently outperforming baselines for molccular quality and property optimization. Additionally, Molecular Dynamics (MD) simulations and ADMET profiling of top generated candidates indicate promising drug-like behav- ior and binding stability, comparable to known Epidermal Growth Factor Receptor (EGFR) inhibitors. Our results demonstrate the strong potential of RL-guided generative diffusion models for advancing automated molecular design. The imple- mentation is available at https: bettps//ggtthu http:/gttthuu..okkkkoee/ee com/Kyle4490. RL-Diffusion
Introduction
The design of novel molecules with desirable properties is fundamental to drug discovery, materials science, and molecular engineering 2 Deep generative models have emerged as promising tools for automated molecular design, enabling efficient exploration of vast chemical spaces that difficult to navigate manually (35). Among these, diffusion models have demonstrated remarkable capabilities in capturing complex molecular distributions and generating diverse, high-quality samples [6]. However, most existing studies have focused on satisfying basic chemical validity constraints without explicitly controlling multiple drug-relevan, property objectives (7). Achieving reliable multi-objective optimization remains major challenge, particularly for therapeutically important targets such as the Epidermal Growth Factor Receptor (EGFR), a key receptor protein involved in cancer progression and drug resistance [8]. The ability to efficiently design molecules with optimal combinations
synthetic accessibility, and binding affinity to receptor proteins critical for advancing precision drug development.
" "Corresponding author. Department of Computer Science. Western University, 1400 Western Road, London. Ontario N6G 2V4, Canada. Email: phu49@ uwo.ca (P.H.)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address the challenges of property optimization, prior research has explored strategies such as flow matching (9 [0] and anergy-guided, generation 12 Although these methods provide effective solutions under specific conditions, they generally require explicit and differentiable reward functions, making them poorly suited for handling black-box objectives, such as Quantitative Estimate of Drug-likeness (QED) [3]. Synthetic Accessibility Score (SAS) [4] and binding affinity [15), which are predicted by external computational tools and are critical for the success of drug development. [7]. In contrast, Reinforcement Learning (RL) has emerged as particularly versatile alternative due to its strong flexibility in managing non-differentiable rewards and dynamically balancing multiple property objectives [16]. RL has been widely applied to guide the optimization of molecular generation models such as Recurrent Neural Networks (RNNs) (7]. Variational Autoencoders (VAEs) [8 20], and Transformers [2T]. However, these advancements have largely focused on ID string representations such as Simplified Molecular Input Line Entry System (SMILES) [22] and 21) molecular graphs. RL.-guided optimization of diffusion models to directly generate de novo 3D molecules has not yet been fully explored. Generating de novo 3D molecules with precise molecular geometries and chemical interactions is essential for drug development-relevanl downstream tasks like molecular docking and Molecular Dynamics (MD) 23 which cannot be achieved using ID or 2D molecular representations.
[2]
In this study, we propose an uncertainty-awarc multi-objectie RL framework to guide the optimiza- tion of 3D molecular generative diffusion models. Our approach leverages property uncertainties predicted by surrogate models to guide reward assignment and stabilize optimization under complex multi-objective constraints. We evaluate method on three widely used molecular datasets: the QM9 Quantum Chemistry Dataset, the ZINC15 Molecular Library, and the PubChem Compound Database, comparing it against multiple strong baselines and conducting comprehensive ablation studies. To further validate the scalability of our method, we also apply it to different diffusion model architectures. Moreover, in addition considering key molecular properties such as bind- ing affinity, we further assess the practical drug development potential of the generated candidate molecules through MD simulations and Absorption, Distribution, Metabolism, Excretion, and Toxic- ity (ADMET) property assessments 244 benchmarking against known EGFR inhibitors. Our results highlight the strong potential of RL-guided diffusion models to design candidate molecules with superior stability and drug- likenisss,ess, providing a new paradigm for diffusion model-driven de novo 3D molecular design.
Our contributions are summarized as follows:
We propose the first end-to-end framework that integrates RL, diffusion models, and uncer- tainty quantification for 3D molecular generation with multi-objective optimization.
We design a novel reward function that integrates uncertainty quantification with three auxiliary components, including reward boosting mechanism, a diversity penalty, and a dynamic cutoff strategy. This design balances optimization trade-off's among multiple objectives and addresses challenges such as reward sparsity and mode collapse when applying RL to optimize diffusion models. As a result, both the overall quality of generated molecules and the number of candidates that satisfy all property requirements are improved.
We conduct extensive experiments on three benchmark datasets, comparing against multiple State-Of-Thae Of-The-Art (SOTA) baselines and performing comprehensive ablation studies to assess the contribution of each component. To validate the generality of our framework, we further apply it to different 3D molecular diffusion model architectures.
We demonstrate the real-world potential of our approach through MD simulations and ADMET profiling of generated candidate molecules, benchmarking against known EGFR inhibitors.
2 Related Works
De novo 3D molecular generation. Early approaches to 3D molecular generation, such as G-SchNet 25 and E(3)-Normalizing Flows (E-NF) (26). applied autoregressive generation and equivariant flows to model spatial symmetries. More recently, diffusion models have emerged as promising paradigm. Equivariant Diffusion Model (EDM) 277 introduced -quiivaavrar denoising to better
capture molecular geometries. Geometrie Latent Diffusion Model (GeoLDM) (8 extended this
by applying latent space diffusion. Molecular Diffusion Model (MDM) 22 and Generative Force

---

Matching Diffusion Model (GFMDiff) 30 further incorporated physics- physics-based constraints to improve structural realism and diversity.
RI.-guided generative model optimization. Prior works on Rl.-guided diffusion models have mainly focused on image generation tasks with single-objective optimization. Fan et al. proposed Shortcut Fine-Tuning with Policy Gradient (SFT-PG) [31] to reduce distributional mismatch and improve sample quality. Denoising Diffusion Policy Optimization (DDPO) [32] treated denoising multi- step decision process and introduced DDPO-Importanie
Sampling (IS) and DDPO-Score Fine-tuning (SF) to incorporate preference inperence-condiioone rewards. DPOK [33 further added Kullback-Leible (KL)
34
regularization to stabilize training and enhance alignment.
Multi-objective optimization methods. Existing multi-objective optimization methods broadly be grouped into scalarivation-based, constraint-based, gradient-based, and uncertainty- ap- proaches. Scalarization methods, such as Weighted Sum (WS) (35), Product Of Objectives
(POO) 36. Max-Min Method (MMM) [37. and Linear Scalarization with Dynamic Weights
(LSDW) (38], convert the problem into single-objective optimization but require careful weight
tuning, Constraint-based methods, including Normalized Manhattan Distance (NMD) [39 NMD-
WS 40 Compromise Programming (CP) 4T, and Penalty Function Methods (PFM) enforce
objectíves as constraints but can be unstable when conflicts arise. Gradient-based methods like
Singular Value Decomposition (SVD) 43 PCGrad 44 CAGrad [45), and GradVac [46] adjust
gradients to resolve conflicts locally, but cannot model the global Pareto front. Recently, uncertainty- based methods have emerged. Chen al. 47 proposed an umcertainty-quantiin
genetic algorithm
for molecular design. Other methods include Upper Confidence Bound (UCB) [8], Expected Improvement (EI) [4),, Maximum Variance Criterion (MVC) [50), and Bayesian Optimization by density-Ratin Estimation (BORE) [51], which have shown promise but remain underexplored in RL-guided generative modeling.
3 Methodology
This section presents the core components of our framework, including the diffusion model back- bone, surrogate models, and RL-guided optimization. The overall architecture of our framework is illustrated Fig. Detailed explanations, mathematical formulations, and derivations are deferred to Appendix A.
Sample trajectories
Randem Aftmns
New Molecules
2,(D
Puel alR,
Estimate
Update
Mulxiply
Clip
Aggregate
Zz
2
Paliyy Lasc
Rewords
Dicvedity
Bonnn
Equivariant Diffusion Model
Reinforcement Learning- Optimization
Figure 1: Architecture diagram of the RL-guided diffusion model. (a) Conditional EDM is trained generate 3D molecules from random atoms, conditioned on target properties. The model learns a forward noising process g( ==//2,-1) 1) and a reverse denoising process Po po(2+-1 1 c). (b) In the RL phase, the pre-trained conditional EDM is used generate molecules by sampling full denoising trajectories. The transition probabilities Pa at each timestep are recorded. For each generated 3D molecule, structural diversity is computed, and a reward boost is applied if the molecule satisfies validity, uniqueness, and novelty criteria. In parallel, surrogate models are used to estimate the probability that each molecular property exceeds its cutoff. These probabilities are multiplied to compute the overall success probability (termed "overall uncertainty", i.e., the likelihood that the molecule satisfies all desired property thresholds.) Rewards are constructed by combining the overall uncertainty, the bonus, and the diversity. These rewards are multiplied along trajectories and used, together with transition pggobbbbbiliiies, compute a clipped policy gradient loss for updating the model.
3

---

3.1 Conditional EDM Backbone
Since most current 3D molecular diffusion models build upon EDM, we select as our backbone to ensure compatibility with future architectural extensions. EDM consists of two components:
Forward difffusion. The forward process adds Gaussian noise the atomic coordinates and features
x (r, h), where E IR M x3 denotes atomic positions and h E I xd denotes atom features. At each timestep E (1..... 7), noisy latents are sampled via q( z x) [zz 0xX, I, where oz
and are schedule parameters. This defines a Markov chain of corrupted states {ze} used to train the denoising model and establish reverse sampling chain.
Backward diffusion. The reverse process approximates the denoising distribution p(z--1 Zr, c) using a parameterized Gaussian:
p(z Zes c) N (2a_1))4a(2,,,,,,,,,,,
where uz( di -zz, - _t ēo (Z L c) e the condition vector concatenated the atom features,
oo
and ég is the predicted noise from an E(n)-equivariant Graph Neural Network (EGNN) [22
3.2 Surrogate Models for Multi-objective Uncertainty Quantification
The surrogate models quantify the uncertainty of molecular property satisfaction during RL-guided optimization. Each model estimates the likelihood that a molecule exceeds predefined threshold for a specific property, providing smooth reward signal for multi-objective policy learning.
Single-property uncertainty modeling. We employ Chemprop's 's Directed Message Passing Neural Network (D-MPNN) (53) the surrogate predictor. Each surrogate model is trained estimate the predictive mean µ(m) and variance o²(m) of a molecular property for a given molecule m. Assuming the prediction follows Gaussian distribution, define the uncertainty-awan reward for a single property as:
Usingle(ma (m; faa 00
(x (lmm) (m)
=
exp
dx,
(2)
a(m) / 2n
where is the target threshold, and E {+1, -13 indicates task directions, with 77 = +1 for properties where higher values are preferred (c.g.. QED) and 17 -1 for properties where lower values are preferred (c.g., SAS and binding affinity).
Multi-objective reward aggregation. Since each property is modeled independently and assumed to be conditionally independent given the molecule, the overall multi-objective reward is defined as the joint satisfaction probability:
A
Umutti (m;; .1,...,,op 6k)
II
U
single (m:; 8.);
where is the number of target properties and U Uanngle the reward for property i. The aggregated score Umuta (m: E [o, 1] provides smooth and interpretable signal that favors molecules likely to satisfy all property objectives simultaneously.
3.3 RL-guided Optimization
RL-guided optimization process consists of three key components: trajectory sampling, reward design, and policy update.
Trajectory sampling, At each episode, we sample n molecules and record their generation trajecto- ries (zz, 2r-1..... ). In RL, an explicit probability density is required for gradient estimation. Therefore, the reverse transition in Equation (1) is rewritten in its Probability Density Function (PDF) form as:
Pa (z,-1 Zr, c) (2moz)dd22 exp (-222 |z--1 - po(z,, t.)l?
(4)
20g

---

where of is the variance at timestep t, and d is the dimensionality of the latent representation. This formulation defines the likelihood of each denoising step in the Markov chain and serves as the basis for RI. policy gradient estimation.
Reward design. We design uncertainty-driven reward function. Considering Equation (3), the total reward for each generated molecule m is defined as:
Rasaal (m; 81, .i......................... 56. tepisode) mmutu 51, .1.....64) Reonus X(fepisade) D(m). (5)
where lepisode denotes the RL training episode index, (m) is determined by the validity, unique- ness, and novelty of m (the more conditions satisfied, the higher bonus obtained), 6, (1 i k) is a property threshold dynamically updated based moving average of the corresponding property values from previously generated molecules, and (mm) denotes the average Tanimoto similarity between m and the other molecules within the same batch. To encourage early-stage exploration and later-stage exploitation, the penalty weight decays over training episodes aS:A(fepisode = Age offepets
Policy update. We adopt Proximal Policy Optimization (PPO)-style [55] strategy to update the parameters of the diffusion model. The objective is to minimize the clipped surrogate loss:
LPO - Bm-pp [min (r(m)- (m), clip(r(m), 1 - 1 c) Reocal (m:)),
(6)
where r(m) =
ae (m) denotes the likelihood ratio between the current policy and the previous
(m(m)
policy and denotes the clipping range.
4 Experiments
4.1
Experimental Settings
This section describes the experimental design of our study, including datasets and data processing, surrogate model training, diffusion model training, RL-guided diffusion model training. baseline comparison, ablation study design, MD simulations and ADMET property prediction, evaluation, and computational resources. Detailed settings, dataset descriptions, hyperparameter configurations, and implementation details can be found in Appendix B.
Datasets and Data Preprocessing. We collected 3D molecular structures and SMILES strings of the three datasets. RDKit [56] was used to compute QED and SAS scores, and AutoDock Vina [57 was applied to evaluate binding affinity to EGFR. Each molecule is associated with a SMILES string, 3D structure, QED score, SAS score, and binding affinity. The datasets were split into training (80%), validation (10%), and test (10%) sets. For diffusion model training. molecules were split by species grouping. For surrogate model training, molecules were split by molecular scaffold. All models were trained and evaluated using standard vrain/validation/test splits.
Train surrogate models. We use Chemprop to train surrogate models predicting molecular properties from SMILES. A separate model is trained for each property each dataset, yielding nine models in total. Hyperparameters are selected via grid search.
Build diffusion models. Conditional EDM trained on 3D molecular structures and property distributions, using the original hyperparameters. GeoLDM and GFMDiff are also trained for comparative analysis.
RL-guided diffusion model training. The model is trained by sampling property combinations exceeding predefined cutoffs from the training conditioned on these properties. Generated molecules are evaluated for validity, uniqueness, novelty, Tanimoto diversity, and uncertainty compute rewards. Rewards are used to update model parameters via policy gradient.
input conditions. It generates molecules
Baseline comparison. We adapt four RL-guided diffusion baselines for image generation (SFT-PG, DDPO-SF, DDPO-IS, DPOK) to the molecular domain. We reformulated the task into a single- objective setting to fit these baselines. All baselines were trained across all datasets. Performance was compared against a vanilla diffusion model and our framework.
Ablation study design. Our ablation study includes two parts. First, we replace the multi-objective optimization module with four categories of alternative strategies: scalarization-basse (WS, POO, MMM, LSDW), constraint- based (NMD, NMD-WS, CP, PFM), gradient-based (SVD, PCGrad,
5

---

CAGrad, GradVac), and uncertainty-baser (UCB, EI. MVC, BORE). Second, we evaluate the effects of disabling key components: reward boosting, diversity penalty, and dynamic cutoff adjustment.
MD simulations and ADMET property prediction. MD simulations were performed using AmberTTools (58) and OpenMM [59]. Ligands and receptors were parameterized with ff14SB [60] and GAFF [61] solvated with TIP3P water [62]. and simulated under standard conditions (300 K. 1 bar) for total physical simulation time of 4,000 ps (approximately 1,000,000 steps). ADMET properties were predicted with ADMET-AI 24
Evaluation. Surrogate models were evaluated using the coefficient of determination (R2, measuring goodness of fit) [63], residual analysis, and Area Under the Calibration Error curve (AUCE, measuring predictive uncertainty calibration) [64]. To evaluate generation performance, 2,000 molecules were generated per run across three independent runs. Evaluation metrics included validity, uniqueness, novelty, molecular stability, atomic stability, and proportion of desired candidates.
Device. All experiments were conducted on NVIDIA A100 GPUs with 80 GB memory.
4.2 Results and Discussion
This section presents and discusses the experimental results, including surrogate model evaluation, diffusion model performance with and without RI.-guided optimization, comparisons with SOTA baselines, ablation studies, and MD) and ADMET analyses. Additional results, including detailed surrogate model performance, diffusion model performance across architectures, extended examples of generated molecules, and further analysis and explanation are provided in Appendix C.
4.2.1 Surrogate Models
We assess the prediction accuracy and uncertainty calibration of the trained surrogate models. The models achieve strong regression performance across all datasets and properties, with values of 0.99 for QED and SAS, and 0.86-0.88 for binding affinity. Residual plots and predictive variance analysis confirm that model uncertainty correlates with prediction errors. Uncertainty calibration is evaluated using reliability diagrams. The AUCE remains low across datasets and properties (0.02-0.10) 0.), indicating well-calibrated uncertainty estimates. These results demonstrate that surrogate models not only accurately predict molecular properties but also provide reliable uncertainty estimates, validating their use as a guidance signal for RL. fine-tuning of diffusion models.
4.2.2 RL-guided Diffusion Models
Fig. 2 shows the training process of the RL-guided diffusion models. As shown in Fig. 2 c the quality metrics, especially validity and molecular stability, are gradually improved over the course of training. The reward curves (Fig. ( grow steadily and eventually stabilize, indicating convergence.
These results confirm that our RL framework enables effective and stable optimization.
Table[compant our uncertainty- RL-guided diffuuson model with state-of-the-ar RL baselines and a vanilla diffusion model across all datasets. On QM9, our method achieves the best overall performance, with the highest validity 198.17%). VUN score (88.90%) and top molecule percentage (28.33%). It improves validity by over 9% compared to RL baselines and demonstrates better balance across validity, uniqueness, and novelty. On ZINC15, our model attains near- pear-perfect validity (99.02%) and outperforms all baselines across metrics, including 33 3.40% top molecule ratio. This aligns with the drug-like nature of ZINC15, which matches our property objectives better than the small organic compounds in QM9. On PubChem, despite its structural complexity, our method maintains 100% uniqueness and novelty, and outperforms baselines by a large margin. The relatively lower validity (16.23%) is attributed to the mismatch between pre-training and the high diversity of PubChem. The discrepancy between training and evaluation (Fig. - vs. Table 1D arises from the evaluation using larger sample size (2000 vs. 128 molecules). Overall, our method consistently outperforms all baselines across datasets, confirming its effectiveness and generalizability.
simpler molecules
As shown in Fig. 3 we compared the property distributions of 2,000 valid molecules generated with and without RL guidance across three trials. RL-guided models consistently improved QED and reduced SAS and binding affinity, indicating better drug-likeness, synthetic accessibility, and binding potential. The extent of improvement varied across datasets due to intrinsic property and structure

---

QM9
ZINC15
PubChem
100
Ratio (%)
94-
00-
40-
70-
valdo
88-
Lewilty
20
levely
60-
Roeely
10
15
20
25
30
35
40
45
50
55
60
20
40
80
100
120
140
160
180
5
10
15
20
25
30
35
40
45
1.45-
1.4.-
tetal Rewards
1.40-
1.2
1.38-
1.4
1.3
135-
1.2
1332
130-
0.6
0
10
20
30
95 60 65
20
40
60
100 Episados
80
120
140
180
5
10
15
25 Episodes
20
35
40
45
50
Episodes
Figure 2: RL.-guieed diffusion model training performance across three datasets. (a-c) Evolution of key molecular quality metrics (validity, uniqueness, novelty, atom stability, and molecular stability) over RL training episodes, where the vanilla diffusion models were pre-trained on QM9, ZINC15, and PubChem, respectively. (d-f) Corresponding total reward curves showing convergence trends.
Table I: Performance of baseline models and our method across three molecular datasets.
Dataset
Method
(%) (1)
Uni (%)
Nov (%) (f)
VUN (%)
ASta (f)
MSta (%) (f)
Top (%)
W/O RL
88.55 0:055
97.57 0.30
99.75 0.15
16 160
99. 34 0:11
95. 0.4
25.17 = 1.17
SFT.PG
88.57
96.80
99.80
85.57
99.24
95.62
25.58
QM9
DDPO-SF
88.65 1.05
97.39 0.44
99.79 0:00
86, 16 1.59
99.25 0.00
95. = 0.4
25.65 1.51
DDPO-1S
88.82
96.59
85,27
86.10
25.77
DPOK
88.10 0:37
97.52 0.02
99.81 8.15
85.75 0.50
99. n.0
95.22 0.15
25.20 = 1.44
Ours
98.17
90.90
99.17
28.33
W/O RL
30,05 1.34
100.00
100.00 =
30.05
88. 36
12.00
8.02
ZINCI5
SFT-PG
1:44
0.00
100.00
41.25
n.0
25.55
10.43 =
DDPO-SF
30,25
100.00
100.00
30.25
88.37 = 0.44
11.97
8.05
DDPO-IS
1.33
0.00
100.00
0,80
30.47
8.13 0:000
DPOK
30.13
100,00 0:00
100.00
30.13
88.42 n.6l
12.01
8.02 0.78
Ours
99.02 0.46
99.75 +: 0.06
100,00 8,80
6.49 98.77
99,86 0.8
98.08 0.6
33.40 = 6.39
W/O RT.
7.18
99.67
100.00
7.17
94.51
38. 38.18
2.23
PubChemt
SFT-PG
7.47 +: 1.40
99.57 +: 0.85
100.00 =: 6.80
7,44 1.37
82.99 0.49
33.22 0.34
2.03 0.76
DDPO-SF
100.00
100.00
DDPO-IS
10.50 6.19
99.90 0.0
100.00 :: 0:80
10.48 6.16
95. 36 n.99
45.37 =: 1.85
2.52 1.22
DPOK
99.67
100.00
38.17
Ours
16.23 9:9.2
100.00 = 0.00
100.00 = 0.80
16.23
9.72
99.04 : 0.13
88.65 0.59
2.97
Note: Each model generates 2,000 mokecules per run, Results are averaged over three independent runs and reported as mean = 95% confidence interval. "W/O RI denotes vanilla difflssion models without RI. "Vai", "Uni" and "Now" represent the percentages of valid, unique, and novel molecules, molocules that simulinnoousy valid, unique. and "MSta" denote atom-level level and molecule-level stability. "Top" indicates the propertoon generated molecules that simullancousdy satisfy all three property constraints, using relaxed cutofi' (QED 0.4, SAS < 8, and binding affinity e -4.3 to avoid missing patentially useful condidates, All metrics are reported percentages, and higher values indicate hetter performance.
respoctivcly. "VUN" their joint metric computed Val Uni x Nov, represening the and
constraints. These results highlight the robustness and effectiveness of our framework in optimizing multiple molecular properties.
4.2.3 Ablation Study
We conducted ablation studies to assess the contribution of each component in our framework (Table 2 Compared to traditional multi-objective strategies as well as other alternative uncertainty- based approaches, our method consistently outperforms across key metrics. While uniqueness slightly decreases, the trade-off is acceptable given the substantial gains other objectives. Simplified variants of our method show clear performance drops, confirming the necessity of the full design.
7

---

QM9
ZINC15
PubChem
38
0.0
85-
6.7
9.81
6.5
2.3
041
0.3
111
5A5
5.2-
-14
46
-75
Figure 3: Distributions of generated valid molecule properties across three datasets with and without RL-guided optimization. (a-c) QED distributions for vanilla diffusion models pre-trained QM9, ZINC15, and PubChem. (d-f) SAS distributions. (g-i) Binding affinity distributions. Each dataset shows results from three independent trials comparing vanilla diffusion models (without RL) and RL-guided diffusion models (with RL).
4.2.4 MD and ADMET analysis
We further evaluated the trug-likeness and stability of top-ranked molecules generated by our RL- guided diffusion models. As shown in Fig. 4 these candidates were compared with known EGFR mutant inhibitors using molecular docking, MD simulations, and ADMET profiling. The reference in-
hibitors showed stable RMSD trajectories and favorable pharmacokinetic profiles (Figs. 4-- Based on ADMET and MD performance, we found all candidates (molecules in Figs. d maintained stable protein-lige ligand complexes, with RMSD values within or below reference ranges. Notably, candidates Fig. 4 and Fig. [4 showed particularly strong conformational stability. ADMET analysis further confirmed good absorption, low CYP inhibition, and minimal toxicity. Overall, the generated compounds matched or outperformed the reference inhibitors, demonstrating the practical utility of our framework in identifying structurally stable, pharmacologically promising molecules for early-stage drug discovery.
Conclusion
We proposed an uncertainty-aware multi-objective RL-guided diffusion model framework for 3D de novo molecular generation. By incorporating predictive uncertainty into optimization, our method effectively balances complex and conflicting molecular design objectives. Experiments across multiple datasets show that RL-guided diffusion models generate high-quality molecules with superior validity, uniqueness, novelty, stability, and multiple drug-relevant properties. Notably, our method identified novel candidates with strong drug-like characteristics, MD stability, and favorable ADMET properties, beyond the reach of standard diffusion models. These results underscore the potential of framework for early-stage drug discovery.

---

Table 2: Ablation analysis results.
Category
Method
Val (%) (t)
Uni (%) (t)
Nov (1)
VUN (T)
ASta (7) (T)
MSta (f)
Top (%) (T)
95.92
99.81
87,86
52
96.80
27.02
Scalarizatin hased
POO
89.13 0.34
87.86 L35
99.45 w0
77.88 144
98.57 tM7
91.27 0:00
24.60
MMM
90,08 u.9
79.55 1:30
99.35
71,20 1:40
98.56 IM
91.33
26.53 2,24
LSDW
76.96
69.95
94.73
21.73
NMD
93.30 a: 2.45
83.57 1:33
99.61 000
77.67 a: 1:37
99.45 mm
96.08 1:03
25.75 1:30
Constraint-
NMD-WS
92.88
76.71
99.39
70.82
99.37
95.20
23.15
bused
CP
1.33
94.20
PFM
91.98 0.8
96.70 1:1
99.77 007
83.75 1 077
99.61 1 n01
97.58 1:978
24.68 1:16
SVD
84.62
95.34
99.57
80.32
97.72
87.70
25.62
Gradient-
POGrad
86.98 0.45
92.60 2:055
0.27 99.46
80.12 0.37
98.22 0:25
90.18 #:0
24.87
based
CAGrad
86.33 033
93.86 0:440
99.61
S0.72 : 0.75
98.30
90.35 0.74
24.27 0.90
VradVic
88,50
96.16
84.83
99.15
94.65
24,43
UCB
86.10
95.59 2:048
99,98 004
82.28 0.48
99.51 000
97. +:055
13:40
Uncertainty-
EI
85.68 uss
92.61 0:4.
99,50
78.95 u.5
98.00
88.88 0.2
26.97 1.72
based
MVC
87.78
97.23
85,18
99.03
94.42
24.03
BORE
89.33 2:477
97.09 4:3
99.81 1415
86.57 114
99.42 104
96.40 0.44
23.73 1.33
Reward Boost
90,00 0:52
96.93 0.0.9
99.68
86,95 20177
99. 22
95.40 9:0:6
25.92
WVO
Diversity Penalty
Static Cutoff
95.73 1.0
94.76 4:33
99.93 104
90.65 176
99.87 1u0
99.03 1:028
24.88 1.9
Ours!
98.17
88.90
99.87
99.17
28.33
This repeats previons results for easier compaiison.
Note: Each model generates 2,000 melecules run. Resalts are averaged over three independent zuns and reported ment a 95% confidence inserval. All experiments in this table concuctedd on the di fllssonn model pre-tmined QMO, "Val", "Uni" and "Nov" represent the percentages of valid, miqqee and novel mulecules, respectively, "VUN' their
the penveniage of molecules that simulumewwlly valid, unique, melecule-level stability. "Toy' indicates the propurtion of genensed molecules simellancossly sutisfy three property construints, using relased cutotis (QED (.4, SAS and binding aftinity 4.55 avoid missing potencially usctul candidates. "Ours refers framowork one modole (roward bocd,, divessity penalty, dynamic catotf) abloted. metrics are reported as percentages, and higher values indicate better performance.
computed as Val Uni Nov,
DEC: B25
0.13
crt:: 0.59
GED 0.79 SAS 5.75
SAS:
SAS: 5.95
6.55
affmty-44 4.4
Altinte SA
Affmnt--b0
AMtrANA 6.5
CIPEER GAATEE
CYPOOR
CYP364 HOTE
CYPSOS
CYNO0
CRUM EOVANS stastia stteppto CY0SA paccusii tar
COPOCIO
CVP20
CYPOEE
CYFS44 In Sheiee lglieer CYPERE untorall
Humah locutinll sssuydes
Figure 4: Comparison between a known EGER inhibitor and top molecules generated by RL- guided diffusion models. (a) Docking pose of known EGFR inhibitor. (b-d) Docking poses of the top-ranked molecules generated by RL-guided diffusion models pre-trained on QM9, ZINC15, and PubChem, respectively. QED, SAS, and binding affinity values shown in each panel. (e-l) Corresponding ADMET radar plots and MD simulation RMSD curves. (e, f) Known inhibitor. (g. h) Molecule from (c). (i.j) Molecule from (b). (k, 1) Molecule from (d). All MD trajectories were verified to have reached equilibrium prior to analysis, as indicated by the stabilization of RMSD values within a stable range of approximately 0.20-0.30 nm after initial relaxation phase and the absence of sustained drift or abrupt fluctuations. The presented molecules were selected based on their overall performance.

---

Limitations. While our approach shows strong overall performance, results on PubChem suggest that existing diffusion architectures may struggle with large, complex molecules containing many heavy atoms. This limitation stems from backbone scalability rather than the RI framework itself. Future work on scalable diffusion architectures may further improve performance on large-molecule datasets.
Broader Impact. This work presents a generalizable framework that may accelerate early-stage drug discovery by efficiently exploring chemical space and optimizing multiple pharmacologically relevant properties. enables the targeted design of molecules that meet predefined therapeutic criteria. Beyond drug discovery. the approach may benefit materials science, catalyst design, and molecular engineering. We acknowledge the ethical implications of powerful generative tools and support their responsible use within appropriate regulatory and safety frameworks.
ACKNOWLEDGMENTE
This work was supported in part by the Canada Research Chairs Tier II Program (CRC-2021- 00482), the Canadian Institutes of Health Research (PLL 185683, PJT 190272, PJT204042), the Natural Sciences, Engineering Research Council Canada (RGPIN-2021-00077) and The Canada Foundation for Innovation (CFI) John R. Evans Leaders Fund (JELF) program (#43481), the Stu- dentship/Fellowship funded by Breast Cancer Canada, and the Vector Scholarship in Artificial Intelligence provided through the Vector Institute.
References
[1] Petra Schneider, W. Patrick Walters, Alleyn T. Plowright, Norman Sieroka, Jennifer Listgarten, Robert A Jr Goodnow, Jasmin Fisher, Johanna M. Jansen, José S. Duca, Thomas S. Rush, Matthias Zentgraf, John Edward Hill, Elizabeth Krutoholow, Matthias Kohler, Jeff Blaney, Kimito Funatsu, Chris Luebkemann, and Gisbert Schneider. Rethinking drug design in the artificial intelligence era. Nature Reviews Drug Discovery, 1955)335-334, 2020.
[2] Benjamin Sánchez-Lengeling and Alán Aspuru-Guzin Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, 361 36400) 2018. Epub 2018 Jul 26.
[3] Rafael Klomez-Bombarelit Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato Benjamín Sánchez-Lengeling Dennis Sheberla, Jorge aguilera-Iparraguinn Timothy D. Hirzel, Ryan P. Adams, and Alán Aspura-Guzik. Automatic chemical design using a data-driven contimuous representation of molecules. ACS Central Science, 4(2):268-276, 2018.
[4] Jiaxuan You, Bowen Liu, Rex Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. arXiv preprint arXiv: 02473, 2018.
[5] Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. In Proceedings of the 9th International Conference on Learning Representations (ICLR), 2021.
[6] Brian L. Trippe, Jason Yim, Doug Tischer, Tamara Broderick, David Baker, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif soaff-scaffoldig problem. arXiv preprint arXiv:2206.044 220222220204110 2022.
[7] Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay Pande. Moleculenet: A benchmark for molecular machine learning. Chemical Science, 92):513-530, 2018.
[8] Trever G. Bivona. Egfr targeted therapy resistance: current status, challenges, and future outlook. Journal of Thoracic Disease, 12(5):2849-2854 2020.
[9] Guanghao Wei, Yining Huang, Chenru Duan, Yue Song, and Yuanqi Du. Navigating chemical space with latent flows. Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS), 2024.
10

---

[10] Alexandra Volokhova, Michat Koziarski, Alex Herándee-Gaarii,, Cheng- Hao Liu, Santiago Miret, Pablo Lemos, Luca Thiede. Zichao Yan, Alán Aspuru- Guzik, and Yoshua Bengio. Towards equilibrium molecular conformation generation with gflownets. volume 3, pages 1038-1047, 2024.
[11] Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, and Diederik Kingma. Learning energy- based models by diffusion recovery likelihood. In Proceedings of the 9th International Confer- ence on Learning Representations (ICLR), 2021.
[12] Yue Cao and Yang Shen. Energy-based graph convolutional networks for scoring protein docking models. Proteins: Structure, Function, and Bioinformatics, 88(8):1091-1099 2020.
[13] G. Richard Bickerton, Giovanni V. Paolini, Julien Besnard, Sorel Muresan, and Andrew L. Hopkins. Quantifying the chemical beauty of drugs. Nature Chemistry, 4(2):90-98, 2012.
[14] Peter Ertl and Ansgar Schuffenhauer, Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminfor- matics, 1(1):8, 2009.
[15] Madhav V. Samudrala. Somanath Dandibhotla, Arjun Kaneriya, and Sivanesan Dakshanamurthy. Plaig: Protein-ligand binding affinity prediction using novel interaction- graph neural network framework. ACS Bio & Med Chem Au, 5(3):447 463, 2025.
[16] Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformaties, 9(1):48, 2017.
[17] Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for de drug design. Science Advances, 4(7)xeaap7885, 2018.
[18) Ragy Haddad, Eleni E. Litsa, Zhen Liu, Xin Yu, Daniel Burkhardt, and Govinda Bhisetti. Tar- geted molecular generation with latent reinforcement learning. Scientific Reports, 15(1):15202, 2025.
[19] Hwanhee Kim, Soohyun Ko, Byung Ju Kim, Sung Jin Ryu, and Jaegyoon Ahn. Predicting chemical structure using reinforcement learning with a stack-augmented conditional variational autoencoder. Journal of Cheminformatics, 14:83, 2022.
[20] Steven Kearnes, Li Li, and Patrick Riley. Decoding molecular graph embeddings with rein- forcement learning. In Proceedings of the ICML Workshop on Learning & Reasoning with Graph-Structured Data, 2019.
[21] Daria Grechishnikova. Transformer neural network for protein-specifie de novo drug generation as machine translation problem. Scientific Reports, 11:321, 2021.
[22] David Weininger. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules, Journal of Chemical Information and Computer Sciences, 28(1):31-36, 1988.
[23] Alessandra Nurisso, Antoine Daina, and Ross C. Walker. A practical introduction molecular dynamics simulations: Applications to homology modeling. In Homology Modeling: Methods and Protocols, volume 857 of Methods in Molecular Biology, pages 137-173. Humana Press, Totowa, NJ, 2012.
[24] Kyle Swanson, Parker Walther, Jeremy Leitz, Souhrid Mukherjee, Joseph C. Wu, Rabindra V. Shivnaraine, and James Zou. Admet-ai: a machine learning admet platform for evaluation of large-scale chemical libraries. Bioinformatics, 40(7):btac416 16, 2024.
[25] Niklas W. A. Gebauer, Michael Gastegger, and Kristof T. Schütt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. Advances Neural Information Processing Systems 32 (NeurIPS), pages 7566-7578, 2019.
[26] Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs, Ingmar Posner, and Max Welling. E(n) equivariant normalizing flows. In Advances Neurul Information Processing Systems 34 (NeurIPS), pages 4181-4192, 2021.
11

---

[27] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In Proceedings of the 39th International Conference on Machine Learning (ICML), 2022.
[28] Minkai Xu, Alexander S. Powers, Ron O. Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In Proceedings of the 40th International Conference on Machine Learning (ICML). Proceedings of Machine Learning Research, vol. 202. pages 38592-38610, 2023.
[29] Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong. Mdm: Molecular diffusion model for 3d molecule generation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
[30] Can Xu, Haosen Wang, Weigang Wang, Pengfei Zheng, and Hongyang Chen. Geometric- facilitated denoising diffusion model for 3d molecule generation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 338-344--- 338-346,20 March 2024.
[31] Ying Fan and Kangwook Lee. Optimizing ddpm sampling with shortcut fine-tuning. In International Conference on Machine Learning, 2023.
[32] Kevin Black, Kimin Lee, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, and Mohammad Ghavamzadeh. Training diffusion models with reinforcement learning, arXiv preprint arXiv:2305. 2023.
[33] Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. In Advances in Neural Information Processing Systems, 2023.
[34] Solomon Kullback and Richard A Leibler. On information and sufficiency. The Annals of Mathematical Statisties, 22(1):79-88) 1951.
[35] Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Aréé Clevert. Efficient multi-objective molecular optimization in a continuous latent space. Chemical Science, (0034))8016-8024 2019.
[36] Zuzana Červeñansáá, Janette Kotianová, Pavel Važan, Bohuslava Juhásová, and Martin Juhás. Multi-objective optimization of production objectives based on surrogate model. Applied Sciences, 10(21):7870, 2020.
[37] Andrzej P. Wierzbicki. The Use of Reference Objectives in Multiobjective Optimization, volume 177 of Lecture Notes in Economics and Mathematical Systems. Apringer-Verlayn Berlin / Heidelberg, Germany, 1980.
[38] Axel Abels, Diederik M. Roijers, Tom Lenaerts, Ann Nowé, and Denis Steckelmacher. Dynamic weights in multi-objective deep reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning (ICML), volume 97, pages 11-20, 2019.
[39] Wei-Yu Chiu, Gary G Yen, and Kang Juan. Minimum manhattan distance approach to multiple criteria decision making in multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 20(6):972-985, 2016.
[40] Kaisa Miettinen. Nonlinear Multiobjective Optimization, volume 12 of International Series in Operations Research & Management Science. Kluwer Academic Publishers, Boston, MA,
1 1999.
[41] R. Timothy Marler and Jasbir Singh Arora. Survey of multi-objective optimization methods for engineering. Structural and Mulidiiscppinnay Optimization, 26(6):369-395, 2004.
[42] Anthony V. Fiacco and Garth P. McCormick. Nonlinear Programming: Sequential Uncon- strained Minimization Techniques. John Wiley & Sons, New York, NY, 1968.
[43] Dmitry Semushkin, Nikolay Patakin, Arseny Kuznetsov, and Anton Konushin. Independent component alignment for multi-task learning. Proceedings of the 2023 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12240-8229,, 2023.
12

---

[44] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman. and Chelsea Finn. Gradient surgery for multi-task learning. In Advances in Neural Information Processing Systems 33 (NeurIPS), pages 9682-9693, 2020.
[45] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In Advances in Neural Information Processing Systems 34 (NeurIPS), 2021.
[46] Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In Proceedings of the 9th International Conference on Learning Representations (ICLR), 2021.
[47] Lung-Yi Yi Chen and Yi-Pei Li. Uncertainty quantification with graph neural networks for efficient molecular design. Nature Communications, 16:3262, 2025.
[48] Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning (ICML), pages 1015-1022, 2010.
[49] Jonas Mockus. Application of bayesian approach to numerical methods of global and stochastic optimization. Journal of Global Optimization, 4:347-365, 1994.
[50] Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient bayesian optimization. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 3627-3635, 2017.
[51] Louis C. Tiao, Aaron Klein, Matthias W. Seeger, Edwin V. Bonilla, Cédric Archambeau, and Fabio Ramos. Bore: Bayesian optimization by density-ratio estimation. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139, pages 10289-10300. PMLR, 2021.
[52] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling, E(n) equivariant graph neural networks. In Proceedings of the 38th International Conference on Machine Learning (ICML), pages
2021.
[53] Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi Jaakkola, Klavs Jensen, and Regina Barzilay. Analyzing learned molecular representa- tions for property prediction. Journal of Chemical Information and Modeling, 59(8):3370-3388, 2019.
[54] P. Murphy, Kevin Machine Learning: A Probabilistic Perspective. Adaptive Computation and Machine Learning. The MIT Press, Cambridge, MA, 2012. Hardcover edition.
[55] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint I
707. 707.06347, 2017.
[56] Greg Landrum. RDKit: Open-source cheminformatics, 2006. http://ww.rrkii rakit. org
[57] Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, (1(2):455 461, 2010.
[58] David A. Case, Hasan M. Aktulga, Kellon Belfon, David S. Cerutti, G. Andrés Cisneros, Vinícius W. D. Cruzeiro, Negin Forouzesh, Timothy J. Giese, Andreas W. Götz, Holger Gohlke, Saced Izadi, Koushik Kasavajhala, Mehmet C. Kaymak, Edward King. Tom Kurtzman, Tai- Sung Lee, Pengfei Li, Jian Liu, Tyler Luchko, Ray Luo, Madushanka Manathunga, Matias R. Machado, Hai Minh Nguyen, Kurt A. O'Hearn, Alexey V. Onufriev, Feng Pan, Sergio Pantano, Ruxi Qi, Ali Rahnamoun, Ali Risheh, Stephan Schott- Verdugo, Akhil Shajan, Jason Swails, Junmei Wang. Haixin Wei, Xiongwu Wu, Yongxian Wu, Shi Zhang, Shiji Zhao, Qiang Zhu, Thomas F. III Cheatham, Daniel R. Roe, Adrian Roitberg, Carlos Simmerling, Darrin M. York, Maria C. Nagan, and Kenneth M. Jr. Merz. Ambertools. Journal of Chemical Information and Modeling, (1200)611836199 2023.
13

---

[59] Peter Eastman, Raimondas Galvelis, Raú P. Peláez, Charlles R. A. Abreu, Stephen E. Farr, Emilio Gallicchio, Anton Gorenko, Michael M. Henry, Frank Hu, Jing Huang, Andreas Krämer, Julien Michel, Joshua A. Mitchell, Vijay S. Pande, João P. G. I.. M. Rodrigues, Jaime Rodriguez- Guerra, Andrew C. Simmonett, Sukrit Singh, Jason Swails, Philip Turner, Yuanqing Wang, Ivy Zhang, John D. Chodera, Gianni De Fabritiis, and Thomas E. Markland. Openmm 8: Molecular dynamics simulation with machine learning potentials and gpu/cpu advances. The Journal of Physical Chemistry B, 128(1):109-116 2024.
[60] James A Maier, Carlos Martinez, Koushik Kasavajhala, Lars Wickstrom, Kevin E Hauser, and Carlos Simmerling. ff14sb: Improving the accuracy of protein side chain and backbone parameters from ff99sb. Journal of chemical theory and computation, 1 11(8):3696331 -7113, 2015.
[61] Junmei Wang, Romain M. Wolf, James W. Caldwell, Peter A. Kollman, and David A. Case. Development and testing of general amber force field. Journal of Computational Chemistry, 25(9):1157-1174, 2004.
[62] William Jorgensen, Jayaraman Chandrasekhar, Jeffry D Madura, Roger Impey, and Michael I. Klein. Comparison of simple potential functions for simulating liquid water. The Journal of Chemical Physics, 19:926-935, 1983.
[63] N. J. D. Nagelkerke. A note on general definition of the coefficient of determination. Biometrika, 78(3):691-692 1991.
[64] Chuan Guo, Geoff Pleiss, Yu Sun, and Q. Weinberger, Kilian' On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 1321-1330, 2017.
[65] Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu. Equivariant energy- guided sde for inverse molecular design. In Proceedings of the I Ithh International Conference on Learning Representations (ICLR), 2023.
[66] Ali Khodabandeh Yalabadi, Mehdi Yazdani-Jahroma and Ozlem Ozmen Garibay. Bokd- iff: Best-of-k diffusion alignment for target-specific 3d molecule generation. arXiv preprint (ttv:2501.156311 2025.
[67] Miroslav Suruzhon, Michael S. Bodnarchuk, Antonella Ciancetta, Ian D. Wall, and Jonathan W. Essex. Enhancing ligand and protein sampling using sequential monte carlo. Journal of Chemical Theory and Computation, 18(6)338443310 2022.
[68] Julian Cremer, Tuan Le, Frank Noé, Djork-Aré Clevert, and Kristof T. Schütt. Pilot: equivariant diffusion for pocket-conditioned de novo ligand generation with multi-objective guidance via importance sampling. Chemical Science, 15:14954-14967, 2024.
[69] Ziqi Chen, Bo Peng, Tianhua Zhai, Daniel Adu-Ampratwum, and Xia Ning. Generating 3d small binding molecules using shape-conditioned diffusion models with guidance. Nature Machine Intelligence, 7:758-770, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the core contributions of the paper, including the design of an uncertainty-an
multi-objective reinforcement learning frame- work for optimizing 3D molecular generative diffusion models. The claims are grounded in both theoretical formulation and empirical validation across multiple datasets. The limita- tions of the method are acknowledged in the last section. Moreover, while the long-term motivation of aiding real-world drug discovery is mentioned, it is clearly separated from the achieved contributions. Therefore, the claims accurately reflect the paper's scope and results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Section includes limitations of our work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This paper does not include theoretical formal proofs.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All experimental settings, including data splits, model backbones, hyperparam- eters, and evaluation protocols, are fully described in Section 4 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The datasets used in our study are publicly available, with access links provided in the Appendix. The source code will be released upon acceptance of the manuscript.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
soi.immmiiiiooooiiioooooo... for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips. www.ccppwwcpcccccceeceoeooooeeee htte://wiisss/eeeeeeeeee www.eoeuiiisioonnoooooooo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All experimental details, including dataset descriptions, data splits, model archi- tectures, optimization procedures, hyperparameter configurations, and evaluation protocols, are clearly documented in Section 4 and further detailed in Appendix. These specifications are sufficient to fully understand and reproduce the reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report 95% confidence intervals across multiple runs for all key experi-
ments.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We mention the computational resources used to conduct the experiments in Section 4 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code of Ethics and confirm that research fully complies with the guidelines. Our work does not involve human subjects, sensitive data, potential misuse.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
18

---

Justification:
Broader impacts are discussed in Section 5.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not involve models or datasets with high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external tools (c.g., RDkit, AutoDock Vina) and models cited and used under their respective licenses.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
19

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperssit paperswi thcode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new code and models, which are documented and will be released with usage instructions upon acceptance.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve research with human subjects and therefore does not require IRB approval.
Guidelines:
20

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs are not used as a component of the core methods in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

Appendix
A MMehhdooogy
This section provides extended details to supplement Section of the main manuscript.
A.1 Conditional Diffusion Model
Notation. We define RMxx as the coordinates of M atoms in 3D space, constrained to the zero center of gravity subspace, meaning the sum the coordinates Σ, x; = The atom features, R2xxd are invariant to E(3) transformations and include attributes like one-hot encoded atom types or charges. The conditioning variable c represents a desired molecular property. At each diffusion step t, the latent variable z,
(l)) combines the noised coordinates z and
features zi The parameters a and or control the balance between signal retention and noise addition, while o(., t, c) is Equivariant Graph Neural Network (EGNN) that predicts noise during denoising, conditioned on both the time step and the property c. Finally, Nigh denotes joint normal distribution over coordinates and features, with coordinates constrained to the zero center of gravity subspace.
Forward Diffusion. The forward diffusion process is the first key component of EDM, designed to gradually corrupt the original data into noise, creating a sequence of latent variables Zu, Zi...., Zr that we can later denoise to generate new samples. In standard diffusion model, the process adds noise to a data point x according to the formula:
(Z: x) = N(z;; 11))
(A.1)
where an determines how much of the original signal is retained, and of controls the variance of the added Gaussian noise. This formula captures the essence of diffusion: increases, the data progressively noised until it resembles pure noise at t= T.
In EDM, this idea is extended jointly model both coordinates and features h, since molecules require both positional and categorical information. The forward process is Markovian, meaning each step depends only on the previous one, and is expressed
q( Zi..... ZT x. h) 9(z0 x. h) II 1=1
g(zz z(-1)-
(A.2)
The initial distribution is defined as:
<(a x, h) = Nun (zo o hl,
(A.3)
and the transition from 2 to Z given by:
g(zz 2.-1) Nun -,
(A.4)
where 0'je-1 01//u- adjusts the signal scaling between steps, and the noise variance is
سă = o² o 021-11221
The joint distribution Neh factorizes into distributions for coordinates and features:
Nat (zz an h), r (x! a,xx, of (Z (h) (,,h, I.)..
(A.5)
Here, N₂ is a normal distribution over the zero center of gravity subspace, ensuring the coordinates remain translation-invariann, and is defined as:
A². [x] μ4, I) (V2mm) (M- exp (-1 - 2
(A.6)
with μ in the zero center of gravity subspace. The feature distribution A is a standard normal distribution, reflecting the invariance of h to E(3) transformations.
22

---

The noise schedule is designed to be variance-prece preserving, meaning -V- ensuring that the total variance of the data remains constant as noise is added. The specific form of a, is chosen to smoothly transition from retaining the signal to adding noise:
a2 = (1 2s) (1 - (t/T)²) +
(A.7)
where 10 is small constant to prevent numerical issues, allowing at to decrease from do R 1 (almost pure signal) to ar s 0 (almost pure noise). To quantify the balance between signal and noise at each step, we define the Signal-to-Noise Ratio (SNR):
SNR(t) = = = a1
(A.8)
The SNR decreases as increases, reflecting the increasing dominance of noise. For computational convenience during training and sampling, we also compute the negative log-SNR, defined as:
(1) = log (NR( = log of + log o²
(A.9)
The purpose of the negative log-SNR is to provide a numerically stable representation of the noise level at each step, which is particularly useful when optimizing the model or scheduling the noise. Using properties of the sigmoid function, we can express aj sigmoid(-7(th) and of (y))o which simplifies calculations and ensures that the noise schedule is well-behaved across all timesteps.
Backwar ward Diffussio. The goal of the reverse process to generate new molecules by denoising the latent variables, starting from pure noise at t = 'I' and iteratively refining them back to a data sample at In a conditional diffusion model, the reverse process approximates the denoising step, conditioned on as:
1 2z, c) = -N(z-1;//0ZZ, 1, (,)), oT
(A.10)
where Ho (Z,, t, c) the mean predicted by EGNN, guiding the denoising process toward samples that satisfy the condition This formula provides high-level view of the reverse process, capturing how the model learns to reverse the noise addition step by step.
We define this process more precisely to ensure E(3) equivariance. The true posterior for the reverse step is:
<(z x. h, z,) (Z, Pe-+o ( h]. o o., where 8 < i, and the mean and variance are:
(A.11)
Me_aa(x h), 2,) (an, (01,00/07)2, + / /o,) hi.
(A.12)
Tt (o)
2 o. adp = ac/os: oge of -
(A.13)
Since and h are unknown during generation, approximate them using EGNN olz,,,, c), which predicts the noise é = [=( (.)) The estimated data is then computed as:
h (z./a,) (a./o,))..
(A.14)
where é, = Φ(zz, t, c). Substituting this into the mean, the reverse transition distribution becomes:
p(z, Ze, c) No (z. |e- (ix h). zt), II
(A.15)
For sampling, we start with 2r ~ NZ (0, I), representing pure noise, and iteratively apply the reverse process for = T down = 1, setting
1. At each step, we sample ~ ~N(0, I), subtract the center of gravity from z, compute the predicted noise é, = o(2,, c). and update:
(a)
2 (1/agj) (o/)s/ oc+oc.
(A.16)
Finally, we sample the data (x, h) p(x, zo, c). EGNN ensures that the predicted noise for coordinates is equivariant:
Re (f(²) (RZ (x) 24 .),
(b)
(A.17)
where R is an orthogonal matrix.
23

---

Training Objective. To train the model, we aim to maximize the conditional log- log-likelhhod logp( p(x c), which is achieved by optimizing a variational lower bound:
logp(x, c) 2 Le,0 + buse +
Le
(A.18)
The term Le,t - -KL(((, x, h, Ze) [lp(z, Zz, c)) measures the divergence between the true and approximate reverse distributions for .,...,7 The term L..0 = log (x, h Zo, c) evaluates the likelihood at the final step, and Lebbase (L x. h) ([p())) compares the forward process at = T to the prior. Since or = 0, base =s O.
For t 2 1, the KI. divergence simplifies to noise prediction objective:
Le,t (o..) [11)))o(t]l o(z,, t, c)
(A.19)
where wt) 1 SNR(t 1)/SNR(t). Setting w(t) 1 simplifies training and improves sample quality, reducing the objective to minimizing the mean-squared eπror between the true noise and the predicted noise σ(z,, t, c). During training. we sample & compute ar xx h] + dtc, and minimize || o(z,, i, c)
T), noise e (O I).
(h) The zero-term Ce.0 splits into contributions from coordinates and features: Ce,0- c.0 + L., For coordinates, we approximate the posterior as:
q(x ) xo Nx 2 /00, (0//a
(A.20)
and the generative distribution is:
p(x zo, c) = -N(x (z) /0o) - (0o/ao)e (000///000
(A.21)
yielding:
(a) = Ee(H)
log 2 - 1 [ - (20) (100)2200,,))
(A.22)
2
where Z (V2n 00/0a) (M-))3
For categorical features, the noising is
( h) (z)
(h) a, ,f
(A.23)
and the generative distribution is categorical distribution, approximating ((h) se for small dg-
Conditional Generation Mechanism The purpose of conditional generation is to produce molecules (x, h) ~ p(x, C, M) that satisfy the desired property e and have M atoms. We first sample (c, MM) p(c, MM) from learned distribution estimated from the training data. Then, we sample pure noise ZT ~ Nun (0, I), iteratively denoise using p(z--1 Z,, c). and finally sample the data (x, h) p(x, h Zo. c). The neural network ((z,, t, c) incorporates by concatenating to the node fcatures, guiding the denoising process toward the desired property.
A.2 Surrogate Models for Multi-objective Uncertainty Quantification
Uncertainty Modeling. The uncertainty Usingle(1n 5) encompasses both aleatoric and epistemic uncertainties, combined into the total variance
iim (m) = o (m) + + (m)
(A.24)
Aleatoric uncertainty o² reflects inherent data noise, such as variability from computational simula- tions, while epistemic uncertainty of arises from model limitations due to limited training data.
In the Chemprop model with evidential learning, these are estimated using the Normal Avvere-Gamma (NIG) parameters:
B
0 where B, a, and are predicted by the model.
and 0²
(A.25)
v(a -
24

---

The Gaussian distribution assumption, with mean ((m) and standard deviation
o amm) V (m))
(A.26)
reasonable because allows the integral in Singlo (m;; 6) to compute a probability, validated by the model's calibration to real-world data variability.
Uncertainty as a Threshold Probability. The uncertainty (m) directly informs the probability that molecule's true property value exceeds the threshold ò, computed as:
Usingle (m; 6) = 1-50
x p(m)) (mm)
2
J o(m) 2m exp
dx.
(A.27)
This integral represents the cumulative probability under a Gaussian distribution, where a larger a (m) reduces the likelihood of exceeding due to increased spread, and a smaller (m) increases u(m) is favorable.
The inclusion of total uncertainty (o ensures that both aleatoric and epistemic contributions are considered, reflecting the overall reliability of the prediction.
Multi-Objective Uncertainty.
The multi-objective uncertainty is defined as:
Umutii (m:; .1,
II U siglle (m; 8,).
(A.28)
This product is justified by assuming conditional independence of property uncertaintics given the molecule, meaning the likelihood of meeting all thresholds 81,... òn is the joint probability of individual successes. Each siggge (m;; 6.) measures the probability for one property, and the product ensures molecule is rewarded only if likely satisfies all objectives simultancously.
This approach avoids trade-offs, penalizing molecules with low probability for any property, which is critical in RI. where a balanced solution is needed, making it a coherent aggregation strategy.
A.3 RL-guided Optimization
Transition Probability. We first employ the trained diffusion model sample of molecules and record their diffusion trajectories. Each trajectory can be treated as Markov chain. The transition probability between two states the chain are defined in Equation (A.4). Its PDF format is given by
(Zt-1 Zt, c) (2 exp (-
||z--1 po(2,, ct,)))
(A.29)
20
relying on the variance of and the latent variables z, and 2,-1. The variance of is computed based on the noise schedule in the diffusion process. Referring to Equation (A.13). define
(A.30)
where of = 1- Ct,, ensuring that the variance increases grows, reflecting the progressive addition of noise in the forward process.
The latent variable is sampled iteratively during the backward process of the diffusion model. Starting from Zr -O(0, I), each z, fort 1,.... is computed using Equation (A.
oi
(A.31)
Odll
Jario
where e ~ IV, and ē 0(z,, t, c). Subsequently. 24-1 is sampled from N(zz-1 (e-s((xx, ([x, h), Zz -
(A.32)
ensuring the denoising trajectory aligns with the Iearned distribution.
25

---

Reward Bonus Design. The reward bonus Reonus (m) is determined by three binary flags: validity (Vm). uniqueness (1m)) and novelty (Rm),, each taking values in {0, 1}. These flags indicate whether molecule m satisfies the respective criteria. The bonus is formulated as weighted linear combination:
Roonss (m) = b Um + by tm + bm
(A.33)
where by: by and b, are positive scalars representing the reward contributions of validity, uniqueness, and novelty, respectively.
This reward- boosting design allows binary objectives related generation quality be integrated without inducing reward sparsity, thereby enabling unified optimization of both binary and continuous rewards multi-objective setting.
Optimization. We optimize the diffusion model using PPO-style clipped surrogate loss. Gradients are accumulated over multiple sampled timesteps based fixed trajectories, enabling stable updates under complex multi-objective rewards. The pseudocode is shown in Algorithm A.1
Algorithm A.1 RL-guided Optimization for Diffusion Models
1: Initialize diffusion model To. optimizer, and learning rate scheduler
2: for episode = to N do
3: Freeze current policy as wa
4: Generate a batch of molecules using Tda
5:
Record complete diffusion trajectories and transition probabilities
6:
Estimate uncertainty and compute reward probabilities
7:
Measure molecular diversity, validity, uniqueness, and novelty Compute total rewards
8:
9:
for each policy update over fixed trajectories do Sample timesteps t1, ....... tx ~ 2(( T)
10:
for each sampled timestep & do
12:
Evaluate log- prg-pobbabilites under at timestep t Compute importance ratio r(m) = exp(logpo log pa..)
14:
Compute clipped surrogate loss Accumulate gradients
16:
end for
17:
Update Ty using accumulated gradients Step learning rate scheduler
18:
19:
end for
20:
end for
26

---

B Experimental Settings
As illustrated Fig. B.1 our pipeline consists of five key stages: model training, diffusion model pretraining, RL-guided optimization of the diffusion model, and molecule generation with subsequent in silico evaluation. The following sections provide additional implementation details for each stage of the pipeline.
data preprocessing, surrogate
Molecular Input Dataset
Data
Train
Diffusion Model
Generate
Novel Molecules
Preprocessing
Train
Guide
Surrogate Model
in silico Evaluation
RL
Figure B.1: Overall workflow of our framework. A molecular dataset is first preprocessed train both the diffusion model and the surrogate models. RL guides the optimization of the diffusion model using predictions from the surrogate models. The optimized diffusion model then generates candidate molecules for in silico evaluation.
B.1 Data Sources and Descriptions
use three molecular datasets (QM9, ZINCI5, and PubChem) as sources for training and evaluating surrogate and diffusion models. In addition, we use curated crystallographic structure of mutant EGFR as the target receptor for molecular docking. Table B.1 summarizes the sources, descriptions, and download options for each dataset used in this study.
B.2
Data Preprocessing
B.2.1 Target Protein Preprocessing
We selected the human EGFR as the docking target in this study due to its central role in cell signaling and clinical importance in various cancers. In particular, we used the 3D crystal structure with PDB ID 6VHN, which corresponds to the kinase domain of mutant form of EGFR bound to smoll-molecule inhibitor. This structure provides a biologically relevant conformation for evaluating ligand binding and offers sufficient resolution (2.5 Å) for structure-based modeling.
To prepare the protein for molecular docking, ensure structural completeness and compatibility with docking software. First, all water molecules, ions, and co-crystallized ligands not involved in the binding site were removed to eliminate potential artifacts and ensure clean receptor surface. The structure was then processed convert any non-standard residues into their canonical forms, and missing side chains were reconstructed using geometry- baseed modeling ensure complete atomic representation. Polar hydrogen atoms were added throughout the protein to restore potential hydrogen bonding interactions typically unresolved crystallographic data. Finally, partial atomic charges were assigned to the protein using a standard method compatible with docking engines. The resulting structure was saved in the appropriate format and used as the receptor input in subsequent docking simulations.
we applied a series of standard preprocessing steps to
B.2.2 Molecular Data Preprocessing
To process each dataset, we first remove invalid molecules (e.g., those with missing 3D coordinates or broken structures) and eliminate duplicates. For the remaining molecules, we annotate three key properties (QED, SAS, and binding affinity) for each molecule. Specifically, QED and SAS are computed using RDKit, while the binding affinity is predicted using AutoDock Vina through docking simulations with the EGFR receptor site. These property annotations will used in both surrogate model training and diffusion model training.
27

---

Table B.1: Sources and descriptions of datasets.
Dataset
Item
Details
Source
http:///eeephemm. wtw.:/hhe......--eeeeeeee.e.. -e3-webbite--ee us weettst-eett-eeeeeeeeeeeeeeeeeee 1aast-1..0 datasets/ /db.. gab..tar........................
amazonaws.
QM9
Description
Dataset containing quantum chemical propertics small organic molecules. QM9 is widely used in the development and evaluation of machine learning models for molecular property prediction.
Download Option
Whole datasct (SDF format)
Source
https://zincil: docking org/ wetg/tranches/homee
ZINCIS
Description
free database commercially available molecules for virtual screening, frequently used in drug discovery research.
Download Option
3D lead- like molecules (SDF format); neutral or +0 charged; mid-range logP;
stock availability; standard reactivity
Source
https: ncbi .nmm nih. htt../////puryyssaaall///olollla tab=compound
PubChem
Description
A public chemical datahase providing bioactivity data and annotations for small molecules. PubChem is extensively used in cheminformatics, drug discovery, and bioinformatics applications.
Download Option
Small molecule subset (SDF format)
Source
https://www.rcet rcsb. http/structureeee
EGFR
Description
Crystallographic structure of the kinase domain of mutant human EGFR (PDB ID: 6VHN), co-crystallized with small-molcoo inhibitor. This structure serves the receptor target for binding affinity prediction via molecular dack- ing.
Download Option
Downloadable in .pdb format from RCSB PDB
After property computation, all hydrogen atoms are removed from the 3D molccular structures. This decision is motivated by the need reduce computational overhead and simplify molecular graphs during preprocessing and training. Hydrogen atoms offer limited topological information and can be accurately reconstructed and spatial information of each molecule remains intact, this step introduces no adverse impact on the training performance. Each molecule is ultimately represented by its SMILES string, 3D structure (including atom types and coordinates), and its drug-related properties. The SMILES string is used as molecular input when training surrogate models, while 3D structure used as input to diffusion model.
later stage based on standard valency rules. Because the core chemical
To train the diffuuson model, we further construct dataset-level configuration files based on the hydrogen-remover molecules, For each dataset, each distinct atom type in the molecule is assigned numerical encoding via an atom encoder and its corresponding decoder. The unique atom types identified in each dataset are listed in Table(B.2 The number of atoms in each molecule is counted to determine the maximum graph size across the dataset. Fig B. illlssrrate the number of molecules in each dataset grouped by heavy atom count. Molecules in QM9 typically contain fewer than 10 heavy atoms, while those 2INC15 and PubChem span a broader range of sizes. Especially in PubChem, some structures exceed 40 atoms.
Table B.2: Unique atom types observed in each dataset. (after hydrogen removal)
Dataset
Unique Atom Types
QM9
C. O F C. P.S. CL Br. C.N.O.F.RS.C..B CL Br,
ZINC15
PubChem
To prepare the data for model training and evaluation, we apply different data splitting strategies tailored to the modeling task.
28

---

QM9
ZINC15
PubChem
120c
136222
3399345
33k
42884
Number of Molscuiee
100k
20k
2640
21k
19k
10k
1160
5k-
3234
31
129
1
Number Alome
Figure B.2: Distribution of molecules by the number of heavy atoms in each dataset. Subfigures (a), (b), and (c) correspond to QM9, ZINC15, and PubChem, respectively.
Scaffold-based splitting, For the surrogate model, we adopt scaffold-based splitting. Specifically, each molecule is first assigned Semis-Murche scaffold by removing its side chains and retaining the core ring and linker structures. Molecules sharing the same scaffold are grouped together. These scaffold groups are then randomly shuffled and assigned to the training, validation, and test in such way that scaffold appears in more than one set. This ensures that the model is evaluated on structurally novel scaffolds not seen during training.
Species- based splitting. For training the diffusion model, we use species-based splitting. Each molecule grouped based on the unique of atom types it contains (i.e., its chemical species). These species groups are shuffled and assigned to disjoint training, validation, and test partitions, ensuring that each subset contains distinct combinations of atomic species. This strategy preserves the global diversity of atom types while promoting generalization to structurally diverse molecules in 3D space.
Table B.3 summarizes the dataset statistics and splitting outcomes, including the total number of molecules, maximum number of heavy atoms, and the number of samples in the training, validation, and test sets for each dataset.
Table B.3: Dataset statistics after preprocessing, Hydrogen atoms are excluded in max atom count.
ZINC15
QM9
PubChem
# total molecules
198,626
132,251
444,287
# max atoms
26
50
# molecules in train set (80%)
158,900
105,800
355,429
# molecules in test set (10%)
19,862
13,225
44,428
# molecules in validate set (10%)
19,864
13,226
44,430
B.3 Surrogate Model Training and Hyper-parameters
We train surrogate models using Chemprop, graph neural network-based molecular property predictor. For each dataset, we use the available SMILES strings and their corresponding property labels (e.g., QED, SAS, or binding affinity) to train separate model. The trained surrogate model estimates not only the predicted property value but also the uncertainty associated with whether the value exceeds predefined threshold. Model hyperparameters are selected via grid search, as summarized in Table B.4 The best hyperparameter configurations selected for each dataset are reported in Table B.5
29

---

Table B.4: Hyperparameter search space for surrogate model training.
Hyperparameter
Search Range
activation
ReLU, SELU, GELU, Leaky ReL.U
hatch size
32, 64, 128, 512, 1024
dropout
0.0. 0.15, 0.1, 0.2
ensemble size
1,5, 10, 20, 30
epochs
20, 30, 40, 50,
evidential regularization
le-3, 5e-3, le-2
ffn_hidden_sse size
200, 300, 400, 600
final learning rate
1c-5, lc-4
hidden size
200, 300, 400, 600
loss function
mse, evidential, mve 2,
ffn__um layers
max learning rate
le-3, le-2
initial learning rate
le-4, le-3
Table B.5: Hyperparameters for surrogate models across datasets and prediction tasks.
Hyperparameter
QM9 SAS
ZINCIS SAS
PubChem
QED
Aflimity
QED
Allinity
QED
SAS
Affinity
activation batch size
PRcLU
PRcLU
PReLU
PReLU
PRcLU
PReLU
PReLU
PReLU
PRcLU
64
64
64
64
64
64
dropout
0.15
0
ensemble size
I
1
opochs
40
40
40
40
evidential regularization
Se-3
Se-3
Se-3
5e-3
Se-3
5e-3
le-2
Se-3
Se-3
Trn_hidden__ttte
linal learning cate
le-5
le-5
le-5
le-5
le-5
le-5
le-5
le-5
le-5
hidden size
300
300
300
300
300
loss function
evidential
evidential
evidential
evidential
evidential
evidential
evidential
evidential
evidential
#fn_aum_ layers
2
2
2
max learning rale
3e-3
3e-3
3e-3
3e-3
3e-3
3e-3
3e-3
3e-3
initial learning rate
1e-4
lc-4
1e-4
le-4
lc-4
lc-4
le-4
lc4
le4
B.4 Hyper-parameters for RL-guided Diffusion Model Training
We perform grid search over key hyperparameters for RL-guided diffusion model training, as summarized Table B.6 The final selected configurations for RL-guided diffusion models pre- trained on each dataset (QM9, ZINC15, PubChem) are reported in Table B.7
Table B.6: Hyperparameter search space for RL-guided diffusion model training.
Hyperparameter
Search Range
_inuu_
0.05, 0.2, 0.3, 0.35
mrique_bonis
0.05, 0.2, 0.3, 0.35
novel_bonue
0.05, 0.2, 0.3, 0.35
clip_param
lc-4, 2e-4, 3e-4
learning rate
le-5, 2e-5, 5e-5
reuse
1, 3, 5, 10
samples
32, 64, 128, 256
B.5 Baseline Implementation
For each baseline, we follow the original framework design proposed in the corresponding papers. To adapt our multi-objective setting these single-objective optimization frameworks, we convert the task into a binary reward formulation: a molecule receives a reward of only if all target properties exceed their predefined cutoffs, and otherwise. The cutoffs are dataset-specific and determined based on the top 10% of molecules ranked by each property, which provides a well-defined reward signal while mitigating the issue of extreme reward sparsity. For SFT-PG, we instead define the reward as the discrepancy between the property distribution of the generated molecules and an
30

---

Table B.7: Final hyperparameter settings for RL.-guided diffusion model training.
Hyperparameter
QM9
ZINC15
PubChem
valid_bonus
0.2
0.2
0.05
unique_boo bonus
0.35
0.35
0.3
novel_bonus
0.05
0.05
0.05
clip_param
3e-4
3e-4
3e-4
rl Ir reuse
le-5
le-5
le-5
3
samples
128
128
64
ideal target distribution derived from the unconditional diffusion model. We adopt hyperparameter configurations similar to the original implementations, using a clipping parameter of 0.2, a learning rate of 10-5, and generating 128 samples per policy update.
B.6 Ablation Study Design
We compare our method against three classes of widely used multi-objective optimization paradigms (scalarization-basee constraint-basee and gradient-based) to validate that the performance gains of our uncertainty-awwer reward are not simply due to reformulating the objectives. These methods represent common and well-established strategies in multi-objective tasks.
We also compare to other uncertainty -rased based methods to demonstrate that our formulation has more actionable signals than generic uncertainty sampling exploration strategies.
Lastly, we ablate key components in our reward module such as reward boosting, diversity penalty, and dynamic cutoff adjustment, which helps to understand their individual contributions and assess how each module improves molecule quality, stability, or balance across objectives.
B.7 MD simulations and ADMET property prediction
To evaluate the dynamic stability and pharmacokinetic viability of candidate molecules beyond static docking, we conducted MD simulations and ADMET property predictions as two complementary computational assays. The MD simulations were implemented using Amber'Tools for system setup and OpenMM for GPU-accelerated production runs. Prgtein-ligand complexes were constructed by first assigning GAFF atom types ligands using Antechamber, while proteins with the ff14SB force field. All complexes were solvated in a rectangular TIP3P water box with minimum À buffer each direction and neutralized using counterions. The system topology and coordinate files generated via tleap, and the entire simulation conducted under periodic boundary conditions.
parameterized
Each system underwent initial energy minimization and equilibration prior to production runs. The cquilibration protocol consisted of a 100 ps NVT heating phase and a 200 ps NPT equilibration phase, gradually increasing the temperature to 300 K using Langevin thermostat. The production simulations were run under the NPT ensemble at atm and 300 K using Langevin dynamics with a fs timestep, totaling 1,000,000 steps (corresponding PME was used for long-range electrostatics with a 10 Á cutoff, and all hydrogen-containing bonds were constrained.
total physical simulation time of 4,000 ps).
The primary readout of MD simulation was the RMSD of protein backbone atoms, calculated over time using the initial apo structure as reference. RMSD trajectories were used assess the stability and equilibration of each protein-ligand complex. All systems showed RMSD stabilization within narrow range of approximately 0.20-0.30 nm after an initial relaxation phase, confirming that equilibrium was reached prior to subsequent analysis. Complexes demonstrating such early convergence, low fluctuation, and the absence of sharp deviations were considered structurally stable. This dynamic screening step ensures that promising compounds maintain consistent binding behavior and do not induce substantial conformational drift or instability under physiologically relevant simulation conditions.
31

---

In parallel. ADMET properties were evaluated using ADMET-AL, graph neural network- based platform trained on diverse experimental datasets. While the tool generates predictions for 41 pharmacokinetic and toxicological endpoints, we focused our analysis on a curated subset of 14 key properties that are particularly relevant for early-stage drug development. These include critical parameters across all five ADMET domains, which are absorption (e.g., human intestinal absorption, Caco-2 permeability), distribution (e.g., volume of distribution, blood-brain barrier permeability), metabolism (e.g., CYP450 isoform inhibition for 3A4, 2D6, and 2C9), excretion (e.g.. renal clearance), and toxicity (e.g., hERG inhibition, Ames mutagenicity, hepatotoxicity, and LD50).
The rationale for selecting these 14 properties stems from their strong empirical association with clinical trial outcomes and their routine use in pharmaceutical pipelines as early-stage go/no-go filters. Many of the remaining 27 endpoints, such as secondary CYP isoforms or advanced carcinogemicity markers, are either redundant with the selected subset less predictive during the hit-to-lead phase. The 14-feature radar profile thus serves as concise yet informative representation of a molecule's 's drug-lik drug-likeness and safety, enabling practical compound triage without overfitting to noisy or weakly generalizable predictions.
Each compound's 's profile was evaluated both as an individual feature vector and as an integrated radar chart to identify potential liabilities. Compounds with strong absorption and distribution but poor metabolic or toxicity profiles were flagged for exclusion. Conversely. molecules with well-balanced and favorable values across all 14 categories were considered pharmacokinetically viable. This ADMET analysis complements the MD results by ensuring that structurally stable ligands also meet essential safety and efficacy criteria, thereby reinforcing their prioritization for further validation.
B.8 Evaluation
B.8.1
Evaluation Metries for Surrogate Models
We adopt a widely used evaluation framework that includes both regression accuracy and uncertainty calibration metrics to assess the performance of surrogate models trained with uncertainty prediction.
Regression Accuracy. We use the coefficient of determination (R²) to assess the accuracy of the surrogate model's predicted mean values. This metric quantifies how well the predicted outputs align with the ground-truth labels. It is defined as
E, 1- 9.) R²=1
(B.1)
E,0 7)2
where y/ and ÿ denote the ground-trui and predicted values, respectively, and ğ is the mean of the ground-truth values. An R² score of 1 indicates perfect prediction, whereas score of suggests that the model performs no better than simply predicting the mean. Higher R² values therefore reflect stronger predictive performance and tighter alignment between the model's outputs and observed data.
Uncertainty Calibration. To quantify how well predicted uncertainties reflect actual confidence, we compute the AUCE. This metric compares predicted confidence levels to the empirical frequency with which the true values fall within corresponding confidence intervals. It is defined as
AUCE= = 1
P(() - de,
(B.2)
where e [0, 1] denotes the confidence level, and P(() is the proportion of predictions whose ground-truth values fall within the predicted c-level confidence interval. Lower AUCE values indicate better calibration.
Visualization. To complement the numerical metrics, we employ two types of visualizations for model diagnostics:
compare predicted values against ground-truth targets, with predictive uncer- tainty shown as color gradients. We additionally include histograms and residual error plots to assess potential bias, variance, and distributional shifts.
Parity plots:
32

---

Calibration curves: plot empirical coverage P(c) versus nominal confidence level allowing a visual assessment of how well the predicted uncertainties align with observed outcomes.
These visualizations provide intuitive insights into both regression accuracy and uncertainty calibra- tion, and help identify systematic errors that may not be apparent from R. or AUCE alone.
B.8.2 Evaluation Metries for Diffusion Models With and Without Optimization
To assess the quality generated molecules, we evaluate them using six key metrics: validity, unique- ness, novelty, atom stability, molecular stability, and the proportion of top molecules. Let Mgen denote the of molecules generated by the model, and Minin the set of training molecules.
Validity. Validity measures the fraction of generated molecules that can be parsed into chemically valid molecular structures. Specifically, we consider molecule valid if can be successfully reconstructed from its atomic coordinates and types, converted into an RDKit molecule object, and translated into a canonical SMILES string without triggering errors during bond inference, sanitization, or stereochemistry assignment. This process includes constructing an XYZ. block from the predicted positions and atom types, inferring bonds using RDKit's s internal heuristics, and applying standard sanitization and stereochemical processing. If these steps succeed without exceptions, the molecule is considered valid. Formally, the validity score is defined as:
Km E MMgan m is valid}] 100%
Validity
(B.3)
[Maen
Uniqueness. Uniqueness denotes the proportion of valid molecules that are distinct from each other. It is computed by removing duplicates among all valid molecules based on their canonical SMILES representations. A higher uniqueness score indicates that the model is capable of generating a chemically diverse set of valid structures, rather than repeatedly producing similar or identical molecules. Conversely, low uniqueness score may suggest issues such as overfitting to the training data or mode collapse, where the model fails explore wide range of chemical space. Formally, is defined as:
Uniqueness- = (Unique(MMaide x 100%
(B.4)
M
Novelty. Novelty quantifies the proportion of unique valid molecules generated by the model that do not appear in the training set. This comparison is performed using canonical SMILES strings to ensure consistency in molecular representation. A higher novelty score indicates that the model is capable of generating novel chemical structures beyond those it has seen during training, which is desirable in de novo molecular design. In contrast, low novelty score may suggest that the model has overfit to the training data and is merely replicating known molecules. Formally, it is defined as:
[m E Unique(Mvaiaa) m E Mimin]l
Novelty
x 100%
(B.5)
(Unique(MMaaa))
VUN. VUN measures the proportion of generated molecules that are simultaneously valid, unique, and novel. It is computed as the product of Validity, Uniqueness, and Novelty. This metric reflects the overall ability of the model to generate truly de novo compounds that chemically correct, structurally diverse, and absent from the training set. A high VUN indicates that a large fraction of the generated molecules can be considered meaningful de novo candidates. In contrast, low VUN suggests that only a small portion of outputs are genuinely novel and chemically usable, limiting the model's utility in de novo molecular design. Formally,
[m E Unique( (Maiia MMrain)
VUN = Validity x Uniqueness x Novelty
x 100% (B.6)
MMgenl
Atom Stability. Atom Stability measures the proportion of atoms in valid molecules that satisfy known valency rules. For each generated molecule, atom types and their 3D coordinates are used to reconstruct local bonding environments. Each atom then evaluated based on whether its inferred number of bonds falls within its chemically allowed valence range. This metric captures whether the
33

---

model respects fundamental atom-level chemical constraints. A high atom stability score indicates that most atoms are placed in chemically reasonable configurations with proper bonding patterns. Formally, the metric is defined as:
Atom Stability Im E Myalid all atoms in m obey valeney) x 100%
(B.7)
[Maatal
Molecular Stability. Molecular Stability measures the proportion of valid molecules that form chemically coherent structures whole. A molecule is considered stable if forms a connected graph without unrealistic fragments, over-bonded atoms, or invalid substructures, Stability is assessed by reconstructing the full molecule from predicted positions and atom types, followed by heuristic bond inference and consistency checks. This metric reflects the model's ability to generate molecules that are not only locally plausible (per atom), but also structurally sound on a global level. high molecular stability score suggests that the majority of generated molecules form valid chemical graphs without structural discontinuities or violations of bonding rules. Formally, the metric is defined as:
Molecular Stability = 1'm E Moalid m is chemically stable] x 100%
(B.8)
MMaldd
Top Molecules. This metric calculates the percentage of generated molecules that simultaneously satisfy all three essential drug-relevant property constraints: QED > 0.4, SAS < 8.0, and binding affinity S -4.5 kcal/mol (QED and SAS scores are computed using RDKit, while binding affinity is estimated using AutoDock Vina). Table describes the meaning of different value ranges for QED, SAS, and binding affinity in the context of drug design. As a composite metric, it reflects the model's capacity to generate candidate compounds that are not only chemically valid but also pharmacologically meaningful and synthetically accessible. We require a minimum QED score of 0.4 to ensure baseline drug-likeness. In practice, molecules with higher QED scores are more likely to exhibit favorable physicochemical properties, including appropriate lipophilicity, molecular weight, and ceddrogen-bondingg pharmacokinetics, reduced toxicity, and enhanced bioavailability in vivo. We also apply a SAS threshold of 8.0 to filter out molecules that are prohibitively difficult to synthesize. This constraint reflects real-world development considerations, where compounds that require low-yield or complex synthetic routes are often deemed infeasible regardless of their predicted activity. Finally, we impose a binding affinity threshold of -4.5 kcal/mol to retain candidates that are thermodynamically capable of engaging the target protein with meaningful strength. For kinase targets like EGFR, effective binding at the Adenosine TriPhosphate (ATP) pocket is a prerequisite for competitive inhibition and downstream signaling blockade. Docking scores are inherently approximate. However, more negative
potential. These characteristics are typically associated with improved
Table B.8: Reference ranges and interpretability of QED, SAS, and binding affinity scores.
Property
Range
Interpretation
QED
(0.0, 0.3)
Low drug-llienesse poor pharmaceutical potential
(0.3, 0.5)
Suboptimal but potentially useful structures
(0.5, 0.7)
Typical lead-like drug candidates
(0.7, 0.9)
High drug- ming-likeness, good design quality
(0.9, 1.0)
Excellent drug-likeness (rare)
SAS
(1.0, 3.0)
Very easy to synthesize; trivial structures Easy synthesize using standard routes Moderately challenging to synthesize
13.0,500)
(5.0, 6.5)
(6.5, 8.0)
Synthesis may require complex procedures and conditions
(8.0, 10.0)
Extremely difficult or infeasible to synthesize
Binding Affinity
(-15.0.-9.0)
Very high binding affinity; near inreversible
(kcal/mol)
(-9.0,
Strong binding; ideal for inhibitors
(-7.0, -5.5)
Moderate binding; potentially active
(-5,5, .4.5)
Binding is weak; potential for initial target engagement
(-4.5, 0.0)
Very weak binding; typically inactive
34

---

binding energies often correlate with stronger interactions and, by extension, higher inhibitory potential. To avoid prematurely discarding promising compounds, particularly during the exploratory phase of molecular screening, we adopt this permissive but principled filtering strategy that balances chemical diversity with essential real-world constraints. By requiring all three properties to be satisfied simultaneously, the metric avoids overestimating molecules that perform well in only one aspect while failing others. At the same time, it ensures that slightly suboptimal values one property do not automatically disqualify molecule, long the overall profile remains within plausible range. This joint criterion enables a more holistic assessment of generated molecules and better reflects the multidimensional requirements of early-stage trug-like candidates. Formally, the Top Molecules score is defined as follows:
Top Molecules Im
Mnovel QED(m) 2 0.4 SAS(m) S 88A Affinit((m) S -4455 x x100%
Mggen
(B.9)
B.9 Devices and Computational Setup
All experiments conducted on NVIDIA A100 GPUs with 80 GB of memory. On the QM9 dataset, both EDM and GeoLDM require approximately 90 seconds per training iteration during pretraining. Due its additional modules and guidance mechanisms, GFMDiff takes around minutes per iteration. During RL-based EDM optimization, which includes molecule generation, property evaluation, and sample reuse, each optimization step takes about minutes. Generating a single molecule using EDM model requires approximately 20 seconds. On larger datasets, the training time increases accordingly. On ZINC15, one pretraining iteration of EDM takes around 7 minutes, while on PubChem takes approximately 10 minutes. During optimization based on pretrained EDM models, each optimization step takes about minutes on ZINC15 and minutes on PubChem.
35

---

C Results and Discussion
C.1 Correlation Analysis
The independence assumption in Equation 3 allows represent the multi-objective reward as a product of individual property-specific probabilities, where each probability corresponds to the likelihood that generated molecule exceeds a predefined cutoff for given property. These probabilities are estimated using uncertainty-aware predictions from Chemprop, and their product provides a smooth and differentiable reward signal suitable for RL.
In our study, we focus on three key properties: QED, SAS, and binding affinity. These properties are designed to reflect different and complementary aspects of molecule quality. Empirically, find that these properties are only weakly correlated. As the correlation matrices are shown Table C all pairwise Pearson correlation coefficients between the properties are less than 0.3 in magnitude, indicating low correlation and supporting the independence assumption in this context.
Table C.I: Correlation matrices of QED, SAS, and binding affinity for different datasets
Q.9
QED
SAS
Affinity
QED
1.00
-0.22
0.12
SAS
-0.22
1.00
0.02
Affinity
0.12
0.02
1.00
ZINC15
QED
SAS
Affinity
QED
1.00
-0.04
-0.01
SAS
-0.04
1.00
0.04
Affinity
-0,01
0,04
1.00
PubChem
QED
SAS
Affinity
QED
1.00
-0.18
0.19
SAS
-0.18
1.00
-0.06
Affinity
0.19
-0.06
1.00
Notably, as the number of properties increases (c.g., more than 5), the assumption of independence may become less reasonable due to the potential rise in inter-property correlations. In such cases, more advanced modeling strategies that explicitly consider property interactions could be beneficial. However, it is common practice in multi-objective molecular design focus small set of key properties.
C.2 The Results of Surrogate Models
We assess the performance and reliability of the trained surrogate models using parity plots (Fig C.1 and calibration curves (Fig C.2p. The parity plots demonstrate strong alignment between predicted and ground-truth values across all datasets and properties, with R² scores consistently above 0.8, indicating high regression accuracy. The accompanying histograms show that the predictions cover wide and balanced range of property values, while the residual plots below each panel reveal no major systematic bias, with residuals centered around zero. These observations confirm that the models are not only accurate but also generalizable across molecular variations. The calibration curves further show that the predicted uncertainties are well calibrated, with AUCE values below 0. in most cases, suggesting that the uncertainty estimates meaningfully reflect the actual prediction emoos.. Together, these results confirm that the surrogate models provide accurate and trustworthy property predictions, supporting their use in uncertainty- reward construction.
36

---

ZINC15
PubChem
R² 0.99
R²
0.99
R
0.95
32
12
04
02
13
1022
42
02
R
6.0
1.00
14
30
Frodictonn
50
35
2.0
2.0
10
0-2
-30
4.0
310
A
0.87
2.0
Dinding Aifintt
5.0
+0.0
15.0
4.0
30
4.0
-24
-40
100
15
1.
0
-.0
-12
Figure C.I: Parity plots comparing ground-truth property values with predictions from surro- gate models. The panels visualize results across three datasets (QM9, ZINC15, PubChem) and three properties (QED, SAS, binding affinity). Each point represents a molecule in the test set, colored by the predicted total uncertainty d The histograms indicate the distribution of predicted and true values. Scatter plots below each parity plot show residual errors. R2 reported in each plot to indicate prediction accuracy, R8 values above 0.8 are generally considered to reflect strong predictive performance.
37

---

Qu8
70ACT5
Propert derk aRth
0.8
0.0
0.5
DE0
0.4
0.4
0.2
0.2
0.2
0.0
0.2
044
0.6
0.0
1.0
00
0.4
5.6
0.8
1.0
0.0
0.2
0.4
0.6
0.0
10
0.0
0.8
0.2
0.5
0.4
022
0.2
0.0
0.2
0.6
0.6
3.0
0/3
0.4
5.6
0.8
0.0
0.4
0.6
0.8
30
1.0
Aoppetii ao
0.5
0.6
0.2
0.2
02
0.0
0.5
Figure C.2: Confidence--asee calibration curves (orange) of our surrogate models. The plots show calibration performance on the test set across three datasets (QM9, ZINC15, PubChem) and three properties (QED, SAS, binding affinity). AUCE shown in each panel, quantifies miscali- bration-lower AUCE values indicate better uncertainty calibration. The gray region denotes the deviation from ideal calibration (diagonal dashed line). AUCE values below 0.1 generally consid- ered indicative of well-calibrated uncertainty estimates.
C.3
Extended Discussion on Baseline Comparisons
Similar to many multi-objective optimization problems, our binary reward design for these baselines creates sparse and discontinuous optimization landscape. Molecules that close to meeting the cutoffs but fail in one property receive no learning signal, making it difficult for the policy to gradually improve. This effect is exacerbated challenging datasets where valid near-optimal molecules are rare in the initial stages training. As a result, during the training of some baselines, nearly all sampled molecules receive reward, leading to slow convergence and limited exploration.
SFT-PG approaches the problem from a different angle by minimizing the distributional gap between the model's generated property distributions and an ideal target distribution. While this avoids the brittleness of hard thresholds, it still suffers from indirect optimization. Without explicit instance-level reward feedback, the model may align distribution statistics while continuing to produce low-quality individual samples. This is because the training objective only penalizes aggregate discrepancies rather than enforcing quality at the individual molecule level. As result, the model can satisfy distributional targets by generating a large number of cerrage-quality samples, even if many of them fail to meet practical thresholds. This limitation is further exacerbated in settings with noisy or imperfect surrogate labels, where lack of instance-level supervision prevents the model from learning fine-grained improvements.
In contrast, our uncertainty-aware RI framework addresses these limitations through a probabilistic reward function that estimates the likelihood of each molecule satisfying individual property con-
38

---

straints. This yields several advantages. First, it provides denser and more informative gradient signals, allowing the policy to improve even for partially successful molecules. Second, it enhances sample efficiency and convergence speed by avoiding reward sparsity. Third, by incorporating both aleatoric and epistemic uncertainty, the model learns to prioritize high-confidence regions of chemical space, which is especially important under noisy or limited data.
Furthermore, we report additional baseline comparisons further validate the effectiveness of our method. Table C.2 presents results obtained using a weighted sum of objective values for the baselines, rather than binary indicators, to enable more comprehensive comparisons. Table C.3 summarizes results from various optimization methods on the QM9 dataset. including Training-Free Free Guidance (TFG) [65]. Best-of-N [66]. Sequential Monte Carlo (SMC) [7], PILOT [68], and DiftSMol [69]. Additionally. Table [.4] reports experiments on two groups of real-valued properties on the QM9 dataset, further validating the proposed method. Overall, the results show that our method consistently achieves superior performance across all settings.
Table C.2: Baseline results with a weighted-sum formulation of objectives on the QM9 dataset.
Method
Val (%) (t)
Uni (%)
Nov (%) (f)
VUN (%) (f)
ASta (%)( (7)
MSta (%) (t)
Top (%) (1)
SFT-PG
90.68 = 1.35
96.12 = 0.34
99.51 0:20
86.73 + 1.22
99.29 0.15
95.92 +04
26.28 1.37
DDPO-SF
91.02 # 1.68
96.23 + 0.41
99.39 + 0.11
87.05 # 1.01
99.35 = 0.22
95.56 + 0.67
26.32 1.18
DDPO-IS
91.23 = 1,55
95.27 =
99.33
86.33
+ 0.79
99.28
89.18 0.37
26.98 0.80
DPOK
90.56 0.4
96.91
99.62 008
87.43
99.02
95.48 0:6.4
26.02 1.90
Ours
98.17
90.90
99.63
88.90
99.87
99.17
28.33 661
Note: Each model generates 2,000 molecules per run. Results are averaged over three independent runs and reported mean 95% confidence interval. Higher values (hold) indicate betler performance.
Table C.3:
More baseline comparisons on QM9 dataset.
Method
Val (%) (f)
Uni (f)
Nov (1)
VUN (%) (1)
ASta (%) (1)
MSta (%) (1)
TTpp (%) (1)
TFG
91.02 0.31
97.10 0.22
99.70 1L10
88.12 0
99.08 0
95.67 00600
25.41 +
Best-of-N
94.53 0.36
93.02
0.17
99.41
0:34
87.41 0:3
99.12 1016
96.56 2.52
26.62 0.8
SMC
95.11 = 0.23
92.34
0.76
99.69
: 0.52
87.55
1 0.46
99.47 1:112
97.51 0:22
26.95 1:08
PILOT
95.20 1 0.23
91.92
1 0.61
99.61
0:38
87.17 0.433
99.13 0.077
97.14 0.35
27.01 1.772
DifiMMol
94.12 0.28
92.88
a 0.31
99.59
0.16
87.10
99.01 0.9
95.97
=
26.51 0.74
Ours
98.17 0.07
90.90
0.72
99.63
0.04
88.90
99.87 0.03
99.17
28.33 0.61
Note: Fach model generates 2,000 molecules per run. Results are avernged over three independent runs and reported as mean 95% confidence interval. Higher values (bold) indieate better performance.
Table C.4: Results on two groups of real-valued properties on the QM9 dataset.
Group
Method
(f)
Uni (1)
Nov (5) (f)
VUN (t)
ASta (f)
MSta (1)
Topp (%)
Group
W/O RL
89.78 = 0.73
97.98 0.36
99.56 0:445
87.58 = 0.75 90.59 +
99.06 0.3
86.02 0.32
32.07 #
Ours
95.96 #
94.80 0.23
99.58 0.52
99.35 000
92.14 0.37
39.84
Group
W/O RI.
87.89
95.32
99.39 1007
83.27
99.01
86.23
066
29.41
Ours
94.47 = 6.19
95.16 = 0.3
99.44 0.16
89.39 =: 0.57
99.52 0.43
92.26 0.49
36.74 = L.1
Note: Each model generates 2, 2,000 molecules per run. Results are avenaeed over three independent runs and reported as mean + 95% confidence interval. Higher values (bold) indicate better performance. Group I: HOMO-LUMO gap. HOMO energy, and LUMO energy. Group 2: Dipole moment (JO), Hest eapacity (C.), Isotropic polarizability (cx).
C.4 Extended Discussion on Ablation Studies
To assess the contribution of each component in our uncertainty-aware RI. framework, we conduct ablation studies that isolate their individual effects. Results show that uncertainty-guided reward scaling is essential for stability and reliability. It prevents the model from overfitting to spurious high scores and ensures consistent improvement across multiple objectives. The diversity penalty further encourages structural exploration by discouraging repeated generation of similar molecules, leading to broader search and a higher yield of top- por-erfognnnng candidates. Dynamic thresholding allows the model to adaptively respond to varying property distributions, avoiding premature convergence caused by rigid cutoffs. Lastly, the reward bonus effectively sharpens selection pressure toward high-quality molecules, enhancing validity while preserving uniqueness and novelty. Each component contributes to a different dimension of generation, and their combination yields robust and pharmaceutically meaningful outputs.
39

---

C,5 Free Energy Perturbation-based Affinity Prediction
To complement the docking results with a more rigorous assessment of strength, free energy perturbation calculations were applied to estimate the binding free energy (AGrina) of the four selected compounds.
binding
Using exponential averaging based on the Zwanzig equation, we computed binding free energy as the difference between the ligand's 's free energy in the bound (complex) and unbound (ligand-only) coannd-olyy..oo All simulations were run in OpenMM.
As shown in the Table C.5 the predicted binding free energy values ranged from -4.14 to -4.29 kcal-mol-1, corresponding to low-micromolar binding affinities. Importantly, the three designed compounds exhibited binding affinities that are comparable to modestly better than the known EGFR inhibitor. While not dramatically superior, these results lend additional support to the docking- based predictions and suggest that the newly generated candidates have acceptable and potentially favorable binding characteristics.
Table C.5: Free energy perturbation-based affinity prediction.
Compound
Comple (kcal/mol)
AGigann (kcal/mol)
AGrind (kcal/mol)
Predicted K2 (μM)
Known inhibitor
-19.5
-15.4
-4.14
112 2 JM
Molecule I
-20.3
-16.0
-4.29
~0.89 μM
Molecule
-20.0
-15.7
-4.26
~=0 JM
Molecule III
-20.2
-15.9
-4.28
~00 μM
Molecule the generated molecule shown Fig. 44 the main Molecule the generated molecule shown of the main text.
Fig. 47 of the main text. Molecule ITI: the generated molecule shown Fig. 4
C.6 Extended Discussion on MD and ADMET Prediction
While the main text highlights the general trend of MD and ADMET results for generated molecules, several specific observations warrant further discussion. In the MD simulations, all three top-ranked candidates formed stable complexes with EGFR over the full 10,000 ps production trajectories. One candidate (Fig. 4c in main manuscript) exhibited excellent structural stability, maintaining low RMSD relative to the equilibrated apo conformation, indicative of strong and persistent binding. Another (Fig. 4t in main manuscript) showed minimal conformational fluctuations across the simulation and preserved critical hydrogen bonds within the ATP-binding site, reinforcing its potential as a potent inhibitor.
Beyond structural dynamics, in silico pharmacokinetic profiling of these candidates using ADMET-AI revealed consistently favorable drug-like properties. All three molecules exhibited high predicted human intestinal absorption, low blood-brain barrier permeability,, and no violations of Lipinski's ss rule. Importantly, one of the molecule (Fig. in main manuscript) showed low predicted hepatotoxicity and high metabolic stability, while another (Fig. 4b in main manuscript) scored favorably across P450 inhibition panels, suggesting low potential for metabolic drug-drug interactions.
These extended findings further support that top molecules generated by our framework are not only structurally valid and property-optimizer but also promising terms downstream pharmacological viability. Incorporating such analyses into gest-generation filtering can substantially improve the relevance of Al-designed molecules for real-world drug discovery workflows.
C.7 Visualization for Generated Molecules
We visualize representative candidate molecules generated by RL-guided diffusion models pre- pre-trained on different datasets in Fig. C.3 For optimized model, we rank all generated molecules based on their multi-objective uncertainty scores. The top eight molecules with the highest scores from each model are shown. This selection strategy ensures that the displayed candidates are not only high-quality in terms of predicted properties, but also robust with respect the model's uncertainty.
40

---

Figure C.3: Representative candidate molecules generated by RL-guided diffusion models. Each model was pre-trained on one of the three datasets (QM9, ZINC15, PubChem) and generated 2,000 molecules. The top molecules selected by composite properties are shown: first row: QM9, second row: ZINC15, third row: PubChem.
C.8 Analysis Other Diffusion Models
Table C.jjsummarizes the performance of our RL-guided optimization framework across other two diffusion models (GeoLDM and GFMDiff) the QM9 dataset. Our method consistently improves the key evaluation metrics over the non-RL baselines, including validity, novelty and top molecules. This demonstrate that our method generalizes well across different diffusion models.
Table C.6: Performance of other diffusion models and method QM9 datasets.
Model
Method
Val (t)
Uni (1)
Nov (t)
VUN (% (f)
ASta (T)
MSta (%) (t)
Top (%) (f)
GeoLDM
W/O RL
91.98 o.T
95.13 004
99.71 nmn
87.25 0.5
99.08 2016
95.65 25.67 0.00
85.40 = oo
Ours
95.11
94.12
99.73
99.12
95.54
87.08
GFMDiff
W/O RL
94.33 6:17
96.34 M41
96.11
87.35 1.72
99.75 = 008
98.23
008
74,50
Ours
96.72 = 1.27
95.93 = 084
96.15 021
89.21 1.39
99.77 = 0.10
98.35
0.56
75.01 # 001
95% confidence interval. "WIO RL. denotes vanilla diffusson models without RL. "Val". "Uni". and "Nov" represent the percentages of valid, unique, and novel of molecules that are simultaneously
Each model generates
molccules per
averaged over independent and reported
molecules, respectivelly. "VUN" their joint metric computed as Val x Uni Nov, representing the alonr-leve molccule-levd stability. "Top" indicates the proportion generated mokocules that simullanevasly sutisfy three property comstraints, using relaxed cutoffs (QED 0.4, SAS < 8, and binding affinity < -4.5) 5) to avoid missing potentially useful candidates. All merrics are reported as percentages, and higher values indicate hetter performance.
unique, and
41


Reviews:
[{"review_id": "VsN7YlB5Hn", "reviewer": "Reviewer_Phx1", "summary": "The paper presents a methodologically sound pipeline, which systematically integrates three key stages: reward design, data sampling, and optimization using Proximal Policy Optimization (PPO). The authors conduct their analysis within a rigorous experimental setting, ensuring a fair and direct comparison against established baselines.", "questions": "1. How is uncertainty computed?\n\n2. Is this method applicable to other architectures, such as GeoLDM?\n\n3. How does the RL-based method compare to training-free guidance [1]?\n\n[1] https://arxiv.org/abs/2209.15408", "limitations": "Please check weakness.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe paper presents a logically sound and well-structured methodological pipeline.\n\nThe experimental evaluation is rigorous, employing fair comparisons against relevant baselines.\n\nA key contribution is the novel integration of an uncertainty metric directly into the reward function, which is a compelling idea.\n\nWeaknesses\nThe primary weakness is the limited methodological novelty. The proposed framework is largely an assembly of well-established components, which may not meet the innovation threshold for this venue.\n\nThe choice to demonstrate the method on an explicit-density model (EDM) instead of a more challenging and widely-used latent diffusion model (like GeoLDM) limits the perceived impact. Applying guidance to latent models is a more difficult and generalizable problem, and the current choice may be seen as targeting a simpler use case. Demonstrating success on a latent model would significantly strengthen the contribution.", "was_revised": true, "final_justification": "Since the method also performs well on GeoLDM, I am willing to raise my score.", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback. We are encouraged that the reviewer finds our pipeline to be logically sound and well-structured, and appreciates the rigorous experimental evaluation and fair comparisons against relevant baselines. We are particularly pleased that the integration of an uncertainty metric into the reward function is seen as a compelling and valuable contribution.\n\nThe reviewer’s main concerns center on: 1. the novelty; 2. the choice of model architecture (EDM vs. GeoLDM); 3. the uncertainty computation; and 4. the comparison to training-free guidance methods.\n\nBelow, we address each of these concerns in detail and provide clarifications and evidence.\n\n**1. The novelty**\n\n1)To the best of our knowledge, at the time of submission, **our work is the first to propose an end-to-end framework that integrates Reinforcement Learning (RL), diffusion models, 3D molecular generation, and uncertainty-based multi-objective optimization.** This combination has not previously been explored in the molecular generation literature. In other application domains, RL has been used to optimize diffusion models, primarily for single-objective tasks such as improving aesthetic scores in image generation. Representative examples include DPOK: RL for Fine-tuning Text-to-Image Diffusion Models and DDPO: Training Diffusion Models with RL, which apply RL to guide diffusion-based image synthesis. Although these works are also not the first to combine RL with diffusion models, they have demonstrated strong performance and clear practical value in the context of image generation and were accepted at NeurIPS and ICLR main tracks.\n\n2)In terms of technical innovation, beyond the novel integration of uncertainty quantification into the reward function to address multi-objective optimization, **we also tackle several challenges faced by RL in the context of optimizing diffusion models in our task**. These challenges include mode collapse, premature convergence, and stagnation during the learning process. To address these issues, we design three auxiliary mechanisms in our reward function:\n\na.**Reward boosting mechanism**. This mechanism integrates additional constraints such as molecular validity, uniqueness, and novelty into the training objective in the form of soft rewards. This enhances the flexibility of the reward structure and explicitly encourages the model to generate chemically feasible molecules while optimizing their desired properties. Besides, it strengthens the reward signal during the early stages of training, allowing the policy to quickly identify viable regions of the chemical space and avoid getting stuck in the sparse-reward phase. Further implementation details were provided in **Appendix A.3**.\n\nb.**Diversity penalty mechanism**. The diversity penalty encourages structural exploration by penalizing overly similar molecules within a batch. This drives the policy to explore new regions of the chemical space and prevents premature convergence to narrow, high-reward clusters. From an RL perspective, this mechanism functions similarly to entropy regularization by promoting broader policy support and mitigating mode collapse.\n\nc. **Dynamic Cutoff Strategy**. When converting uncertain property predictions into probabilistic improvement signals, it is necessary to define a cutoff threshold for each property. In existing studies, these thresholds are typically set as static values. However, such static thresholds can quickly become misaligned with the evolving distribution of generated molecules during training, leading to reward saturation or sparsity. These issues may undermine training stability or result in premature convergence. To ensure that the reward signal remains both discriminative and informative throughout the training process, we propose a dynamic cutoff mechanism. This method adaptively computes property-specific cutoff values based on the shifting distribution of predicted values in each training segment. Specifically, we determine the quantile used for cutoff calculation using cosine interpolation between 50% and 10%, depending on the current stage of training. This quantile is then applied to the property distribution of newly generated molecules to compute updated cutoffs (top x% values). These adaptive cutoffs are subsequently used when evaluating the probabilistic improvement of generated molecules.\n\nThe contribution of each module can be referenced from the ablation study presented in **Table 2** of the manuscript. The results show that our RL-optimized model consistently improves performance across multiple metrics and datasets. Notably, on the ZINC15 dataset, we observe substantial gains in key evaluation criteria: the validity of generated molecules increases by 70%, and the proportion satisfying all property constraints rises by 25%.\n\nWe thank the reviewer for pointing this out and will incorporate the corresponding updates in the camera-ready version.\n\n3)**Our work closely follows the Guidelines for Writing a Good NeurIPS Paper in relation to the application theme**: \n\na.The guidelines state: “Application papers should describe your work on a ‘real’ as opposed to ‘hypothetical’ application; specifically, it should describe work that has direct relevance to, and addresses the full complexity of, solving a non-trivial problem.” Our work directly targets a clinically relevant and complex challenge in drug design. Specifically, we propose an AI-based framework for generating novel therapeutic molecules that simultaneously optimize multiple critical molecular properties, including drug-likeness, synthetic accessibility, and binding affinity to a target protein. As a result, we successfully discovered novel candidate molecules with performance comparable to known EGFR inhibitors. This work is part of a larger interdisciplinary collaboration. Currently, our partner chemists are synthesizing and experimentally validating the generated molecules to assess their real-world therapeutic potential.\n\nb. The guidelines also emphasize that “Authors are encouraged to convey insight about the problem, algorithms, and/or application,” and that “Techniques shown to be uniquely fitted to specific popular applications, leading to improved performance or more accurate solutions” are particularly valued. As discussed in our points 1) and 2), our method was carefully designed to match the specific challenges of 3D molecular generation and multi-objective drug discovery, satisfying these criteria.\n\nc. Finally, the guidelines state that “A NeurIPS application paper should be comparable in quality to papers in the corresponding application domain conference.” We believe our work meets this bar. Beyond the algorithmic contributions, we also developed a user-friendly interface to broaden accessibility. Users without programming experience can upload a target receptor structure and easily obtain candidate therapeutic molecules, as detailed in **Appendix D**.\n\n\n**2. The choice of model architecture (EDM vs. GeoLDM)**\n\nWe chose to demonstrate our framework on the EDM because it is a seminal work in applying diffusion models to 3D molecular generation. Subsequent research efforts, such as GeoLDM, have been built upon this framework. Therefore, we consider EDM to be a robust and highly representative baseline, which makes it a natural and appropriate starting point for our experimental design. \n\nTo further demonstrate the broad applicability of our framework, **we have also evaluated it on other more advanced diffusion models, including GeoLDM**. The corresponding results were reported in **Appendix C.6**, and for convenience, we include the relevant excerpt below. The results indicate our framework can be effectively applied across different 3D molecular diffusion models.\n\n**Table 1**\n\n| Model   | Method   | Val (%) ↑     | Uni (%) ↑     | Nov (%) ↑     | VUN (%) ↑     | ASta (%) ↑    | MSta (%) ↑    | Top (%) ↑     |\n|---------|----------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|\n| GeoLDM  | W/O RL   | 91.98 ± 0.17  | 95.13 ± 0.64  | 99.71 ± 0.07  | 87.25 ± 0.52  | 99.08 ± 0.16  | 95.67 ± 0.60  | 85.40 ± 0.00  |\n|         | **Ours** | 95.11 ± 1.24  | 94.12 ± 0.31  | 99.73 ± 0.13  | 89.28 ± 1.57  | 99.12 ± 0.17  | 95.54 ± 0.49  | 87.08 ± 0.02  |\n| GFMDiff | W/O RL   | 94.33 ± 0.17  | 96.34 ± 0.81  | 96.11 ± 1.03  | 87.35 ± 1.72  | 99.75 ± 0.03  | 98.23 ± 0.03  | 74.50 ± 0.00  |\n|         | **Ours** | 96.72 ± 1.27  | 95.93 ± 0.94  | 96.15 ± 0.21  | 89.21 ± 1.39  | 99.77 ± 0.10  | 98.35 ± 0.56  | 75.01 ± 0.01  |\n\n\n**3. The uncertainty computation**\n\nThe uncertainty is computed by combining aleatoric and epistemic components to estimate the total prediction variance. Based on this, the probability that a molecule’s property exceeds a target threshold is estimated using a Gaussian assumption. In multi-objective settings, the overall uncertainty is defined as the product of individual threshold probabilities across properties, assuming independence. This yields a unified reward signal reflecting the likelihood of satisfying all objectives simultaneously. Details were provided in **Appendix A.2**.\n\n**4. The comparison to training-free guidance methods**\n\nWe established a Training-Free Guidance (TFG) baseline. The results are summarized in the table below and demonstrate that our method achieves better performance.\n\n**Table 2**\n\n| Method   | Val (%) ↑     | Uni (%) ↑     | Nov (%) ↑     | VUN (%) ↑     | ASta (%) ↑    | MSta (%) ↑    | Top (%) ↑     |\n|----------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|\n| TFG      | 91.02 ± 0.31  | 97.10 ± 0.22  | 99.70 ± 0.10  | 88.12 ± 0.37  | 99.08 ± 0.16  | 95.67 ± 0.60  | 25.41 ± 0.87  |\n| **Ours** | 98.17 ± 0.07  | 90.90 ± 0.72  | 99.63 ± 0.04  | 88.90 ± 0.68  | 99.87 ± 0.03  | 99.17 ± 0.27  | 28.33 ± 0.61  |"}, {"review_id": "mIDSn6CPHZ", "reviewer": "Reviewer_cfze", "summary": "In this work RL is incorporated in the training of guided-diffusion models to facilitate generation of 3D molecules satisfying \nmultiple properties, waiving the need to have differentiable surrogate models of the properties of interest. A conditional diffusion model is first trained on the molecular dataset then further optimized using RL with an uncertainty-based multi-objective reward to encourage the generation of molecules satisfying multiple constraints.", "questions": "1) Clarify task directions in line 133. Does it refer to minimization or maximization of a property?\n\n2) In the forward process there is conditioning on C in Figure 1 caption, is this needed? This does not match the description of forward process in Appendix A.1.\n\n3) In RL-guided training, it is explained that property combinations satisfying predefined cutoffs (QED > 0.4, SAS < 8, binding affinity < -4.5) are used as conditions.\nIn generation, how the combinations of property values (QED, SAS, Binding affinity) are chosen for conditioning? \n\n4) Does RL-guidance help only with better conditional generation or it is helping with property optimization. i.e., maximizing/minimizing properties? If the later is not the case, how can your method be altered for this purpose?\n\n5) Based on Figure 3 (WO/RL vs W/RL), for ZINC15 dataset, my guess is that the failure of other methods, in satisfying all three properties (TOP % Metric), lies in failing the QED property which is also reflected in the validity metric (Table 1). How is ZINC15 dataset different from other datasets that could justify this significant difference in performance?\n\n6) (Out of curiosity) How does the RL-guided optimization after pretraining on each dataset, affects the quality of conditional generation for samples used for pretraining?", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper has nice narrative and experiments to demonstrate the capabilities of the proposed model.", "was_revised": true, "final_justification": "All my questions were addressed by the authors.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback. We are glad that the reviewer found our paper technically sound, with a clear presentation and well-designed experiments. We also appreciate the recognition of the method’s quality, originality, and significance, as well as the acknowledgment that our experimental results effectively demonstrate the model’s capabilities.\n\nThe reviewer’s concerns mainly center around clarifications related to task directions, the conditioning scheme in the forward process, the selection of property combinations during generation, and the role of RL in conditional generation versus property optimization. We thank the reviewer for raising these important points, and we address each concern in detail below.\n\n\n**1. Clarify task directions in line 133. Does it refer to minimization or maximization of a property?**\n\n**Yes**. Our experiments consider three key properties: Quantitative Estimate of Drug-likeness (QED), Synthetic Accessibility Score (SAS), and binding affinity. In our task, the objective is for all three properties of a molecule to simultaneously exceed predefined cutoff values, rather than optimizing a single property in isolation. Once a molecule satisfies all three cutoffs, we prefer QED to be as high as possible (indicating better drug-likeness), SAS to be as low as possible (indicating easier synthesis in laboratory conditions), and binding affinity to be as low as possible (indicating stronger binding to the target protein).\n\nAccordingly, in line 133, the task direction for QED is set to \"maximize\", while the directions for SAS and binding affinity are set to \"minimize\".\n\n**2. In the forward process there is conditioning on C in Figure 1 caption, is this needed? This does not match the description of forward process in Appendix A.1.**\n\nThank you for pointing this out. This is a typo in the figure. **The forward process in Figure 1 does not require conditioning**. We will correct this in the camera-ready version.\n\n**3. In RL-guided training, it is explained that property combinations satisfying predefined cutoffs (QED > 0.4, SAS < 8, binding affinity < -4.5) are used as conditions. In generation, how the combinations of property values (QED, SAS, Binding affinity) are chosen for conditioning?**\n\nIn generation, we need to input the combination of multiple properties into the model to guide molecule generation. Since we consider multiple properties simultaneously, there are infinitely many possible combinations. To avoid feeding unreasonable or invalid property combinations into the model, **we first filter the test set using predefined cutoff values to obtain reasonable property combinations. We then sample from these filtered combinations to serve as conditional inputs for generation**.\n\n**4. Does RL-guidance help only with better conditional generation or it is helping with property optimization. i.e., maximizing/minimizing properties? If the later is not the case, how can your method be altered for this purpose?**\n\nThank you for this insightful question. Our RL-guided training primarily improves conditional generation by encouraging the model to generate molecules that satisfy multiple property constraints simultaneously. We do not explicitly formulate the task as direct maximization or minimization of individual properties.\n\nThis design choice is motivated by the fact that the properties (QED, SAS, binding affinity) we consider are often conflicting. For example, aggressively minimizing binding affinity (which is desirable from a binding perspective) may lead to complex or unrealistic molecules with poor synthetic accessibility or low drug-likeness. Optimizing a single property in isolation can therefore harm the overall quality and viability of the generated molecules.\n\nTo avoid such trade-offs, our reward is defined as the probability that a generated molecule satisfies all predefined property cutoffs. These probabilities are estimated using uncertainty-aware predictions from surrogate models, resulting in a scalar reward value in the [0, 1] range. This approach enables us to softly integrate multi-objective constraints without explicitly requiring the maximization/minimization of any one property at the expense of others.\n\nThat said, if one wishes to adapt our method for pure property optimization (e.g., maximizing binding affinity), the reward function can be modified accordingly. For instance, one could define the reward as a weighted combination of raw property scores or use property-specific gradients where feasible. However, this would require careful consideration of property conflicts and trade-offs.\n\n**5. Based on Figure 3 (WO/RL vs W/RL), for ZINC15 dataset, my guess is that the failure of other methods, in satisfying all three properties (TOP % Metric), lies in failing the QED property which is also reflected in the validity metric (Table 1). How is ZINC15 dataset different from other datasets that could justify this significant difference in performance?**\n\nThe ZINC15 dataset differs from other datasets in two key aspects:\n\n1)**ZINC15 is a drug-like molecular dataset**, in which molecules generally possess favorable pharmacological properties. As a result, it provides training samples that are inherently closer to the target distribution of the optimization task, making it easier for the model to generate molecules that meet the desired property constraints.\n\n2)**Molecules in ZINC15 typically contain a moderate number of heavy atoms** and tend to exhibit less complex topological structures and conformational variability. This makes it easier for the model to preserve structural validity during both the noise perturbation and reverse generation processes.\n\nIn contrast, QM9 is not a drug-oriented dataset, and many of its molecules lack desirable pharmacological characteristics. This increases the distributional shift and structural adaptation challenges the model must overcome during learning and optimization. As for PubChem, although its molecules often exhibit good drug-like properties, they frequently contain a large number of heavy atoms, complex functional groups, and intricate ring systems. These features result in a combinatorially large structural space, making the reverse diffusion process more prone to accumulating minor prediction errors, which can ultimately lead to structural collapse.\n\n**6. (Out of curiosity) How does the RL-guided optimization after pretraining on each dataset, affects the quality of conditional generation for samples used for pretraining?**\n\nThank you for this thoughtful question. After pretraining, we further optimize the model using Reinforcement Learning (RL), where the conditional inputs are high-quality property combinations sampled from the test set. During both RL optimization and evaluation, the same set of high-quality combinations is used as conditioning input. Before RL, the model’s performance under these conditions was relatively limited. **With RL optimization, the model significantly improves its conditional generation quality**. For example, we observe higher validity and a greater proportion of top molecules. This indicates that RL effectively enhances the model’s ability to generate molecules under conditions related to the pretraining distribution. It also highlights a gap between exposure during pretraining and actual controllability under target conditions, which RL helps to close by directly optimizing generation performance under those conditions.\n\nThank you again for your question and for your recognition of our work. Please feel free to share any further questions or suggestions you may have."}, {"review_id": "iXLsDxPw3C", "reviewer": "Reviewer_qJWH", "summary": "The paper proposes an uncertainty-aware RL framework to post-train a molecular diffusion model towards multiple objectives. A surrogate model predicts uncertainty estimation, which is used to reweight reward functions to keep objectives balanced. Experiments on three molecule generation benchmarks as well as MD simulations and ADMET profiling tasks validates the effectiveness of the framework.", "questions": "Please refer to the weaknesses.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 1, "strengths_and_weaknesses": "Strengths: the framework introduces predictive uncertainty into multi-objective RL framework.\n\n\n\nWeakness: \n\n1. Although there are many terms and components in the title, the technical details of most components are not novel, making the title and the paper confusing.\n\n   * Diffusion Models for 3D De Novo Molecular Design: simply a conditional EDM, which has been deployed by hundreds of papers.\n   * Uncertainty-Aware: the predictive uncertainty is obtained by an integration with threshold of predicted mean and variance via D-MPNN, which is simple, black-box and not strict. There are more established methods in uncertainty quantification, such as [1]. \n   * Multi-Objective: the aggregation assumes independency and simply multiplies the single-objective rewards. Again, there are tons of papers in multi-objective optimization.\n   * Reinforcement Learning: seems like the major contribution of the paper. The policy update method resembles PPO, so the only novel part is the reward design. What is the real difference between proposed method and baselines?\n\n2. The comparison between proposed method and baselines is unclear. There are many methods to handle multi-objective RL, but the authors ignore them and the baselines are treated in a single-objective manner with a ridiculous binary reward, which is clearly unfair. Also, please include inference-time reward-guided sampling methods as alternative baselines, such as Sequential Monte Carlo (SMC) and Best-of-N.\n\n3. For experiments, QM9 is a dataset with internal 19 types of molecule properties, including some chemical and thermodynamic quantities. I strongly encourage to include this task (predicting multiple real-valued molecule properties) and compare with conditional molecule generation baselines mentioned in GeoLDM [2].\n   \n\n   \n[1] Huang, K., Jin, Y., Candes, E., & Leskovec, J. (2023). Uncertainty quantification over graph with conformalized graph neural networks. *Advances in Neural Information Processing Systems*, *36*, 26699-26721.\n\n[2] Xu, M., Powers, A. S., Dror, R. O., Ermon, S., & Leskovec, J. (2023, July). Geometric latent diffusion models for 3d molecule generation. In *International Conference on Machine Learning* (pp. 38592-38610). PMLR.", "was_revised": true, "final_justification": "Based on the rebuttal responses and discussions, I raised my score from 2 to 3. While the authors resolved part of my concerns, I'm still concerning about the novelty and the mixed performance among metrics.", "rebuttal": "We appreciate the reviewer’s recognition of our contribution, including the integration of uncertainty into multi-objective RL and comprehensive validation across multiple benchmarks, MD simulations, and ADMET analysis.\n\nThe concerns raised by the reviewer mainly stem from: the novelty, the baseline setup, and the experimental design on the QM9 dataset. Below, we respond to these concerns in detail.\n\n**1. Although there are many ...**\n\nWhile some components have been explored separately, our main contribution is their integration into a unified, end-to-end framework for 3D therapeutic molecule design. Besides, we address some common challenges in using RL to optimize diffusion models in our task.\n\n**Diffusion Models for 3D De Novo ...**\n\nWe chose to build our framework on EDM because **it is a seminal work in applying diffusion models to 3D molecular generation**. Based on our prior extensive literature review, we found that only around 10 papers have extended EDM, most of which were published at top-tier venues such as NeurIPS and ICLR. This demonstrates that EDM is a well-recognized and foundational framework, making it a suitable and credible starting point for our work.\n\nFurthermore, most existing works based on EDM focus on improving generation quality or conditioning only on the target protein. In contrast, **our study is distinct in its goal of designing therapeutic molecules that satisfy multiple drug-relevant properties simultaneously**. This is a **practically motivated** setting that aims to produce candidate compounds suitable for synthesis and experimental testing by our collaborating chemists.\n\nWe also acknowledge the reviewer’s mention of alternative diffusion backbones, such as GeoLDM. To demonstrate the generality of our method, **we additionally evaluated our framework on other advanced diffusion models, including GeoLDM and GFMDiff**, as detailed in **Appendix C.6**. \n\n\n**Uncertainty-Aware: the predictive ...**\n\nWe appreciate the reviewer’s suggestion regarding ConFormalized (CF)-GNN. While it may offer theoretical advantages, CF-GNN has practical limitations for RL, including slow inference due to calibration set access and unstable reward signals caused by threshold effects. Given these factors, **CF-GNN-based methods are not well suited for RL deployment**.\n\nWhile we acknowledge the reviewer’s concerns regarding the use of D-MPNN, we would like to emphasize that **we conducted a rigorous evaluation of our surrogate models to ensure their reliability**. As shown in **Appendix C.1**, we provided detailed assessments including parity plots, calibration curves, and quantitative metrics such as R² and AUCE. All surrogate models achieved high R² scores (> 0.8), indicating strong predictive accuracy, and low Area Under the Calibration Error (AUCE) values (< 0.05), demonstrating well-calibrated uncertainty estimates. **These results confirm that our D-MPNN-based models are both accurate and reliable**.\n\nIn this context, the D-MPNN-based approach compensates well for any theoretical limitations through strong empirical performance, making it a more practical and stable choice for RL deployment.\n\n\n**Multi-Objective: the aggregation ...**\n\nWe acknowledge the reviewer’s concern regarding the independence assumption in our reward aggregation. To support this design choice, we computed the Pearson correlation coefficients between all pairs of properties and found that all coefficients are below 0.3. **This indicates weak correlations among the properties and justifies the assumption of independence in our experimental setup**.\n\nBy aggregating the multi-objective reward as the product of individual property-specific rewards, we simplify the reward formulation while preserving a smooth and interpretable optimization landscape. **This design aligns with the goal of enabling efficient and stable policy learning in the RL framework**. Empirically, this approach has proven to be highly effective, as demonstrated by our results.\n\nWe agree that multi-objective optimization is a well-studied area. To ensure a fair and comprehensive comparison, **we included 16 commonly used multi-objective optimization methods** in our ablation study. Our method consistently outperformed these other optimization strategies, further validating the effectiveness of our approach.\n\n\n**Reinforcement Learning: seems ...**\n\nWe thank the reviewer for recognizing the contributions of our work, particularly in RL and reward design. To the best of our knowledge, at the time of submission, **our work is the first to present an end-to-end framework that integrates RL, diffusion models, 3D molecular generation, and uncertainty-based multi-objective optimization**. This design has not been explored in prior molecular generation literature. As mentioned earlier, our framework addresses a real-world application in designing novel EGFR inhibitors as alternatives to known inhibitors, which aligns with the NeurIPS application theme.\n\nIn terms of technical innovation, beyond incorporating uncertainty into the reward function, we addresses several challenges in using RL to diffusion models in our tasks, including mode collapse, premature convergence, and sparse or unstable reward signals. To this end, we introduce three auxiliary mechanisms:\n\n**a. Reward Boosting Mechanism**: Designed to mitigate sparse reward signals in the early stages of training, this mechanism incorporates soft auxiliary rewards based on molecular validity, uniqueness, and novelty. It enables the policy to quickly identify viable regions in chemical space, accelerates learning, and improves sample efficiency.\n\n**b. Diversity Penalty Mechanism**: Aims to prevent mode collapse and premature convergence by penalizing structural redundancy within a batch of generated molecules. This mechanism encourages exploration of diverse chemical structures, helping the model avoid overfitting to narrow high-reward regions and improving coverage of the chemical space.\n\n**c. Dynamic Cutoff Strategy**: Addresses reward saturation or instability caused by static thresholds during training. By adaptively updating the property-specific cutoffs based on the evolving distribution of generated molecules, this mechanism ensures that the reward signal remains discriminative and informative throughout training. As a result, it supports stable learning dynamics and better alignment with multi-objective constraints.\n\nThe contribution of each module is validated in our ablation study (see **Table 2** in the main paper).\n\n\n**2. The comparison between proposed ...** \n\nOur task focuses on RL-guided optimization during the training phase of diffusion models. In this context, and following established practices in the field, we adopt gradient-based policy optimization methods such as PPO. These methods are generally preferred for direct integration with neural networks in an end-to-end fashion, as they allow for efficient backpropagation, unlike many other RL methods. Among gradient-based RL methods, the most common strategies for handling multiple objectives include weighted sum and Pareto-based methods such as PCGrad. As mentioned earlier, we have already included these strategies in our ablation analysis.\n\nWe acknowledge that the use of binary rewards in baseline designs may introduce challenges in RL training, such as sparse early rewards and lack of feedback for near-miss candidates. However, this design aligns with the practical goal of therapeutic molecule design, where only molecules that meet all required property thresholds are considered viable. In real-world drug discovery pipelines, it is common to retain only compounds that satisfy all desired criteria. Therefore, **while a binary reward may not be ideal from an RL optimization perspective, it is a principled and task-aligned formulation rather than an unfair or incorrect one**.\n\nWe also agree that it is difficult to identify prior works that are directly comparable in this our setting. Nonetheless, we appreciate the reviewer’s suggestion and have Best-of-N and SMC as baselines. The table below show that our method outperforms both approaches, supporting its effectiveness.\n\n| Method     | Val (%) ↑     | Uni (%) ↑     | Nov (%) ↑     | VUN (%) ↑     | ASta (%) ↑    | MSta (%) ↑    | Top (%) ↑     |\n|------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|\n| Best-of-N  | 94.53 ± 0.36  | 93.02 ± 0.17  | 99.41 ± 0.24  | 87.41 ± 0.38  | 99.12 ± 0.16  | 96.56 ± 0.52  | 26.62 ± 0.85  |\n| SMC        | 95.11 ± 0.23  | 92.34 ± 0.76  | 99.69 ± 0.52  | 87.55 ± 0.46  | 99.47 ± 0.12  | 97.51 ± 0.27  | 26.95 ± 0.83  |\n| **Ours** | 98.17 ± 0.07  | 90.90 ± 0.72  | 99.63 ± 0.04  | 88.90 ± 0.68  | 99.87 ± 0.03  | 99.17 ± 0.27  | 28.33 ± 0.61  |\n\n\n**3. For experiments, QM9 is a dataset ...**\n\nUnder the application theme, our work is focused on designing novel molecules that can serve as alternatives to known EGFR inhibitors. In this context, we prioritized the properties of binding affinity, synthetic accessibility, and drug-likeness. **In contrast, many of the typical chemical and thermodynamic properties included in QM9, such as HOMO/LUMO energy levels and heat capacity at constant volume, are either unrelated or only indirectly related to drug discovery and are not considered key decision-making factors in therapeutic molecule design.** Therefore, these properties were not prioritized in our study.\n\nWe used QM9 as part of our experiments primarily because its simplicity and small molecular size (up to 9 heavy atoms) make it a practical starting point for model development and early-stage validation. Our task formulation does not depend on the specific molecular properties provided by the QM9 dataset.\n\nThat said, incorporating property prediction tasks into molecular design could be an interesting direction. We appreciate the reviewer’s suggestion and will consider exploring this in future work."}, {"review_id": "UwmU0HIOjD", "reviewer": "Reviewer_2S15", "summary": "This paper presents a reinforcement learning framework integrating uncertainty for de novo molecular design diffusion models. The proposed method leverages predictive uncertainty from surrogate models to construct multi-objective reward functions, enabling the generation of drug-like molecules that balance key properties such as drug-likeness, synthetic accessibility, and binding affinity. Evaluations across benchmark datasets and multiple multi-property aggregation methods show performance gains over existing RL baselines. The approach is further validated through molecular dynamics simulations and ADMET profiling.", "questions": "- What are the implications of assuming independence in Eq. 3? How reasonable is this assumpton when a big number of properties are considered in multi-objective optimization?\n- How much influence do the authors think the way they aggregate the multi-objective task to a single-objective in the baselines has on the results?\n- Why do the authors think the pre-trained models on ZINC have so low permance? Why is still possible to rescue these models with Reinforcement Learning and reach quite good performance?", "limitations": "-\tThe MD simulation time parameters are poorly reported as steps or frames. Reporting total physical simulation time would be much more informative. It's unclear right now, if the reopted simuations are equilibrated.\n-\tImprovements in already good docking score do not necessarily correlate with better affinity. Confirming high affinity with a more reliable affinity prediction method (such as Free Energy Perturbation) on the 4 molecules in Figure 4 would strengthen the paper.\n-\tGiven that most of the work on fine-tuning diffusion models for optimized chemical spaces has been done in the context of diffusion guidance, I think it would strengthen the contribution to compare against some these works. Some examples are [1,2,3].\n\n[1] PILOT: equivariant diffusion for pocket-conditioned de novo ligand generation with multi-objective guidance via importance sampling. Julian Cremer, Tuan Le, Frank Noé, Djork-Arné Clevert and Kristof T. Schütt. Chem. Sci., 2024,15, 14954-14967. DOI\thttps://doi.org/10.1039/D4SC03523B\n\n[2] Unified Guidance for Geometry-Conditioned Molecular Generation. Sirine Ayadi, Leon Hetzel, Johanna Sommer, Fabian Theis, Stephan Günnemann. 38th Conference on Neural Information Processing Systems (NeurIPS 2024). https://arxiv.org/abs/2501.02526\n\n[3] Generating 3D small binding molecules using shape-conditioned diffusion models with guidance. Ziqi Chen, Bo Peng, Tianhua Zhai, Daniel Adu-Ampratwum and Xia Ning. Nat Mach Intell 7, 758–770 (2025). https://doi.org/10.1038/s42256-025-01030-w", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n-\tPaper is clear and well written.\n-\tSimple method outperforming baselines.\n-\tAuthors extensively compare both against other RL baselines and multi-property optimization methods.\n\nWeaknesses:\n\n-\tRelevance and presentation of MD results (addressed during revisions).\n-\tAuthors do not compare against stablished methods in the molecular domain (addressed during revisions)", "was_revised": true, "final_justification": "The authors have successfully addressed my comments. I raise the score accordingly.", "rebuttal": "We sincerely thank the reviewer for the detailed and thoughtful feedback. We are pleased that the reviewer found our paper to be clear, well written, and of good technical quality. We especially appreciate the recognition that our method is effective in outperforming baselines, and that the experimental evaluation includes extensive comparisons with both RL baselines and multi-property optimization approaches. The positive assessment across quality, clarity, significance, and originality is encouraging and valuable to us. \n\nThe reviewer’s concerns primarily relate to the presentation and interpretation of molecular dynamics results, the assumptions and design choices in our multi-objective reward formulation, the aggregation strategy used in baseline comparisons, and the comparison with existing diffusion-guided molecular optimization methods. \n\nWe provide detailed responses below.\n\n**1. What are the implications of assuming independence in Eq. 3? How reasonable is this assumpton when a big number of properties are considered in multi-objective optimization?**\n\n**The independence assumption in Eq. 3 allows us to represent the multi-objective reward as a product of individual property-specific probabilities**, where each probability corresponds to the likelihood that a generated molecule exceeds a predefined cutoff for a given property. These probabilities are estimated using uncertainty-aware predictions from Chemprop, and their product provides a smooth and differentiable reward signal suitable for RL.\n\nIn our study, we focus on three key properties: QED (drug-likeness), SAS (synthetic accessibility), and binding affinity. These properties are designed to reflect different and complementary aspects of molecule quality. **Empirically, we find that these properties are only weakly correlated**. As shown in the correlation matrix below, all pairwise Pearson correlation coefficients between the properties are less than 0.3 in magnitude, indicating low correlation and supporting the independence assumption in this context.\n\n**Correlation Matrix**\n\n|QM9     | QED   | SAS   | AFFINITY |\n|--------|-------|-------|----------|\n| **QED**     | 1.00  | -0.22 | 0.12     |\n| **SAS**     | -0.22 | 1.00  | 0.02     |\n| **AFFINITY** | 0.12  | 0.02  | 1.00     |\n\n|ZINC15    | QED   | SAS   | AFFINITY |\n|--------|-------|-------|----------|\n| **QED**     | 1.00  | -0.04 | -0.01    |\n| **SAS**     | -0.04 | 1.00  | 0.04     |\n| **AFFINITY** | -0.01 | 0.04  | 1.00     |\n\n|PubChem | QED   | SAS   | AFFINITY |\n|--------|-------|-------|----------|\n| **QED**     | 1.00  | -0.18 | 0.19     |\n| **SAS**     | -0.18 | 1.00  | -0.06    |\n| **AFFINITY** | 0.19  | -0.06 | 1.00     |\n\nWe agree that as the number of properties increases (e.g., more than 5), the assumption of independence may become less reasonable due to the potential rise in inter-property correlations. In such cases, more advanced modeling strategies that explicitly consider property interactions could be beneficial. However, it is common practice in multi-objective molecular design to focus on a small set of key properties.\n\n\n**2.How much influence do the authors think the way they aggregate the multi-objective task to a single-objective in the baselines has on the results?**\n\nAggregating multiple objectives into a single scalar reward, as done in baseline methods, may introduce challenges in RL. For example, it may lead to sparse early reward signals, resulting in slow learning, and can fail to provide positive feedback for “promising candidate molecules” that are close to satisfying all objectives but fall just short on one. This limits the model’s ability to explore near-optimal solutions and may reduce the effectiveness of optimization.\n\nHowever, in practical molecular design tasks, it is often required that generated molecules simultaneously meet all specified property thresholds, as only such molecules are typically retained for subsequent stages of evaluation. Therefore, using an aggregation strategy actually aligns with common practice in drug discovery workflows.\n\nIn addition, to the best of our knowledge, there are very few baseline methods in the molecular generation literature that are directly compatible with our task setting. Among the limited available approaches, the most commonly used strategies for multi-objective optimization are weighted sum and gradient alignment. We included both in our ablation analysis, and our method consistently outperforms them. \n\n\n**3.Why do the authors think the pre-trained models on ZINC have so low permance? Why is still possible to rescue these models with Reinforcement Learning and reach quite good performance?**\n\nThe relatively low performance of the pre-trained models on the ZINC dataset may stem from the fact that the general training objective used during pretraining is not sufficiently aligned with the highly specific multi-property criteria required for high-quality molecule generation. As a result, the pre-trained models may learn to capture overall data distribution patterns but fail to prioritize regions of chemical space corresponding to desirable property combinations.\n\nThrough RL optimization, the model is further refined by using desirable property combinations collected from the test set as conditioning inputs, together with a goal-directed reward function to guide generation. This setup explicitly directs the model to generate molecules that satisfy the required constraints, enabling it to effectively adjust its generation strategy. As a result, the quality of generated molecules under specific desirable conditions is significantly improved, leading to better overall performance.\n\n**4.The MD simulation time parameters are poorly reported as steps or frames. Reporting total physical simulation time would be much more ......**\n\nTo improve clarity, we have revised the reporting of MD simulation time throughout the paper. All relevant figures and descriptions now use total physical simulation time in picoseconds (ps), rather than steps or frames. This change provides a more interpretable and standard measure of simulation progress. \n\nTo confirm equilibration, we re-ran the MD simulations for the four selected compounds and analyzed the RMSD of each protein–ligand complex. The RMSD profiles showed clear stabilization after an initial adjustment period, indicating that the systems reached equilibrium before further analysis. This confirms the reliability of the trajectories used for evaluating binding stability (as the RMSD values plateaued within a stable range of approximately 0.20–0.30 nm without signs of sustained drift or abrupt fluctuations).\n\n\n**5.Improvements in already good docking score do not necessarily correlate with better affinity. Confirming high affinity with a more reliable affinity prediction method ......**\n\nWe appreciate the reviewer’s suggestion and agree that improvements in docking scores should be supported by more rigorous affinity predictions. To that end, we applied Free Energy Perturbation (FEP) calculations to estimate the binding free energy (ΔG bind) of the four compounds\n\nUsing exponential averaging based on the Zwanzig equation, we computed binding free energy as the difference between the ligand’s free energy in the bound (complex) and unbound (ligand-only). All simulations were run in OpenMM.\n\nAs shown in the table below, the predicted binding free energy values ranged from –4.14 to –4.29 kcal/mol, corresponding to low micromolar binding affinities. Importantly, the three designed compounds exhibited binding affinities that are comparable to or modestly better than the known inhibitor. While not dramatically superior, these results lend additional support to the docking-based predictions and suggest that the newly generated candidates have acceptable and potentially favorable binding characteristics.\n\n| Compound            | ΔGcomplex (kcal/mol) | ΔGligand (kcal/mol) | ΔGbind (kcal/mol) | Predicted Kd (μM) |\n|---------------------|----------------------|----------------------|-------------------|-------------------|\n| known inhibitor     | -19.5                | -15.4                | -4.14             | ~1.2 μM           |\n| mol from b (QM9)    | -20.3                | -16.0                | -4.29             | ~0.89 μM          |\n| mol from c (ZINC15) | -20.0                | -15.7                | -4.26             | ~1.0 μM           |\n| mol from d (PubChem)| -20.2                | -15.9                | -4.28             | ~0.9 μM           |\n\n**6.Given that most of the work on fine-tuning diffusion models for optimized chemical spaces has been done in the context of diffusion guidance, I think it would strengthen the contribution to compare against some these works. Some examples are [1,2,3].**\n\nWe have carefully reviewed all three papers. We note that the second paper does not appear to provide publicly available source code, which makes reproducing their method for comparison difficult. As for the first and third papers, we observe that their approaches rely on large-scale pretraining on protein–ligand complex datasets, whereas our method is pretrained only on molecular data. As a result, it is challenging to directly apply their models to our dataset.\n\nBesides, their methods are designed for conditional generation based on protein inputs. We considered the possibility of using our target proteins as input to their pretrained models to perform a practical comparison. However, the distribution of our target proteins is substantially different from those used in their training data, making this approach infeasible in practice.\n\nThat said, we appreciate the reviewer’s suggestion and will continue to explore whether these works can be meaningfully included in future comparisons, as well as investigate other potential baselines that could better align with our task setting.\n\nAll the updated figure and analysis will be included in the camera-ready version of the paper."}]

Author rebuttal:

Meta-review: This paper proposes a 3D de-novo molecular design pipeline combining multi-objective RL with guided diffusion models.

All the reviewers are positive about the work, and the only reviewer who was negative, qJWH, also increased their score after the rebuttal. While there were several questions about the experiments, the authors gave convincing answers to all the questions and also added new supporting results.

One of the major criticisms against the paper was the lack of novelty. I do not think algorithmic novelty is necessary for the applications track. The paper does a great job in putting together a complex pipeline for 3d de-novo molecular design and that is worthy enough to be published!
Decision: Accept (poster)