Decomposed Direct Preference Optimization for Structure-Based Drug Design
Authors: Xiwei Cheng
Xiangxin Zhou
Yuwei Yang
Yu Bao
Quanquan Gu
Venue: Accepted by TMLR ()

Abstract: Diffusion models have achieved promising results for Structure-Based Drug Design (SBDD). Nevertheless, high-quality protein subpocket and ligand data are relatively scarce, which hinders the models’ generation capabilities. Recently, Direct Preference Optimization (DPO) has emerged as a pivotal tool for aligning generative models with human preferences. In this paper, we propose DecompDpo, a structure-based optimization method aligns diffusion models with pharmaceutical needs using multi-granularity preference pairs. DecompDpo introduces decomposition into the optimization objectives and obtains preference pairs at the molecule or decomposed substructure level based on each objective’s decomposability. Additionally, DecompDpo introduces a physics-informed energy term to ensure reasonable molecular conformations in the optimization results. Notably, DecompDpo can be effectively used for two main purposes: (1) fine-tuning pretrained diffusion models for molecule generation across various protein families, and (2) molecular optimization given a specific protein subpocket after generation. Extensive experiments on the CrossDocked2020 benchmark show that DecompDpo significantly improves model performance, achieving up to 98.5% Med. High Affinity and a 43.9% success rate for molecule generation, and 100% Med. High Affinity and a 52.1% success rate for targeted molecule optimization.

Published in Transactions on Machine Learning Research (09/2025)
Decomposed Direct Preference Optimization for Structure-Based Based Drug Design
Xiwei Cheng"
cheng.siiddhorrthaatee...ll
Khoury College of Computer Sciences, Northeastern University
Xiangxin Zhou"
DytcD Seed
School of Artificial Intelligence, University of Chinese Academy of Sciences
New Laboratory of Pattern Recognttion (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)
Yu Bao
Gxog..001@bytedoree
ByteDance Seed
Yuwei Yang ByteDance
goweek.ynng@@yytedoocccoom
Quanquan Gu ByteDance Seed
quanguan.guuugggecacccccm
Reviewed on OpenReview: https: // openreview. net/ per/forum? id- duSpo5DRk8
//
Abstract
Diffusion models have achieved promising results for Structure-Bases Drug Design (SBDD). Nevertheless, high-quality protein subpockets and ligand data are relatively scarce, which hinders the models' generation capabilitics. Recently, Direct Preference Optimization (DPO) has emerged as pivotal lool for aligning generative models with human preferences. In this paper, we propose DECOMPDPO, structure-based optimization method that aligns diffusion models with pharmaceutical needs using multi-gramularity preference pairs. DECOMPDPO introduces decomposition into the optimization objectives and obtains preference pairs at the molecule or decomposed substructure level based on each objective's decomposability. Additionally, DECOMPDPO introduces physics-informed energy term ensure reasonable molecular conformations in the optimization results. Notably, DECOMPDPO used for two main purpose: (1) fine-tuning pretrained diffusion models for molecule genera- tion across various protein families, and (2) molccular optimization given specific protein subpocket after generation. Extensive experiments on the CrossDocked2020 benchmark show that DECOMPDPO significantly improves model performance, achieving up to 98.5% Med. High Affinity and a 43.9% success rate for molecule generation, and 100% Med. High Affinity and 52.1% success rate for targeted molccule optimization. Code is available at wtt.iiit....omlaaaaaalalll..a
be effectively
1 Introduction
is strategic approach in medicinal chemistry and pharmaceutical research that utilizes 3D structures of biomolecules to guide the design and optimization of new therapeutic agents. The goal of SBDD is to design molecules that bind to specific protein targets. Recent studies viewed this task as a data-driven conditional generative problem, introducing powerful generative
Structure-based drug design (SBDD) (Anderson 2003
"Equal contribution (this work done during Xiwei Xiangxin's internship RyteDance).

---

Published in Transactions on Machine Learning Research (09/2025)
models with geometric deep learning Powers et al. 2023
For example, Peng et al. (2022) : Zhang & Liu
(2023) proposed generating atoms or fragments sequentially by an Seu3)-qquvvariant aulo-regressive model,
Luo et al. (202]); Peng et al. (2022); ; Guan et al. (2023a) introduced diffusion models (Ho et al. 2020) to
model distributions over ligand atom types and positions.
A significant bottleneck for the development of generative models in SBDD is the scarcity of high-quality
protein-ligand complex data /Vamathevan et al.) 2019). While large-scale datasets have spurred rapid advances in fields such as natural language processing, the collection of protein-ligand binding data is notably more challenging and limited due to the complex and resource-intensive experimental procedures. Notably, the CrossDockod2000 dataset (Francoeur et al.) 2020p, widely-used dataset for SBDD, augments existing data by docking ligands into similar protein pockets in the Protein Data Bank. Although this increases dataset size, it may unavoidably introduce some low-quality data. As highlighted by Zhou et al. (2024a), the ligands in the CrossDockod2020 dataset have moderate binding affinities, which do not meet the stringent demands of drug design. Morcover, the number of unique ligands remains the same before and after this data augmentation, limiting generative models from learning diverse and high-quality molecules.
To address the aforementioned challenge, Xie et al. (2021); Fu et al. 2022 provided straightforward method for searching molecules with desired properties in the extensive chemical space. However, pure searching or optimization methods lack generative capabilit ies and fall short in the diversity of the designed molecules. Zhou et al. (2024a) integrated a conditional diffusion model with optimization by iteratively selecting better molecular substructures as generation conditions. This method achieves better properties while maintaining a certain level of diversity. Nonetheless, the performance of this method is still limited due to fixed model parameters during the optimization process.
To break the bottleneck, we introduce DECOMPDPO, a multi-objective optimization framework that aligns diffusion models with practical pharmaceutical requirements using generated data. Inspired by the decompo- sition nature of ligand molccules, DECOMPDPO introduces decomposition into the optimization objective to provide more flexibility in preference selection and alignment. Based on each objective's decomposability, DECOMPDPO directly aligus the model with full-molecule level preferences using G1.OBALDPO or LoCAL,PO with coroompooed-sussrruueee conformations in drug discovery, DECOMPDrO integrates physics-informed energy terms to penalize molecules with poor conformations. Additionally, a linear beta schedule is proposed for improving optimization elficiency. We demonstrate the effectiveness of DECOMPDPO in two scenarios: structure-bases molecule generation and molecule optimization, showing that DECOMPDPO significantly outperforms existing baselines. We highlight our contributions as follows:
level preferences. Recognizing the importance of physically realistic molecule
We propose DECOMPDPO, which introduces decomposition into the optimization objectives for more effective and flexible alignment of diffusion- based generative models with real-world pharmaceutical requirements using multi-gramulann
preferences.
Our approach is applicable to both structure-based molecule generation and optimization. Notably, DECOMPDPO achieves 98.5% Med. High Affinity and a 43.9% success rate for molecule generation, and 100% Med. High Affinity and a 52.1% success rate for molecule optimization on CrossDocked2020 dataset.
To the best of our knowledge, we are one of the first works to introduce preference alignment. to structure- based drug design. Our approach aligns the generative models for SBDD with the practical requirements of drug discovery.
Recently, an independent concurrent work by Gu et al. (0024) also employs preference alignment for fine-tuning diffusion models in SBDD. Specifically, they regularized the DPO objective to mitigate overfitting on winning data. However, they primarily focused on binding affinity optimization and do not check the sanity and conformational soundness of generated molecules, which are essential in practical drug design. Compared to Gu et al. (2024), we construct preference pairs at both the full-molecule and decomposed-subssrrecct levels for enhanced optimization performance and flexibility, and integrate physics-informed energy terms lo penalize unreasonable molecular conformations. Beyond binding affinity optimization, the effectiveness of DECOMPDPO is demonstrated under broader multi-objective sctting to generate molecules better aligned with pharmaccutical needs, with further evidence of its strong performance through iterative fine-tuning for molecule optimization.
2

---

Published in Transactions on Machine Learning Research (09/2025)
2
Related Work
Structure-based Drug Design Structure-based drug design (SBDD) aims to design ligand molecules that can bind to specific protein targets. Recent efforts have been made to enhance the efficiency of modeling molecule distributions. Ragoza et al. (2022) employed variational autoencoder to generate 3D molecules in
atomic density grids. Autoregressive approaches (Luo et al. 2021 Peng et. al.
2022 Liu et al. 2022) generate
3D molecules atom by atom, while Zhang et al. (2022) extends to predict molecular fragments in an auto- regressive way. Recently, Guan et al. (2023a): Schneuing et al. (2022); Lin et al.
(2022) introduced diffusion
models to SBDD. Building upon these developments, some recent studies have sought to further enhance SBDD methods by incorporating biochemical prior knowledge. DecompDiff (Guan al. 2023b) dccomposed ligands into substructures and generated molecules with decomposed priors and validity guidance in diffusion. DrugGPS (Zhang & Liu 2023) incorporated subpocket similarities to augment molecule generation. IPDiff (Huang et al. 2023 addressed the inconsistency between forward and reverse diffusion by leveraging a pre-trained protein-ligand interaction prior. Despite these advancements, effectively generating molecules that meet real-world drug development criteria remains challenging due to the fundamental mismatch between distribution learning and reward-guided generation. In this work, we aim to bridge this inconsistency by aligning diffusion model distributions directly with real-world pharmaceutical objectives, enhancing the molecule generalion efficiency.
Structure-based Molecule Optimization In addition to simply generative modeling of existing protein- ligand pairs, some researchers leveraged optimization algorithms to design molecules with desired properties. AutoGrow 4 (Spiegel & Durrant 2020), RGA (Fu et al. 2022,, and DecompOpt (Zhou et al. 2024a] take a specific protein target as input and perform optimization using oracle functions to evaluate the fitness of generated molecules. Specifically, RGA (Fu et al.
2022 used neural networks to guide the genetic algorithm with reinforcement learning. DecompOpt (Zhou et al. 2024a employed controllable diffusion model conditioned on protein subpockets and substructures for iterative optimization. In addition, [Ridenbach (2024): Shen et al. (2023) performed structure-based optimization in lower dimensions, such as latent vectors or 2D graphs. Beyond optimizing towards a single protein target, methods such as PIT.TT (Cremer et al.] 2024 reweighted diffusion trajectories toward particular objectives using importance sampling, KGDiff (Qian et al, 2024 guided diffussin trajectories with gradients from an additional expert network, and TAGMol (Dorna
et al. 2024 guided continuous coordinates in both the training and sampling processes.
A complementary line of work optimizes ligand-only 2D graphs SMILES strings without explicit 3D protein couditioning during generation. REINVENT (Oiveecrona et al.] 2017) uses reinforcement learning to fine-tune SMILES generators with property-specife reward signals. GraphGA [Jensen 2019) employs a
genetic algorithm over molecular graphs guided by reward functions. MARS (Xie et al. 2021) adopts Monte Carlo tree search for fragment--ased graph optimization. GEAM (Lee et al. 2023b) learns a lask-specific fragment vocabulary for reward-guided molecular assembly, while Saturn (Guo & Schwaller, 2024) combines Mamba models with memory-basee experience replay for efficient optimization. These methods have shown effectiveness in property-driven optimization tasks but operate without explicit 3D protein information, where spatial complementarity and interaction geometry are essential.
In this work, we target SBDD and therefore benchmark against SBDD baselines to ensure a fair comparison in inputs, hypothesis space, and evaluation protocol. Our model directly aligns the diffusion model with preferences, and is applicable to both target-specific and general-purpose SBDD optimization settings.
Learning from Human/Al AI Feedback Likelihood-based training alone can fail to meet user-defined preferences for generative models. Reinforcement learning from human AI feedback (Ziegler et al. 2019
Stiennon et al. 2020 Ouyang et al. 2022 Lee et al. 2023a Bai et al. 2022) addressed this by first learning reward model from data annotated by human or AI, then fine-tuning the generative model through policy-gradient methods (Christiano et al. 2017 Schulman et al.) 2017). Similar techniques have also been introduced to diffusion models for text-to-image generation (Black et al. 2023 Fan et al., 2024 Zhang et al.) 2024j. Recently, Direct Preference Optimization (DPO) (Rafailov et al. 2024) further simplified the procoss by directly optimizing on preference pairs without an explicit reward model. Wallace et al. (2023) re-formultted DPO and derived Diffusion-DPO for aligning text-to-image diffusion models. While these methods primarily
3

---

Published in Transactions on Machine Learning Research (09/2025)
Global Properties
Decomposable Properties
yo Aaaitsiieeneaea arm2
FooD A-Trmaiii Aaaumr
arm1
scaffold
0.452
0.592
Diffusion Model
Sampling
Togi
3.63
4.54
3.25
Selection
-0.422
-0.232
4.68
4.09
2.75
Preference Allgnment
Multi-Objective Preference
Physics-Informed Energy Constraint
expectation and standard deviation from training data
Teorokrninn
Exood
Bungle
maxe (0, Ly Plaa ,o
))*
max (0, Age - 12,,,, 30 30,
Figure 1: Overview of DECOMPDPO. (a) Sample molecules and select molecule pairs for each target protein using a pre-trained diffusion model; (b) Construct physically constrained preference for each optimization objective based on its decomposability; (c) Compute the DECOMPDPO loss and align the diffusion model with the multi-objective preference.
focus on natural language or image generation, Zhou et al. (2024b) proposed to finc-tune diffusion models for antibody design by DPO targeting low Rosetta energy. our work, we introduce preference alignment improve the desired properties of generated molecules and propose specialized methods to improve the performance of DPO in the scenario of SBDD.
3 Method
In this section, introduce DECOMPDPO, which aligns diffusion models with pharmaceutical needs using physically constrained multi-gramularity preferences (Figure D We first define the SBDD task and introduce the decomposed diffusion model in Section 3.1 We then incorporate decomposition into the optimization objectives for multi-objective preference alignment in Section 3.2 Recognizing the importance of maintaining physically realistic molecular conformations during optimization, we introduce physics-ines informed energy terms for penalizing rewards in Section 3.3 Finally, we propose linear beta schedule improve the optimization efficiency for diffusion models (Section 3.4
3.1 Preliminaries
In the context of SBDD, generative models aim generate ligands M {(x; u.M. (MMMMMM)) )ise((,... NM) that Here, Np and NM are the
bind specific protein binding site, represented as P {(x? u " he( number of atoms in the protein and ligand, respectively; E R v E R bij E B& represents the 3D atom coordinates, the atom types, and the bonds between atoms, where K₃ and K₁ represent the number of atom and bond types.
Following the decomposed diffusion model introduced by Guan et al. (203b)). each ligand is decomposed into fragments K, comprising several arms A connected by at most one scaffold S (I.A) 2 1, S1 > 1, |C [A] + (SS).. Based on the decomposed substructures, informative data-dependent priors Op = (MI:K Σ1.k H} are estimated from atom positions by maximum likelihood estimation, where μy R represents the prior center, Σg E Rax3 represents the prior covariance mairix, and H {n {0, 1] 1x Σ 1/k 1} represents the prior-atom mapping for M. This data-dependent prior enhances the training efficiency of the diffusion model, where M is gradually diffused with fixed schedule (Ae)tat....... We denote a, = =1 and o₁ I['s 0t- The i-th atom position is shifted to its corresponding prior center: X x (H') Tt.
4

---

Published in Transactions on Machine Learning Research (09/2025)
The noisy data distribution at time derived from the distribution at time t 1 is computed follows:
NM (V
x =
- [),
NM
vv.. = II c(vil - Ac/Kaa),
(1)
==1
NM
(bbb(br-1,))
II C(b;](1-)ebb) -,)b; A./K)).
The perturbed structure is then fed into the prediction model. The reconstruction loss time can be derived from the KL divergence as follows:
[(x)_ =,, [|x00 x0l1 [() E, Σ c(v,, vo)s log cVv,, volk
Ka
K2 Σ c(br, bo) log c(b,, bo)a c(b,, bo)k
(
E.
(2)
c(ve, vo)a
where (Xo, Vo, bo); (Xe; Vt, by), (Ro, Vo, bo), represent true atoms positions, types, and bonds types at time O time t, and predicted atoms positions, types, and bonds types at time t U10 T): denotes mixed categorical distribution with weight ag and 1 0z. The overall loss is I, = T,(=) + +% T. + T,))), with Yos "o as weights of reconstruction loss of atom and bond type. We provide more details for the model architecture in Appendix To better illustrate decomposition, we show a decomposed molecule with the arms highlighted in Figure
3.2 Direct Preference Optimization Decomposed Space
Decomposable Optimization Objectives In real pharmaceutical applications, drug candidates should possess multiple desirable properties, yet such molecules rare among all known drug- molecules, making distribution learning less efficient in generating desired molecules. Although previous methods like DecompOpt [hou et al. 2024a) fully exploit the power of conditional diffusion models through iterative generation, the model's performance is limited by static parameters learned from offline data. Direct preference optimization offers a simple yet efficient way to align models with pairwise preference data. Inspired by the success of ligand decomposition in improving generative power (Guan et al. 2023b Zhou et al. 2024a), we introduce decomposition into optimization objectives in DECOMP DPO for greater flexibility in preference selection and alignment.
An optimization objective is considered decompos- able if the overall score of molecule is proportional to the sum of substructure- scores, which im- plies that a substructure with a higher score will lead the molecule to have a higher overall score. For example, Vina Minimize Score largely based on pairwise atomic interactions, with each substructure contributing its interactions with the protein target and negligible inter-substructure interactions, making it decomposable. As shown in Figure 2 we validated the proportional relationship of Vina Min- imize Score in our dataset. However, objectives like QED and SA are non-decomposable, as their calcu- lations involve non-linear operations. We provide more statistical evidence in Appendix C
Molecule Vina Mine-6.05
X Substruccures Vima Minimize Score
Vina Milmm2 36
Vina Min--1.55
-5.
Vina Minn-1130
Molecule Vina Minimize Score
Figure 2: Illustraioon of decomposable objectives. De- compose molecule into two arms (purple and pink) and a scaffold (yellow), where the sum of the substruc- tures' Vina Minimize Scores equals to the molecule's (left). The Pearson correlation between molecule's 's and sum of substructure's Vina. Minimize Scores the training dataset (right).
GlobaIDPO To align the model with practi- cal pharmaceutical preferences, following RLHF
OOuyang et al. 2022). the pre-trained model is fine-tuned by maximizing certain reward functions with the

---

Published in Transactions on Machine Learning Research (09/2025)
Bealback-Leibber (KL) divergence regularization:
max E P-D, [r(M, P)] BDDL (Po(M) (P) || Pret(M P)), (MIP(
(3)
where >> 0 is hyperparameter controlling the deviation from the reference model Pref- Recently,
DPO RRfailov et al. 2024) derives a pairwise training loss from equation 3 providing a simpler way to fine-tune the model with pairwise preference data:
LDo = (P لーر Togo (Bog PM IP)
Pr P)) Bogg Po IP) ]
Pref(M [P))
(4)
where M and M represent the preferred and less preferred molecules, respectively.
As (M P)) is intractable for diffusion modcls, following Diffusion-DPO Wallace et al 2023 we define the reward over the entire dilfussin process r( r(M, P) (pa(M.r (.M,MM [MM,, ( P)J, where MieT denotes
the diffusion trajectories from the reverse process pg. By utilizing the evidence lower bound, the DPO loss is converted to:
log Pa 0.T
IP)
log Po(M OT PP) D]
loga BE M.
DDitfusion-DPO = E (P.M*.M MnD
Pref(Morr IP) Prot(M [P)
(M M Mir (M M
(5)
Following Wallace et al. 2023 we further approximate the intractable reverse probability pa with forward probability q and use Jensen's inequality to externalize the expectation:
logor (3 (8(1og (a M PP) -log Po(Mi-IlMM.P9
Pral(Mt MM Pre MT, ,)) [M ]]
CDiflusion-DR DDO (P. MMMMMM""WD M.D.---MM.
(6)
MM (M5).
Mr MiNu(M;11M))
The Dislusion-DPO loss is applied to align models with non-decomposable objectives using molecule-level preferences, which will refer to as GLOBALDPO hereafter for clarity.
LocalDPO According to the decomposition in drug space, a molecule's probability factorizes over sub-
structures (Guan et al. 2023b). As a result, we reformulate the Diffusion-DPO loss as:
PalM |M P). log
log Pa(M |M P) (-- |M P
[]]
LDarrssoN =
(P.M JM MM I- My sLM (MM)M MolM, (M6))
loga $
(M M((+.p))
(7)
where Mdfht and M are the i-th decomposed substructure extracted from the winning and losing molecule, respectively. Here, the substructures of the preferred molecule are always considered the winning side, even though it is not always the case that they have better propertics.
For decomposable objectives, we introduce decomposition into preference alignment by directly constructing preference pairs based on substructure-level properties, and define the LOCALDPO loss as:
K
[og (33 sign -(M0)))))))()))))))))))
LLOCALDPO = -E
(P.M.,M M.D.NNT Ming(MtiM) ~o(M; (Mn)
(8)
(M ++ M P)
Pa (M(
M .P)
where 10 log
log
Pret (M M (0+ .P)
Pref (M - ,P)
and ((M represents the reward of the decomposed substructure MD) LOCALDPO can then be reformulated as GLOBAADDPO on reconstructed molecule pairs assembled from substructures with higher- or lower-ranking
properties. Please refer to Appendix A for the proof.
In multi-objective optimization, different objectives can interfere with each other, leading to suboptimal results. By allowing fine-grained guidance at the substructure level, LOCALDPO can alleviate conflicts multi-objective optimization by offering more flexible and diverse optimization pathways, ultimately enhancing alignment performance with complex design goals.

---

Published in Transactions on Machine Learning Research (09/2025)
Table Summary of different properties of reference molecules and molecules generated by DECOMPDPO and other generative models (Gen.) and general purpose optimization methods (Opt.). (1) / (4) denotes a larger smaller number is better. Top results are highlighted with bold text and underlined text, respectively.
Vina Score (1)
Viaa Min (1)
Vins Dock (L)
High Affinity (t)
QED (1)
SA (f]
Diversiy (f)
Size
Success
Methoda
Avg.
Mrd.
Avg.
Med.
Avg.
Med.
Avg.
Mod.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Rate (1)
Refereace
-6.36
-6.46
-6.71
-6.49
-7.45
-7.26
0.48
0.47
0.73
0.74
22.8
25.0%
LIGAN
-6.33
-6.20
21.1%
11.1%
0.33
0.30
0.59
0.57
0.86
0.67
19.9
3.9%
GraphBP
+4.80
+4.70
14.2%
6.7%
0.43
0:45
0.49
0,48
0.79
0.78
0.1%
ARt
-5.75
-5.64
-6.18
-5.88
-6.75
-6.62
37.9%
31.0%
0.51
0.50
0.63
0.63
0.70
0.70
17.7
7.1%
Gen.
2ock
-4.70
-7.15
-6.79
48.4%
51.0%
0.56
24.4%
TirgetDif
-5.47
-6.30
-7.80
-7.91
50.1%
0.48
0.71
10.5%
-6.42
-7.01
-7.48
-8.57
75.5%
0.52
0.53
0.59
0.74
0.73
17.7%
McCCRAFT
-6.50
-7.01
-7.27
-7.26
-7.92
-8.01
59.0%
62.6%
0.50
0.51
0.69
0.68
0.72
0.73
26.8
22.7%
DocompDif
-5.96
-7.05
-7.60
-7.88
-8.88
.8.88
72.3%
87.0%
0.44
0.43
0.60
0.60
0.80
0.60
29.4
28.0%
TAGMol
-7.02
-7.77
-7.95
-8.07
-8.59
-8.69
69.89
76.4%
0.55
0.56
0.56
0.56
0.89
0.70
24.7
11.19
Opt.
DECoMPDro
-6.13
-7.54
-8.30
-8.57
-9.60
-9.68
85.8%
98.5%
0.48
0.46
0.67
0.67
0.83
0.62
31.6
43.9%
DecompDpo Based on the ideas introduced above, we define a unified loss that applies LOCALDPO to decomposable objectives and GLOBALDDP to non-decomposable objectives with a weighted sum:
DDEcoMPDro
W;LLoCALDPO()) +
Σ
w.LLGLOBA.DPO(.)
(9)
iE DEQDesomp
jE JeCNon-Decomp
where Decomp and QNon- (Non-Decomp represent the set of decomposable and non-dccomposable properties, and Wi, wy are weighting coefficients. This dual-granularity alignment accommodates both types of objectives and provides more precise control over the optimization process to meet the diverse requirements of molecule design.
3.3 Physically Constrained Optimization
An important aspect preference alignment in drug design is ensuring the gencrated molecular conformations remain physically plausible. Inspired by Wu al. (2022). we define physics-informed energy terms that. penalize bonds and angles which deviate significantly from empirical values, formulated as:
Ebond = Σ
max (0, Lij - Mineos -30t 30').,,)))
JJB
(10)
Eangle
Σ
(max (0, |AyR - Moreesee -00 3o
i.J.EA
where denotes the set of bonds the molecule, and A denotes the set angles formed by two adjacent bonds in B. Here, Lij is the bond length between atoms and j, and p de are the expectation and standard deviation of bond lengths for those atom types, which are calculated from the training data. Similarly, for each angle (i,j, k) E A, Aijk measures the radian of the angle, and Pe, of N..gy. represent the empirical angle statistics for the corresponding atom types. The overall energy term is defined Tconstraint = Eband + Eaglle. To constrain the model from learning unrealistic molecular conformations, we adjust the reward by penalizing it with this energy term: r*MMPP) r(M,P) - (M, P), where À is a weighting factor that balances the original reward against structural validity. Additional evaluation A Appendix demonstrates the effectiveness of this constraint preserving reasonable molecular conformations
during optimization.
3.4 Linear Beta Schedule
As shown in Equation (3) the parameter 3 regulates how aggressively the model deviates from the reference distribution Pref in pursuit of higher rewards. During the diffusion process, carlier steps influence the subsequent ones. Larger B in early diffusion steps can stabilize training and preserve adherence to the
7

---

Published in Transactions on Machine Learning Research (09/2025)
Table 2: Summary of different properties of reference molecules and molecules generated by DECOMPDPO and other target-specifie molecule optimization methods. (t) / (4) denotes a larger smaller number better. Top 2 results are highlighted with bold text and underlined text, respectively.
Vima Score (4)
Vina Min (4)
Vina Dock (L)
High A Affiniy (t)
QED (f)
SA (*)
Diversily (1)
Surcess
Methods
Avg.
Med.
Avg.
Mod.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Rate (1)
RGA
-8.01
-8.17
64.4%
89.3%
0.57
0.57
0.71
0.73
0.41
0.41
46.2%
Opc
-5.87
-6.81
-7.35
-7.72
-8.98
-9.01
73.3%
93.3%
0,48
0.15
0.65
0.65
0.60
0.61
52.5%
DECOMPDPO
-7.27
-7.93
-8.91
-8.88
-9.90
-10.08
88.5%
100.0%
0.48
0.42
0.60
0.62
0.62
52.1%
distribution learned from offline data with stronger regularization. As we progress through the final steps, where atom types and precise positions have crucial impact on molccular properties, becomes advantageous to relax this regularization to allow more aggressive optimization towards target objectives. To achieve this balance, we propose línear beta schedule, 8 8r, where 3r is the maximum regularization weight at the initial stage = 0 and decreases linearly approaches T. This schedule effectively balances the influence of the reference model and optimization objectives throughout the diffusion process.
4 Experiments
We implement DECOMPDPO in two important pharmaceutical scenarios: (1) fine-tuning the model for molecule generation across diverse protein families, and (2) optimizing the model for specific protein target.
4.1 Experimental Setup
Dataset We followed prior work (Luo et al. 2021 Peng et al. 2022 Guan et al. 2023a bj, using the Cross- Docked2020 dataset (Francoeur et al. 2020) to pre- prait our reference model and evaluate the performance of DECOMPDPO. According to the protocol established by Luo et al. (2021), we filtered complexes to retain only those with high-quality docking poses (RMSD < 1A) and diverse protein sequences (sequence identity < 30%), resulting in a refined dataset comprising 100,000 high-quality training complexes and 100 novel proteins for evaluation.
To fine-tune the model for general-purpose molecule generation, we first generate 10 candidate molecules for each training protein. Each molecule's favorability is measured by multi-objective score Tmulti Ex.ex Ti, where X is the set of normalized optimization objectives. For each protein, we select the top and bottom scoring molecules to form preference pairs, resulting in 63,092 valid pairs overall. To optimize the model for targeted molecule optimization, we sample 500 molecules for each test protein and construct preference pairs by similarly selecting the top and bottom 100 molecules according to their scores Tmulti-
Basclines To assess the capability of DECOMPDPO in general-purpose molecule generation, we compare it with several representative generative models. LiGAN (Ragoza et al. 2022) employs CNN-based variational autoencoder to encode receptor and ligand into a latent space, then generates ligands' atomic densities. Atom-based autoregressive models such as AR (Luo et al. [021), Pocket2Mol (Peng et al. 2022), and GraphBP (Liu et al. 2022 update atom embeddings using a graph neural network (GNN). Difg (Guan et al. (0233)) and DecompDiff (Guan et al., 2023b) are GNN-based diffusion models, with the latter incorporating decomposed priors. IPDiff (Huang et al. 2023) integrates protcin-ligand interaction priors into both forward and reverse diffussin processes. MoICRAFT (Qu et al.) 2024) introduces bayesian flow network to SBDD. We also compare with several general-purpose optimization methods. TAGMol (Dorna et al. 2024) optimizes continuous coordinates throughout training and sampling, while AliDiff (Gu et al. 2024) applies preference alignment, and KGDiff (Qian et al., 2024 uses an expert- network --ased gradient guidance. Since AliDiff and KGDiff primarily focus on binding-affinity optimization, we compare with them under this setting.
To benchmark DECOMPDPO's capability in targeted optimization, we compare it with two strong baselines: RG A (uu et al. )022), a reinforced genetic algorithm that optimize molecules with policy network through
8

---

Published in Transactions on Machine Learning Research (09/2025)
evolutionary processes, and DecompOpt (Zhou et al. 2024a which employs conditional diffusion model to iteratively replace substructures for improved properties.
Evaluation Following previous studies (Gun et al. 2023a Luo et al., 2021 Ragoza et al. 2022) we evaluate molecules from two aspects: target binding affinity and molecular properties, and molecular conformation. We use AutoDock Vina to assess target binding affinity. Vina Score quantifies the direct binding affinity between molecule and the larget protein, Vina Min measures the affinity after local structural optimization via force fields, Vina Dock assesses the affinity after re-docking the ligand into the target protein, and High Affinity measures the proportion of generated molecules with higher Vina Dock score than that of reference ligands. For molecular properties, we calculate drug- likeeisss (QED) (Bickerton et al. 2012), hetic accessibility (SA) (Frtl & Schuffenhauer 2009), and diversily. Following Jin et al. (2020); Xie et al. 2021), the overall quality of generated molecules evaluated by Success Rate (QED 0.25, SA 0.59, Vina Dock < -8.18). To evaluate molecular conformation, we compute Jensen-Shannon divergence (JSD) between the distributions of the generated molecules with reference ligands. We also evaluate median RMSD and energy difference of rigid fragments and the whole molecule before and after optimizing molecular conformations with Merck Molecular Force Field (MMFF) (Halgren 1996 We provide more evaluation of the sanity of generated molecules in Appendix C
Implementation Details The bond-first noise schedule proposed by Peng et al. 2023 effectively addresses the inconsistency between atoms and bonds when using predicted bonds for molecule reconstruction. We adapt this noise schodule for DccompDiff, resulting an enhanced model as our reference model, termed DecompDiff*. We select QED, SA, and Vina Minimize Score as our optimization objectives. Please refer to Appendix B for more implementation details.
4.2 Main Results
Molecule Generation We first evaluate the ef- fectiveness of DECOMPDPO in optimizing models to- wards generating desirable molecules across various protein families. As shown in Table [ DECOMPDrO significantly improves all metrics over the reference model, DecompDiff* underscoring its effectiveness in multi-objective optimization. Notably, DECOMPDrO achieves the highest score for most binding-alfinily- related metrics, confirming its robust ability to gen- erate molecules that bind well across diverse protein families. Figure 3 shows examples of molecules gen- erated by DECOMPDPO, illustrating that DRCOM- PDPO is capable of preserving realistic molecular conformations while achieving better scores. Addi- tional visualization results are shown in Appendix
Reference
DecompDift
DecompDPO
OED-0.71 SAJO Vina-9.74
0ED:0:46 54/0.8 Vina.4..9 DecompDiff*
QED:051 88/0.72 Vina.-10.23
Reference
DecompDFO
44 SA Vine-8.55
QED:D 38 SA:0.85 Vine--0.66
QED:0.75 BA:28 Vima:-10
Figure 3: Visualization of reference binding ligands and the molecule generated by DECOMPDIFE* and DECOM- PDo on protein 4D70 (lop) and 1UMD (bottom).
To demonstrate that DECOMPDPO preserves struc-
lurally realistic conformations while improving performance, we measure the Jensen Shannon divergence (JSD) of all-atom pairwise distance distributions between generated molecules and reference ligands. As shown in Table 3 DECOMPDPO closcly matches DecompDifft, achieving the lowest JSD among all models evaluated. We further compute the median energy and RMSD differences pre- and post-MMFF optimization for both rigid fragments without rotatable bonds and whole molecules. The results indicate that DECOMPDPO generally performs par with DecompDiff* and notably achieves the lowest whole-molecule median energy difference, underscoring DECOMPDPO 's capability in balancing property optimization with physically realistic structures. We provide additional distributions of these metrics, along with bond distance and bond angle JSD statistics, in Appendix C

---

Published in Transactions on Machine Learning Research (09/2025)
Table 4: Summary of results of single-objective optimization for affinity-rlatted metrics. (1) / (4)
denotes a
larger / smaller number better. The best result is highlighted with bold text.
Vina Score (4)
Vina Min (1)
Vina Dock (4)
High Affinity (1)
QED (t)
SA (1)
Diversity (t)
Size
Success
Method
Med.
Avg,
Med.
Avg.
Med.
Avg.
Med.
Avg.
Ved.
Avg.
Medl
Avg.
Med.
Avy.
Rate
KGDiff
-8.01
-8.61
-8.78
-8.85
-9.43
-9.43
79.2%
87.0%
0.51
0.51
0.51
0.54
0.75
24.5
13.7%
AliDir
-7.07
-7.55
-8.00
-8.17
-8.00
-8.81
73.4%
81.4%
0.50
0.57
0.73
24.4
12.3%
DECOMPDrO
-6.52
-8.01
-8.97
-9.15
-10.50
-10.29
91.8%
100.0%
0.16
0.13
0.67
0.67
0.70
0.70
31.7
16.8%
Table 5: Ablation study of decomposing DPO loss and linear beta schodule. (1) / (4) denotes a larger smaller number is better. The best result. is highlighted with bold text.
Vina Soore (4)
Vina Min (L)
Vina Dock (1)
High Aflinity (1)
QED (t)
SA (f)
Diversity (1)
Succres
Method
Avy.
Med.
Aug.
Med.
Aug.
Med.
Aog.
Ved.
Avg.
Med.
Avg.
Med.
Med.
Rate
w/ Constant Beta Weight
-5.97
-7.14
-7.78
-8.04
-9.04
-9.09
74.9%
91.8%
0.48
0.44
0.62
0.62
0.61
0.61
32.1%
Molecule-level DPO
-5.81
-7.10
-7.75
-8.03
-9.06
-9.10
76.2%
92.3%
0.49
0.63
0.61
0.62
35.5%
Drcoxeeroo
-6.13
-7.54
-8.30
-9.68
85.8%
98.5%
0.48
0.67
0.67
0.62
43.9%
Targeted Optimization To evaluate DECOM- PDpo effectiveness in larget-specific molecule op- timization, we further optimize the fine-tuned model for each target protein in the test set. As shown in Table 2 DECOMPDPO outperforms all baselines in metrics and achieves Success Rate comparable to DecompOpt, illustrating its strong ability to optimize ligands for specific protein targets. In practice, DECOMPDPO can also be integrated into DecompOpt, potentially unlocking even greater im- provements by combining preference alignment with iterative optimization.
Table Summary of conformation related metrics of molecules (Mol.) and corresponding rigid fragments (RF.) generated by DECOMPDPO and other diffusion models. The top 2 results are highlighted with bold text and underlined text.
TirgetDiff
IPDif
DecompDiff
DecompDPO
JSD - All Atom
0.09
0,08
0.07
0.07
Energy Diff- RF.
1355.94
1159 45
39.39
38.38
Energy Diff- Mol.
6116.37
21491.71
8833.80
672.02
RMSD RF.
0.13
0.14
0.13
0.11
RMSD Mol.
1.02
1.04
1.10
1.08
4.3 Ablation Studies
Single-Objective Optimization To further demonstrate DECOMPDPO 's effectiveness in Landing-affinite optimization, we use LOCALDPO for Vina Minimise optimization. As shown in Table DECOMPDre achieves the highest scores on most affinity-related metrics among general-purpose binding-affinity optimization methods, with improvements of 9.4% in Vina Score, 18.0% in Vina Minimize, and 18.2% in Vina Dock over the reference model. Notably, DECOMPDPO also achieves improvements in QED and SA, which we attribute to the negative correlations (-0.12 and -0.11) between substructure-level Vina Minâmize differences and corresponding molecule-level QED and SA differences in our training preference pairs. Additionally, DECOMPDPO achieves 78.9% Complete Rate, defined as the percentage of valid and connected molecules among all generated molecules, indicating that its performance boost does not come at the cost of general molecular properties.
Benefits of Decomposed Preference Our primary hypothesis is that decomposing the optimization objectives improves training efficiency by offering more flexibility in preference selection, particularly in multi-objective settings. We verify this claim by fine-tuning the reference model with only molecule-level preference pairs for all optimization objectives, which we term as Molecule-level DPO. As shown in Table 5 DECOMPDPO outperforms Molccule-level DPO across all metrics, validating that decomposed preference could provide greater flexibility and diversity in preference selection for enhanced optimization effectiveness.
Benefits of Lincar Beta Schedule We also examine the effectiveness of the lincar beta schedule by comparing it with a constant-3 baseline for molecule generation. As shown in Table 5 the linear schedule consistently enhances all reported metrics, illustrating its effectiveness in balancing adherence to the reference
10

---

Published in Transactions on Machine Learning Research (09/2025)
distribution with high-reward optimization, leading to more efficient preference alignment for diffusion models.
5 Conclusion
In this work, we introduced preference alignment to SBDD, developing DECOMPDPO to align pre-trained diffusion models with multi-grane gulliiggammlarttyy preference, which provides more flexibility during the optimization process. The physics-infrmmed energy term penalizing the reward is beneficial for maintaining reasonable molecular conformations during optimization. The linear beta schedule effectively improves optimization efficiency by progressively reducing regularization during the diffusion process. DECOMPDPO shows promising results in molecule generation and molecule optimization, highlighting its ability to meet practical needs of the pharmaccutical industry.
Broader Impact Statement
Our contributions to structure-based drug design have the potential to significantly accelerate the drug discovery process, thereby transforming the pharmaceutical research landscape. Furthermore, the versatility of our approach allows for its application in other domains of computer-aided design, inchuding, but not limited to, protein design, material design, and chip design. While the potential impacts are ample, we underscore the importance of implementing methods responsibly to prevent misuse and potential harm. Hence, diligent oversight and ethical considerations remain paramount in ensuring the beneficial utilization of our techniques.
References
Amy C Anderson. The process of structure-based drug design. Chemistry & biology, 10(9):787 797, 2003.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint or orXiv:2212.08075 2022.
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs, Nature chemistry, 4(2):90-98 98, 2012.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXio preprinl ar Xrx::23005.133 13301, 2023.
Martin Buttenschoen, Garrett M Morris, and Charlotte M Deane. Posebusters: Ai-based docking methods fail to generate physically valid poses generalise to novel sequences. Chemical Science, 15(9))3100-3139, 2024.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement. learning from human preferences. Advances in neural information processing systems, 30, 2017.
Julian Cremer, Tuan Le, Frank Noé, Djork-Arné Clevert, and Kristof T Schütt. Pilot: Equivariant diffusion for pocket conditioned de novo ligand generation with multi-objective guidance via importance sampling. arXiv preprint ar 14925,, 2024.
Vineeth Dorna, Subhalingam, Keshav Kolluru, Shreshth Tuli, Mrityunjay Singh, Saurabh Singal, NM Kr- ishnan, and Sayan Ranu. Tagmol: Target-aware gradient-guided molecule generation. ar preprint arXiv:2406.0100 01650, 2024.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug- like molecules basod molecular complexity and fragment. contributions. Journal of cheminformalics, 1(1) 111::1111, 2009.
11

---

Published in Transactions on Machine Learning Research (09/2025)
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Tee. Reinforcement learning for fine-tuning text-lo-image diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jía, Richard B lovanisci, lan Snyder, and David R Koes. Throe-dimensional convolutional neural networks and cross-docked data set for structure-based based drug design. Journal of chemical information and modeling, (0(9):420042215 2020.
Tianfan Fu, Wenhao Gao, Connor Coley, and Jimeng Sun. Reinforced genetic algorithm for structure-based drug design. Advances in Neural Information Processing Systems, 35:12325- -23388, 2022.
Siyi Gu, Minkai Xu, Alexander Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, and Stefano Ermon. Aligning target-aware molecule diffusion models with exact energy optimization. arXiv preprinl XrX::22477006665 2024.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In International Conference Learning Representations, 2023a.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. DecompDiff: Diffusion models with decomposed priors for structure-based drug design. In Andreas Krause, Emıa Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 11827-11846. PMLR, 23-29 Jul 2023b. URL https:
\/proceedings.u wwwpang.mmm...... www.s//202/2222 guan23a.haee hmml
Jeff Guo and Philippe Schwaller. Saturn: Sample-efficient generative molecular design using memory manipulation. arXiv preprinl Niv: 17066, 2024.
Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmf994. Journal of computational chemistry, 17(5-6):490-51) 519, 1996.
Charles Harris, Kieran Didi, Arian Jamasb, Chaitanya Joshi, Simon Mathis, Pietro Lio, and Tom Blundell. Posecheck: Generative models for 3d structure-based drug design produce unrealistic poses. In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
Jonathan Ho, Ajay Jain, and Picter Abbocl. Denoising diffusion probabilistic models. Advances in newral information processing systems, 33:6840 6851, 2020.
Zhilin Huang, Ling Yang, Xiangxin Zhou, Zhilong Zhang, Wentao Zhang, Xiawu Zheng, Jie Chen, Yu Wang, CUI Bin, and Wenming Yang. protein-ligand interaction prior for binding-aware 3d molecule diffusion models. In The Twelfth International Conference on Learning Representations, 2023.
Jan H Jensen. A graph-based genetic algorithm and generative model/ model/montee carlo tree search for the exploration of chemical space. Chemical science, 10(12)3366-3357, 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objcetive molecule gencration using interpretable substructures. In International conference on machine learning, pp. 4849 4859. PMLR, 2020.
Joseph Katigbak, Haotian Li, David Rooklin, and Yingkai Zhang. Alphaspace 2.0: Representing concave biomolecular surfaces using I-clusters. Journal of chemical information and modeling, 60(4):1444-155 1508, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint arXiu:1412.600 6980, 2014.
Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellic Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint or Xxxx::330000066 2023a.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Seul Lee, Scanie Lee, Kenji Kawaguchi, and Sung Ju Hwang. Drug discovery with dynamic goal-aware fragments. Xiv preprint orXiv:2310.0083 2023b.
Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Z Li. Diffbp: Generative diffusion of 3d molecules for target protoin binding. arXiv preprint arXiv:221 11214, 2022.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. arXiv preprint arXio:2204.0944 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229 6239, 2021.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1 14, 2017.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neurol information processing systems, 35:27730 27744, 2022.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xic, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molccular sampling based on 3d protein pockets. In International Conference Mochine Learning, pp. 17644-17655. PMLR, 2022.
Xingang Peng, Jiaqi Guan, Qiang Liu, and Jianzhu Ma. Moldiff: addressing the atom- bond inconsistency problem in 3d molecule diffusion generation. arXiv preprint arXiv:2305.077 075008 2023.
Alexander S Powers, Helen H Yu, Patricia Suriana, Rohan Koodli, Tianyu Lu, Joseph M Paggi, and Ron O Dror. Geometric deep learning for cructure-based ligand design. ACS Central Science, (12): 2023.
Hao Qian, Wenjing Huang, Shikui Tu, and Lei Xu. Kgdiff: towards explainable target-aware molecule generation with knowledge guidance. Briefings in Bioinformatics, 25(1):bbad433, 2024.
Yanru Qu, Keyue Qiu, Yuxuan Song, Jingjing Gong, Jiawei Han, Mingyue Zheng, Hao Zhou, and Wei- Ying Ma. Molcraft: Structure-based drug design in continuous parameter space. In ICML, 2024. URL
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 13(9):2701 2713, 2022.
Danny Reidenbach. EvoSBDD: Latent evolution for accurate and efficient structure-basde drug design. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024. URL ntttps://opereviier net/forrumiidesLLUUWoo...
Arne Schneuing, Ili Tgashov, Adrian W Dobbelstein, Thomas Castiglione, Michael M Bronstein, and Bruno Correia. Multi-domain distribution learning for de novo drug design. In The Thirteenth International Conference on Learning Representations.
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Mla Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia. Structure-based drug design with equivariant diffusion models. arxiv preprint Xix::210011699. 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. or Xiu preprint ar arXio: 2017.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Tony Shen, Mohit Pandey, and Martin Ester. Tacogfn: Target conditioned gflownet for structure- based drug design. Xiv preprint orXiv:2310. 0320223, 2023.
Jacob O Spiegel and Jacob D Durrant. Autogrow4: open-source genetic algorithm for de novo drug design and lead optimization. Journal of cheminformatiic, 12(1):1 16, 2020.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008 3021, 2020.
Jessica Vamathevan, Dominic Clark, Paul Czodrowski, lan Dunham, Edgardo Ferran, George Lee, Bin Li, Anant Madabhushi, Parantu Shah, Michaela Spitzer, et al. Applications of machine learning in drug discovery and development. Nature reviews Drug discovery, 18(6):463 477, 2019.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model aligmment using direct preference optimization. arXiv preprinl or Xix:223111 1. 12908, 2023.
Lemeng Wu, Chengvue Gong, Xingchao Liu, Mao Ye, and Qiang Liu. Diffusion-based molecule generation with informative prior bridges. Advances in Newral Information Processing Systems, 35:36533 36545, 2022.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. (MARS): Markov molec- ular sampling for multi-objectivee drug discovery. In International Conference on Learning Representations, 2021. URL. https://openrevvee/ http://www...ettteooooe////aaaae
Yinan Zhang, Eric Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. arXiv preprint 0rXiv:2401.12244, 2024.
Zaixi Zhang and Qi Liu. Learning subpocket prototypes for generalizable structure-based drug design. arXiv preprint ar Xiv:2300. 13997, 2023.
Zaixi Zhang, Yaosen Min, Shuxin Zheng, and Qi Liu. Molecule generation for target protein binding with structural motifs. In The Eleventh International Conference on Learning Representations, 2022.
Xiangxin Zhou, Xiwei Cheng, Yuwei Yang, Yu Bao, Liang Wang, and Quanquan Gu. Decompopt: Controllable and decomposed diffusion models for structure-based molecular optimization. In The Twelfth International Conference on Learning Representations, 2024a. URL http:::/ppenreriie..eeeee/////ee
Xiangxin Zhou, Dongyu Xue, Ruizhe Chen, Zaixiang Zheng, Liang Wang, and Quanquan Gu. Antigen-specific antibody design via direct ensrgy-based preference optimization. arXiv preprint 2XXi:2403.165100 2024b.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fino-tuning language models from human preferences. arxio preprint ar Xiv: 19099 08593, 2019.
A Theoretical Analysis
According to DecompDiff (Guan et al. 2023b), the reverse diffusion probability of a molecule can be factorized over its substructures as:
p(M,-1 Mp) p(x-1 x,) p(v.-1 Vi. p(be-1 b bu)
NNr
Nos Nar
Ha p(x]__ xx. i p(v lvt. v()
II
Ha H p(b2 bb b
(11)
(M (
14

---

Published in Transactions on Machine Learning Research (09/2025)
Under the assumption of decomposable molecular probability, we can then reformulate LOCALDPO loss
K
[log Po(M [M ,P)
log Po(M M ,P) Pia(M IM,
LIoCAMIP Eist log (r(
sign sigu(r( (r(M)()))) -(M()))))
PoalM - IM ,P)
log Do(Mi 1.M
Prot(M--1 |M log po(M
Baata log (
(12)
Prel(M/11 ( |M,
po(M MM ,P)
log polME MT,P), P)D]
-aaaat
log (
[log
MoeelMi Mi Pro(M Mi .P)
where Easta
M and M denote the i-th substructures decomposed from Mi MMMM)MM M.MMM moMM
(P.M MMMM"MMMM-BB (-)-----o07)
the original winning and losing molecules, M and M refer to the i-th substructures with higher and lower individual property, M'+ and M denote the molecules reconstructed from substructures with higher and lower property scores across all decomposod substructures. We can then reinterpret LOCALDPO GLOBALDPO over recombined molecule pairs with higher and lower properties than original molecules, enabling the model to learn optimization more extreme cases.
B Implementation Details
B.1
Featurization
Following DecompDiff (Guan al.) 2023b), we characterize each protein atom using set of features: a one-hot indicator of the element lype (H, C, N, O, S, Se), a one-hot indicator of the amino acid type to which the atom belongs, one-dimensional indicator denoting whether the atom belongs to the backbone, and a one-hot indicator specifying the arm/ arm/scaffold region. We define the part of proteins that lies within 10A of any atom of the ligand as the pocket. Similarly, a protein alom is assigned to the arm region if it lies within a ι0Å radius of any arm; otherwise, it is categorized under the scaffold region. The ligand atom is characterizod with a one-hot indicator of element type (C, N, O, F, P, S. CI) and a one-hot arm arm/scaffold indicator. The partition of arms and scaffold is predefined by decomposition algorithm proposed by DecompDiff.
We use two types of messago-passin passing graphs model the protein-ligand complex: k-ncarest neighbors (kun) graph for all atoms (we choose k = 32 in all experiments) and fully-connected graph for ligand atoms only. In the knn graph, edge features are obtained from the outer product of the distance embedding and the edge type. The distance embedding is calculated using radial basis functions centered at 20 points between OÅ and 10Á. Edge types are represented by a 4-dimensional onc-hot vector, categorizing edges as between ligand aloms, protein aloms, ligand-protein aloms or protein-ligand atoms. For the fully-connected ligand graph, edge features include one-hot bond type indicator (non-bond, single, double, triple, aromatic) and a feature indicating whether the bonded atoms belong to the same arm scaffold.
B.2
Model Details
Our base model used DECOMPDPO is the model proposed by Guan et al. (202bb), incorporating the bond first noise schedule presented by Peng al. (2023). Specifically, the noise schedule is defined as follows:
8T 81
sigmoid( w) sigmoid(w)
b 81 ST +
ār = 8 sigmoid(-
+
For atom types, the parameters of noise schedule are set as $1 = 0.9999, ST = 0.0001, w = 3. For bond types, a two-stage noise schodule is employod: the initial stage (t [1, 600),, bonds are rapidly diffused with parameters $1 0.9999, sT = 0.001, 3. In the subsequent stage (t [600. 1000)), the parameters are set 81 0.001, ST 0.0001, 2. The schedules of atom and bond type are shown in Figure
15

---

Published in Transactions on Machine Learning Research (09/2023)
1,0
bond
0.8
0.2
0.0
200
1000
Timestaps
Figure 4: Noise schedule of atom and bond types.
B.3 Molecular Fragmentation
Following DecompDiff (Guan et al. 2023b), we fragment a molecule into arms and scaffold using RDKit and Alphaspace2 (Katigbak et al. 2020) toolkit. Specifically, subpockets for the target protein are extracted using Alphaspace2 and ligands are decomposed into fragments using BRICS. Then terminal fragments with only one connection site are assigned to subpockets by linear sum assignment. Arms centers are defined the centroids of terminal fragments and any remaining subpockets, and scaffold center is defined as the farthest fragment from all arm centers. Finally, the nearest neighbor clustering is performed to tag fragments as arms or the scaffold.
B.4 Training Details
Pre-training We use Adam (Kingma Baj 2014) for pre-training, with init Seaaninggraaeere and betas= 0. 999)). The learning rate is scheduled to decay exponentially with factor of 0.6 with minimize Leriiiieeeerrrnnn_eeee Beaeminn_rateee-666 The learning rate is decayed if there is improvement for the validation loss in 10 consecutive evaluations. We set batch size=8 and clip_grad gradient__..... -enddiirm-t-....... During training, a small Gaussian noise with a standard deviation of 0.1 to protein atom positions is added as data augmentation. To balance the magnitude of different losses, the reconstruction losses of atom and bond type are multiplied by weights To = 100 and 7 = 100, respectively. We perform evaluations every 2000 training steps. The model pre-trained on a single NVIDIA A6000 GPU, and it could converge within 21 hours and 170k steps.
Fine-tuning and Optimizing For both fine-tuning and optimizing model with DECOMPDPO, we use the Adam optimizer with init geat__leaning_rateeeeee
and betas= (0. 0.5,0.9991 We maintain a constant learning rate throughout both processes. We set Birch_size=4 and clip_gradient norm=8. Consistent with pre-training, Gaussian noise is added to protein atom positions, and we use weighted reconstruction loss. For fine-tuning the model for molecule generation, we set Br = 0.001 and trained for 30,000 steps on one NVIDIA A40 GPU. For molecular optimization, we set BT = 0.02 and trained for 20,000 steps on one NVIDIA V100 GPU, We perform evaluation every 1,000 steps.
B.5
Experiment Details
The scoring function for selecting training molecules is defined S QED SA + Vina Min/(-12), Vina Minimize Score is divided by -12 to ensure that it generally ranges between 0 and 1. For molccule generation, we exclude molecules that cannot be decomposed or reconstructed,, resulting in a total of 63,092 preference pairs available for fine-tuning. The weight each objective in multi-objective optimization is set to 1. In molccular optimization, to ensure that the model maintains a desirable completion rate, we include an additional 50 molecules that failed in reconstruction in the losing side of preference pairs. To tailor the optimization to specific protein, the weights of the optimization objectives are defined as w, e (#-w.),
16

---

Published in Transactions on Machine Learning Research (09/2025)
where is the mean property of the generated molecules and 2, is the threshold of the property used in Success Rale. For both molecule generation and molecular optimization, in DecompDift. Opt Prior is defined as mixture of Ref Prior, which is determined by the reference ligand, and Pocket Prior, which is defined by a prior gencration algorithm using AlphaSpacc2 (Katigbak et al. 2020 depending on whether Ref Prior passes the Success threshold. The À used for penalizing rewards with energy terms proposed in Section 3.3 is set 0.1.
employ the same Opl Prior used
In evaluating the performance of DECOMPDPO, for each checkpoint, generate 100 molecules for the molecule generation task and 20 molecules for the molecular optimization task across each target protein in the test set. For both molecule generation and optimization, we select the checkpoint with the highest weighted Success Rale, which defined as the product of the Success Rate and the Complete Rate.
C Additional Results
C.1 Full Evaluation Results
Table 6: Jensen-Shannon Divergence of the bond distance distribution between the generated molecules and the reference molecule by bond type, with a lower value indicating better. *-", "=", and 5:" represent single, double, and aromatic bonds, respectively. The top 2 results are highlighted with bold text and underlined text.
Bond
liGAN
GraphBP
Pocket2Mol
TargetDiff
DecompDiff
IPDifE
DECOMPDrO
C-C
0.601
0.368
0.609
0.496
0.369
0.359
0.451
0.535
C=C
0.665
0.530
0.620
0.561
0.505
0.537
0.530
0.546
C-N
0.634
0.456
0.474
0.416
0.363
0.344
0.411
0.404
C=N
0.749
0.693
0.635
0.550
0.584
0.567
0.598
C-O
0.656
0.467
0.492
0.454
0.421
0.376
0.489
0.411
C=0 0.661
0.471
0.558
0.516
0.461
0.374
0.431
0.319
C:C 0.497
0.407
0.451
0.416
0.263
0.251
0.221
0.281
C:N
0.638
0.689
0.552
0.487
0.235
0.269
0.255
0.265
Table 7: Shasnn-Shannon Divergence of the bond angle distribution between the generated molecules and the reference molecule by angle type, with a lower value indicating better. The top 2 results are highlighted with bold text and underlined text.
Bond
EGAN
GraphBP
AR
TifgetDiff Pocket2Mol
DecompDiff
IPDiff
DECOMPDPO
CCC
0.598
0.424
0.340
0.323
0.328
0.314
0.402
0.488
CCO
0.637
0.354
0.442
0.401
0.385
0.324
0.451
0.428
CNC
0.604
0.469
0.419
0.237
0.367
0.207
0.407
0.360
OPO
0.512
0.684
0.367
0.274
0.303
0.217
0.388
0.185
NCC
0.621
0.372
0.392
0.351
0.354
0.294
0.399
0.375
CC=0
0.636
0.377
0.476
0.353
0.356
0.259
0.363
0.265
COC
0.606
0.482
0.459
0.317
0.389
0.339
0.463
0.427
Molecular Conformation To provide a more comprehensive evaluation of molecular conformations, we compute the JSD distances for different types of bonds and angles between molecules from generative models and reference molecules. shown Table and Table DECOMPDPO generally achieves comparable JSD values to those of DecompDiff, demonstrating that DECOMPDrO generally maintains desirable molccular conformations during preference alignment.
In Figure ligands, along with their corresponding Jensen-Shannon divergence (JSD). DECOMPDPO and DecompDiff* both achieve low JSD scores, indicating minimal deviation from the distribution of real molecules. We further investigate the structural quality of generated molecules by examining both rigid fragments and whole molecules before and after MMFF optimization. As shown in Figure G DECOMPDPO shows lower or bomparable median energy differences relative to DecompDiff*, and consistently lower RMSD differences across various fragment sizes, validating its improvements in optimization objectives coincide with maintaining
compare the pairwise distance distributions between generated molecules and reference
17

---

Published in Transactions on Machine Learning Research (09/2025)
improving conformational quality. In Figure 7 the median RMSD and energy differences for whole molecules further confirm DECOMPDPO preserves physically realistic geometries.
Ret Pockutalot: (250:8.14)
Targethiff (:42:8069)
Deco120111 (:50:0.67)
(150:9.68)
Becusjaro (399:8.07)
10
Distance of all atom pairs (A)
Figure 5: Compare pairwise distance distributions between all atoms in generated molecules and reference
molecules from the test set. Jensen-Shamp divergence (JSD) between two distributions is reported.
Median Energy Difference (kcalmol)
AR
Median Energy Difference (kcalmol)
20000
AR
PocketZMol TargetDiff
Pocket2Mol TargetDiff DecompDir
DecompDrtt
250
IPDIff
DecompDPO
6000
DecompDPO
2000
50
hhlld dull
Fragment Size
Number of rocatable bonds
Figure 6: Median energy difference for rigid fragments (left) and generated molecules (right) before and after optimizing with the Merck Molecular Force Field.
2.5
KGAM
DecomoDit
PocketzMol
Pochak2Mel "T
Decomsoro
Median RMSD (A)
IPDHT
Decomater
Median RMSD (A)
Decomatio
0.6
1.0
Fragment Size
Fragment Size
Figure 7: Median RMSD for rigid fragments (left.) and generated molecules (right) before and after optimizing with the Merck Molecular Force Field.
Molecular Properties To provide a comprehensive evaluation, we have expanded our evaluation metrics which primarily focus molecular properties and binding affinities.
beyond those discussed in Section 4.1
18

---

Published in Transactions on Machine Learning Research (09/2025)
To assess the model's efficacy in designing novel and valid molecules, calculate the following additional metries:
Complete Rate is the percentage of generated molecules that are connected and valid, which defined by RDKit.
Novelty is defined as the ratio of generated molecules that are different from the reference ligand of the corresponding pocket in the test set.
Similarity is the Tanimoto Similarity between generated molecules and the corresponding reference ligand.
Uniqueness the proportion unique molecules among generated molecules.
Table 8: Summarr of the models' ability in designing novel and valid molecules. (1) (4) denotes a larger smaller number is better.
Methods
Complete Rate (t)
Novelty (f)
Similarity (1)
Uniqueness (*)
LiGAN
99.11%
100%
0.22
87.82%
Generate
AR
92.95%
100%
0.24
100%
2ool
98.31%
100%
0.26
100%
Tiff
90.36%
100%
0.30
99.63%
DECOMPDIFF*
72.82%
100%
0.27
99.58%
DECOMPDRO
74.05%
100%
0.26
99.57%
Optinize
RGA
100%
0.37
96.82%
DecompOpt
71.35%
100%
0.36
100%
DECOMPDrO
63.05%
100%
0.26
99.63%
As reported in Table 8 in molecule generation, DECOMPDPO fine-tuned model achieves better Complete Rate and Similarity compared to the base model. In molecular optimization, DECOMPDPO maintains relatively acceptable Complete Rate and the lowest similarity among all optimization methods.
Recent works (Qu et al. 2024 Schneuing et al.) have further emphasized the importance of ensuring the physical realism beyond optimizing for molecular properties. assess the performance of DECOMPDPO in generating valid molecules, we use PoseBuster (Buttenschoen et al., 2024 and PoseCheck (Harris al, 2023 to compute several additional metrics:
Strain Energy reflects the internal energy cost of the generated conformation relative to a relaxed state. We report the 25%, 50%, and 75% quartiles to give a more global view of ligand stability.
Clash is the average number of steric clashes between ligand atoms and protein residues, thus highlighting potential conflicts in the binding pose.
HB Donors and HB Acceptors quantify the number of hydrogen-bond interactions formed by the ligand, while Hydrophobic and vdWs are the number of non-polar and van der Waals interactions, which often correlate with effective binding.
PB- Valid reports the percentage generated molecules that pass POSEBUSTER's built-in validation checks, accounting for geometry and potential clashes.
As shown in Table DECOMPDrO exhibits the most favorable Strain Energy distributions among all the diffusson--bssed generative models, indicating that its generated ligands tend to adopt more stable conformations. Although the average number of steric clashes (Clash) remains somewhat high, DECOMPDPO slightly improves upon DECOMPDIFF, suggesting that the remaining anomalics can be mitigated in future work by refining reference distributions. In addition, DECOMPDPO shows an increase in the PB valid rate, confirming that its improvements in molecular design do not come at the cost of producing invalid structures.
19

---

Published in Transactions on Machine Learning Research (09/2025)
Table Summary different properties of reference molecules and molecules generated by DECOMPDPO and other generative models. (t) (4) denotes larger smaller number is better. Top 2 results are highlightod with bold text and underlined text, respectively.
Strain Energy (U)
Clash (U)
HB Domors (t)
HB Acceptor (f)
Hydrophobic (1)
vdWs (t)
PB-Valid (t)
25%
50%
75%
Avg.
Avg.
Avg.
Avg.
Avg.
(%)
Reference
34
107
196
5.51
0.87
1,42
5.06
6.61
95.0%
AR.
259
595
2286
4.49
0.51
0.90
3.78
5.54
55.6%
Mocket2Mol
102
189
374
6.24
0.32
0.63
4.53
5.25
73.1%
TargetDiff
369
1243
13871
10.84
0.63
0.98
5.43
7.92
50.8%
DecompDidt
162
354
16.42
0.59
1.48
7.96
11.06
54.6%
DecompDPO
15.05
0.43
8.25
11.01
56.6%
Overall, these results reinforce that preference alignment in DECOMPDPO not only improves optimization objectives but also helps maintain chemically and structurally reasonable molecular conformations.
We also draw boxplots to provide confidence intervals for the performance in molecule generation, which are shown in Figure
Wina Score
Vina Mr.
Vina Dook
DED
LIGAN
Pocket2Mol
TargesDDff
DecompOlft
DecompOPO
Figure 8: Boxplots of QED, SA, Vina Score, Vina Minimize, and Vina Dock of molecules generated by DECOMPDPO and other generative models.
To further illustrate the potency of the generated molecules, draw scatter plot of heavy atom numbers versus Vina Dock score to demonstrate the effect of heavy atom nunbers on the binding affinity of generated molecules.
Brpstii
IRD
Devengbr
-20
Figure 9: Scatter Plots of heavy atom numbers versus Vina Dock scores for Target IPDiff, DecompDiff, and DECOMPDPO.
C.2 Benefits of Physics-informed Energy Terms
As shown in Table [0 DecompDPO achieves comparable optimization results with and without using physics-informed energy terms. However, as suggested Table [T] without using physics-informed energy terms, the energy difference of both whole molecule and rigid fragments significantly increases, demonstrating the necessity of physics-informed energy terms in maintaining reasonable conformations during optimization.
20

---

Published in Transactions on Machine Learning Research (09/2025)
Table 10: Summary of results of DECOMPDPO with and without using peysics-informed energy terms, (1)
(L) denotes a larger / smaller number better. The better result is highlighted with bold text.
Vina Score
Vina Min
Vina Dock
High Affinity (t)
QED (1)
SA
Diversity
Success
Method
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Rate (1)
w/o phys
-6.17
-7.37
-8.07
-8.3
-9.15
-9.36
80.7%
95.7%
0.47
0.44
0.65
0.65
0.61
0.62
38.9%
DecoMrDro
-6.13
-7.54
-8.30
-8.57
-9.60
85.8%
98.5%
43.9%
Table 11: Evaluation of physical realism with and without physics-informed energy terms. Lower energy
differences and RMSD indicate better structural consistency.
Method
JSD Dist (L)
Energy Diff rigid frage (L)
Energy Dif Mol (L)
RMSD rigid Fragg (J)
RMSD Mol (L)
w/o plys
0.07
48.00
887.15
0.12
1.10
DecompDPO
672.02
Top-6 Bond Length Distributions
C-C [ho
C-N
C-O (h:3 00:311660
5093153
66025
340009
130000
00009
200000
50000
300300
20000
3.0
1.
(n-98026)
(1-101233
Itw (0740)74)
22300
20000
100000
1920
2009
2.0
26 2.
2.0
2.2
2.0
Dondd leggo 20
Danc langth 14)
Bond kingch
Figure 10: Histogram of bond lengths for the six most frequent atom pairs.
Top-6 Angle Distributions
(hu: 1561935)))))
[ho 92-7991799
0n: 6051691
10030
13000
60900
468300
5008S
25088
20930
40
389
120
340
100
43
100
120
1.00
00
1.00
343
166
Angle
A1g
Angle
C-N-C
0.P-0
N-C-M
157563
0000
65300
22300
20030
25300
2038
LO008
18930
L83
242
283
106
Figure 11: Histogram of bond angles for the six most frequent atom triplets.
21

---

Published in Transactions on Machine Learning Research (09/2025)
To further substantiate the rationale for physics-informed energy constraint, visualize the training-set distributions of the six most frequent bond-length substructures and bond-angle substructures in Figure T0 and Figure [1 which we adopt as empirical priors to penalize geometries that deviate from physically realistic ranges during optimization.
C.3 Evidence for Decomposability of Properties
As illustrated in Section 3.2 QED and SA are non-decomposable due to the non-linear processes involved their calculations. We validate this no1-decomposaaiilty on our training set. As shown in Figure [2] the Pearson correlation coefficients between the properties of molecules and the sum of the properties of their decomposed substructures higher QED or SA do not necessarily lead the molecule to have better properties. Therefore, we choose molecule-level preferences for QED and SA.
very low, not exceeding 0.1. These results indicate that substructures with
GED Substructures
Σ SA of Substructures
Correlation: 0.10
Correlation: 0.07
62
QED Malecules
Molecules
Figure 12: The Pearson correlation between molecule's and sum of substructures' SA (left) / QED (right) Scores in the training dataset.
C.4 Trade-off in Multi-objective Optimization
Given the multiple objectives in DECOMPDPO, inherent trade-offs between different properties are unavoidable. In molecular optimization, as rated in Figure 13 molecules generated by DECOMPDPO exhibit significantly improved properties compared to those generated by DecompDiff. However, DECOMPDPO encounters a notable trade-off between optimizing the Vina Minimize Score and SA.
C.5 Training Set Distribution
We further provided winning and losing molecules' distribution, as well as the distribution of the difference in QED, SA, and Vina Minimize in the training set, as shown in Figure 14 and Figure 15
22

---

Published in Transactions on Machine Learning Research (09/2025)
Cocomoo
3.5
Figure 13: Pairplots of molecules' properties before and after using DECOMPDrO for molecular optimization protein 4Z2G (left) and 2HCJ (right).
win
lose
OFG
SA
Vina Min
20.0
1750
2533
1500-
2033
1250
150m
1233
500-
250
Figure 14: Distribution of QED, SA, and Vina Minimize of the winning and losing molecule in the training set.
Vina Difference
Difference Preference
1200
1200
20000
10:0
1009
Obunt
600
19000
400
400
-50
-30
0.2
0.9
Vina
Difeemnce
Difterence
5A Diftermnce
Figure 15: Distribution of difference in QED, SA, and Vina Minimize of the winning and losing molecule in the training set.
23

---

Published in Transactions on Machine Learning Research (09/2025)
C.6
Optimization Trend
To demonstrate that our method steadily moves loward the preferred distribution, plot the success rate over training steps, As shown in Figure 16 the curve exhibits clear upward trend throughout the training process.
Success rate training
Bide DM0
1009 1900
1969 100 Wil D9h 19m0 1707
Figure 16: Success rate along the training process of DECOMPDPO on CrossDocked2020 test set.
C.7 Examples of Generated Molecules
Examples of reference ligands and molecules generated by DecompDiff and DECOMPDPO, which are shown
in Figure 17
DecompDiff
DecompDPO
DecompDiff
QED:0.7 34.9.54 5.5
QED:8.49 3AJ0 V1
QED B3 BALO.50 V.0::978
QED SA.0.33 Vina-110.54
QED:9.66 SA.0.7 Vina::10.19
QED- SA.0.83 Vina.9.72 9.72
Figure 17: Additional Examples of reference binding ligands and the molecule with the highest property among all generated molecules of DECOMPDIPP* and DECOMPDPO on protein 1GG5 (left) and 3TYM (right).
24


Reviews:
[{"review_id": "NK2lbPaUob", "reviewer": "Reviewer_cUBn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- The work is theoretically justified and well presented.\n- The model produces strong results for docking score optimisation compared to a number of state-of-the-art baseline methods for 3D SBDD generation.\n- Ablation studies seem to show that including decomposition DPO objectives improves the predicted binding affinity of sampled molecules.\n\n## Weaknesses\n\n- While docking scores are improved with the authors' method, the QED and SA do not seem to show any improvement from applying decomposed DPO. At a minimum the authors should provide a discussion on this and attempt to provide some possible explanations for why this may be the case.\n- The paper could do a better job of positioning itself with existing work, especially in the introduction and related works sections. The authors state their method aligns the model with \"real-world pharmaceutical requirements\". Which requirements? Why are they relevant? And how is this method different from the many other existing methods for SBDD which fine tune or align to a given scoring function, eg. reinvent. The authors also say they are \"one of the first to introduce preference alignment to SBDD\". This is not entirely true - while DPO is not commonly used in SBDD, many methods have been proposed to align the model's samples with a scoring function or with high-scoring, preferred molecules. The related work section of the paper should also provide a more thorough discussion on these existing methods, such as reinvent, GraphGA, MARS, GEAM, Saturn, etc. Additionally, given that both decomposition into molecular fragments and DPO have both been previously proposed for SBDD, I would recommend the authors to outline the specific contributions of their method more clearly.\n- The authors provide some ablations studies on their design decision but more are needed to validate their ideas. They show results for a model trained without localDPO but DecompDiff* seems to perform on par with this model and has no linear $\\beta$ schedule and no DPO fine-tuning, making these results puzzling. More thorough ablation studies, removing only one feature at a time from the core DecompDPO model, are needed here. These should include examine the impact of the local and global DPO objectives individually, and the physically constrained optimisation terms.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a number of changes to 3D structure-based drug design ligand generative models and 3D structure-conditioned molecular optimisation models. These changes include combining existing ideas on decomposing the ligand into fragments with reinforcement learning preference alignment methods, as well including energy based guidance terms and an improved DPO schedule. Collectively these changes aim to improve the 3D quality of generated ligand structures, as well as aligning the samples of the model to optimise pharmaceutically relevant scoring functions such as drug-likeness, synthetic accessibility, and docking scores. The authors show that assigning global scores to decomposed substructures of the molecule and fine-tuning the model using their proposed objective can improve docking scores of generated samples. They also show that penalising high-energy structures helps their model to produce more realistic structures, and they provide an improved schedule for $\\beta$, the parameter which controls the deviation from the reference distribution in DPO.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Provide a more in-depth discussion or investigation into why QED and SA scores do not seem to be improved much by introducing DPO. This is also closely related to the additional ablation studies, below.\n- A more thorough discussion on related work and improved positioning of this paper's contributions (see above for more detail). Ideally, this would also include running well-known models such as reinvent on their molecular optimisation or de novo generation tasks.\n- Additional ablation studies focused on each of the specific changes they are proposing to make to the model (see above for more details).\n- The evaluation should show the average size of ligands sampled for all models where possible. Since vina score is highly correlated with ligand size, it is important to include these numbers to allow a reasonable comparison between methods.\n- The success rate definition makes no chemical sense and should modified or removed. Firstly, a QED of 0.25 is far too low to be considered drug-like, and, secondly, each target will have a very different distribution of vina scores, even for reference binders, so it does not make sense to apply the same threshold to every protein.\n\n## Other Clarifications\n\n- How do you deal with molecules in the training dataset that cannot be decomposed by BRICS?\n- In equation 1 p is used to define a one-step forward noising process, but then in equation 6, q is used for a multi-step noising process and $p\\_{\\theta}$ is used for a multistep denoising process. I think the notation here could be clarified to reduce confusion.\n- The notation for H in section 3.1 is not clear. What does H actually correspond to? What does the $i$ in $\\eta\\_{ik}$ refer to?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6eoxGWlicB", "reviewer": "Reviewer_9zyk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Strong performance as compared to a number of existing SOTA methods\n2. Simple yet sensible methods: align DPO at both global and local structure level\n\nWeaknesses:\n1. Relatively weak contribution: though technically sound, the main contribution seems to be just adding an additional local level DPO alignment. This weakness does not affect my decision as TMLR does not focus on novelty, but rather technical correctness\n2.  There are several technical inaccuracies or confusions that need to be clearly articulated\n\n    a. The authors claim on page 5, just before the definition of Equation (5), that \"$p_{theta}(M_0|P)$ in tractable for diffusion models\" yet all the subsequent equations all still use the same $p_{theta}(M_0|P)$ and to make things worse jointly modelling over all time steps as $p_{theta}(M_{1:T}|P)$. There is no clear explanation of why this works, it seems non-sensical\n   \n    b. In their physically constrained optimization, the authors penalize any generated structure whose bond lengths or bond angles deviate markedly from the mean values observed in the training set. This approach seems questionable. Bond lengths and angles between adjacent atoms can vary widely for several well-known reasons—conformational isomerism, hybridization differences, resonance/delocalization, steric and torsional strain, and so on. It is therefore unreasonable to assume a single, symmetric (e.g., Gaussian) distribution for these geometric parameters. Deviation from the global mean of a multimodal distribution does not necessarily indicate an abnormal structure; the value may lie near a different local mode. Although this penalty apparently improves results, its theoretical basis is unclear. I would like to see histograms of bond lengths and angles for a representative set of substructures from the full dataset. If these distributions are indeed multimodal, the current formulation is not well-justified.\n\n3. One of the claims of using DPO is its data efficiency to address data scarcity issue in medicinal chemistry. Author have repeatedly mentioned this in the first few section of the introduction. However, there isn't a single experiment convincingly demonstrating this. I would like to see how efficient DPO is to fully justify it usage. \n\n4. Section 3.4 for linear beta schedule seems non-sensical. The exact words used by author are \" we propose a linear beta schedule, $β_t = t/T β_T$ , where $β_T$ is the maximum regularization weight at the initial stage t = 0 and decreases linearly as t approaches T.\" This is kind of absurd. According to the schedule, $β_t$ actually increases overtime as t increases from 0 to T. How does this work? Furthermore, $β_t$ in equations 4,5,6,7 and 8 are merely just a multiplication factor over the entire objective in log. How does it control deviation from reference? Authors need to clearly define these terms and their rationale rigorously.\n\n 4. There are few spelling and grammatical errors throughout the manuscript. A careful review is warranted to correct these mistakes", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a combination of techniques for fine-tuning a diffusion-based 3D molecular generative model so it docks to a known protein pocket. The primary method selected is Direct Preference Optimization (DPO), originally developed for language models and valued for its simplicity—no explicit, complex reward design is required—and its data efficiency.\n\nThe authors realize that aligning the model with DPO only at the global molecule level may be insufficient to incentivize the generation of high-quality substructures, which they have shown contribute to better overall scores. Consequently, they also compute the DPO objective at the substructure level. This modification is straightforward: they simply sum the DPO objectives computed for each substructure (the same objective applied to smaller sets of atoms and bonds). They argue that this provides finer control over generation and leads to improved results.\n\nAnalogous to annealing the noise schedule in diffusion models to obtain higher-quality samples, the authors further propose annealing the weight of the KL divergence between the reference model and the optimized model to ensure training stability.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see my comments above", "broader_impact_concerns": "The broader impact provided by the authors are more like advertising their contribution and potential of their methods again. The real impact part relating to responsible use of this method is vague.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vN40yLQdl4", "reviewer": "Reviewer_Vc4d", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The experimental results on the CrossDocked2020 dataset show DECOMPDPO’s superiority in molecule generation and optimization tasks. The paper is well-structured, addresses a clear problem, employs advanced generation techniques, indicating it meets the basic requirements for publication. However, the authors need to address some issues before considering further decision.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes DECOMPDPO, a novel structure-based drug design (SBDD) model. It introduces decomposition into optimization objectives and aligns diffusion models with pharmaceutical needs using multi-granularity preference pairs. The loss function combines GLOBALDPO and LOCALDPO terms, handling both decomposable and non-decomposable targets and providing precise control for multi-objective optimization. Additionally, DECOMPDPO uses physics-informed energy terms to penalize unrealistic molecular conformations, ensuring the generated molecules’ structural rationality.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The authors should conduct a more in-depth analysis of DECOMPDPO’s theoretical properties, such as proving its convergence and stability during optimization. This will enhance the paper;s theoretical depth and provide stronger support for other researchers, promoting methodological development in the field.\n2. It’s advisable to include more comparisons with the latest methods, particularly in handling complex drug design tasks. A more detailed analysis of the experimental results would also be beneficial, such as discussing performance changes under different target protein families or molecular attribute weights. This will demonstrate DECOMPDPO’s robustness and adaptability under various conditions.\n3. The paper lacks a detailed evaluation of DECOMPDPO’s computational efficiency. Given the high computational cost of training and sampling in diffusion models, DECOMPDPO’s extra optimization steps may further increase time and computation costs. The authors should explore optimization strategies, such as model compression and accelerated sampling procedure, to enhance the method’s feasibility and efficiency in practical applications.\n4. Although high-quality data scarcity is a bottleneck for generative models in SBDD, the paper doesn’t adequately discuss other challenges DECOMPDPO may face in practical applications, like protein structure dynamics and differences between targets. Moreover, it lacks detailed discussions on integrating DECOMPDPO into the entire development process and the practical problems and solutions that may arise.\n5. The paper notes that in multi-objective optimization tasks, different targets can interfere with each other, leading to suboptimal results. DECOMPDPO uses a combination of LOCALDPO and GLOBALDPO to address this issue. However, there seem to be no ablation experiments to validate the combined loss function’s effectiveness.\n6. The paper contains a few grammatical errors. Please refine them to improve readability.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)