Decomposed Direct Preference Optimization for Structure-Based Drug Design

## Abstract

Abstract Diffusion models have achieved promising results for Structure-Based Drug Design (SBDD). Nevertheless, high-quality protein subpocket and ligand data are relatively scarce, which hinders the models’ generation capabilities. Recently, Direct Preference Optimization (DPO) has emerged as a pivotal tool for aligning generative models with human preferences. In this paper, we propose DecompDpo , a structure-based optimization method aligns diffusion models with pharmaceutical needs using multi-granularity preference pairs. DecompDpo introduces decomposition into the optimization objectives and obtains preference pairs at the molecule or decomposed substructure level based on each objective’s decomposability. Additionally, DecompDpo introduces a physics-informed energy term to ensure reasonable molecular conformations in the optimization results. Notably, DecompDpo can be effectively used for two main purposes: (1) fine-tuning pretrained diffusion models for molecule generation across various protein families, and (2) molecular optimization given a specific protein subpocket after generation. Extensive experiments on the CrossDocked2020 benchmark show that DecompDpo significantly improves model performance, achieving up to 95.2% Med. High Affinity and a 36.2% success rate for molecule generation, and 100% Med. High Affinity and a 52.1% success rate for molecular optimization.

## 1 Introduction

Structure-based drug design (SBDD) (Anderson, ) is a strategic approach in medicinal chemistry and pharmaceutical research that utilizes 3D structures of biomolecules to guide the design and optimization of new therapeutic agents. The goal of SBDD is to design molecules that bind to specific protein targets. Recent studies viewed this problem as a conditional generative task in a data-driven way, and introduced powerful generative models equipped with geometric deep learning (Powers et al., ). For example, Peng et al. (); Zhang & Liu () proposed to generate the atoms or fragments sequentially by a SE(3)-equivariant auto-regressive model, while Luo et al. (); Peng et al. (); Guan et al. () introduced diffusion models (Ho et al., ) to model the distribution of types and positions of ligand atoms.
However, the scarcity of high-quality protein-ligand complex data has emerged as a significant bottleneck for the development of generative models in SBDD (Vamathevan et al., ).
The success of deep learning typically relies on large-scale datasets. In fields like computer vision and natural language processing, the proliferation of the social media has greatly simplified the collection of text, images, and videos, thereby accelerating advancements.
In contrast, the collection of protein-ligand binding data is challenging and limited due to the complex and resource-intensive experimental procedures.
Notably, the CrossDocked2020 dataset (Francoeur et al., ), a widely-used dataset for SBDD, consists of ligands that are docked into multiple similar binding pockets across the Protein Data Bank using docking software. This may be regarded as a form of data augmentation; while it expands the dataset’s size, it may unavoidably introduce some low-quality data. As highlighted by Zhou et al. (), the ligands in the CrossDocked2020 dataset have moderate binding affinities, which do not meet the stringent demands of drug design. Moreover, the number of unique ligands remains the same before and after this data augmentation, limiting generative models from learning diverse and high-quality molecules.

To address the aforementioned challenge, Xie et al. (); Fu et al. () provided a straightforward method for searching molecules with desired properties in the extensive chemical space. However, pure searching or optimization methods lack generative capabilities and fall short in the diversity of the designed molecules.
Zhou et al. () integrated conditional diffusion models with iterative optimization by providing molecular substructures as conditions and iteratively replacing the substructures with better ones. This method achieves better properties while maintaining a certain level of diversity. Nonetheless, the performance of this method is still limited due to fixed model parameters during the optimization process.

To break the bottleneck, we propose a method for multi-objective optimization, aligning diffusion models with practical pharmaceutical requirements of drug discovery using generated data. Inspired by the decomposition nature of ligand molecules, DecompDpo introduces decomposition into optimization objective to provide more flexibility in preference selection and alignment. Based on each objective’s decomposability, DecompDpo directly aligns model with preferences using GlobalDPO with molecule pairs or LocalDPO with decomposed substructure pairs. Recognizing the importance of maintaining reasonable molecular conformations during optimization, DecompDpo integrates physics-informed energy terms to penalize molecules with poor conformations. Additionally, a linear beta schedule is proposed for improving optimization efficiency. We apply DecompDpo to two scenarios: structure-based molecule generation and structure-based molecular optimization. Under both settings, DecompDpo can significantly outperform baselines, demonstrating its effectiveness.
We highlight our contributions as follows:

- •
We propose DecompDpo, which introduces decomposition into the optimization objectives to improve optimization effectiveness and flexibility, directly aligning generative diffusion models with practical pharmaceutical requirements using multi-granularity preferences.
- •
Our approach is applicable to both structure-based molecular generation and optimization. Notably, DecompDpo achieves 95.2% Med. High Affinity and a 36.2% success rate for molecule generation, and 100% Med. High Affinity and a 52.1% success rate for molecular optimization on CrossDocked2020 dataset.
- •
To the best of our knowledge, we are the first to introduce preference alignment to structure-based drug design. Our approach aligns the generative models for SBDD with the practical requirements of drug discovery.

Recently, an independent concurrent work by Gu et al. () proposed a different framework that also uses preference alignment methods to fine-tune diffusion models for SBDD. Notably, they regularized the DPO objective to alleviate overfitting on the winning data. However, they primarily focused on optimizing affinity-related metrics for molecule generation and did not evaluate optimized molecular conformations, which is an important aspect in drug design. Compared to Gu et al. (), we directly formulate preference alignment in SBDD as a multi-objective optimization problem, which is more aligned with pharmaceutical needs, and introduce decomposition into the optimization objectives to provide more flexibility in multi-objective optimization. In addition, we incorporate physics-informed energy terms to penalize unreasonable molecular conformations, thereby maintaining desirable conformations during optimization. Moreover, we demonstrate the effectiveness of DecompDpo in molecular optimization through iterative fine-tuning, achieving superior performance compared to existing optimization methods.

## 2 Related Work

#### Structure-based Drug Design

Structure-based drug design (SBDD) aims to design ligand molecules that can bind to specific protein targets.
Recent efforts have been made to enhance the efficiency of generating molecules with desired properties. Ragoza et al. () employed variational autoencoder to generate 3D molecules in atomic density grids. Luo et al. (); Peng et al. (); Liu et al. () adopted an autoregressive approach to generate 3D molecules atom by atom, while Zhang et al. () proposed to generate 3D molecules by predicting molecular fragments in an auto-regressive way.
Guan et al. (); Schneuing et al. (); Lin et al. () introduced diffusion models to SBDD, which first generate the types and positions of atoms and subsequently determine bond types by post-processing.
Some recent studies have sought to further improve efficacy of SBDD methods by incorporating biochemical prior knowledge.
DecompDiff (Guan et al., ) proposed decomposing ligands into substructures and generating atoms and bonds simultaneously using diffusion models with decomposed priors and validity guidance. DrugGPS (Zhang & Liu, ) considered subpocket-level similarities, augmenting molecule generation through global interaction between subpocket prototypes and molecular motifs.
IPDiff (Huang et al., ) addressed the inconsistency between forward and reverse processes using a pre-trained protein-ligand interaction prior network. In addition to simply generative modeling of existing protein-ligand pairs, some researchers leveraged optimization algorithms to design molecules with desired properties. AutoGrow 4 (Spiegel & Durrant, ) and RGA (Fu et al., ) optimized the binding affinity of ligand molecules towards specific targets by elaborate genetic algorithms. RGA (Fu et al., ) viewed the evolutionary process as a Markov decision process and guided it by reinforcement learning. DecompOpt (Zhou et al., ) proposed a controllable and decomposed diffusion model that can generate ligand molecules conditioning on both protein subpockets and reference substructures, and combined it with iterative optimization to improve desired properties by iteratively generating molecules given substructures observed in previous iterations. Our work also focuses on designing molecules with desired properties by optimization. Differently, we optimize the parameters of the model that generate molecules instead of molecules themselves, which has been demonstrated to be more effective and efficient.

Learning from Human/AI Feedback Maximizing likelihood optimization of generative models cannot always satisfy users’ preferences. Thus introducing human or AI assessment to improve the performance of generative models has attracted significant attention. Reinforcement learning from human/AI feedback (Ziegler et al., ; Stiennon et al., ; Ouyang et al., ; Lee et al., ; Bai et al., ) was proposed to align large language models to human preference, consisting of reward modeling from comparison data annotated by human or AI and then using policy-gradient methods (Christiano et al., ; Schulman et al., ) to fine-tune the model to maximize the reward. Similar techniques have also been introduced to diffusion models for text-to-image generation (Black et al., ; Fan et al., ; Zhang et al., ), where the generative process is viewed as a multi-step Markov decision process and policy-gradient methods can be then applied to fine-tuning the models. Recently, Direct Preference Optimization (DPO) (Rafailov et al., ), which aligns large language models to human preferences by directly optimizing on human comparison data, has attracted much attention. Wallace et al. () re-formulated DPO and derived Diffusion-DPO for aligning text-to-image diffusion models. The above works focus on aligning large language models or text-to-image diffusion models with human preferences. Recently, Zhou et al. () proposed to fine-tune diffusion models for antibody design by DPO and choose low Rosetta energy as preference. In our work, we introduce preference alignment to improve the desired properties of generated molecules given specific protein pockets and propose specialized methods to improve the performance of DPO in the scenario of SBDD.

Figure: Figure 1: Illustration of DecompDpo. This process can be summarized as: (a) Sample molecules and select molecule pairs for each target protein using a pre-trained diffusion model; (b) Construct physically constrained preference for each optimization objective based on its decomposability; (c) Compute the DecompDpo loss and align the diffusion model with the multi-objective preference.
Refer to caption: x1.png

## 3 Method

In this section, we present our method, DecompDpo, which aligns diffusion models with pharmaceutical needs using physically constrained multi-granularity preferences (Figure 1). We first define the SBDD task and introduce the decomposed diffusion model for this task in Section 3.1. Then, we incorporate decomposition into the optimization objectives and propose DecompDpo for multi-objective optimization in Section 3.2. Recognizing the importance of maintaining reasonable molecular conformations during optimization, we introduce physics-informed energy terms for penalizing the reward in Section 3.3. Finally, we introduce a linear beta schedule to improve the efficiency of optimizing diffusion models (Section 3.4).

### 3.1 Preliminaries

In the context of SBDD, generative models are conditioned on the protein binding site, represented as $\mathcal{P}=\{({\bm{x}}_{i}^{\mathcal{P}},{\bm{v}}_{i}^{\mathcal{P}})\}_{i\in
\{1,\cdots,N_{\mathcal{P}}\}}$, to generate ligands $\mathcal{M}=\{({\bm{x}}_{i}^{\mathcal{M}},{\bm{v}}_{i}^{\mathcal{M}},{\bm{b}}_
{ij}^{\mathcal{M}})\}_{i,j\in\{1,\cdots,N_{\mathcal{M}}\}}$ that bind to this site.
Here, $N_{\mathcal{P}}$ and $N_{\mathcal{M}}$ are the number of atoms in the protein and ligand, respectively.
For both proteins and ligands, ${\bm{x}}\in{\mathbb{R}}^{3},{\bm{v}}\in{\mathbb{R}}^{h},{\bm{b}}_{ij}\in{
\mathbb{R}}^{5}$ represents the coordinates of atoms, the types of atoms, and the bonds between atoms.
Here we consider $h$ types of atoms (i.e., H, C, N, O, S, Se) and 5 types of bonds (i.e., non-bond, single, double, triple, aromatic).

Following the decomposed diffusion model introduced by Guan et al. (), each ligand is decomposed into fragments $\mathcal{K}$, comprising several arms $\mathcal{A}$ connected by at most one scaffold $\mathcal{S}$ ($|{\mathcal{A}}|\geq 1,|{\mathcal{S}}|\leq 1,K=|{\mathcal{K}}|=|{\mathcal{A}}|+
|{\mathcal{S}}|$).
Based on the decomposed substructures, informative data-dependent priors ${\mathbb{O}}_{\mathcal{P}}=\{{\bm{\mu}}_{1:K},{\bm{\Sigma}}_{1:K},{\mathbf{H}}\}$ are estimated from atom positions by maximum likelihood estimation. This data-dependent prior enhances the training efficacy of the diffusion model, where $\mathcal{M}$ is gradually diffused with a fixed schedule $\{\lambda_{t}\}_{t=1,\cdots,T}$.
We denote $\alpha_{t}=1-\lambda_{t}$ and $\bar{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{t}$.
The $i$-th atom position is shifted to its corresponding prior center: $\tilde{{\mathbf{x}}}_{t}^{i}={\mathbf{x}}_{t}^{i}-({\mathbf{H}}^{i})^{\top}{
\bm{\mu}}$. The noisy data distribution at time $t$ derived from the distribution at time $t-1$ is computed as follows:

$$ $\displaystyle p(\tilde{{\mathbf{x}}}_{t}|\tilde{{\mathbf{x}}}_{t-1},\mathcal{P})$ $\displaystyle=\prod_{i=1}^{N_{\mathcal{M}}}\mathcal{N}(\tilde{{\mathbf{x}}}_{t }^{i};\tilde{{\mathbf{x}}}_{t-1}^{i},\lambda_{t}({\mathbf{H}}^{i})^{\top}{\bm{ \Sigma}}),$ (1) $\displaystyle p({\mathbf{v}}_{t}|{\mathbf{v}}_{t-1},\mathcal{P})$ $\displaystyle=\prod_{i=1}^{N_{\mathcal{M}}}\mathcal{C}({\mathbf{v}}_{t}^{i}|(1 -\lambda_{t}){\mathbf{v}}_{t-1}^{i}+\lambda_{t}/K_{a}),$ (2) $\displaystyle p({\mathbf{b}}_{t}|{\mathbf{b}}_{t-1},\mathcal{P})$ $\displaystyle=\prod_{i=1}^{N_{\mathcal{M}}\times N_{\mathcal{M}}}\mathcal{C}({ \mathbf{b}}_{t}^{i}|(1-\lambda_{t}){\mathbf{b}}_{t-1}^{i}+\lambda_{t}/K_{b}),$ (3) $$

where $K_{a}$ and $K_{b}$ represent the number of atom types and bond types used for featurization. The perturbed structure is then fed into the prediction model, then the reconstruction loss at the time $t$ can be derived from the KL divergence as follows:

$$ $\displaystyle L^{(x)}_{t}=||{\mathbf{x}}_{0}-\hat{{\mathbf{x}}}_{0}||^{2},\ L^ {(v)}_{t}=\sum_{k=1}^{K_{a}}\bm{c}({\mathbf{v}}_{t},{\mathbf{v}}_{0})_{k}\log \frac{\bm{c}({\mathbf{v}}_{t},{\mathbf{v}}_{0})_{k}}{\bm{c}({\mathbf{v}}_{t}, \hat{\mathbf{v}}_{0})_{k}},\ L^{(b)}_{t}=\sum_{k=1}^{K_{b}}\bm{c}({\mathbf{b}} _{t},{\mathbf{b}}_{0})_{k}\log\frac{\bm{c}({\mathbf{b}}_{t},{\mathbf{b}}_{0})_ {k}}{\bm{c}({\mathbf{b}}_{t},\hat{\mathbf{b}}_{0})_{k}},$ $$

where $({\mathbf{x}}_{0},{\mathbf{v}}_{0},{\mathbf{b}}_{0})$, $({\mathbf{x}}_{t},{\mathbf{v}}_{t},{\mathbf{b}}_{t})$, $(\hat{{\mathbf{x}}}_{0},\hat{{\mathbf{v}}}_{0},\hat{{\mathbf{b}}}_{0})$, represent true atoms positions, types, and bonds types at time 0 0, time $t$, predicted atoms positions, types, and bonds types at time $t$; $\bm{c}$ denotes mixed categorical distribution with weight $\bar{\alpha}_{t}$ and $1-\bar{\alpha}_{t}$. The overall loss is $L_{t}=L^{(x)}_{t}+\gamma_{v}L^{(v)}_{t}+\gamma_{b}L^{(b)}_{t}$, with $\gamma_{v},\gamma_{b}$ as weights of reconstruction loss of atom and bond type. We provide more details for the model architecture in Appendix C. To better illustrate decomposition, we show a decomposed molecule with the arms highlighted in Figure 2.

### 3.2 Direct Preference Optimization in Decomposed Space

#### Decomposable Optimization Objectives

In realistic pharmaceutical scenarios, potential drug molecules should possess multiple desirable properties, which is rare among all known drug-like molecules. As illustrated in previous work (Zhou et al., ), directly learning the distribution from training data is suboptimal, making it inefficient in generating desired molecules. Despite DecompOpt (Zhou et al., ) fully exploits the power of conditional diffusion models through iterative generation, the model’s upper limit remains constrained by the static parameters learned from offline data. Direct preference optimization offers a simple yet efficient way to align models directly with pairwise preferences. Inspired by the success of introducing decomposition in the drug space (Guan et al., ; Zhou et al., ), we introduce the concept of decomposition into optimization objectives in DecompDpo, providing greater flexibility in preference selection and alignment.

Figure: Figure 2: Illustration of decomposable objectives. Decomposition of a molecule into two arms (purple and pink) and a scaffold (yellow), where the sum of the substructures’ Vina Minimize Scores equals to the molecule’s (left). The Pearson correlation between molecule’s and sum of substructure’s Vina Minimize Scores in the training dataset (right).
Refer to caption: x2.png

We define an optimization objective as decomposable if the property of a molecule is proportional to the sum of the properties of its decomposed substructures. This means that a substructure with a higher property will lead the molecule to have a higher overall property. For example, Vina Minimize Score is largely based on pairwise atomic interactions, with each substructure contributing its own set of interactions with the protein target and negligible inter-substructure interactions, making it decomposable. As shown in Figure 2, we validated the proportional relationship of Vina Minimize Score in our training dataset. Unfortunately, not every optimization objective is decomposable. Molecular properties such as QED and SA are non-decomposable, as their calculations involve non-linear operations. We provide more statistical evidence in Appendix D.

#### GlobalDPO

To align the model with practical pharmaceutical preferences, following RLHF (Ouyang et al., ), the pre-trained model is fine-tuned by maximizing certain reward functions with the Kullback–Leibler (KL) divergence regularization:

$$ $\displaystyle\underset{p_{\theta}}{\max}\ \mathbb{E}_{{\mathcal{P}}\sim{ \mathcal{D}},{\mathcal{M}}\sim p_{\theta}({\mathcal{M}}\mid{\mathcal{P}})}r({ \mathcal{M}},{\mathcal{P}})-\beta D_{\mathrm{KL}}\left[p_{\theta}({\mathcal{M} }\mid{\mathcal{P}})\parallel p_{\text{ref}}({\mathcal{M}}\mid{\mathcal{P}}) \right],$ (4) $$

where $\beta>0$ is a hyperparameter controlling the deviation from the reference model $p_{\text{ref}}$. Recently, Rafailov et al. () proposed Direct Preference Optimization (DPO), deriving the DPO training loss from the RLHF loss and providing a simpler way to fine-tune the model with pairwise preference data:

$$ $\displaystyle{\mathcal{L}}_{\text{DPO}}=-\mathbb{E}_{({\mathcal{P}},{\mathcal{ M}}^{+},{\mathcal{M}}^{-})\sim{\mathcal{D}}}\ \text{log}\sigma\bigg{(}\beta \text{log}\frac{p_{\theta}({\mathcal{M}}^{+}\mid{\mathcal{P}})}{p_{\text{ref}} ({\mathcal{M}}^{+}\mid{\mathcal{P}})}-\beta\text{log}\frac{p_{\theta}({ \mathcal{M}}^{-}\mid{\mathcal{P}})}{p_{\text{ref}}({\mathcal{M}}^{-}\mid{ \mathcal{P}})}\bigg{)}.$ (5) $$

Here, ${\mathcal{M}}^{+}$ and ${\mathcal{M}}^{-}$ represent the preferred and less preferred molecules, respectively.

As $p_{\theta}({\mathcal{M}}_{0}\mid{\mathcal{P}})$ is intractable for diffusion models, following Diffusion-DPO (Wallace et al., ), we define the reward over the entire diffusion process $r({\mathcal{M}},{\mathcal{P}})=\mathbb{E}_{p_{\theta}({\mathcal{M}}_{1:T}\mid{
\mathcal{M}}_{0},{\mathcal{P}})}[R({\mathcal{M}}_{1:T},{\mathcal{P}})]$, where ${\mathcal{M}}_{1:T}$ denotes the diffusion trajectories from the reverse process $p_{\theta}$. Consequently, the DPO loss is reframed as:

$$ ${\mathcal{L}}_{\text{Diffusion-DPO}}=-\mathbb{E}_{({\mathcal{P}},{\mathcal{M}} ^{+},{\mathcal{M}}^{-})\sim{\mathcal{D}}}\ \text{log}\sigma\bigg{(}\beta \mathbb{E}_{{\mathcal{M}}_{1:T}^{+},{\mathcal{M}}_{1:T}^{-}}\bigg{[}\text{log} \frac{p_{\theta}({\mathcal{M}}_{1:T}^{+}\mid{\mathcal{P}})}{p_{\text{ref}}({ \mathcal{M}}_{1:T}^{+}\mid{\mathcal{P}})}-\text{log}\frac{p_{\theta}({\mathcal {M}}_{1:T}^{-}\mid{\mathcal{P}})}{p_{\text{ref}}({\mathcal{M}}_{1:T}^{-}\mid{ \mathcal{P}})}\bigg{]}\bigg{)}.$ (6) $$

Following Wallace et al. (), we further approximate reverse probability $p_{\theta}$ with forward probability $q$, and utilize Jensen’s inequality to externalize the expectation:

$$ ${\mathcal{L}}_{\text{Diffusion-DPO}}=-\mathbb{E}_{\begin{subarray}{c}({ \mathcal{P}},{\mathcal{M}}^{+},{\mathcal{M}}^{-})\sim{\mathcal{D}},t\sim{ \mathcal{U}}(0,T),\\ {\mathcal{M}}^{+}_{t}\sim q({\mathcal{M}}^{+}_{t}\mid{\mathcal{M}}^{+}_{0}),\\ {\mathcal{M}}^{-}_{t}\sim q({\mathcal{M}}^{-}_{t}\mid{\mathcal{M}}^{-}_{0}) \end{subarray}}\text{log}\sigma\bigg{(}\beta\bigg{[}\text{log}\frac{p_{\theta} ({\mathcal{M}}_{t-1}^{+}\mid{\mathcal{M}}_{t}^{+},{\mathcal{P}})}{p_{\text{ref }}({\mathcal{M}}_{t-1}^{+}\mid{\mathcal{M}}_{t}^{+},{\mathcal{P}})}-\text{log} \frac{p_{\theta}({\mathcal{M}}_{t-1}^{-}\mid{\mathcal{M}}_{t}^{-},{\mathcal{P} })}{p_{\text{ref}}({\mathcal{M}}_{t-1}^{-}\mid{\mathcal{M}}_{t}^{-},{\mathcal{ P}})}\bigg{]}\bigg{)}.$ (7) $$

The Diffusion-DPO loss is applied to align our model with non-decomposable optimization objectives using molecule-level preferences. For clarity, we refer to this molecule-level preference alignment as GlobalDPO hereafter.

#### LocalDPO

According to the decomposition in drug space, the probability of a molecule is equivalent to the product of the probabilities of its decomposed substructures. As a result, we reformulate the Diffusion-DPO loss as:

$$ ${\mathcal{L}}_{\textsc{Diffusion-DPO}}=-\mathbb{E}_{\begin{subarray}{c}({ \mathcal{P}},{\mathcal{M}}^{+},{\mathcal{M}}^{-})\sim{\mathcal{D}},t\sim{ \mathcal{U}}(0,T),\\ {\mathcal{M}}^{+}_{t}\sim q({\mathcal{M}}^{+}_{t}\mid{\mathcal{M}}^{+}_{0}),{ \mathcal{M}}^{-}_{t}\sim q({\mathcal{M}}^{-}_{t}\mid{\mathcal{M}}^{-}_{0}),\\ {\mathcal{M}}^{+}_{t}=\overset{K}{\underset{i}{\bigcup}}{\mathcal{M}}_{t}^{(i) +},{\mathcal{M}}^{-}_{t}=\overset{K}{\underset{i}{\bigcup}}{\mathcal{M}}_{t}^{ (i)-}\end{subarray}}\text{log}\sigma\bigg{(}\beta\overset{K}{\underset{i}{\sum }}\bigg{[}\text{log}\frac{p_{\theta}({\mathcal{M}}_{t-1}^{(i)+}\mid{\mathcal{M }}_{t}^{(i)+},{\mathcal{P}})}{p_{\text{ref}}({\mathcal{M}}_{t-1}^{(i)+}\mid{ \mathcal{M}}_{t}^{(i)+},{\mathcal{P}})}-\text{log}\frac{p_{\theta}({\mathcal{M }}_{t-1}^{(i)-}\mid{\mathcal{M}}_{t}^{(i)-},{\mathcal{P}})}{p_{\text{ref}}({ \mathcal{M}}_{t-1}^{(i)-}\mid{\mathcal{M}}_{t}^{(i)-},{\mathcal{P}})}\bigg{]} \bigg{)},$ (8) $$

where ${\mathcal{M}}^{(i)}$ represents the $i$-th decomposed substructure. Here, the decomposed substructures of the preferred molecule are always considered the winning side, even though it is not always the case that they have better properties.

For decomposable optimization objectives, we incorporate decomposition into preference alignment by directly constructing preference pairs based on the properties of substructures. Using substructure-level preferences, we derive the training loss for LocalDPO as follows:

$$ $\begin{aligned} {\mathcal{L}}_{\textsc{LocalDPO}}&=-\mathbb{E}_{\begin{ subarray}{c}({\mathcal{P}},{\mathcal{M}}^{+},{\mathcal{M}}^{-})\sim{\mathcal{D }},t\sim{\mathcal{U}}(0,T),\\ {\mathcal{M}}^{+}_{t}\sim q({\mathcal{M}}^{+}_{t}\mid{\mathcal{M}}^{+}_{0}),{ \mathcal{M}}^{-}_{t}\sim q({\mathcal{M}}^{-}_{t}\mid{\mathcal{M}}^{-}_{0}),\\ {\mathcal{M}}^{+}_{t}=\overset{K}{\underset{i}{\bigcup}}{\mathcal{M}}_{t}^{(i) +},{\mathcal{M}}^{-}_{t}=\overset{K}{\underset{i}{\bigcup}}{\mathcal{M}}_{t}^{ (i)-}\end{subarray}}\text{log}\sigma\bigg{(}\beta\overset{K}{\underset{i}{\sum }}{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\text{sign }(r({\mathcal{M}}^{(i)+})>r({\mathcal{M}}^{(i)-}))}\big{[}A^{(i)}\big{]}\bigg{ )},\\ \text{where}\ A^{(i)}&=\text{log}\frac{p_{\theta}({\mathcal{M}}_{t-1}^{(i)+} \mid{\mathcal{M}}_{t}^{(i)+},{\mathcal{P}})}{p_{\text{ref}}({\mathcal{M}}_{t-1 }^{(i)+}\mid{\mathcal{M}}_{t}^{(i)+},{\mathcal{P}})}-\text{log}\frac{p_{\theta }({\mathcal{M}}_{t-1}^{(i)-}\mid{\mathcal{M}}_{t}^{(i)-},{\mathcal{P}})}{p_{ \text{ref}}({\mathcal{M}}_{t-1}^{(i)-}\mid{\mathcal{M}}_{t}^{(i)-},{\mathcal{P }})},\end{aligned}$ (9) $$

where $r({\mathcal{M}}^{(i)})$ represents the reward of the decomposed substructure ${\mathcal{M}}^{(i)}$. Compared to Diffusion-DPO, LocalDPO behaves differently when the substructure-level preference is inconsistent with the molecule-level preference, that is when the sign function yields $\text{sign}(\cdot)<0$. In such cases, if the model’s preference, denoted as $A^{(i)}$ in Equation 9, conflict with the substructure-level preference, the loss for LocalDPO increases because the function $-\log\sigma$ is monotonically decreasing. Conversely, if the model’s preference aligns with the substructure-level preference, the loss decreases. As a result, LocalDPO more effectively corrects misaligned substructure-level preferences.

In multi-objective optimization, different objectives can interfere with each other, leading to suboptimal results. By leveraging decomposed preferences, LocalDPO offers more flexible and diverse optimization pathways, indirectly mitigating conflicts inherent in multi-objective optimization and enhancing overall performance.

#### DecompDpo

Based on GlobalDPO and LocalDPO introduced above, we construct preference pairs for each optimization objective according to its decomposability. By taking a weighted sum of all the preference alignment losses, we derive the overall loss for DecompDpo:

$$ $\displaystyle{\mathcal{L}}_{\text{{DecompDpo}}}=\sum_{i\in{\mathcal{Q}}_{\text {Decomp}}}w_{i}{\mathcal{L}}_{\textsc{LocalDPO}}(i)+\sum_{j\in{\mathcal{Q}}_{ \text{Non-Decomp}}}w_{j}{\mathcal{L}}_{\textsc{GlobalDPO}}(j),$ (10) $$

where ${\mathcal{Q}}_{\text{Decomp}}$ and ${\mathcal{Q}}_{\text{Non-Decomp}}$ represent the decomposable and non-decomposable properties, and $w_{i}$, $w_{j}$ are weighting coefficients. This dual-granularity alignment allows for more precise control over the optimization process and offers greater flexibility in selecting preferences to meet the diverse requirements of molecular design.

### 3.3 Physically Constrained Optimization

An important aspect of preference alignment in drug design is to maintain reasonable molecular conformations that obey physical rules. Inspired by Wu et al. (), we define physics-informed energy terms that penalize bonds and angles which deviate significantly from empirical values, formulated as:

$$ $\displaystyle E_{\text{bond}}$ $\displaystyle=\sum_{i,j\in{\mathcal{B}}}\big{(}\max\big{(}0,\ \left|L_{ij}-\mu _{v_{i},v_{j}}^{l}\right|-3\sigma_{v_{i},v_{j}}^{l}\big{)}\big{)}^{2},$ (11) $\displaystyle E_{\text{angle}}$ $\displaystyle=\sum_{i,j,k\in{\mathcal{A}}}\big{(}\max\big{(}0,\ \left|A_{ijk}- \mu_{v_{i},v_{j},v_{k}}^{a}\right|-3\sigma_{v_{i},v_{j},v_{k}}^{a}\big{)}\big{ )}^{2},$ (12) $$

where ${\mathcal{B}}$ denotes the set of bonds in the molecule, and ${\mathcal{A}}$ denotes the set of angles formed by two neighboring bonds in ${\mathcal{B}}$. Here, $L_{ij}$ is the bond length between atoms $i$ and $j$, and $A_{ijk}$ is the radian of the angle formed by atoms $i$, $j$, and $k$; $\mu_{v_{i},v_{j}}^{l}$ and $\sigma_{v_{i},v_{j}}^{l}$ are the expectation and standard deviation of bond lengths between atom types $v_{i}$ and $v_{j}$, obtained from the training data. Similarly, $\mu_{v_{i},v_{j},v_{k}}^{a}$ and $\sigma_{v_{i},v_{j},v_{k}}^{a}$ represent the expectation and standard deviation of bond angles formed by atom types $v_{i}$, $v_{j}$, and $v_{k}$. The overall energy term is defined as $r_{\text{constraint}}=E_{\text{bond}}+E_{\text{angle}}$. To prevent the model from learning unrealistic molecular conformations, we adjust the reward by penalizing it with this energy term: $r^{*}({\mathcal{M}},{\mathcal{P}})=r({\mathcal{M}},{\mathcal{P}})-\lambda r_{
\text{constraint}}({\mathcal{M}},{\mathcal{P}})$, where $\lambda$ is a weighting factor that balances the importance of the physical constraints.

### 3.4 Linear Beta Schedule

Drawing from Equation 4, the parameter $\beta$ serves as a form of regularization, balancing the exploration of high-quality molecules with adherence to the pre-learned prior distribution. During the diffusion sampling process, earlier steps influence the subsequent ones. Moreover, the final few steps are crucial in determining the atoms’ types and positions, which significantly affect the molecules’ properties and make optimization in the later steps more critical. To improve optimization efficiency, we propose a linear beta schedule $\beta_{t}=\frac{t}{T}\beta_{T}$, where $\beta_{T}$ is the beta parameter at the final time step $T$ of the reverse process. This schedule ensures a progressive reduction in the impact of regularization, enhancing alignment with the desired properties as the diffusion process progresses.

## 4 Experiments

Considering the practical demands of the pharmaceutical industry, we implement DecompDpo to address two critical needs: (1) fine-tuning the reference model for molecule generation across various protein families, and (2) optimizing the reference model specifically for targeted protein subpockets.

### 4.1 Experimental Setup

#### Dataset

We followed prior work (Luo et al., ; Peng et al., ; Guan et al., ; ) in using the CrossDocked2020 dataset (Francoeur et al., ) to pre-train our base model and evaluate the performance of DecompDpo. According to the protocol established by Luo et al. (), we filtered complexes to retain only those with high-quality docking poses (RMSD $<1$Å) and diverse protein sequences (sequence identity $<30\%$), resulting in a refined dataset comprising 100,000 high-quality training complexes and 100 novel proteins for evaluation.

To fine-tune with DecompDpo, we sample $10$ molecules for each protein in the training dataset using pre-trained base model. The favorability of each molecule was evaluated based on a multi-objective score defined as $r_{\text{multi}}=\sum_{x_{i}\in X}x_{i}$, where $X$ denotes the set of normalized optimization objectives. For each protein, we select the molecules with the highest and lowest scores to form preference pairs for the fine-tuning process, resulting in 63,092 valid pairs. For molecular optimization, we sample 500 molecules for each target protein with fine-tuned model in the test dataset and and construct preference pairs from the top 100 and bottom 100 molecules based on their scores.

#### Baselines

To assess the capability of DecompDpo fine-tuned model in generating high-quality molecules across various protein families, we compare it with several representative generative models. liGAN (Ragoza et al., ) employs a CNN-based variational autoencoder to encode both ligand and receptor into a latent space, subsequently generating atomic densities for ligands. Atom-based autoregressive models such as AR (Luo et al., ), Pocket2Mol (Peng et al., ), and GraphBP (Liu et al., ) update atom embeddings using a graph neural network (GNN). TargetDiff (Guan et al., ) and DecompDiff (Guan et al., ) utilize GNN-based diffusion models, the latter innovatively incorporates decomposed priors for predicting atoms’ type, position, and bonds with validity guidance. IPDiff (Huang et al., ) integrates the interactions between pockets and ligands into both the diffusion forward and sampling processes. In addition, to evaluate the molecular optimization capabilities of DecompDpo, we compared it with two strong optimization methods. RGA (Fu et al., ), which utilizes a reinforced genetic algorithm to simulate evolutionary processes and optimize a policy network across iterations, and DecompOpt (Zhou et al., ), which leverages decomposed priors to control and optimize conditions in a diffusion model.

#### Evaluation

Following the methodology outlined by Guan et al. (), we evaluate molecules from two aspects: target binding affinity and molecular properties, and molecular conformation. Following the established protocol from previous studies (Luo et al., ; Ragoza et al., ), we use AutoDock Vina to assess target binding affinity. Vina Score quantifies the direct binding affinity between a molecule and the target protein, Vina Min measures the affinity after local structural optimization via force fields, Vina Dock assesses the affinity after re-docking the ligand into the target protein, and High Affinity measures the proportion of generated molecules with a Vina Dock score higher than that of reference ligands. Regarding molecular properties, we calculate drug-likeness (QED) (Bickerton et al., ), synthetic accessibility (SA) (Ertl & Schuffenhauer, ), and diversity. Following Jin et al. (); Xie et al. (), the overall quality of generated molecules is evaluated by Success Rate (QED $>$ 0.25, SA $>$ 0.59, Vina Dock $<$ -8.18). To evaluate molecular conformation, Jensen-Shannon divergence (JSD) is employed to compare the atom distributions of the generated molecules with those of reference ligands. We also evaluate median RMSD and energy difference of rigid fragments and the whole molecule before and after optimizing molecular conformations with Merck Molecular Force Field (MMFF) (Halgren, ).

#### Implementation Details

The bond-first noise schedule proposed by Peng et al. () effectively addresses the inconsistency between atoms and bonds when using predicted bonds for molecule reconstruction. We adapt this noise schedule for DecompDiff, resulting in an enhanced model that we used as our base model, termed as DecompDiff*. Details about the bond-first noise schedule are provided in Section C.2. For multi-objective optimization, we select QED, SA, and Vina Minimize Score as the optimization objectives. During fine-tuning, we assign a weight of 1 to each objective, and for molecular optimization, we weighted each objective by the reciprocal of the distance from the current objectives’ mean to the success rate threshold. Please refer to Appendix C for more details.

Table: Table 1: Summary of different properties of reference molecules and molecules generated by DecompDpo and other generative models. ($\uparrow$) / ($\downarrow$) denotes a larger / smaller number is better. Top 2 results are highlighted with bold text and underlined text, respectively.

Figure: Figure 3: Visualization of reference binding ligands and the molecule generated by DecompDiff* and DecompDpo on protein 4D7O (left) and 1UMD (right).
Refer to caption: x3.png

### 4.2 Main Results

#### Molecule Generation

We evaluate the effectiveness of DecompDpo in terms of target binding affinity and molecular properties. As shown in Table 1, after a single epoch of fine-tuning with DecompDpo, the performance is significantly improved across all metrics, demonstrating the effectiveness of DecompDpo in multi-objective optimization. Notably, DecompDpo achieves the highest score in Vina Minimize, Vina Dock, High Affinity, and Success Rate among all generative methods, and also improves other metrics compared to the base model, indicating its superior ability to generate high-quality molecules across various target proteins. Figure 3 shows reference ligands and molecules generated by DecompDiff* and DecompDpo. As shown, molecules generated by DecompDpo achieve better performance while maintaining desired molecular conformations. More visualize results are provided in Appendix D.

Regarding molecular conformation, we plot the all-atom pairwise distance distribution of generated molecules and compute the JSD with the distribution obtained from reference ligands. As shown in Figure 4, DecompDpo has performance comparable to DecompDiff*, achieving the lowest JSD relative to the distribution of reference molecules among all generative models. We also calculate the JSD of bond distance and bond angle distributions, observing that DecompDpo does not significantly compromise molecular conformation while achieving superior optimization results. These results are reported in Appendix D. To further evaluate molecular conformation, we calculate the median energy difference before and after conformation optimization by MMFF for rigid fragments that do not contain rotatable bonds. As shown in Figure 4, DecompDpo performs comparably to DecompDiff* when with fewer rotatable bonds and achieves the lowest energy differences among all generative methods when with more rotatable bonds. These results indicate that DecompDpo generates more reasonable conformations with lower energy while optimizing towards desired properties. Results of the median RMSD differences for rigid fragments and whole molecules are provided in Appendix D.

Table: Table 2: Summary of different properties of reference molecules and molecules generated by DecompDpo and other optimization methods. ($\uparrow$) / ($\downarrow$) denotes a larger / smaller number is better. Top 2 results are highlighted with bold text and underlined text, respectively.

Figure: Figure 4: Compare pairwise distance distributions between all atoms in generated molecules and reference molecules from the test set. Jensen-Shannon divergence (JSD) between two distributions is reported (left). Median energy difference for rigid fragments of generated molecules before and after optimizing with the Merck Molecular Force Field (right).
Refer to caption: x4.png

#### Molecule Optimization

To validate the capability of DecompDpo in molecular optimization, we perform iterative DPO, optimizing the DecompDpo fine-tuned model for each target protein in the test set. As shown in Table 2, DecompDpo achieves the highest scores in affinity-related metrics among all optimization methods. Compared to DecompOpt, DecompDpo achieves a comparable Success Rate, demonstrating its effectiveness in continuously enhancing molecule performance toward a specific protein of interest in practical pharmaceutical applications. Additionally, DecompDpo can be adapted to DecompOpt, potentially providing stronger results by combining the benefits of preference alignment with iterative optimization.

### 4.3 Ablation Studies

#### Single-Objective Optimization

To further validate the effectiveness of DecompDpo, we test its performance in single-objective optimization. As AliDiff (Gu et al., ) aims to align diffusion models with high binding affinity, we select Vina Minimize as the optimization objective and compare the performance of DecompDpo with AliDiff. As shown in Table 4, DecompDpo achieves higher Vina Minimize, Vina Dock, High Affinity, and SA compared to AliDiff.
Notably, while only optimizing towards high Vina Minimize, DecompDpo attains remarkable improvements in molecular properties compared to the base model. Specifically, DecompDpo achieves 5.9%, 11.7%, and 11.8% improvement in Vina Score, Vina Minimize, and Vina Dock, respectively, and improvements of 6.7% and 10% in QED and SA.
Additionally, we computed the Complete Rate, defined as the percentage of valid and connected molecules among all generated molecules. The Complete Rate of DecompDpo is 83.6%, representing a 14.8% improvement compared to the base model. These results indicate that DecompDpo not only enhances the targeted optimization objective but also improves overall molecular quality and validity, demonstrating its effectiveness in single-objective optimization.

#### Benefits of Decomposed Preference

Our primary hypothesis is that introducing decomposition into the optimization objectives enhances training efficiency by providing greater flexibility in preference selection and multi-objective optimization. We verify this hypothesis in the molecule generation setting by fine-tuning the base model with molecule-level preference pairs for all optimization objectives, which we term Molecule-level DPO. As shown in Table 4, DecompDpo outperforms Molecule-level DPO on most of the affinity-related metrics and achieves a higher Success Rate, validating that decomposed preference enhances optimization effectiveness and efficiency. Besides, DecompDpo achieves a higher SA, indicating that it potentially mitigates conflicts in multi-objective optimization by providing greater flexibility and diversity in preference selection.

#### Benefits of Linear Beta Schedule

To validate the effectiveness of the linear beta schedule proposed in Section 3.4, we evaluate the performance of DecompDpo when the value of $\beta$ remains constant in molecule generation setting. As shown in Table 4, employing the linear beta schedule improves all metrics, with only a negligible decrease in Diversity, indicating its effectiveness in enhancing optimization efficiency.

Table: Table 3: Summary of results of single-objective optimization for affinity-related metrics. ($\uparrow$) / ($\downarrow$) denotes a larger / smaller number is better. The best result is highlighted with bold text.

Table: Table 4: Ablation study of decomposing DPO loss and linear beta schedule. ($\uparrow$) / ($\downarrow$) denotes a larger / smaller number is better. The best result is highlighted with bold text.

## 5 Conclusion

In this work, we introduced preference alignment to SBDD for the first time, developing DecompDpo to align pre-trained diffusion models with multi-granularity preference, which provides more flexibility during the optimization process. The physics-informed energy term penalizing the reward is beneficial for maintaining reasonable molecular conformations during optimization. The linear beta schedule effectively improves optimization efficiency by progressively reducing regularization during the diffusion process. DecompDpo shows promising results in molecule generation and molecular optimization, highlighting its ability to meet practical needs of the pharmaceutical industry.

## Appendix A Impact Statements

Our contributions to structure-based drug design have the potential to significantly accelerate the drug discovery process, thereby transforming the pharmaceutical research landscape. Furthermore, the versatility of our approach allows for its application in other domains of computer-aided design, including, but not limited to, protein design, material design, and chip design. While the potential impacts are ample, we underscore the importance of implementing our methods responsibly to prevent misuse and potential harm. Hence, diligent oversight and ethical considerations remain paramount in ensuring the beneficial utilization of our techniques.

## Appendix B Limitations

While we demonstrate that DecompDpo excels in improving models in terms of several prevalently recognized properties of molecules. A more comprehensive optimization objectives properties still require attention. Besides, we simply combined the various objectives into a single one by using a weighted sum loss, without investigating the optimal approach for multi-objective optimization. Extending the applicability of DecompDpo to more practical scenarios is reserved for our future work.

## Appendix C Implementation Details

### C.1 Featurization

Following DecompDiff (Guan et al., ), we characterize each protein atom using a set of features: a one-hot indicator of the element type (H, C, N, O, S, Se), a one-hot indicator of the amino acid type to which the atom belongs, a one-dimensional indicator denoting whether the atom belongs to the backbone, and a one-hot indicator specifying the arm/scaffold region. We define the part of proteins that lies within 10Å of any atom of the ligand as pocket. Similarly, a protein atom is assigned to the arm region if it lies within a 10Å radius of any arm; otherwise, it is categorized under the scaffold region. The ligand atom is characterized with a one-hot indicator of element type (C, N, O, F, P, S, Cl) and a one-hot arm/scaffold indicator. The partition of arms and scaffold is predefined by a decomposition algorithm proposed by DecompDiff.

We use two types of message-passing graphs to model the protein-ligand complex: a $k$-nearest neighbors (knn) graph for all atoms (we choose $k=32$ in all experiments) and a fully-connected graph for ligand atoms only. In the knn graph, edge features are obtained from the outer product of the distance embedding and the edge type. The distance embedding is calculated using radial basis functions centered at 20 points between 0Å and 10Å. Edge types are represented by a 4-dimensional one-hot vector, categorizing edges as between ligand atoms, protein atoms, ligand-protein atoms or protein-ligand atoms. For the fully-connected ligand graph, edge features include a one-hot bond type indicator (non-bond, single, double, triple, aromatic) and a feature indicating whether the bonded atoms belong to the same arm or scaffold.

### C.2 Model Details

Our based model used in DecompDpo is the model proposed by Guan et al. (), incorporating the bond first noise schedule presented by Peng et al. (). Specifically, the noise schedule is defined as follows:

$$ $\displaystyle s$ $\displaystyle=\frac{s_{T}-s_{1}}{\text{sigmoid}(-w)-\text{sigmoid}(w)}$ $\displaystyle b$ $\displaystyle=\frac{s_{1}+s_{T}+s}{2}$ $\displaystyle\bar{\alpha}_{t}$ $\displaystyle=s\cdot\text{sigmoid}(-w(2t/T-1))+b$ $$

For atom types, the parameters of noise schedule are set as $s_{1}=0.9999$, $s_{T}=0.0001$, $w=3$. For bond types, a two-stage noise schedule is employed: in the initial stage ($t\in[1,600]$), bonds are rapidly diffused with parameters $s_{1}=0.9999$, $s_{T}=0.001$, $w=3$. In the subsequent stage ($t\in[600,1000]$), the parameters are set as $s_{1}=0.001$, $s_{T}=0.0001$, $w=2$. The schedules of atom and bond type are shown in Figure 5.

Figure: Figure 5: Noise schedule of atom and bond types.
Refer to caption: x5.png

### C.3 Training Details

#### Pre-training

We use Adam (Kingma & Ba, ) for pre-training, with init_learning_rate=0.0004 and betas=(0.95,0.999). The learning rate is scheduled to decay exponentially with a factor of 0.6 with minimize_learning_rate=1e-6. The learning rate is decayed if there is no improvement for the validation loss in 10 consecutive evaluations. We set batch_size=8 and clip_gradient_norm=8. During training, a small Gaussian noise with a standard deviation of 0.1 to protein atom positions is added as data augmentation. To balance the magnitude of different losses, the reconstruction losses of atom and bond type are multiplied with weights $\gamma_{v}=100$ and $\gamma_{b}=100$, separately. We perform evaluations for every 2000 training steps. The model is pre-trained on a single NVIDIA A6000 GPU, and it could converge within 21 hours and 170k steps.

#### Fine-tuning and Optimizing

For both fine-tuning and optimizing model with DecompDpo, we use the Adam optimizer with init_learning_rate=1e-6 and betas=(0.95,0.999). We maintain a constant learning rate throughout both processes. We set batch_size=4 and clip_gradient_norm=8. Consistent with pre-training, Gaussian noise is added to protein atom positions, and we use a weighted reconstruction loss. For fine-tuning model for molecule generation, we set $\beta_{T}=0.001$ and trained for 1 epoch, 16k steps on one NVIDIA A40 GPU. For molecular optimization, we set $\beta_{T}=0.02$ and trained for 20,000 steps on one NVIDIA V100 GPU, and perform evaluation every every 1,000 steps.

### C.4 Experiment Details

The scoring function for selecting training molecules is defined as $S=QED+SA+Vina_Min/(-12)$. Vina Minimize Score is divided by -12 to ensure that it is generally ranges between 0 and 1. For molecule generation, we exclude molecules that cannot be decomposed or reconstructed, resulting in a total of 63,092 preference pairs available for fine-tuning. In molecular optimization, to ensure that the model maintains a desirable completion rate, we include an additional 50 molecules that failed in reconstruction in as the losing side of preference pairs. To tailor the optimization to a specific protein, the weights of the optimization objectives are defined as $w_{x}=e^{-(x-x_{s})}$, where $x$ is the mean property of the generated molecules and $x_{s}$ is the threshold of the property used in Success Rate. For both molecule generation and molecular optimization, we employ the same Opt Prior used in DecompDiff. The $\lambda$ used for penalizing rewards with energy terms proposed in Section 3.3 is set to 0.1.

In evaluating the performance of DecompDpo, for each checkpoint, we generate 100 molecules for the molecule generation task and 20 molecules for the molecular optimization task across each target protein in the test set. For both molecule generation and optimization, we select the checkpoint with the highest weighted Success Rate, which is defined as the product of the Success Rate and the Complete Rate.

## Appendix D Additional Results

### D.1 Full Evaluation Results

#### Molecular Conformation

To provide a more comprehensive evaluation of molecular conformations, we compute the JSD of distances for different types of bonds and angles between molecules from generative models and reference molecules. As shown in Table 5 and Table 6, DecompDpo achieves the lowest or second lowest JSD for bond types such as ’C=O’, ’C-N’, and ’C-O’, and for angle types such as ’OPO’, ’NCC’, and ’CC=O’. For other types of bonds and angles, the JSD generally remains similar to that of DecompDiff, demonstrating that DecompDpo generally maintains desirable molecular conformations during preference alignment.

Table: Table 5: Jensen-Shannon Divergence of the bond distance distribution between the generated molecules and the reference molecule by bond type, with a lower value indicating better. “-”, “=”, and “:” represent single, double, and aromatic bonds, respectively. The top 2 results are highlighted with bold text and underlined text.

Table: Table 6: Jensen-Shannon Divergence of the bond angle distribution between the generated molecules and the reference molecule by angle type, with a lower value indicating better. The top 2 results are highlighted with bold text and underlined text.

We also evaluate the median RMSD of rigid fragments before and after optimizing molecular conformations with MMFF. As shown in Figure 6, DecompDpo consistently achieves lower RMSD differences than the base model, DecompDiff, across all fragment sizes. The median RMSD and energy differences for whole molecules are presented in Figure 7 and Figure 8, respectively. Generally, DecompDpo achieves comparable or even better results than DecompDiff, indicating that it can generate molecular conformations with lower energy without explicitly optimizing for it.

#### Molecular Properties

To provide a comprehensive evaluation, we have expanded our evaluation metrics beyond those discussed in Section 4.1, which primarily focus on molecular properties and binding affinities. To assess the model’s efficacy in designing novel and valid molecules, we calculate the following additional metrics:

- •
Complete Rate is the percentage of generated molecules that are connected and vaild, which is defined by RDKit.
- •
Novelty is defined as the ratio of generated molecules that are different from the reference ligand of the corresponding pocket in the test set.
- •
Similarity is the Tanimoto Similarity between generated molecules and the corresponding reference ligand.
- •
Uniqueness is the proportion of unique molecules among generated molecules.

Figure: Figure 6: Median RMSD for rigid fragments of generated molecules before and after optimizing with the Merck Molecular Force Field
Refer to caption: x6.png

Figure: Figure 7: Median RMSD of generated molecules before and after optimizing with the Merck Molecular Force Field
Refer to caption: x7.png

Figure: Figure 8: Median energy difference of generated molecules before and after optimizing with the Merck Molecular Force Field
Refer to caption: x8.png

As reported in Table 7, in molecule generation, DecompDpo fine-tuned model achieves better Complete Rate and Similarity compared to the base model. In molecular optimization, DecompDpo maintains a relatively acceptable Complete Rate and the lowest similarity among all optimization methods.

Table: Table 7: Summary of the models’ ability in designing novel and valid molecules. ($\uparrow$) / ($\downarrow$) denotes a larger / smaller number is better.

Figure: Figure 9: Boxplots of QED, SA, Vina Score, Vina Minimize, and Vina Dock of molecules generated by DecompDpo and other generative models.
Refer to caption: x9.png

We also draw boxplots to provide confidence intervals for the performance in molecule generation, which are shown in Figure 9.

### D.2 Evidence for Decomposability of Properties

As illustrated in Section 3.2, QED and SA are non-decomposable due to the non-linear processes involved in their calculations. We validate this non-decomposability on our training set. As shown in Figure 10, the Pearson correlation coefficients between the properties of molecules and the sum of the properties of their decomposed substructures are very low, not exceeding 0.1. These results indicate that substructures with higher QED or SA do not necessarily lead the molecule to have better properties. Therefore, we choose molecule-level preferences for QED and SA.

Figure: Figure 10: The Pearson correlation between molecule’s and sum of substructure’s SA (left) / QED (right) Scores in the training dataset.
Refer to caption: x10.png

### D.3 Trade-off in Multi-objective Optimization

Given the multiple objectives in DecompDpo, inherent trade-offs between different properties are unavoidable. In molecular optimization, as illustrated in Figure 11, molecules generated by DecompDpo exhibit significantly improved properties compared to those generated by DecompDiff. However, DecompDpo encounters a notable trade-off between optimizing the Vina Minimize Score and SA.

Figure: Figure 11: Pairplots of molecules’ properties before and after using DecompDpo for molecular optimization on protein 4Z2G (left) and 2HCJ (right).
Refer to caption: x11.png

### D.4 Examples of Generated Molecules

Examples of reference ligands and molecules generated by DecompDiff* and DecompDpo, which are shown in Figure 12.

Figure: Figure 12: Additional Examples of reference binding ligands and the molecule with the highest property among all generated molecules of DecompDiff* and DecompDpo on protein 1GG5 (left) and 3TYM (right).
Refer to caption: x12.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Figure 4 (right): what does fragment size on x-axis mean? (Average?) number of atoms in the fragments? Or (average?) number of fragments in the molecules?
2. How are the fragments without rotatable bonds computed (in the section about MMFF evaluation)?
3. It would be interesting to look at the distributions of the scores in winning and loosing examples in the created dataset for preference alignment.
4. Do I understand correctly that linear $\beta$-schedule means that in cases when t is large enough (i.e., closer to the beginning of the denoising trajectory) the DPO loss is scaled by a large number? And does it mean that effectively the model learns to align only the later stages of the denoising process?
5. Did authors try to use (11) and (12) as penalties while training the diffusion model itself? Looks like it is a nice additional penalty in general
6. How do authors select the size of the sampled molecules? Is the distribution of sizes in samples the same with and without DPO?
soundness: 1
strengths: I find the proposed idea on the local DPO for molecular fragments very interesting and original and can potentially help in specific tasks. I also like the idea about constrained optimisation and believe it can be used in diffusion models in general to imporve the quality of the molecules. Overall, applying DPO to improve specific properties of the generated molecule makes total sense and can help generate high-quality molecules in cases when the quality of the training data is not sufficiently high.
summary: The authors propose to apply DPO to structure-based drug design. Authors formulate the DPO objective for diffusion models and suggest to apply it separately to different molecular fragments. Besides, authors propose to use the linear scheduling of the DPO regularization. Finally, authors incorporate physically-constrained optimisation to improve the geometry of the generated molecules. Overall, in my opinion, the work has interesting ideas and suggestions. However, the chosen evaluation methodology and the current results do not prove that the proposed techniques are beneficial at all. I discuss the weaknesses in details below, and believe there is a substantial scope for improvement. I will be happy to reconsider my decision after the rebuttal.
weaknesses: **Major issues:**
1. LocalDPO looks like the main contribution of this paper. However, the evaluation of this component does not prove that the proposed mechanism is crucial for sampling molecules with optimised properties. In fact, based on Table 4, it looks like the model without LocalDPO solves the task equally well (the differences in the scores are too small). I can imagine that the chosen evaluation method does not highlight the advantages of the LocalDPO and its unique feature – locality. So my main question for which I struggle to find an answer in the paper: why would I use LocalDPO instead of the global one?

2. The same applies to the physical constrained optimisation. Tables 5 and 6 suggest that DPO doesn't improve the geometry of the molecules.

3. I am also confused about utilisation of different objectives in LocalDPO and GlobalDPO. In the implementation details, authors mention that they use QED, SA, and Vina Minimize Score. Does it apply to both local and global DPO components? If yes, then I do not understand the point of LocalDPO since neither QED nor SA are additive by fragments. If not, then all the main results concern only Vina optimisation, and I would recommend to make it more clear.

4. I am not entirely convinced by the claim regarding the improvement of molecular conformers in terms of energy (lines 428 - 458). First, in my opinion, the difference between distributions of values of DecompDiff and DecompDPO in Figures 4 (right), 5 and 6 is too small to say that DecompDPO is better. Second, why would it be better? The only geometric properties the model is optimized for are bond distances and bond angles. But I believe that having "correct" (i.e. within the distribution of the real data) bond distances and angles is only the necessary condition for the low conformational energy, but not sufficient. Besides, it is clear from Tables 5 and 6 that DecompDPO doesn't outperform DecompDiff in this regard. Therefore, the argument for improved molecular conformation feels unsubstantiated.

5. Single objective optimisation section (and Table 3) confuses me a lot:
* 5.1 Why do authors discuss improvements of QED and SA of the model that was optimized for affinity score only? How can these improvements be explained?
* 5.2 Why not to optimise separately for SA and for QED and then show how these scores improve after the corresponding optimisation? I believe that it is important to demonstrate the consistent ability to optimise different unrelated scores in separate experiments.
* 5.3 How does connectivity and validity depend on the fact that the model is optimised for Vina scores? To me this effect looks arbitrary and doesn't have any rationale behind (similar to what I discussed in 4.1). I agree that it is a good idea to discuss validity and connectivity, but I would suggest to do it in a different context – as a sanity check that demonstrates that the molecules generated by DecompDPO are in general adequate (for example, as an Appendix table).

**Minor comments and suggestions:**
1. Figure 4 (left) is not very informative – in my opinion it can be put to the appendix.
2. For clarity of the made arguments and to improve the overall structure of the paper, I believe that it would be beneficial to have another table in the main text with the key geometric scores: i.e. JS divergence between distances (values from the Figure 4 (left) legend), JS between angles, avg. conformer energies. At least, in my opinion, some comparison of angles should be in the main text, since authors provide equation (12).
3. Maybe we cannot see the DPO effects on bond distances and angles because $3\sigma$ (in equations 11 and 12) is a very loose threshold? What if to try just $\sigma$?
4. Line 472: Table 4 -> Table 3
5. Docking evaluation:
* 5.1. I believe that reporting scores after minimization or docking may not fully align with the goal of the proposed model, which is to generate 3D molecular structures that inherently adopt good binding conformations. Rather than presenting redocked and minimized structures, I would suggest evaluating the number of steric clashes between proteins and ligands, as this could provide a more direct assessment of the generated conformations’ suitability for binding.
 * 5.2. Due to the high correlation between the docking scores and molecule sizes, I would recommend reporting binding efficiency scores (i.e. docking score divided by the number of heavy atoms). I believe it provides a better signal of the potency of the generated molecules.
6. I would be curious to look at the validity and connectivity metrics - as an overall quality assessment of the DecompDPO molecules. Also would interesting to see the percentage of the samples that pass PoseBusters filters.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Could the authors please provide the complete rates for the DecompDPO and DeompDiff baseline molecules? When running AutoDock Vina, in 100 samples for a pocket, only the successfully reconstructed molecules (usually fewer than 100 and sometimes around 50–60/100 for larger pockets) proceed to the vina docking process. This means the vina results in the table for all methods are actually calculated from fewer than 10,000 samples. It would be helpful to benchmark the method using this complete rate alongside the vina results. Thanks!
soundness: 3
strengths: Thanks for the good work on exploring the DPO method in decomposed diffusion for SBDD. I found the work to have the following strengths:
1) The method achieved decent performance on vina score, vina min, and vina dock without significantly compromising on QED and SA scores compared with the baselines selected in the paper.
2) The paper is clearly written, easy to follow, and has well-designed figures.
3) The paper introduces the use of Local DPO for optimization, which goes beyond global DPO.
summary: In this paper, the authors propose DecompDPO, a method that applies Direct Preference Optimization (DPO) in a diffusion model for structure-based drug design (SBDD). In addition to the global DPO loss, which focuses on binding affinity as evaluated by AutoDock Vina, the paper introduces local DPO and a bond angle penalty to complement global DPO, aiming to improve ligand-protein binding affinity. DecompDPO achieves competitive performance on vina results compared to other selected baselines.
weaknesses: 1. Performance claims in the paper

DecompDPO did outperform all selected baselines in terms of vina score, vina min, and vina dock, as mentioned in the paper. However, the paper did not benchmark against some other existing works that demonstrate better performance in improving the conditional sampling of diffusion models for SBDD. For example, KGDiff[1], a guidance-based method, has better performance on vina score, vina min, and vina dock. Both KGDiff and DecompDPO were benchmarked on the same test set with 100 test pockets and 100 samples per pocket. While I understand that vina results are not a precise measure of binding affinity and that not being SOTA in vina results does not necessarily mean the method is not good, I would still be cautious in stating that "DECOMPDPO achieves the highest score in Vina Minimize, Vina Dock, High Affinity, and Success Rate among all generative methods" in the paper.

2. Performance on Vina Score

DecompDPO shows good performance on vina min and vina dock, but the marginal improvement on vina score is small. Vina score directly reflects the quality of the sampled pose, while vina dock evaluates the quality of the pose sampled by the model and then optimized by AutoDock Vina. The results in the paper indicate that the improvement in the pose directly sampled from the generative model is minimal.

3. Effectiveness of Local DPO

Although local DPO is a novel idea compared to global DPO (e.g., AliDiff), Table 4 shows that the improvement brought by local DPO is marginal compared to global DPO. This raises questions about the effectiveness and real contribution of local DPO toward the overall improvement of local+global DPO.

Reference:
[1] KGDiff: towards explainable target-aware molecule generation with knowledge guidance

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: - Why is the model only fine-tuned for one epoch? How does longer training affect the target metrics and other molecular properties?

- Is there a trade-off between the target metrics and other molecular properties? Does the general molecular quality degrade when the reward is optimized more aggressively? The PoseBusters [5] filters are widely accepted criteria to perform general sanity checks for molecules, and could be used to answer this question.

- What molecular fragmentation method is used? I could not find this important methodological detail in the paper.

- I would advise to add an ablation study to systematically evaluate the contribution of the physics-informed reward term.

- Optionally, a more extensive evaluation could also include methods like REINVENT [6] or other reinforcement learning/GFlowNets-based approaches that typically do not model molecules explicitly in 3D but can still optimize binding oracles implicitly and are usually strong baselines for molecule optimization tasks.



### Minor comments
- Figure 1: all symbols should be explained in the Figure legend
- lines 177-178: all symbols need to be defined ($\mu_{1:K}$, $\Sigma_{1:K}$, $H$); it would also be useful to describe the data-dependent priors in more detail
- Section 3.1: this section introduces loss components $L_t$ per time step but, in my opinion, should also provide the overall loss function that involves sampling and aggregation of time steps
- Figure 2: font size of axis labels should be increased
- Table 1: in my opinion, the most interesting comparison is the one between the base model, DecompDiff*, and the fine-tuned model to assess the effect of the proposed preference alignment approach. It would be easier to compare the metrics if these two models were presented right below one another in the table.
- When discussing baselines in Section 4.1, it would be interesting to know what rewards RGA and DecompOpt are optimizing for. This would help put the results discussed later into perspective.
- Line 472: the referenced table should be _Table 3_
- Equation 9: sign$(x>y)$ is quite unconventional. I recommend to use sign$(x-y)$ instead
- Equations 8 and 9: the definition of $M_t^+$ and $M_t^-$ as the union of $M^{(i)}$'s can be moved out of the expectation index


### References

[5] Buttenschoen, Martin, Garrett M. Morris, and Charlotte M. Deane. "PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences." Chemical Science 15.9 (2024): 3130-3139.

[6] Loeffler, Hannes H., et al. "Reinvent 4: Modern AI–driven generative molecule design." Journal of Cheminformatics 16.1 (2024): 20.
soundness: 2
strengths: ### Motivation & ideation

I believe user preference-guided optimization of generated molecules is an important frontier in structure-based drug design. The presented work is a promising extension of the capabilities of recent machine learning models for this task. It builds on various ideas established in prior works on decomposed diffusion models [1], decomposed molecule optimization [2], and preference optimization for molecules [3], and is a sensible extension thereof. Applying the diffusion DPO loss at a substructure level for decomposable objectives makes sense and likely leads to more efficient training. Some benefits of this approach are also shown empirically in this study (Table 4).
Furthermore, to the best of my knowledge the paper represents one of the first applications of DPO to small molecule design (together with Ref. [4]) and thus provides a novel view on the drug design problem.


### Experimental setup

The chosen experimental setup looks reasonable to me and includes useful comparisons and baselines.
It also already contains most relevant ablation studies, including local (substructure) DPO vs purely global DPO and the proposed schedule for the regularization parameter $\beta$.
The presented results are consistent with the underlying hypotheses and support the proposed preference alignment strategy for molecular diffusion models.


### References

[1] Guan, Jiaqi, et al. "DecompDiff: diffusion models with decomposed priors for structure-based drug design." arXiv preprint arXiv:2403.07902 (2024).

[2] Zhou, Xiangxin, et al. "DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization." arXiv preprint arXiv:2403.13829 (2024).

[3] Zhou, Xiangxin, et al. "Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization." arXiv preprint arXiv:2403.16576 (2024).

[4] Gu, Siyi, et al. "Aligning target-aware molecule diffusion models with exact energy optimization." arXiv preprint arXiv:2407.01648 (2024).
summary: This paper presents a new diffusion model for structure-based drug design that improves molecular properties (docking score, drug-likeness score, and synthesizability score) of a base model using Direct Preference Optimization (DPO). In addition to straightforward application of diffusion-DPO on the molecular level, the authors propose to apply a more fine-grained version at the fragment level to further improve their results. 
The method's performance is assessed in a multi-objective fine-tuning experiment as well as a target-specific optimization setup.
weaknesses: ### Empirical results

While the conceptual arguments for the proposed techniques are strong and well-motivated, the empirical results are currently less convincing. The performance improvement compared to the base model is only moderate in most reported metrics (e.g. QED and SA scores in Table 1). Here, it would be useful to put these results into perspective somehow. The authors could, for example, discuss potential ways to increase the performance gap between base model and optimized model. It would be natural to ask if more DPO fine-tuning iterations/epochs can further increase the optimized scores. If that is not the case, it would at least be interesting to know why.

Moreover, when directly optimizing a limited number of target metrics, it is easy to "break things". The evaluation should therefore include additional sanity checks for the generated molecules. The reported _Complete Rate_ is a good first step but a larger set of diverse metrics and checks (e.g. PoseBusters filters as mentioned in the questions below) would increase my confidence in the findings.

### Baselines

The main benchmark in Table 1 contains a representative set of popular structure-based drug design models. However, I would consider these baselines rather weak in this context as most of them were trained to reproduce the training data distribution whereas DecompDPO directly aims to optimize Vina score, QED and SA. More relevant baselines are only considered in the _Molecule Optimization_ section.
Maybe the authors could consider including the same or similar baselines methods in Table 1 as well.

Given the arguably low bar, I find it surprising that DecompDPO does not show larger improvements compared to the presented baseline methods. To give a concrete example, by simply matching the SA values from the dataset (_Reference_) Pocket2Mol seems to achieve better SA scores than DecompDPO which includes SA in its preference alignment scheme.


### Ablation studies

The paper already investigates the impact of many design choices. However, I would also like to see how longer DPO training affects the results. It is currently unclear why the models are only fine-tuned for one epoch. Would longer training improve results or only degrade other molecular properties?
The paper also introduces a physics-inspired constraint on bond and angle geometries and makes claims such as "The physics-informed energy term penalizing the reward is beneficial for maintaining reasonable molecular conformations during optimization" (Conclusion section), but no empirical evidence is provided.

The ablation of the multi-objective optimization approach produced another unexpected result. QED and SA values in the single-objective case (only optimizing for Vina score) are as good or even better than in the multi-objective setup. 
It would be good to see an attempt at explaining this outcome and/or a discussion of the implications for the multi-objective approach. 

### Discussion

Given some of the surprising results stated above, I believe the paper would benefit from a more in-depth discussion of its empirical findings. It would certainly help to better understand the current limitations of the method.

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 2
questions: For most questions, see weaknesses.

- DecomDPO performs iterative DPO, optimizing the model for each target protein in the test set. Does this mean a new model is trained for each test set pocket? If so, what happens if you only do standard DPO fine-tuning as done in AliDiff?

- On line 350 it appears to describe that DecompDPO samples 500 molecules per 100 test pocket for table 1. Do the benchmarks reflect the average over all 500 or is it some filtered subset? 

- What is the inference time, and how does it compare to prior methods factoring in the complete rate?

- What happens if you fine-tune TargetDiff on the DPO fine-tuning dataset as done in AliDiff? It would be interesting to see of the decomposed objective yields better training data.

- Where are results showing fine-tuning pretrained diffusion models for molecule generation across various protein families (line 21-22)? There is no notion of protein families discussed in existing CrossDocked2020 benchmarks.

Nit: Line 77 says DecompDPO was the first, but line 80 says AliDiff was the first to use preference alignment.
soundness: 2
strengths: Overall, DecompDPO shows a well-done approach to taking complex decomposed molecule diffusion models and optimizing them for easy-to-calculate chemical properties. Many components of the DPO optimization are specific to the way DecompDiff frames the arms and scaffolds of molecule generation. The work in Appendix D.2 is really interesting and would suggest being a primary focus of the paper as this is where the most novelty as it pertains the DPO setup is.

- DecompDPO shows 8% improvement over base model DecompDiff due to DPO fine-tuning with a single epoch of  63K pairs with a pretraining dataset of 100K.
- DecompDPO brings in several innovations from prior methods, including DecompDiff and the bond-first noise schedule, into a single model.
- DecompDPO adds structural queues from bond distances and angles to derive the DPO reward
- From Table 2, it is clear that fine-tuning for multi-objective preferences improves base unaltered predicted structure compared to prior optimization methods on top of DecompDiff.
summary: DecompDPO introduces substructure decomposition into the Direct Preference Optimization of a diffusion model for structure-based drug design. It has two primary uses: fine-tuning diffusion models across various protein families and secondary optimization post-generation. DecompDPO also directly aligns itself with preferences using two forms of DPO: Global over molecule pairs and Local over substructures. DecompDPO was benchmarked on existing CrossDocked2020 benchmarks.
weaknesses: Overall, the paper combines many techniques from prior works, but the benchmarking is quite sparse regarding critical ablations. Many architectural and DPO-specific optimization choices are not ablated, although the proposed novelty is such. As a result, it's hard to understand where the method works and why.

- Relevant prior work [1] is not included, which discusses [3] and CrossDocked Benchmark implications and diffusion multi-objective optimization. 
- Vina Dock, as it randomizes all torsions to start, has little bearing on the strength of the structure aspect of the generative model. 
  - [2] demonstrated that an LLM without 3D information can achieve a success rate of 86.4% and an average Vina Dock of -10.27 kcal/mol by optimizing for docking score directly 25.6x faster than DecompOpt.
  -  The molecular properties like QED and SA are also improved by leveraging a pretrained LLM that only guides Vina Dock, whereas the property-aware optimization of DecompDPO does not yield significant gains.
-  The ground truth CrossDocked2020 data was generated with SMINA docking, and as a result, most of the cross-docked ligands (i.e., non, co-crystal) do not exhibit the desired protein-ligand interaction. As a result, benchmarks such as PoseCheck [3] allow for a better but still imperfect comparison but are not included.
- Given that the structures generated from the AliDiff model have better affinity according to the Vina Score, the fact that DecompDPO structures could be optimized with external force fields or dynamic torsion sampling is not all that valuable compared to non-diffusion methods. 
   - **Q:** Why does the multi-objective DPO fair worse than the energy-focused version of AliDiff? Could the regressions stem from the IPDiff vs DecompDIff underlying architecture and training? A deeper study of the underlying DPO method agnostic to the architecture would be very interesting as unless one is to use DecompDiff can someone use DecompDPO?
  -  If Vina Dock, QED, and SA score are desired, there are prior methods that are much faster to achieve that [2]. 
- The complete rate is the lowest of all tested methods, which is the first and most important benchmark as it shows how well the model can function beyond whether the molecules are good or not.
  -  **Q:**  Is there a deeper analysis as to why 35% of the generated molecules are not connected and/or valid? 
  - This seems unexpected as the bond diffusion schedule that was used was specifically chosen to generate connected molecules. 
  - **Q:**  For evaluation, does this mean that of the 100 molecules generated, the benchmarks are calculated over the 65 valid molecules?
- A lot of focus is put on adding energy terms to improve the molecule's conformation, but DecompDPO has worse bond distances than DecompDiff for 5/8 bond types and 6/7 bond angles. 
  - **Q:**  Does this part of the reward add anything of value if it hurts the generated structures?
- Overall, it is clear DecompDPO is better than DecompOpt, but more ablations are needed to understand where the improvements and regressions are coming from.
  - **Q:**  Could a similar DPO scheme be applied to TargetDiff, and would similar regressions occur?

[1] PILOT: Equivariant diffusion for pocket conditioned de novo ligand generation with multi-objective guidance via importance sampling https://arxiv.org/abs/2405.14925

[2] EvoSBDD: Latent Evolution for Accurate and Efficient Structure-Based Drug Design https://openreview.net/pdf?id=sLhUNz0uTz

[3] PoseCheck: Benchmarking Generated Poses: How Rational is Structure-based Drug Design with Generative Models? https://arxiv.org/abs/2308.07413

Meta-review: In this paper, authors propose DecompDPO, a method that applies Direct Preference Optimization (DPO) to diffusion models for structure-based drug design (SBDD). The key technical contributions include: 1) Decomposed preference optimization that operates at both global molecular and local substructure levels, 2) Physics-informed energy terms to maintain reasonable molecular conformations, and 3) A framework that can be used for both fine-tuning pretrained models and post-generation optimization. The authors evaluate their method on the CrossDocked2020 benchmark and report improvements in binding affinity metrics while maintaining drug-likeness properties.

The work has several strengths as follows. The paper addresses an important problem in SBDD by developing preference-guided optimization of generated molecules. The application of DPO to molecular generation is novel and timely. The decomposed approach that operates at different granularities (global vs local) is technically interesting. The authors provide extensive ablation studies and experimental results comparing against relevant baselines. The physics-informed constraints represent a thoughtful addition to maintain molecular validity during optimization.

However, there are several weaknesses for the work as follows. The empirical improvements demonstrated by the method are notably modest, particularly for the core metrics that the approach explicitly aims to optimize. Reviewer HgKr emphasized this point, noting that the multi-objective optimization sometimes performs worse than single-objective optimization, which raises fundamental questions about the effectiveness of the proposed methodology. Reviewer koEE highlighted that the heavily emphasized Vina Dock metric may not be the most meaningful measure, as it involves re-docking that operates independently of the generated 3D structure. The raw Vina Score shows only a marginal improvement of 0.14 kcal/mol over the baseline DecompDiff. The analysis remains incomplete in several crucial aspects - multiple reviewers requested additional experiments and comparisons that were not fully addressed in the rebuttal, including comparisons with 2D methods like TacoGFN and a more thorough investigation of why DPO appears to degrade performance when applied to different architectures like TargetDiff. A significant practical limitation lies in the validity issues - the method achieves lower complete rates compared to some baselines, with up to 35% of generated molecules being invalid or disconnected. While the authors provided some explanation in their rebuttal, this remains a concerning limitation for practical applications. These weaknesses collectively suggest that while the underlying ideas may have merit, the current implementation and analysis fall short of demonstrating a clear advance in the field of structure-based drug design.

While the paper presents interesting ideas and makes a reasonable attempt at improving molecular generation through preference optimization, the empirical results do not convincingly demonstrate the effectiveness of the proposed method. The modest gains in primary metrics, coupled with concerning regressions in some scenarios, suggest the approach may need substantial refinement before it represents a clear advance in the field. The authors' rebuttal, while thorough, did not fully address core methodological concerns raised by reviewers regarding evaluation metrics and architectural choices. The incomplete rates and limited improvement over simpler baselines make it difficult to justify acceptance at this time.

For future submissions, I recommend: 1) More thorough investigation of why multi-objective optimization underperforms single-objective in some cases, 2) Addressing the validity rate issues, 3) Expanding comparisons to include both 2D and 3D baselines while clearly articulating the tradeoffs, and 4) Providing more rigorous analysis of the method's behavior when applied to different architectures.

Decision: Reject