TurboHopp: Accelerated Molecule Scaffold Hopping with Consistency Models
Authors: Kiwoong Yoo
Owen Oertell
Junhyun Lee
Sanghoon Lee
Jaewoo Kang
Venue: NeurIPS 2024 poster (2024)
Keywords: Scaffold Hopping
Consistency Models
Diffusion Models
3D Structure-Based Drug Design
Reinforcement Learning
Drug Discovery
Generative Models
Primary area: machine_learning_for_healthcare
TL;DR: Fast, and efficient E(3)-equivariant scaffold-hopping model utilizing consistency models for rapid generation additionally powered by RL

Abstract: Navigating the vast chemical space of druggable compounds is a formidable challenge in drug discovery, where generative models are increasingly employed to identify viable candidates. Conditional 3D structure-based drug design (3D-SBDD) models, which take into account complex three-dimensional interactions and molecular geometries, are particularly promising. Scaffold hopping is an efficient strategy that facilitates the identification of similar active compounds by strategically modifying the core structure of molecules, effectively narrowing the wide chemical space and enhancing the discovery of drug-like products. However, the practical application of 3D-SBDD generative models is hampered by their slow processing speeds. To address this bottleneck, we introduce TurboHopp, an accelerated pocket-conditioned 3D scaffold hopping model that merges the strategic effectiveness of traditional scaffold hopping with rapid generation capabilities of consistency models. This synergy not only enhances efficiency but also significantly boosts generation speeds, achieving up to 30 times faster inference speed as well as superior generation quality compared to existing diffusion-based models, establishing TurboHopp as a powerful tool in drug discovery. Supported by faster inference speed, we further optimize our model, using Reinforcement Learning for Consistency Models (RLCM), to output desirable molecules. We demonstrate the broad applicability of TurboHopp across multiple drug discovery scenarios, underscoring its potential in diverse molecular settings.The code is provided at https://github.com/orgw/TurboHopp

TurboHopp: Accelerated Molecule Scaffold Hopping with Consistency Models
Kiwoong Yoo!, Owen Oertell", Junhyun Lee", Sanghoon Leel and Jaewoo Kang" 113
AIGEN Sciences
Cornell University
Korea University
kiwoong- soooogggooolggeescieeneesss com, ojo2@cormell.eee edu, 1jhyun33@korea.ac.o ac. sanghoon leeeaagenscienness. com, kangj@kores kc.
Abstract
Navigating the vast chemical space of druggable compounds formidable challenge in drug discovery, where generative models are increasingly employed identify viable candidates. Conditional 3D structure-based drug design (3D- SBDD) models, which take into account complex three-dimensional interactions and molecular geometries, are particularly promising. Scaffold hopping an efficient strategy that facilitates the identification of similar active compounds by strategically modifying the core structure molecules, effectively narrowing the wide chemical space and enhancing the discovery of drug-lik like products. However, the practical application 3D-SBDD generative models is hampered by their slow processing speeds. To address this bottleneck, celerated nocket-conditiones 3D scaffold hopping model that merges the strategic effectiveness of traditional scaffold hopping with rapid generation capabilities of consistency models. This synergy not only enhances efficiency but also signifi- cantly boosts generation speeds, achieving up to 30 times faster inference speed well as superior generation quality compared to existing diffusion-based models, establishing TurboHopp as a powerful tool in drug discovery. Supported by faster inference speed, we further optimize our model, using Reinforcement Learning for Consistency Models (RLCM), output desirable molecules. We demonstrate the broad applicability of TurboHopp across multiple drug discovery scenarios, underscoring potential diverse molecular settings. The code is provided https: / thub. www.orgw/oweec..t.. orgs/TurboHHpp
introduce TurboHopp,
Introduction
The vast chemical space, with up 1063 potential molecules (molecular weight under 500 Da) (Kirkpatrick and Ellis [2004]), presents immense challenges in drug discovery, where the process often extends over decade and incurs costs in the billions. Recent strides in molecular design have introduced advanced generative models that excel in de-novo generation and optimization within 3D molecular space (Isert al. [2023]). Despite these technological advances, challenges persist, such as ensuring synthesizability (Gao et al. [2021], Fialková et al. [2021]) and maintaining desirable drug-like properties. The scarcity of protein-ligand complex data further complicates these demanding issues. While 1D/2D models attempt to address these challenges though various methods (Loeffler et al. [2024]), they often fall short in representing true 3D structures, and typically
"Corresponing author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

necessitating additional steps for proper conformation generation. On the other hand, 3D models (Ragoza al. [2022], Luo et al. [2022], Baillif al. [2023], Peng et al. [2022]), which directly incorporate spatial structures, show greater promise in activity prediction regarding 3D structures but are hindered by significantly slower inference speeds (Baillif et al. [2023]), which limit their practical applications such as fine-tuning via Reinforcement Learning (Ding and Jin [2024], Baillif et al. [2023], Wallace et al. [2023]). Recent research efforts, such as those by Lee and Cho [2024], have started fine-tuning 3 3D-autoreerrsssvve models with specialized reward functions to align with these stringent standards. Yet, notably absent are attempts involving Structure- DMs). as the prolonged inference times remain formidable barrier, precluding the adoption of similar methodologies. In this work, we address this gap by successfully applying reinforcement learning to 3D-SBDD DMs, overcoming the challenge of prolonged inference times and demonstrating the feasibility of such approaches in practical drug design scenarios.
Drug Design
Diffusion Models(3D-SBB))
3D-SSBDD-DMs (Guan et al. [2023, 2024], Schneuing et al. [2022], Torge et al. [2023], Igashov et al. [2024]) exhibit powerful generative capabilities but suffer from slow processing speeds due to their reliance on iterative sampling. Addressing these speed limitations is critical for several reasons. Primarily, faster processing speeds are essential for accelerating hit discovery phases, which could significantly increase the likelihood of identifying quality hits within shorter time period. Additionally, the high computational demands of these models not only escalate costs but also limit access to advanced optimization techniques, thereby restricting the generation of molecules with desirable properties such as synthesizability, high binding affinity, and drug- likeness. Furthermore, real-time applications, such interactive modeling sessions with human experts, require rapid model inference to provide effective feedback and facilitate iterative testing (Sundin et al. [2022]). By enhancing the speed of these models, it may be possible to enable real-time applications, allowing for on-the-fly modifications and adaptations during interactive sessions. Addressing the speed limitations of diffusion models is thus essential to fully realize these objectives and leverage the full potential of 3D-SBDD-DMs in practical applications.
To address slow inference of diffusion models, various strategies have been adopted to enhance their efficiency, in the computer vision domain such as improvement of solvers, optimization of diffusion processes, and model distillation techniques (Cao et al. [2024]). The emergence of consistency models and its variants (Song and Dhariwal [2023], Ding and Jin [2024]. Song et al. [2023]), heralds rapid, efficient alternative capable of delivering high-quality results with significantly reduced computational demands. However, these acceleration attempts have been minimally explored in the context of 3D drug design.
In this work, we present an efficient E33)eqquivariant consistency model that combines the rapid generation capabilities of consistency models with the effectiveness of scaffold hopping (Fig. I). To substantiate the reliability and robustness of model performance, subsequent sections will detail results and comparisons with established diffusion and consistency models. Morcover, incorporate reinforcement learning (RL) techniques (Oertell et al. [2024]) to further refine our model based on objectives that challenging to address through direct modeling alone, demonstrating its capability to surmount limitations encountered by previous 3D-SBDD models (Harris et al. [2023]).
Our contributions are as follows:
We propose target-aware equivariant consistency model tailored for saffold-hoppingg achieving speeds up to 30 times faster than traditional DDPM-based models while achieving higher likenss,,,,ss, synthesizability, connectivity and binding affinity.
By leveraging faster inference speeds, we apply Reinforcement Learning (RL) to enhance binding affinity scores and reduce steric clashes without redocking in 3D-SBDD-a strategy not previously explored with diffusion models. This approach enables targeted optimizations in molecule design and opens up opportunities to tackle metrics that were previously challenging to address through direct modeling.

---

Mid Misvactiviyy
Mid activity
activity
High activity
Low activity
Low activity
a) Diffusion Model+ De Novo generation
b) Consistency Model Scaffold Hopping
Figure 1: (a) Previous diffusion-basee SBDD models methodically explore vast chemical space for pocket-active molecules, with short arrows symbolizing gradual, step-wise inference process. (b) TurboHopp efficiently generates active ligands using a consistency model, which accelerates inference and reduces the number of steps, illustrated by the longer arrows. Moreover, it strategically leverages the functional groups of high-activity reference molecules, shown as colored areas on the diagram, to optimize the exploration within targeted chemical space.
QED
DiffHopp
TurboHopp
0.8
DiffHopp
0.6
TurboHopp
0.4
0.2
200
250
500
100
200
300
400
500
Generation Steps
Generation Steps
Comparison QED over Generation Steps
Progression Generated Outputs Generation Steps
Figure 2: (a) Comparison of QED scores, with TurboHopp (blue) reaching peak values faster compared to DiffHopp (red), throughout the generation process. (b) Progression of generated outputs of both models. Final steps are highlighted with red boxes.
2 Related Works
2.1 Scaffold Hopping
Scaffold hopping key strategy in drug discovery aimed at identifying new compounds that share biological activities but differ structurally from known molecules. This approach is crucial for enhancing drug properties and navigating intellectual property challenges. includes techniques like heterocycle replacements, ring modifications, peptidomimetics for stability, and topology-based hopping, which innovates on molecular shape while maintaining activity (Sun al. [2012]). Deep- learning models, particularly in topology-basee hopping, explore complex molecular structures to predict novel scaffolds, driving forward drug discovery by enabling the design of therapeutics with optimized efficacy and patentability (Torge et al. [2023]).
2.2 curtusion-Basee Molecular Generative Models and Limitations
Diffusion-based molecular generative models (DMs) apply noise processes and their reversal to generate novel molecular structures. These range from de-novo, non-conditional molecule creation to

---

precision drug design within protein pockets, and extend to specific tasks like scaffold hopping and linker generation (Hoogeboom et al. [2022], Vignac et al. [2023], Huang et al. [2022], Guan et al. [2023, 2024], Huang et al. [2024]. Torge et al. [2023], Igashov et al. [2022]).
Despite their broad applicability, these models encounter substantial constraints. First, the iterative reversal of noise through SDEs or ODEs often requires extensive computation, typically hundreds to thousands of iterations, resulting in slow model inference. This significantly constrains the speed at which new, active molecules can be identified, diminishing the practical efficacy of these models in dime-sensitive applications (Baillif al. [2023]). Second, diffusion models often produce molecular poses that fail to comply with key biophysical constraints, as highlighted by recent evaluations (Harris al. [2023]) Guan et al. [2023, 2024] struggle significantly in generating biophysically realistic molecular poses. Their strain energies are notably high, with median values around 1241.7 kcal/mol and 1 kcal/mol, substantially exceeding the preferable baseline of 102.5 kcal/mol from the CrossDocked dataset (Francoeur et al. [2020].. Additionally, both models exhibit considerable issues with protein clashes: Schneuing et al. [2022] shows a high rate of steric clashes with a mean score of 15.33 before redocking, and Guan et al. [2023] also has problematic level, with a mean score of 9.08 prior to redocking. These high levels of steric clashes suggest that the molecules often have conformations that are too cramped or improperly aligned for effective protein binding, highlighting significant challenges in their ability to produce physically and biologically plausible interactions.
2.3 Consistency Models
Initially utilized in the vision domain, consistency models are either developed by distilling knowledge from existing diffusion models or trained as standalone entities. Defined as distinct class of generative models (Song et al. [2023]). they address the computational inefficiencies of traditional diffusion models that require multiple iterative steps to remove noise and generate data. By directly transforming noise into data, consistency models significantly reduce generation time and offer the flexibility of multi-step sampling to improve output quality.
2.4 RL for Consistency Models
Reinforcement Learning for Consistency Models (RLCM) is a framework that enhances the speed and efficacy of training generative models through reinforcement learning (Oertell et al. [2024]). It builds upon the fast inference speed of consistency models, enabling rapid training and inference cycles. RLCM is particularly useful as fine-tunes models optimize for specific reward functions, especially those hard to capture through direct modeling. In the molecule generation domain, to our knowledge there have been no attempts fine-tuning diffusion or consistency models for task-specific purposes yet. By leveraging RLCM, we show promising evidence for mitigating issues in 3D-SBDD DMs (Harris et al. [2023]), and believe that future iteration on the reward functions with our methods will enhance their practical applicability and effectiveness.
3 Methods
3.1 Notations and Problem Definition
Notations Atoms within proteins and ligands are represented by the graph G = (h,æ), x). where h NxF are node features and R-Nx3 are the Cartesian coordinates of the atoms. The atomic elements are encoded as one-hot vectors within h. We consider only those protein pocket atoms that are within an 8 Ångström radius of the ligand, simplifying the protein structure to a Ca representation with node features h p encoding residue types one-hot vectors. Edges between atoms are defined for those within Angström radius, incorporating both the distance dis and normalized direction vector
Problem Definition Given a context which integrates the functional groups of a molecule denoted as g. and a protein pocket p. our focus is to generate scaffold, a key substructure of a molecule Z. A generative model, defined as Po (Z (u) maps uniform Gaussian noise to conditional probability distribution given u. Ultimately, we aim to make sampling of scaffolds for any given molecular context within the set as efficient possible, without losing overall quality.

---

Dataset We follow the dataset preprocessing scheme regarding filtering of compounds and deter- mining of scaffolds done Torge al. [2023], filtering those above QED of 0.3, training on 19,378 protein-ligand complexes in PDBBind. We also adopt the same scaffold extraction method, using Murko-Bemis method (Bemis and Murcko [1996]).
3.2 Background
Diffusion Models In DDPMs, noise is incrementally added to the original data, represented as Zo. until it is completely transformed into a noise-dominated state Zr after ' steps. This process is governed by an SDE:
dz f(z, () d())dw
where w represents standard Brownian motion, f(Z, 1) is a drift function, and u(l) is a diffusion coefficient associated with the noise process. The reverse diffusion process is described by another SDE which incorporates the backward flow of time through W, a Brownian motion in reverse, and modifies the forward SDE with term involving the gradient of the log probability density.:
dz [[Zz,t u(t)t Vz log pe [Z) u(t)dw
22
The denoising process can be mathematically framed an ordinary differential equation (ODE), also known as the Probability Flow ODE (Song et al. [2020]), whose solution trajectories have the same marginal density at time t:
1 dz = [F t) - =tt) log pe (Z))dt
(3)
During training phase, Diffusion Models (DMs) learn to estimate the score, given by Vz log py (Z) with score model sy (z, t). Following the modifications by Karras et al. [2022], the drift term fZZ, t) is set to zero and the diffusion term is defined as u(t) /2r. Consequently, the empirical Probability Flow ODE becomes:
dz
dt = tso (Z,
This form is used to solve the ODE backwards in time, starting from a noise state sampled from (I). Using numerical ODE solvers like Euler and Heun, the model computes the trajectory (Zz)eto resulting in Zo as an approximate reconstruction of the original data distribution.
We develop an equivariant consistency function f (Zz, f u) - Z. for solution trajectory Z, of an ODE across t E [c, where is near zero. This function meets both self-consistency, ensuring f(Z,, & u) is uniform for all (Ze, t) on the trajectory, and boundary Z, at the smallest time step e. Successfully achieving these conditions guarantees that each solution Z, can be traced back to its original state Zo. The function maintains SE(3) equivarrii, adjusting centers as per methodologies outlined in Xu al. [2022]. and outputs both coordinates and node features.
Consistency Function
conditions, mandating f(Z,,
3.3 Model Architecture
We wish to construct a new sample scaffold given a molecular context u ( is the concatenation of the pocket p and functional groups g). An equivariant consistency function fo(Z[u) h) is parameterized by a free-form deep neural network Fg which is (2,4) related SE (3)-equivariant model. Theoreti- cally, it can be any architecture that is -quivvainn We use adaptation of the equivariant Geometric Vector Perceptron (GVP) architecture (Jing et al. [2020]). As done (Torge et al. [2023]), all features are embedded into shared feature space using separate MLPs. The time f'ccorr is embedded with linear layer and added together as the input. As done in Torge et al. [2023], hidden node features hi' and x' are obtained though message-passing layers. The outputs of the consistency function are 21,, h/ z'g. Pouz(h'z) where Pouiss an MLP to map from embedding space to original data space. As shown in Eq. (5), we separate coordinates and node features and parameterize the consistency model with learnable function fo using skip connections.
fo & u)) Zeip (t) Zout (t)zz
fo(b,, u) = (t)h, + Zour (t),
(5)
5

---

Be
Bryar
3
Scaffold
Notse
best
Spest
Referance
Functional group
Pooket
coordinates
features
context
point
generateedd
Figure 3: Model Architecture of Topp Given a reference ligand and its corresponding protein pocket, an equivariant consistency model samples scaffolds conditioned on pocket substructure and functional groups. Models are trained to map points the same PF-ODE path to the original data given context.
Odata - c)
Zakip(t) (t)
data
Zout (t)
(6)
e)²
data
6)2 + data
Zskip (t) and (tur(t) are differentiable time-dependent functions satisfying Zskip(E) (e) = 1 and Zour (e) thus ensuring the function fe adheres to the boundary condition and retains differrentiaiiliyy. Given data the standard deviation data, when e, setting Zskip(e) and 2 (E) the condition fo Ze) Ze is naturally satisfied. The functions 2kiip Bous and Fg are differentiable. As stated Fan al. [2023a], since 2 and Fg are both SE-(3) equivariant, and zsti(() and Zou(()
ensures that
are invariant scalars, the SE equuiiivaree is guaranteed.
3.4 Model Training
We apply Algorithm for training our consistency model. Scaffold Z extracted from the dataset segmented into its coordinate and feature components, and h. respectively. Next, adjustments are made to the coordinates and features along the ODE trajectory (xtere)).T) and {he he respectively, by incorporating perturbations c, and Ç₁- The loss function L(0) reduces discrepancies between successive time steps fo fo(Zn+1, 1 (u) and fe( uu) ensuring self-consistency.
Let fn and fni. denote the coordinate and feature components of Sa( Ln lu), respectively. Similarly, let fo p=+1,z and fm
fm+1,h denote the coordinate and feature components of fo (Zn+1, tn+1
The loss function L()) is defined
L(Ø) MSE on f. MSE(S + . f2ah))
To enhance the stability of the training process for fo. a secondary function fo-, which utilizes parameters 0- as the exponential moving average (EMA) of 0, is employed. This methodology effectively reduces the variability between consecutive predictions by accounting for incremental perturbations, as detailed in Song and Dhariwal [2023]. The combined loss function measures the mean squared error (MSE) between adjacent states for both coordinates x and features h:
Parameter updates are conducted using stochastic gradient descent for 0 while 0- is incrementally ad- justed using EMA protocol where μ indicates the decay rate, enhancing the model's responsiveness and accuracy.
L(O, 0-) MSE(F ASSE,f+112
mn+1,
f/ + MSE( #1+1 cn,h
(8)
0- 1r + 1))
(9)

---

Table 1: DiffHopp VS ToppoHopp VS TorpoHopp Metric Sampling Results models trained on PDBBind. Mean and standard deviation of the common molecular metrics for the molecules from the baseline models as well as time-step variations of our model. "metric" refers that inference was done with metric- based sampling. QVina score (kcal/mol) refers to estimated binding affinity measured by QVina2. Time refers to average time (seconds) required to generate batch of molecules per complex. Best metrics are in bold. 2nd best are underlined.
Method
Connectivity (T)
Diversity (1)
Novelty (1)
(T)
SA (7)
QVina (J)
Steps
Time
Dilllopp DiAtHoppeSNo
0.918=00:2
0.589=00117
0.999+00.01
0.621=0.12
0.662++0.34
-7.923#3.29
500
107.10
0.75240041
0.641=0.15
0.00
0.510=0.12
0.59240.11
-7.210=1145
DittHoppaers
0.927:10.26
0,597:0.20
1.000
0.634=0.18
0.670:0.13
-7.783-2.78
500
442.48
TurboHopp-50
0.872:10.19
0.562:10.18
1.000
0.576=0.12
0.635:1:25
-7.823
50
3.19
Turbotlopp-10 100
0.948++0.22
0,563++0.25
0
0.589=0.19
0.724+0011
-8.272+1.21
100
5.69
Torholoop--1 150
10000:0.00
0.573+00.12
1.000:0.000
0.618=0.19
0.715+0.13
8.277=1.87
150
TurhoHopp-500 gagp-
0.917=10.88
0.612:0:221
1.000:0.00
0.583=0:20
0.631
17.865-11772
50
3.83
Topp 100metii
0.997++006
0,561
1.000::0.00
0.737-0.11
100
TurhoHopp- 150
1.000:2.000
0.569
1.000::00000
0.648=0.19
0.727
-8.219
150
20.88
PDBBind Test
1.000:10000
1,000::0.00
0.599=0.17
0.74210.11
-8.712:10.18
3.5 Reinforcement Learning Architecture
We adopt RLCM from Oertell et al. [2024], where we biject a Markov decision process multistep consistency model inference. More formally, we adopt this mapping:
8t (7iTo TE A Zre.
m(a:(s:) - So Tl/) + = ().,
P(s a.) 2 (8 r 5n)
R=(sH)=r(( = TH; u) u)
where E represents the noise added during the noising procedure of the consistency model.
As with Oertell al. [2024], we create mapping from MDP states noised states along the consistency model trajectory. Likewise, we follow a deterministic transition from the state to the predicted action. This action is combination of the output from the consistency model convolved with random noise in accordance with the DDPM stochastic differential equation (Ho et al. [2020], Song al. [].21]) Because this policy now becomes stochastic, can apply Algorithm 3 (a modified version of reinforce with a clipping). The agent then receives reward at the end of the trajectory corresponding to a combination of desirable chemical attributes. We use PPO (Schulman et al. [2017]) for the core optimization strategy but others like REBEL (Gao et al. [2024]) have been shown to work for RLCM optimization with sometimes stronger results.
3.6 Metric-based Sampling
By leveraging the inherent efficiency of consistency models, we tailor molecule designs to precise standards. Contrary to the fixed-step approach suggested by Song al. [2023], strategy utilizes flexible, customizable metric that adapts to the variability in protein conditions, as evidenced by our empirical findings shown in Fig. 8. This approach allows for dynamic adjustment during generation, ensuring optimal results by selecting scaffolds with the highest evaluated scores. This not only boosts efficiency but also significantly enhances molecule properties, achieving superior docking and QED scores that surpass those found in the initial dataset.
In Algorithm 2, initial samples are drawn from random Gaussian noise 7, ~ (2T,Ta))Ja is then applied to generate initial scaffolds from these samples: Z, fe T). Across sequence of time points (TI: 72 ... - we iteratively refine the generated scaffold. After reaching designated step m, we score metrics to assesses the quality of each Z,. Then select the one with the best metric at the end of all the iterations as the final output. Our metric function based on drug-likeness and synthesizability.
7

---

Table 2: ToppoHopp VS TurboHopp-RI.CN with Metric Sampling. We optimized TurboHopp- 100 with RLCM using rdocking Single optimization task is based on multiple reference protein- ligand complexes of the PDBBind testset; we do not optimize one protein at a time. To fully support faster parallel multi ligand multi protein docking, we used AutoDockGPU instead of QVina2. Binding affinity scores surpassed reference docking scores without losing drug-likeliness and synthesizability.
Method
Connectivity (f)
Diversity
Novelty (1)
QED (1)
SA (1)
Vina
Stops
Time
TupboHopp- getii TurboHoppL--Smmen
0.997=0.06
2:
00000:0000
0.664:0119
0.737=0:11
18.298:1183
100
7.14
0,980
0.86910.17
0.936:1.117
0.61910.18
0.680=0.13
-9.804112844
50
3.69
PDBBind
1.000++0.00
1.000+0000
0.599
0.742=0.11
-8.64344-2008
4 Experiments and Results
4.1 Performance Comparison with DDPM-based baseline
In this section, we compare 3D scaffold generation qualities between TorboHopp and DiffHopp (Torge et al. [2023]), a DDPM-based scaffold-hopping model. For TurboHopp, we also vary the number of steps in multi-step generation.
Evaluation Metries We evaluate sampling quality using several metrics outlined in Torge et al.
[2023]. Drug-likeness and synthetic accessibility are assessed with Bickerton et al. [2012] and Ertl and Schuffenhauer [2009] respectively. Connectivity is defined as the percentage of fully connected molecules without fragments. Diversity is calculated using Tanimoto Dissimilarity, according to Alhossary et al. [2015]. Novelty is determined by the proportion of molecules that were not present in the training dataset. Additionally, use Vina Score from QVin22 Alhossary et al. [2015] to estimate binding affinity.
Results and Discussion Under the same context conditions, for a batch of 150 molecules, Turbo- Hopp generates on average 5-30 times faster than baseline model (Torge et al. [2023]). Furthermore, as shown in Table 1, compared to DMs, performance is superior in all metrics except diversity, which most likely falls compared to DMs due to the fact that consistency models are trained to have consistent results for data on the same trajectory. Also as total steps for multi-step generation increases, we can see an increase in diversity, due to outputs being refined less as steps decrease.
We applied the same score- score-based based sampling method to the baseline model (DifiHopPsooeal) and compared to the model without using (DiffHopp). All metrics rose except binding affinity. However, in DiffHopp, this method significantly reduces sampling efficiency, as it necessitates more frequent evaluations of the generated products towards the end. contrast, TurboHopp had a major increase in all metrics with small downside of increased generation time of only few seconds. This is possible due to the fact that there are less steps required to generate during multi-step generation, thus less steps evaluate afterwards. This illustrates how, with the right methods, efficient sampling can significantly enhance the search for high-quality molecules within chemical space.
4.2 Performance Optimization using RLCM
In this section, we describe the optimization of our model using two specifically designed reward functions, each aimed at achieving distinct objectives in molecular design:
Binding A Affinity: To enhance binding affinity, we began training with TurboHopp- 100 and reduced the sampling steps to 50. The reward function is defined as:
ddocking score (8) = 2 (s) - RS({)) + QED(s) + 10 -A(s) + C(s)
(10)
9
where DS(s) and RS(s) represent the docking scores of the generated and reference molecules, respectively, highlighting the importance of relative score differences during learning. Ass) signifies synthetic accessibility, and C(s) denotes connectivity. Following the insights from Lee et al. [2023], we incorporate QED and A to mitigate reward hacking, with connectivity included to ensure the structural validity of the molecules.
8

---

Protein Steric Clashes: Steric clashes are defined as interactions where the distance between a protein and ligand atom is less than their combined van der Waals radii by a tolerance of 0.5 A, indicating an energetically unfavorable and often physically implausible pose. crucial for SBDD performance evaluation. We refer to Harris et al. [2023] for estimating number of clashes. To robustly manage steric clashes, we retained the original 100 steps from TurboHopp-100 100. The corresponding reward function is:
sssetic clash (s) C(s) QED(s) (SC(s) RSC(s))
(11)
where SC' and RSC denote the steric clash scores for the generated and reference molecules, respectively, with the difference between these scores included to weigh their importance accurately.
For both models, we initially optimize QED and Connectivity for 50 and 100 steps, respectively, and subsequently finetune each task with the defined rewards metries. Table 2 shows that though metric-based sampling and RICMM, docking scores surpass those of the training dataset.
Unlike in the reward conditioned image generation setting (Fan et al. [2023b], Black et al. [2024]), we omit any KL regularization to the base model during training. This decision stems from the nature of drug discovery where overfitting is less of a concern as long as the model maintains high scores targeted areas. Furthermore, we empirically found that our approach does not significantly compromise diversity and novelty.
Optimization Metrics We optimize TurboHoppRI. for the same test set we used for evaluating HorboHopp (refer to Table 2). Optimization encompasses QED, synthesizability. connectivity, docking score and number of protein clashes. For training stability and computational efficiency, we utilize AutodockGPU (Tang et al. [2022]) to estimate binding affinities, replacing QVina2 (Alhossary et al. [2015]) used for Section 4.1.
Fig. 4 and Table 2 compares TurboHopp- 100 and TurboHoppRL- 50. During training, we achieved higher binding affinity scores of about 2-3 kcal/mol compared to TurboHopp- 100. Examples can be found at Fig. 10.
TurboHopp
Uniqueness
TurboHopppR
Connectivity
Novelty
As highlighted in Fig. we also fine-tuned a model to minimize steric clashes, achieving promising results as shown in Fig. 11. However, op- timizing the reward function further could further enhance clash minimiza- tion and improve the balance between metrics, which we propose as direc- tion for future research. While RL fine-tuned models are not necessar- ily expected to generalize to unseen molecules in drug discovery, there is potential for some generalization capabilities (Oertell et al. [2024]), topic also reserved for future explo- ration.
Vadidi
Infaronce Spond
Score
94
Lipinski Score
Docking Score
Figure 4: Compared to TurboHopp-100 (Red), TurboHoppRL--0 (Blue) has enhancing binding affin- ity without losing performance other metrics.
4.3 Performance Comparison with diffusion-based 3D-SBDD inpainting models
Our model is compared with other diffusion based 3D-SBDD models, Guan al. [2023] and Guan et al. [2024]. Inpainting was applied (Lugmayr et al. [2022], Algorithm 4) on recent conditional de-novo diffusion models to create variations suitable for fair comparison. The same Bemis-Murcko scaffold determines scaffolds and functional groups. With regards to sampling conditions, the number of atoms fixed to the reference scaffold for all models. For Guan al. [2024], since additionally uses bond diffusion, bond mask created accordingly, and reference priors are used while

---

Table 3: TurpoHopp VS 3D-SBDD inpainting models. Results on models trained on CrossDocked. "Inpainting" refers to models using inpainting method explained Algorithm 1. "metric" refers that inference was done with metric-based sampling. Qina score (kcal/mol) refers to estimated binding affinity measured by QVina2. Time refers average time (seconds) required to generate batch of molecules per complex. Best metrics are in bold. 2nd best are underlined.
Method
Validity(1)
Connectivity (1)
Divensity (1)
Novelty (1)
QED (1)
SA (1)
QVina (U)
Time
Targe:Difiugsziis
0.927=0.13
0.826=0,12
0.84
0.91420008
0.424 II
0,66
15.89621.93
740.33
DecompDiffapiniin
0.836:2..33
0.722:00117
0.856:1021
0.895
0.420:10007
0.648:10.15
-6.225 1.087
1263.72
TurboHopp-1U0 100
0.990
0.35
0.484++0227
0.936-1-.24
0.488-+021
0.702-+0.13
-7.051 #:214
6.17
TurboHopp- 100
0.993:20088
0.90
0:486:1:028
0.935
0.502:10.21
0.710:10.14
-7.204::2.00
Cros-Dacked Test
1.00000.00
1. 1.000:0000
0.599-10.17
0.476
0.727
-7.51022.56
sampling. The sampling hyperparameters for inpainting (resampling and jump length parameters) were determined by finding the ones with best validity. Also, since these models were trained on CrossDocked, our model was additionally trained on CrossDocked for fair comparison. The same train-test split suggested in Guan et al. [2024] is followed, with an additional QED minimum filter of 0.3 when constructing the dataset. Only the alpha carbon residues of the protein pocket atoms are used to reduce computational burden.
Results and Discussion Table shows that despite having lower diversity, model has much faster generation speed as well as relatively high docking score compared to inpainting versions of reference SBDD models. Our findings show that a custom scaffold-hopping model outperforms repurposed de-novo model.
Geometric properties and ring distributions (Appendix E) were also compared. Results show that our model has closer bond length/anggle/atom-aalle length distributions to the reference molecules compared to Guan et al. [2023], but poorer results compared in bond/torsion angles. In all aspects, Guan al. [2024] was outstanding largely because it learns the distribution of bonds while ours do not. For ring distributions, results show that our model is capable of generating similar ring types compared to the reference. In future research, we plan to design our model to learn bond properties well.
5 Conclusion
We introduced TurboHopp, equivariant conditional consistency model that significantly boosts generation efficiency and outperforms traditional DDPM-based models various metrics. Empirical results indicate that model achieves 5-30 times improvement in generation speed while main- taining or enhancing traditional quality metrics. Additionally, by leveraging this increased inference speed, demonstrated that reinforcement learning (RL) is feasible for 3D-SBDD-DD DBD, enabling the fine-tuning of generative models to meet specific requirements. Most importantly, our method has the potential to be adapted numerous variations explored within 3D-SBDD-DD -MMs, increase speed and efficiency.
Key areas for improvement and future research include enhancing the denoising model, expanding dataset size, and diversifying reward functions for drug discovery tasks. Incorporating bond diffusion (Peng et al. [2023]) and including hydrogens in proteins and molecules could significantly bolster the robustness the denoising model. In the consistency model, techniques such as eliminating the Exponential Moving Average (EMA), adopting Pseudo-Huber loss, and refining noise and loss schedules may yield more robust outcomes (Song and Dhariwal [2023]). Furthermore, for RLCM, exploring reward functions that integrate steric energies and interaction fingerprints presents promising avenue for future research.
Acknowledgements
This work was supported in part by the National Research Foundation of Korea INRF- 2023RIA2C300411 the Ministry of Health & Welfare, Republic of Korea [HR20C0021(3)];
10

---

the ICT Creative Consilience program through the Institute of Information & Communications Technology Planning & Evaluation (ITTP) grant funded by the MSIT LUTP-2024-2020-0-0155
the Ministry of Health & Welfare and Ministry of Science and ICT, Republic of Korea [RS-2024- 00462471]; the Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT (MSIT, Korea) & Gwangju Metropolitan City; and the Information and Communications Promotion Fund through the National Industry Promotion Agency (NIPA), funded by the Ministry of Science and ICT (MSIT), Republic of Korea.
References
Peter Kirkpatrick and Clare Ellis, Chemical space. Nature, 432(7019):223-24, 2004.
Clemens Isert, Kenneth Atz, and Gisbert Schneider. Structure-based drug design with geometric deep learning. Current Opinion in Structural Biology, 79:102548, 2023.
Wenhao Gao, Rocío Mercado, and Connor W Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. arXiv preprint arXiv:2110.06389, 2021.
Vendy Fialková, Jiaxi Zhao, Kostas Papadopoulos, Ola Engkvist, Esben Jannik Bjerrum, Thierry Kogej, and Atanas Patronov. Libinvent: reaction-based generative scaffold decoration for in silico library design. Journal of Chemical Information and Modeling, 62(9))2046-2033 2021.
Hannes H Loeffler, Jiazhen He, Alessandro Tibo, Jon Paul Janet, Alexey Voronov, Lewis Mervin, and Ola Engkvist. Reinvent 4: Modern ai-driven generative molecule design. Journal of Chemin- formatics, 16(1):20, 2024.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, (3(0):2701-2713, 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d molecule generative model for structure- based drug design. arviv preprint arXiv:2203. 10446, 2022.
Benoit Baillif, Jason Cole, Patrick McCabe, and Andreas Bender. Deep generative models for 3d molecular structure. Current Opinion in Structural Biology, 80: 10:10256 2023.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning. pages 17644-17655 PMLR, 2022.
Zihan Ding and Chi Jin. Consistency models as rich and efficient policy class for reinforcement learning, 2024.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. arXiv preprint MXXi:2331.119000 2023.
Daeseok Lee and Yongjun Cho. Fine-tuning pocket-conditioned 3d molecule generation via rein- forcement learning. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng. Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equiv- ariant diffusion for target-aware molecule generation and affinity prediction. arXiv preprint arXiv:2303.03543. 2023.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: diffusion models with decomposed priors for structure-based drug design. arXiv preprint arXiv:2403.07900 2024.
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, al. Structure-baas based drug design with equivariant diffusion models. arXiv preprint arXiv:2210.155: 13695, 2022.
Jos Torge, Charles Harris, Simon V Mathis, and Pietro Lio. Diffhopp: A graph diffusion model for novel drug design via scaffold hopping, arXiv preprint arXiv:2308.074 074116 2023.
11

---

Ilia Igashov, Hannes Stärk, Clément Vignac, Ame Schneuing, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia. Equivariant 3d od-conniti diffusion model for molecular linker design. Nature Machine Intelligence, pages 1-11, 2024.
liris Sundin, Alexey Voronov, Haoping Xiao, Kostas Papadopoulos, Esben Jannik Bjerrum, Markus Heinonen, Atanas Patronov, Samuel Kaski, and Ola Engkvist. Human-in-the-loop assisted de novo molecular design. Journal of Cheminformatics. 14(1):86, 2022.
Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Li. A survey generative diffusion models. IEEE Transactions Knowledge and Data Fingineering 2024.
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310. 141449,,,, 2023.
Yang Song. Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint w:X1::2303.01469, 2023.
Owen Oertell, Jonathan D Chang, Yiyi Zhang, Kianté Brantley, and Wen Sun. RI for consistency models: Faster reward guided text-to-image generation. arXiv preprint urXiv:2404.03673 2024.
Charles Harris, Kieran Didi, Arian Jamasb, Chaitanya Joshi, Simon Mathis, Pictro Lio, and Tom Blundell. Posecheck: Generative models for 3d structure-based drug design produce unrealistic poses. In NeurlPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Hongmao Sun, Gregory Tawa, and Anders Wallqvist. Classification of scaffold hopping approaches. Drug discovery today, 17(7-8)3310-33 324, 2012.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pages 8867-8887. PMLR, 2022.
Clement Vignac, Nagham Osman, Laura Toni, and Pascal Frossard. Midi: Mixed graph and 3d denoising diffusion for molecule generation, 2023.
Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong. Mdm: Molecular diffusion model for 3d molecule generation, 2022.
Zhilin Huang, Ling Yang. Zaixi Zhang, Xiangxin Zhou, Yu Bao, Xiawu Zheng, Yuwci Yang, Yu Wang, and Wenming Yang. Binding-adaptive diffusion models for structure-based drug design. arXiv preprint arXiv:2402. 188883 2024.
Ila Igashov, Hannes Stärk, Clément Vignac, Vietor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia. Equivariant 3d-conditional diffusion models for molecular linker design. arXiv preprint 10.0v:2210.05274, 2022.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. fitee-dimensional convolutional neural networks and cross-docked data set for structure-based drug design. Journal of chemical information and modeling, 6000):420044215, 2020.
Guy W Bemis and Mark A Murcko. The properties of known drugs. 1. molecular frameworks. Journal of medicinal chemistry, 39(15):2887-2893 1996.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances Neural Information Processing Systems, 35:26565-26577, 2022.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923, 2022.
12

---

Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. International Conference on Learning Representations, 2020.
Zhiguang Fan, Yuedong Yang, Mingyuan Xu, and Hongming Chen. Ec-conf: An ultra-fast diffu- sion model for molecular conformation generation with equivariant consistency. arXiv preprint arXiv:2308.000 002377. 2023a.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben
Poole. Score-based generative modeling through stochastic differential equations, 2021.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. httpt://t///atttt. wtwpaaawwww....aaa.aaarraarraaae wwt.a/////////..0 1707.06347.
Zhaolin Gao, Jonathan D. Chang, Wenhao Zhan, Owen Oertell, Gokul Swamy, Kianté Brantley, Thorsten Joachims, Andrew Bagnell, Jason D. Lee, and Wen Sun. Rebel: Reinforcement learning via regressing relative rewards, 2024.
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1:1-11, 2009.
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina Bioinformatics, 31(13):214-2216, 2015.
Seul Lee, Jachyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of- distribution generation. In International Conference on Machine Learning, pages 18872-18892. PMLR, 2023.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. arXiv preprint arXiv:2305. 16381, 2023b.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. 2024.
Shidi Tang, Ruiqi Chen, Mengru Lin, Qingde Lin, Yanxiang Zhu, Ji Ding, Haifeng Hu, Ming Ling, and Jiansheng Wu. Accelerating autodock vina with gpus. Molecules, 27(9):3041, 2022.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 11461-11471, 2022.
Xingang Peng. Jiaqi Guan, Qiang Liu, and Jianzhu Ma. Moldiff: addressing the atom-bond in- consistency problem in 3d molecule diffusion generation. arXiv preprint arXiv:2305.0m 075088 2023.
13

---

A Algorithm Pseudocode
In this section we present the pseudocode for consistency model training (Algorithm 1), consistency model sampling (Algorithm 2), RL training (Algorithm 3), and inpainting. (Algorithm 4)
Algorithm Training
1:
Input: dataset D {(Z,, u4) NEM, where M refers to the number of data points in D, Z₁ is composed of 24 and h4, initial model parameter 6, learning rate 17, step schedule MV.), EMA decay rate schedule p(-), 0
and t 0:
2: repeat
3: Sample 2, u D, and (11 ((k) 1];
4: Decompose into x and h; (r,
5:
Sample 6, ~ (0, I) for
6:
Subtract center of gravity from Exi
7:
Define fn. fo(v + in Ev: in ()) for (x, h):
8:
L(O, 0 t Eve(z.)) MSE( en+ S5
9:
O + 0 - (oL0(0,
O + p(k)0 =u())0;;
11:
k<c+1
12:
until convergence
Algorithm Sampling with Scoring and Selection
Input: Consistency model fo(:.) = (fr.fe)) sequence of time points (Ti, T2: ..... -N-1) where
> > > TN-1, evaluation starting point m, where 1 S m S N -
2: Sample e, No, I) and c, (O,
3: Subtract center of gravity from 6z
4: < (c)
5: Zr t
6: 2 +-fo(Zr, T())
7: Initialize max__cor_ -00
8: Initialize bes + null
9: for 1 N do
10:
Sample ~ ~( /) and eд ~ I)
Subtract center of gravity from ex Et [Ez,EN]
12:
13:
Zrn + VTn Tn+1"
2 + So(2,,, Tn
15:
if n 2 m then
score +- CustomScore(z) if score max_ then
17:
18:
max Score +- score
19:
Zhest +-
20:
end
21:
end if
22:
end for
23:
output: Zhest
14

---

Algorithm 3 Policy Gradient Version of RLCM
1: Input: Consistency model policy me = fo(.,) :) + E, finetune horizon H, context set set C, batch
size b, inference pipeline P
2: for = 1 to M do
3:
Sample b contexts from C, ~ C.
4: Z < P(fe. H,
where the batch of molecules
5:
Normalize rewards r(:; per context
6: Split Z into & minibatches.
7:
for each minibatch do
8:
for / = 0 to H do
9:
Accumulate gradients of using rule:
min r( c)
To (at (s) ne. (0r(8t)
r( c) clip (a 8, +e) )}
e
(a)l88))
10:
end for
11:
Update parameters based on accumulated gradients.
12:
end for
13:
end for
14:
Output trained consistency model fo(::))
Algorithm 4 TargetDiff/DccompDif Inpainting Procedure
Set number of atoms to generate based on reference scaffold. Get scaffold mask m.
Move center of mass of protein atoms zero.
Sample initial molecular atom coordinates
NO 7)
4: Sample initial bond types/atom types by uT one me_hoot(rrg arr max; g:), where Gumbel 1)
for = T. T...,, do
for I,...,U do
1) Get known
8:
known /afag + - dg)e, where N(O, I)
log c log (1 K))
0, 1 = one hot(arg max; (gi + log cil), where ~ Gumbel(O, 1)
2) Get unknown
12:
unknown
1-1
(1x vI-a, eo 1)) + 012, 10, Tau
pukknown Compute the posterior atom types c(v,, eo (12, t))
3) Merge according to mask m
= m @ x 2
known
+ (1 m)
1
unknown
16:
U: u and t
known U
(1 m) ut-1
unknown
then
Sample from the posterior qe (zt-11 It) Sample Yú from the posterior qo (1-1 v1)
20:
end if
end for end for
return M with coordinates vp and atom types/ bond types vo
15

---

B Information on Hyperparameters and Experiment Details
Parameter setting for TurboHopp
Setting
Parameters
TurboHopp
timesteps: 150, 100, 50, 25
Denoiser: GVP
batch size: 256
layers:
Ir: le-4
hidden features: 256
schedule: Reducel.ROnPlaaee (min: le-6, factor: 0.9)
GNN layers: 7
Attention: True
num epochs: 5500
Embedding size: 64
min' 0,002
Optimizers: Adam
maxx 80.0
Y: le-3
Odata:
3: (0.9, 0.995)
p: 7.0
Dalaset: PDBBind filtered
Device: 4x NVIDIA A100 GPUs
Parameter setting for RLCM
Setting
Parameters
Parameters for Docking Objective and Steric
gradient accumulation steps: 1
Clashes
batch size: 215
Dataset: PDBBind filtered test
num epochs: 200
Device: 8x NVIDIA A A00 GPUs
sample iters:
buffer size: 32
min count: 16
train batch size per gpu: 215
num inner epochs:
le-5
clip range: le-4
max grad norm: 10
16

---

C RLCM and Consistency Model Training Curves
For reproducibility, we present the curves from our training runs for both the consistency model and the use of RLCM.
340
Mrrhy / LAand
10.
And /
M Map
1100
40-
M
Spoda
Figure 5: Training curves for the metrics which compose the loss function. Notice that all either increase maintain approximately the value. Connectivity and QED score slightly decrease because we start from a previously RL finetuned checkpoint which optimizes only for connectivity, SA, and QED score.
TurbuHoye15)
TerboHfipp100
Villaas 2hed
1400
1900
1900
250
TurhoRicgg00
TorisH
675
0.5
033
Figure 6: Training curves of total validation loss for different step size variants. We train a number of consistency models empirically determine the optimal tradeoff between step size fidelity and speed. Turbohopp25 had low validity and proved to be too unstable for consistent generations.
17

---

Initial Checkpoint
Final Cheeispoins
200
200
Chedar
Checks
Figure 7: Plot of clashes before and after finetuning with reward function mentioned in the main text. The initial Turbohopp-100 model was RLCM finetuned model for connectivity, synthesizability. and QED score (I'seric clash- Notice the shift toward smaller number of clashes during training. However, we believe that further iteration of the reward function will lead to more effective finetuning but we leave this to future work.
18

---

D Sample Molecules
0.846
0.720
0.877
Thest
MI
0.461
0.449
0.431
0.429
0.295
HyN
Flast
M
Figure 8: Examples of samples generated for PDB 6QQW with drug-likeliness. 1st row samples collected during multi-step phase with best scores and 2nd row indicates samples from the final step. Connectivity and overall metrics increased when we adopted custom score-based sampling.
Reference: -7.60 kcal/mol
-8.44 kcal/mol
68 kcal/mol
7777 kcal/mll
-8.56 kcal/mol
Figure 9: Scaffolds generated for PDB ID 6E6W by TurboHopp-100 Reference molecule is in yellow, while generated molecules in green. Functional groups the carboxylic acid and hydroxyl groups in the upper corner. Dotted lines in refer to ligand-receptor intermolecular interactions: green, blue, yellow, purple being pi-cation, pi-pi stacking, hydrogen bonds, halogen bonds respectively. Compared to reference molecule, generated molecules had new interactions and higher binding affinity, while maintaining similar binding pose.
19

---

6G20
-9.34 kcal/mol
-9.62 kcal/mol
-11.85 kcal/mol
6N8X
-5.23 kcal/mol
-5.41 kcal/mol
-9.36 kcal/mol
6E6W
8.15 kcal/mol
-8.24 kcal/mol
10.64 kcal/mol
6NPP
7.55 kcal/mol
7.68 kcal/mol
-10.09 kcal/mol
Figure 10: Comparison of Reference molecule(Yellov)) and molecules generated by Turbo- Hopp(Green) and PrrboHopp-RL(Orannnn Notice that Turbohopp and TurboHopp-RL generate molecules that have higher binding affinity with the protein.
20

---

6JSF
6QR9
6IBZ
6QMR
Figure 11: Comparison of Reference molecule(Yellove) and molecules generated by Turbo- Hopp(Green) and Red box indicates collision points with protein atoms. TurboHopp-Ri RL. generates molecules that has less clashes with the protein.
21

---

E Geometric Properties Comparison between TurboHopp and 3D-SBDD inpainting models
Table 4: Jensen-Shannon divergence between bond distance distributions of generated VS refer- DnceeC))ss)ocke))
Bond
TargetDiff inrretDDffippaatiinng inpainting
DecompDiff inpuinting
TurboHopp-100
C-C
0.4984
0.2516
0,4376
C=C
0.6303
0.2962
0,4743
C-N
0.4833
0.2188
0,4178
C=N
0.6300
0.3913
0.5339
C-O
0.3882
0.2841
0.3809
C=O 0,3953
0.2395
0,3552
C:C
0.6829
0.2058
0,5562
C:N
0.6705
0.2734
0,5743
Table 5: -hasen-Shannon divergence between atom-atom distance distributions of generated VS reference((rossDDoceee))
Bond
Tf
DecompDiffimpaantng
TurboHopp-100
C-C
0.55
0.28
0.45
all
0.17
0.08
0.13
Table 6: molecules.
Percentage of different ring sizes for referenceeCrossDoccee)) and model-generated
Ring Size
Ref.
Tf impainting
DecompDiffimpanne impainting
TurboHoppp-100
1.70
4.77
0.00
7.85
0.00
1.12
4.00
0.81
30.20
30.08
37.21
21.16
67.40
57.91
41.70
62.51
0.70
5.28
14.06
6.19
0.00
0.81
2.54
1.12
0.00
0.05
0.48
0.31
Table 7: Divergence of Top Torsion Angle Distributions (CrossDocked)
Metric
DecompDifl impinttnn
AargetDiffinpaintee
TurboHopp
CCNC 0.184
0.204
0.207
CCCC 0.264
0.249
0.197
C.C.C.C
0.178
0.258
0.383
Table 8: Jensen-Shannon Divergence of Top 5 Bond Angle Distributions (CrossDocked)
Metric
DecompDiffmaaine mpainting
Target infintiii
TurboHopp
CNC
0.208
0.274
0.321
CCN
0.168
0.207
0.332
C:C.C
0.096
0.120
0.324
CCC
0.231
0.231
0.298
CCO
0.229
0.263
0.319
22

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have claimed our model's performance and contributions based on empiri- cal findings and experimental results. Our model can be easily adapted to similar frameworks using diffusion models for drug discovery.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
Justification: have added limitations and possible further research studies the conclu- sion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [NA]
Justification: Our focus remains on empirical findings and application, bypassing the need for formal theoretical frameworks validations.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We will share our code/checkpoints upon acceptance. In the paper, we provide the algorithms for training model, RL, and inference. We also share training curves for model training and RL well as hyperparameters required reproduce our results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide the data used to train our model, and upon code release, will specify preprocessing code ,modifications, as well as environment for reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips. cc/ cablic/guides/Cont soredeeSbbissioooooo...
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. wttpc//pccpppccccccccoo gtt////dde/////// (CodeSubmissien weisnnoliccyyo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify the hyperparameters in the appendix. Further, we use a standardized dataset.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All claims of statistical significance were justified through a number of trials. Multiple seeds are not needed for RLCM since we are applying method previously shown work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We specify the compute resources required in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: httpa///////uus...// http://euurrp.p.cpp..iicccc/ csGuidelines!
Answer: [Yes]
Justification:
Yes we read the ethics guidlines and are abiding by them.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please check the conclusion section of the paper.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
26

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Models not yet going to be released and will take into account proper safeguards when releasing models with high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All terms of use and licences are respected.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datastts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets are released
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not do any crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do do crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "X5rdlr0hMu", "reviewer": "Reviewer_Cq9Y", "summary": "Given a protein pocket and a reference ligand, the authors suggest a method to generate different scaffolds to be able to eventually come up with new ligands with similar or even improved properties. Precisely, the authors learn a consistency function which maps noise to a scaffold (created as a 3D-conformation) while being aware of the protein pocket and the functional groups of the reference ligand. This created scaffold together with the functional groups build a new potential ligand. By combining the consistency-based model with RL, the generation process can be biased towards scaffolds which build together with the functional groups ligands with optimized properties, e.g. binding affinity or protein steric clashes. The proposed method is compared with a baseline. Also the impact of adding RL to the approach is evaluated.", "questions": "- **(Q)**: For SMILES based goal-directed optimization [1] show that there is a risk that the generator learns to find blind spots in the reward function and rather learns to trick the scoring function than to optimize real-world properties. Do the authors think this might be an issue also for their RL-based approach?\n\n[1] Renz, Philipp, et al. \"On failure modes in molecule generation and optimization.\" Drug Discovery Today: Technologies 32 (2019): 55-63.", "limitations": "The authors describe potential areas for improvement in the conclusion section.", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**Originality**:\n- **(S-O)**: As far as I know, scaffold hopping with consistency models and combining them with goal-directed RL has not been done before. Therefore applying consistency models to scaffold hopping and combining them with RL to optimize chemical properties is novel. \n\n**Quality**:\n- **(S-Q1)**: In terms of clarity and writing style the quality is very high (see clarity section)\n- **(S-Q2)**: The results section shows that the proposed method is promising and helpful for scaffold hopping.\n\n**Clarity**:\n- **(S-C1)** The paper is written very well. This together with a good paper structure, a good introduction and nice figures achieves that key take aways get very clear.\n- **(S-C2)** The introduction and the related work section set the stage very well for the proposed method. The authors give a good overview about recent work along with their strengths and weaknesses. Figure 1 highlights nicely the idea of the proposed method and Figure 2 shows its effectiveness.\n- **(S-C3)** The generation approach is described very well. Also Figure 3 is good and helpful. The auhors clearly describe which components of their pipeline are learnable.\n\n**Significance**:\n- **(S-S1)**: The results are significant because the suggested approach outperforms the compared baseline.\n- **(S-S2)**: Error bars are reported.", "weaknesses": "**Quality and Clarity**:\n- **(W-QC)**: The mathematical notation / the formulas sometimes seem a bit cluttered and inconsistent:\n    * In formula (8), $f_\\theta^{n+1, x}$ is used for $f_\\theta(Z_{n+1}, t_{n+1}|u)$ without having mentioned that one is the shorthand for the other.\n    * Equation (5) indicates that $F$ has two outputs. The way this equation is written is rather code-style notation than a well defined mathematical expression since $x^\\prime_t, h^\\prime_t$ is from a mathematical viewpoint not well defined. This is why this equation style should be avoided.\n    * $\\sigma_\\text{data}$ is not introduced but used in (7).\n\n**Significance**:\n- **(W-S)**: Because of (Q) the relevance for the RL-based scaffold generation might be limited for real-world scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's positive feedback on our model. Thank you for highlighting the typos and mathematical notations to fix; we will ensure they are corrected in the final draft.\n\n**Q1. Concerns regarding reward hacking (Table 2)**\n\n**A1.** \n\nAs the reviewer mentioned, authors agree that it is true that generators are prone to learning blind spots in the reward function, and this might indeed affect how useful it is in real-world properties.  SMILEs-based RL algorithms exploit their state and action spaces to design chemically trivial molecules with exceptionally high docking scores. To counteract reward hacking in standard docking tasks, we follow the methods of [2], using a combination of QED, synthesizability and docking scores to mitigate this issue (Equation 11). In future research, we plan to create a more robust reward function aligned to better tackle reward hacking further. \nTo this end, KL regularization to the original model usually fixes this problem (a common practice in LLM alignment with RLHF) and could be used if other metrics / real world fabrication showed issues. Other methods [3] have also been developed for these issues and likely would be extendable to consistency models. As shown below (please refer to Table 2 of PDF in global rebuttal), TurboHopp RL maintains other metrics other than docking score and also has high diversity compared to TurboHopp. Furthermore, TurboHoppRL trained on PDBBind test sets had competent generation quality on CrossDocked test sets, without training on CrossDocked test sets, exceeding the reference docking score, which indicates that our model generalizes well to new data. Contrary to other works related to RL for diffusion [4] which optimize towards a single protein pocket, this may be because we optimize on an array of target metrics to ensure that overfitting does not occur.\n\n\n| Method                        | Connectivity (↑)   | Diversity (↑)      | Novelty (↑)        | QED (↑)           | SA (↑)            | Vina (↓)           | Steps | Time |\n|-------------------------------|--------------------|--------------------|--------------------|-------------------|-------------------|--------------------|-------|------|\n| **TurboHopp-100_metric**      | 0.997        | 0.561        | 1.000         | 0.664       | 0.737        | -8.298        | 100   | 7.14 |\n| **TurboHoppRL-50_metric**     | 0.980        | **0.869**     | 0.936        | 0.619        | 0.680        | **-9.804**    | 50    | **3.69** |\n| **PDBBind Test**              | 1.000       | -                  | 1.000        | 0.599        | 0.742        | -8.643        | -     | -    |\n\n[2] Ghugare, Raj, et al. \"Searching for high-value molecules using reinforcement learning and transformers.\" arXiv preprint arXiv:2310.02902 (2023).\n\n[3] Uehara, Masatoshi, et al. \"Fine-tuning of continuous-time diffusion models as entropy-regularized control.\" arXiv preprint arXiv:2402.15194 (2024).\n\n[4] Zhou, Xiangxin, Liang Wang, and Yichi Zhou. \"Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Process.\" arXiv preprint arXiv:2403.04154 (2024).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UPFU0DCt1l", "reviewer": "Reviewer_VN4j", "summary": "This paper proposed a pocket-conditioned 3D molecular scaffold hopping model based on the well-established consistency models. The framework is superior in terms of inference speed. Besides, the authors also proposed a corresponding RL method to fine-tune the model towards generating molecules with desirable properties. The experimental results show the effectiveness of the proposed approach compared with DiffHopp.", "questions": "See the weaknesses.", "limitations": "The author have discussed the limitations.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- This work first introduced consistency models to molecular scaffold hopping and achieved promising results.\n- The evaluation was done from various perspectives, including connectivity, QED, SA, Vina, etc.\n- Introducing RL for optimizing the generated molecules towards desired properties is useful in practice.", "weaknesses": "- More baselines are needed. There are also some other methods for scaffold hopping, such as [1,2,3], etc. Comparison with these methods is necessary to show the significance of the proposed method in the practice of drug discovery.\n- Some related works are missing. For example, this work introduced Reinforcement Learning for Consistency Models to improve the properties of generated molecules. There are related works in the field of diffusion models for molecular science that utilize the similar idea. For example, [4] use a RL method (e.g., actor-critic) to fine-tune the diffusion model to generate molecules with higher binding affinity, and [5] also uses RL-like methods to improve the quality of sampled docking poses in the protein-ligand docking task.\n- Lack of some ablation studies. The effectiveness of some proposed modules is not clear, e.g., the metric-based sampling methods.\n- Though many evaluation metrics are utilized in this work. some are still needed. For example, the geometric properties (e.g., bond lengths, bond angles, and torsion angles) are needed to be checked.\n\nReferences:\n\n[1] Hu, Chao, Song Li, Chenxing Yang, Jun Chen, Yi Xiong, Guisheng Fan, Hao Liu, and Liang Hong. \"ScaffoldGVAE: scaffold generation and hopping of drug molecules via a variational autoencoder based on multi-view graph neural networks.\" Journal of Cheminformatics 15, no. 1 (2023): 91.\n\n[2] Yu, Yang, Tingyang Xu, Jiawen Li, Yaping Qiu, Yu Rong, Zhen Gong, Xuemin Cheng et al. \"A novel scalarized scaffold hopping algorithm with graph-based variational autoencoder for discovery of JAK1 inhibitors.\" ACS omega 6, no. 35 (2021): 22945-22954.\n\n[3] Zhou, Xiangxin, Xiwei Cheng, Yuwei Yang, Yu Bao, Liang Wang, and Quanquan Gu. \"DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization.\" arXiv preprint arXiv:2403.13829 (2024).\n\n[4] Zhou, Xiangxin, Liang Wang, and Yichi Zhou. \"Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Process.\" arXiv preprint arXiv:2403.04154 (2024).\n\n[4] Corso, Gabriele, Arthur Deng, Benjamin Fry, Nicholas Polizzi, Regina Barzilay, and Tommi Jaakkola. \"Deep confident steps to new pockets: Strategies for docking generalization.\" ArXiv (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We first thank the reviewer for the constructive feedbacks regarding additional baselines/metrics for the authors to consider.\nPlease refer to global rebuttal as well as PDF attached!\n\n**Q1. Other scaffold-hopping baselines (Table 3,4)**\n\n**A1**.\n Thank you for adding valuable information regarding baseline models. Regarding the first two models the reviewer mentioned: Scaffold-GVAE[1], GraphGMVAE[2] are both VAE models built for scaffold hopping. DecompOpt[3] is a recently published diffusion-based model for molecular optimization and has scaffold-hopping related experiments with results. Baselines regarding Decompopt are in the global rebuttal section.\n\n**1) Comparison with VAE-based models (Table 4)**\n\nUnfortunately, most VAE based scaffold-hopping models are either SMILEs-based which makes comparison with our model difficult. Also, the codes/data for most of them are missing. The authors of ScaffoldGVAE[1] provide a trained model, but the vocabulary necessary for tokenization, encoding, and decoding is missing. Consequently, we couldn't use the model with the provided checkpoint alone. To properly implement ScaffoldGVAE, we would need to generate a new vocabulary and retrain the model using the 1.9 million ChEMBL compounds mentioned in their paper.\nGiven the limited time frame for the review process, we found it challenging to experiment with multiple baselines that require such extensive setup and training. To ensure fairness in terms of training data, we decided to train ScaffoldGVAE on the PDBbind dataset, which is the same dataset used for our model. This approach allowed us to evaluate ScaffoldGVAE under comparable conditions to our model, maintaining consistency in the data used across experiments. Our approach involved pretraining on the training data followed by target-specific fine-tuning for evaluation on the test set. The results are as follows:\n\n| Method              | Validity(↑) | Connectivity (↑) | Diversity (↑) | Novelty (↑) | QED (↑) | SA (↑) | QVina (↓) | Time   |\n|---------------------|:-----------:|:----------------:|:-------------:|:-----------:|:-------:|:------:|:---------:|:------:|\n| ScaffoldGVAE        | 0.489        | 0.894            | 0.584         | 0.702       | **0.577**   | 0.703  | -5.270    | -      |\n| TurboHopp-100_metric| **0.993**    | **0.906**        | 0.486         | **0.935**   | 0.502 | **0.710** | **-7.204** | _8.18_ |\n\nTo define connectivity for 2D molecules, we considered a molecule to have connectivity if the generated scaffold could be successfully substituted for the original core and form bonds with the remaining functional groups. Despite the high QED values, most scaffolds generated failed to connect with functional groups, leading to low validity and connectivity. Furthermore, most of the scaffolds had low similarity with the reference scaffold. This is presumably due to the fact that the SMILEs based models do not consider 3D pocket structure during generation. In addition most VAE based scaffold-hopping models require tailoring on specific targets. Our model's performance can be attributed to its ability to utilize 3D structural information of target proteins. This allows for generation of molecules even for pockets with few ligands known to bind .\n\n**2) Comparison with other SBDD diffusion models (including Decompopt) (Table 3)**\n\nPlease refer to the **global rebuttal** section.\n\n**Q2.  Related works regarding further research on RL-applied molecular science tasks. (Table 2)**\n\n**A2**\nAuthors thank the reviewer for highlighting these works which we will cite. We do note that these works are different from ours in that [4] proposes to use a critic function which we found to be superfluous for consistency models. Further, we highlight that they focus on a single reward metric, while we create an array of target metrics to ensure that no metrics are sacrificed. Also during a single optimization task, we optimize our model towards multiple targets while [4] focuses on single target. Moreover, we differ from [5] since [5] is related to molecule docking tasks while we are covering a different problem regarding molecule generation.  Nevertheless, we agree these works are important related research to cite and will include these works. \n\n**Q3.  Metric Based Sampling (Table 1)**\n\n**A3**\nMetric-based sampling is a technique that selects the best result towards the end of the sample trajectory, rather than selecting the final output. Figure 8 in Appendix D of our paper illustrates an example between conventional end-of-trajectory sampling and metric-based sampling. We extended **Table 1** to ablate the effectiveness of this method, which shows a slight improvement. This is further demonstrated by enhancements observed in DiffHopp (Table 1: DiffHopp_scored vs. DiffHopp), indicating that it can yield higher quality samples. However, in DiffHopp, it significantly reduces sampling efficiency, as it necessitates more frequent evaluations of the generated products towards the end(100 sec vs 440 sec)\n\n**Q4.  Geometric property metrics evaluation (Table 5,6,7,8,9)**\n\n**A4**\nAuthors compared geometric properties including those related to bond (Please refer to **Table 5,6,8,9**)  and ring distributions(Please refer to **Table 7**). Results show that our model has closer bond length/angle/atom-atom length distributions to the reference molecules compared to TargetDiff, but poorer results compared in bond/torsion angles. In all aspects, DecompDiff was outstanding largely because it learns the distribution of bonds. For ring distributions, results show that our model is capable of generating similar ring types compared to the reference. In future research, we plan to design our model to learn bond properties , and we expect better results regarding geometric properties.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9BrfE7Mq0p", "reviewer": "Reviewer_M9sk", "summary": "This paper presents TurboHopp, an accelerated pocket-conditioned 3D scaffold hopping model designed to enhance the efficiency and speed of drug discovery. It addresses the slow processing speeds of 3D-SBDD generative models by offering up to 30 times faster inference speed while maintaining or improving on key metrics like drug-likeness, synthesizability, connectivity, and binding affinity. Additionally, it incorporates reinforcement learning to further optimize molecule designs, demonstrating its potential in various drug discovery scenarios.", "questions": "1. Could the authors provide a more detailed analysis of the trade-off between inference speed and the quality of the generated molecules?\n2. How does the novelty exceed 1 at Table 1?\n3. The part that reduces time is mainly due to the consistency model. How did the author transfer the consistency model? What improvements have been made after the transfer?\n4. What are the advantages of this model over SBDD models, such as Targetdiff [1] and Decomdiff [2]?\n5. In terms of performance comparison, the enhancement provided by reinforcement learning to the model is significant. Can it be understood that surpassing DiffHopp primarily relies on reinforcement learning? Is it reasonable to use evaluation metrics directly for optimization? Can you conduct an ablation study focused solely on the integration of reinforcement learning?\n\n[1] Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. arXiv, 2023.\n\n[2] Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: diffusion models with decomposed priors for structure-based drug design. arXiv, 2024.", "limitations": "The authors state that they have improved generation efficiency, but they do not compare their approach with methods that generate molecules from scratch.", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Accelerated Generation: TurboHopp's inference speed is 5-30 times faster than that of DDPM-based models, greatly improving the efficiency of drug discovery.\nCombination with Reinforcement Learning: By leveraging the fast inference speed of consistency models, TurboHopp applies reinforcement learning to 3D-SBDD-DMs, enabling fine-tuning of generative models based on specific objectives, such as improving binding affinity or reducing steric clashes, for more refined molecule design.", "weaknesses": "The consistency model and reinforcement learning are well-established techniques, each with a robust body of research. The integration of reinforcement learning into diffusion models has been explored in the literature. In this paper, the authors concatenate these two methodologies for the purpose of scaffold hopping, without explicitly detailing any novel strategies or innovations in their implementations. The empirical comparison can be done in a more thoroughly by comparing with other latest state-of-the-art drug discovery algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First, we thank the reviewer for the great insights for authors to consider. As stated in your limitations, we acknowledged that comparison with de-novo generative molecules were important. However, since direct comparison would be unfair, we had to build repurposed versions of de-novo models which we plan to additionally release. Please refer to the global rebuttal section for further details.\n\n**Q1.  Regarding inference speed and generation quality (Table 1 main, Table 3,5,6,7,8,9 as additional reference)**\n\n**A1.** In order to explain the relationship between speed and quality, authors extended previous Table 1 that compares DiffHopp, TurboHopp and variations. As shown in Table 1, fewer steps in TurboHopp leads to overall poor quality compared to versions with more steps. Among the 3 variants of TurboHopp, TurboHopp100 seemed to have the best efficiency regarding sample quality as well as generation time. Table 1 as well as Table 3 show that our model is efficient with comparable metrics compared to diffusion-based models despite the reduce in time . However, in Table 5,6,7,8,9, where we compared geometric properties with existing conditional diffusion models, we see some geometric properties remain a property to improve. In future research if we train the model to learn bond distributions, this may resolve this issue. \n\n**Q2.  Novelty issue**\n\n**A2.** We apologize if we confused the reviewer. Novelty is capped at one and the digits after the plus-minus sign is the standard deviation.\n\n**Q3.  Consistency models and improvements**\n\n**A3.** We did not train using consistency distillation but trained a consistency model from scratch using consistency training [3]. We referred to improved techniques used for consistency training as well and found that MSE loss instead of psuedo-huber loss is better and that  training without skip connection in the model is more stable [4] . In addition, we had to change the original consistency model so that it is suitable for conditional molecule generation. It had to take account conditions(protein, functional groups etc.) as well as expand the model to be multimodal and SE(3) equivariant. Overall,we believe our framework can be broadly adapted to many of the molecule generative diffusion models existing and hopefuily improve sampling efficiency.\n\n**Q4.  Advantages over de-novo conditional generative diffusion models(Algorithm 1, Table 3)**\n\n**A4.** Scaffold-Hopping aims at finding novel scaffolds that connect key functional groups (interacting with protein residues) while de-novo generative models focus on building the whole molecule given a protein pocket. Both models are, SBDD models, and they both aim on building molecules that have great potential in finding novel, potent molecules.  However,  because of the lack of 3D-binding complex data, it is a realistically hard task for even advanced models to learn the vast chemical space of potent molecules. Therefore, by conditioning the functional groups with potential interactions, we can reduce the chemical space the model should learn, which leads to more efficient learning and sampling(**Figure 1**, paper). This is additionally powered by the speed boost of our consistency model in allowing 1) faster, efficient generation which is often requested in the pharmaceutical domain and 2) faster optimization in certain molecular metrics which opens possibilities of applications in human-in-the-loop model optimization. \n\nModel structure-wise, straight comparison between de-novo SBDD models and scaffold-hopping models is unfair since for input conditions, scaffold-hopping models have an extra ligand condition to consider during generation. This may be more harder for the model to learn since there's an extra constraint, but on the flip-side also mean less chemical space for the model to learn.  We do, however, compare inpainting variations of de-novo conditional generative diffusion models which we have additionally built for [1] and [2]. Scaffold hopping can be seen as an inpainting task, where models trained on generating de-novo molecules treat the scaffold as a missing piece that needs to be filled in, while the known functional groups of the molecule are provided as context. In a gist, as shown in Table 3, we have better outcomes in overall molecular metrics. Also, inpainting variations of SBDD models are too slow to practically use compared to our model. For more information, please refer to the global rebuttal. \n\n**Q5. Questions regarding RL performance(Table 1,2)**\n\n**A5.**\nThe authors agree that there is a significant impact of reinforcement learning. However, we do not believe that surpassing DiffHopp primarily relies on RL. In particular, by leveraging the consistency model’s ability to give a number of high quality predictions within significantly less steps, we can use metric sampling to achieve competitive results(**Table 1**). However, we also showed that the inclusion of RL provided significant gains over diffusion based models (**Table 2**). To counteract reward hacking in standard docking tasks, we followed the methods of [5], using a combination of QED, synthesizability and docking scores to mitigate this issue (Equation 11). In future research, we plan to create a more robust reward function aligned to better tackle reward hacking further. While it is true that to some extent we are overoptimizing for these metrics, we found that many metrics excluded from the reward function were able to maintain quality.  This hints that the molecules were not being overoptimized. See the response for reviewer Cq9Y for more information\n\n\n[3] Song, Yang, et al. \"Consistency models.\" arXiv preprint arXiv:2303.01469 (2023).\n\n[4] Song, Yang, and Prafulla Dhariwal. \"Improved techniques for training consistency models.\" arXiv preprint arXiv:2310.14189 (2023).\n\n[5] Ghugare, Raj, et al. \"Searching for high-value molecules using reinforcement learning and transformers.\" arXiv preprint arXiv:2310.02902 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate all the valuable feedbacks from the reviewers. Here we answer a question asked in common about comparing our model with de-novo molecule generative diffusion models.

**Q. Comparison with other SBDD diffusion models (Table 3)**

**A:** Although there exists a plethora of de-novo 3D-SBDD models , direct comparison with a scaffold-hopping model wasn't easy, but we have tried our best to compare as fairly as possible. In order to expand our baselines, we additionally applied inpainting[1]  (refer to **Algorithm 1** of PDF) on recent conditional de-novo diffusion models to create variations suitable for scaffold hopping. In terms of inpainting for scaffold hopping, "knowns" refer to functional groups conditioned, while "unknowns" represent the scaffolds that need to be generated. We use the same Bemis-Murcko scaffold to determine scaffolds and functional groups. With regards to sampling conditions, we fix the number of atoms to the reference scaffold for all models. For Decompdiff, since it additionally uses bond diffusion, we had to create a bond mask accordingly, and we use reference priors instead of applying Alphaspace. The sampling hyperparameters for inpainting (resampling and jump length parameters) were determined by sampling and finding the ones with best validity.  

Also, since these models were trained on CrossDocked, we additionally trained our model on CrossDocked for fair comparison. We follow the same train-test split suggested in DecompDiff, but we add an additional QED minimum filter of 0.3 when constructing the dataset, resulting in training/validation dataset size of 84057/251 molecules . We only use the alpha carbon residues of the protein pocket atoms in order to reduce computational burden. 

Please note that the code for Decompopt was released most recently (August, 2024), and scaffold hopping code for reproducing the results are missing. Decompopt uses Vina Score for validation, while ours use QVina2, and values denoted in the table below are those reported in the paper. "Inpainting" refers to models using inpainting method.  The parameters for "metric" refers that inference was done with metric-based sampling. QVina(kcal/mol) refers to estimated binding affinity measured by QVina2.

Below are the results (we omit the standard deviations due to space issues, for the full table, refer to Table 3 of PDF):

| Method                     | Validity (↑) | Connectivity (↑) | Diversity (↑) | Novelty (↑) | QED (↑) | SA (↑) | QVina (↓) | Time   |
|----------------------------|--------------|------------------|---------------|-------------|---------|--------|-----------|--------|
| **TargetDiff_inpainting**  | 0.927        | 0.826            | *0.841*       | 0.914       | 0.424   | 0.661  | -5.896    | 740.33 |
| **DecompDiff_inpainting**  | 0.876        | 0.722            | **0.856**     | 0.895       | 0.420   | 0.648  | -6.225    | 1263.72|
| **DecompOpt_inpainting**   | -            | -                | -             | -           | 0.490    | 0.710   | -7.280     | -      |
| **TurboHopp-100**          | *0.990*      | *0.853*          | 0.484         | **0.936**   | *0.488* | *0.702*| *-7.051*  | **6.17**|
| **TurboHopp-100_metric**   | **0.993**    | **0.906**        | 0.486         | **0.935**   | **0.502**|**0.710**|**-7.204**| *8.18* |
| **CrossDocked Test**       | 1.000        | -                | 1.000         | 0.599       | 0.476   | 0.727  | -7.510    | -      |

Despite having lower diversity compared to other diffusion models, our model has much faster generation speed as well as relatively high docking score close to DecompOpt, which optimizes molecules for multiple rounds, meaning it would probably take longer than DecompDiff. Please do note that inpainting in general increases generation time (but our model is still much faster without inpainting). Consequently, our findings show that custom scaffold-hopping model outperforms a repurposed de-novo model. 

[1] Lugmayr, Andreas, et al. "Repaint: Inpainting using denoising diffusion probabilistic models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

Meta-review: The authors present the application of consistency models and reinforcement learning to the problem of scaffold hopping. The original paper is well written and overall presentation is good. Reviewers raised concerns in particular regarding the lack of baselines and proper ablation. During the rebuttal phase these concerns have mostly been addressed. In particular, the authors approach outperform current SOTA on this task and do so with much faster inference time. Due to the importance of fast inference times in drug discovery and lead optimization, the latter contribution on its own warrants significant recognition. I recommend the authors include these baseline and ablations in their revised manuscript, and ensure that the reviewers concerns are properly addressed.
Decision: Accept (poster)