3D Molecular Generation via Virtual Dynamics
Authors: Shuqi Lu
Lin Yao
Xi Chen
Hang Zheng
Di He
Guolin Ke
Venue: Accepted by TMLR ()

Abstract: Structure-based drug design, a critical aspect of drug discovery, aims to identify high-affinity molecules for target protein pockets. Traditional virtual screening methods, which involve exhaustive searches within large molecular databases, are inefficient and limited in discovering novel molecules. The pocket-based 3D molecular generation model offers a promising alternative by directly generating molecules with 3D structures and binding positions in the pocket. In this paper, we present VD-Gen, a novel pocket-based 3D molecular generation pipeline. VD-Gen features a series of carefully designed stages to generate fine-grained 3D molecules with binding positions in the pocket cavity end-to-end. Rather than directly generating or sampling atoms with 3D positions in the pocket, VD-Gen randomly initializes multiple virtual particles within the pocket and learns to iteratively move them to approximate the distribution of molecular atoms in 3D space. After the iterative movement, a 3D molecule is extracted and further refined through additional iterative movement, yielding a high-quality 3D molecule with a confidence score. Comprehensive experimental results on pocket-based molecular generation demonstrate that VD-Gen can generate novel 3D molecules that fill the target pocket cavity with high binding affinities, significantly outperforming previous baselines.

Published in Transactions on Machine Learning Research (03/2024)
3D Molecular Generation via Virtual Dynamics
Shuqi Lu DP Technology
lusqūdd..er tech
Lin Yao DP Technology
yool@dp.ted
Xi Chen
chena@dp. tech
DP Technology
Hang Zheng DP Technology
shengh@ddp.ee tech
Di He Peking University
lihe@pku.edu.cr
Guolin Ke DP Technology
kegl@dp.ted
Reviewed on OpenReview:
https: openreview. net/ forum? QuipGVdE6L.
Abstract
Structure-basd based drug design, a critical aspect of drug discovery, aims to identify high-affinity molecules for target protein pockets. Traditional virtual screening methods, which involve exhaustive searches within large molecular databases, are inefficient and limited in discovering novel molecules. The pocket-based 3D molecular generation model offers a promising alterna- live by directly generating molecules with 3D structures and binding positions in the pocket. In this paper, present VD-Gen, novel pocket-based 3D molecular generation pipeline. VD-Gen features a series of carefully designed stages to generate fine-grained 3D molecules with binding positions in the pocket cavity end-to-end. Rather than directly generating or sampling atoms with 3D positions in the pocket, VD-Gen randomly initializes multiple virtual particles within the pocket and learns to iteratively move them to approximate the distribution of molccular atoms 3D space. After the itcrative movement, a 3D molocule extracted and further refined through additional ilerative movement, yielding high-quality 3D molecule with confidence score. Comprehensive experimental results pocket- pocket-based molecular generation demonstrate that VD-Gen can generate novel 3D molecules that fill the target pocket cavity with high binding affinities, significantly outperforming previous basclines.
1 Introduction
Structuro-based (pocket-based) drug design, i.c., finding a molocule to fill the cavity of the protein pocket
with a high binding allinity (Kubinyi 1993 DesJarlais el. al. 1988 DeWitte et al. 1997 DeWitte & Shakhnovich 1996), is one of the most critical tasks in drug discovery. The most widely used method is virtual screening WWlters al., 1998 Shoichet 2006 2004). Virtual screening iteratively places molecules
from a molecular database into the target pocket cavity and evaluates molecules with good binding based on
rules such energy estimation (de Ruiter & Oostenbrink 2011 Chipot & Pohorille, 2007 Christ al. 2010

---

Published in Transactions on Machine Learning Research (03/2024)
Pocket Condition
Particle Initialization
Particle Movement
Molecule Extraction
Molecule Refinement
Confidence Prediction
x R
R2
The VD-Gen pipeline. Given pocket the input, VD-Gen first initializes the Virtual Particles (VPs) and iteratively moves them to approximate the atom distribution in 3D space. Subsequently, a 3D molecule is extracted from the VPs after their movement. The atoms within the extracted 3D molecule are then further refined through additional iterations of movement. Lastly, a confidence score for the refined 3D molecule is predicted.
Figure 1:
However, virtual screening is inefficient for the exhaustive search and is infeasible to generate new molecules that are not in the database. Recently, molecular generative models have become a potential solution to address the problem as they could generate novel molecules in an efficient way. The early attempts focused on ligand-based molecular generation (Kusner et al. 2017 Dai et al. 2018 Winter it al. 2019), which trains models to learn the underlying distribution of the molecules in training data and generate similar molccules. However, those methods did not consider conditional information, such as the shape of the pocket. Later, more efforts were paid to studying how to generate molecules conditioned on the information of protein pockets. Some pocket-based generative models simply generate molecules in the form of SMILES or graphs (Skalic et al. 2019 Xu et al. 2021), without considering the 3D geometric position of the molecule and pocket, which is closely related to binding affinity.
Michel & Essex 2010).
Given the 3D structure of a pocket, the ultimate goal of the task is to generate 3D molecules which contain a set of atoms, each with an atom type and the corresponding 3D position. Previous works can be broadly classified into two categories. 1) 3D density grid generation (Ragoza et al. 2022), in which pockets and molecules are converted to 3D density grids with coarse-grained positions, and then generative model is used to predict the density at cach grid. Since the model can only generate grid-level positions, these approaches cannot produce high-quality 3D molecules. 2) Auto-regressive 3D generation {Luo et al. 2022 Liu et al. 2022: Peng et al., 2022), in which atoms (with a 3D position and an atom type) are generated one by one. However, since it is hard to define the gencration order of atoms during training, these models usually achieve inferior performance.
In this paper, we propose a new method, VD-Gen, which efficiently generates high-quality 3D molecules. The central concept of VD-Gen involves using distribution of Virtual Particles (VPs) in 3D space to approximate the distribution of molecular atoms. As depicted in Fig. [ the VD-Gen pipeline comprises five stages for end-to-end generation of 3D molecules: 1) Given protein pocket, VD-Gen initializes multiple VPs. 2) The VPs are iteratively moved until equilibrium. Tdeally, the distribution of equilibrated VPs closely approximates the distribution of molecular atoms. 3) A 3D molecule is extracted from the equilibrated VPs. 4) The atoms in the extracted molecule are refined through iterative movement. 5) A confidence score predicted for the generated 3D molecule. By employing this pipeline, VD-Gen non-autoorerressivel generates high-quality 3D molccules and addresses issucs in the previous works. In comparison to 3D grid-bascd generative models, VD-Gen produces high-quality 3D molecules with fine-grained coordinates. Furthermore, when contrasted with autoregressive generative models, VD-Gen efficiently generates all atoms simultaneoully, resulting in improved performance that is not dependent on the generation order.
We have conducted comprehensive experiments utilizing various evaluation metrics, such as Vina Trot & Olson 2010), MM-PBSA (Yang et al., 2022), and 3D Similarity (Roy & Skolnick] 2015). in order to thoroughly benchmark VD-Gen. The experimental results indicate that our model capable of generating diverse drug-like molceules exhibiting high binding affinities 3D space, along with favorable binding poses, thereby significantly surpassing all baseline models. To further substantiate the effectiveness of VD-Gen, we
2

---

Published in Transactions on Machine Learning Research (03/2024)
have designed ablation studies, case studies, and visualizations. Moreover, the VD-Gen pipeline can be readily extended to pocket-based 3D molecular optimization, achieving exceptional performance in this task as well.
2
Method
The aim of structure-based molecular generation is to learn a probabilistic mapping - where P is the space of protein pockets, M is the space of all molecules, and D(M) is the probability distributions over molecules. Within this framework, a protein pocket, denoted as P E P, is described by a set of u atoms f(> 1 where each atom represented by a one-hot encoded vector x. E R' that identifics its atomic type among! possible lypes, and vector yf R specifying its 3D spatial coordinates. For a given protein pocket P, the objective is to produce probability distribution over molecules, which assigns higher probabilities to molccules likely to bind to P. This is achieved through a generative model h(P;0), where h()50) is a function modeling the generative process, parameterized by 0. This model accepts a specific pocket, P as input and outputs a corresponding distribution over M, where we can sample a molecule instance, denoted as M. The sampled molecule M is defined as a set of m atoms, M = {( y:))m
Given the challenges associated with directly generating the distribution, we prefer an iterative refinement approach that begins with random initialization. Inspired by Molecular Dynamics, we initially randomize both the types and coordinates of atoms. These atoms are then iteratively refined by a model to approximate their ground-truth positions. For the success of this methodology, it is crucial to map each input atom to a unique target within the ground truth atoms, ideally by minimizing displacement. That is, we can randomly initialize the same number of atoms as in the ground-truth molecule and use the Wasserstein distance to assign a unique training larget for each atom. While this approach is straight we observed unstable training (i.e., divergence) and inferior performance. Further details are delineated in Appendix B
To avoid training instability, we introduce Virtual Particles (VPs) and use many VPs (several times the atom number) to approximate the atom distribution of the ground-truth molecule in 3D space. In particular, set of VPs is randomly allocated within the pocket cavity, serving as a distribution in 3D space. Subsequently, the VPs are gradually moved based on learnable dynamics, approximating the ground truth molecular atoms. We refer to this iterative process as Virtual Dynamics. As this dynamics is easily learnable, the training is stable, effectively addressing the unstable training issue in our initial attempt.
To extract valid molecules from the distribution formed by VPs, we design a filter-thnn-meree approach and subsequently refine the atoms in the extracted molecule using iterative movement. Moreover, considering real-world applications, VD-Gen predicts confidence scores for the generated 3D molecules. We describe the generation pipeline of VD-Gen in the following subsections and Algorithm 3
2.1 VD-Gen Pipeline
The objective of this stage is to initialize VPs within the protein pocket cavity. First, the cavity space must be detected, where the VPs will be inilialized. Given the aloms of a pocket, we use a 3D U-Net model (Ciçek et al., 2016) to predict the grids of cavity space. It is important to note that this 3D U-Net is trained independently, rather than in an end-to-ond manner with the VD-Gen pipeline, as it solely detects the cavity space. The details of this 3D U-Net can be found in Appendix [A.2] Next, we need to determine the number of initialized VPs. A neural network model is used to predict the number of atoms in the generated molecule based on pocket atoms, denoted as m = han (P; 0un), where han is neural model with learnable parameter 6a. and m is the predicted atom count. The number of VPs is then set n = mkkpp where kup E N is hyperparameter. Finally, we randomly and uniformly distribute n VPs within the predicted cavity space and denote the initialized VPs as Vo- It is worth emphasizing that we utilize kip times the predicted atom count to ensure comprehonsive VP coverage within the cavity. Sufficient coverage permits the nearest assignment approach for VPs, subsequently minimizing the learning complexity.
Particle Initialization
The objective of this stage is to iteratively move the initially randomized VPs with their respective ground-truth atoms. atoms.Given an initial set of VPs, Vo, we update their distribution by moving them in 3D space, to approximate the ground-truth atom distribution. This movement process is iterative, similar to Molecular Dynamics. Specifically, at each iteration, the model takes the positions and types of
Particle Movement
3

---

Published in Transactions on Machine Learning Research (03/2024)
VPs from the previous iteration inputs and produces updated positions and types. This process can be represented as Vr+1 = Spm (V,, P; Pp)); where V, [( (err.,]) v[) denotes the set of n VPs predicted at the r-th iteration, fym is an SE(3) equivariant model that accepts 3D coordinates as imputs, and 0 0pm are the learnable parameters. The number of iterations in the Particle Movement stage is denoted by R1.
Molecule Extraction The objective of this stage is to extract molecules from VR by merging VPs. Given that VPs gravitate towards ground-truth atoms, positions with denser VP concentrations are probable atomic locations. Consequently, the merging strategy primarily leverages the 3D coordinates of the VPs, allowing us to select the top-m densest. locations for molecular extraction. To achieve this, we have designed a neural network- betdork-based model.Formally, this model can be represented as Wo = fime( : mm)), where One denotes learnable parameters, and Wa = {(0). { is the set of m atoms in the extracted 3D molccule. The model constructs m aloms from n VPs through a two-step process: filtering and merging. Both these steps predominantly utilize the atomic coordinates with the intention of first eliminating VPs with significant errors and then merging VPs to produce stable representative atoms.
First, some VPs might not align precisely to their targeted positions; our goal is to eliminate such outliers. To achieve this, we predict the errors (distances between VPs and their target positions) for VPs, and exclude those with significant errors. Specifically, errors surpassing C (a predetermined hyper-parameter) are deemed significant.
Second, we merge the filtered VPs into atoms. For every pair of VPs, the model predicts merging probability. With the predicted pairwise merging probability matrix, we establish threshold, resulting in binary merging matrix. VPs are then grouped into clusters based on this matrix. However, determining an appropriate threshold is challenging. Notably, we have already predicted the number of atoms, m, during the Particle Initialization stage. As such, We employ a binary search to determine a merging threshold that results in a cluster count approximating m.
We then obtain several (ideally m) merged clusters, denoted as w; for the set of indices of the i-th cluster's VPs. To compute Wo = {(6). }/) within each cluster, we use t.- (f E w.)) and ef- = Mean((y,"1) E w.)) to sample an atom type and obtain an average coordinate, respectively. It's essential to note that while the average coordinate of the VPs in a cluster offers a stable representative position, atom types are discrete. Therefore, determining an average type is not possible, which leads us to select a type from the VPs within the cluster. Notably, we refer the particles in Wa as atoms to distinguish them from VPs in Vo.
Molecule Refinement The objective of this stage is to further refined the atoms in Wo. to obtain a high-quality 3D molecule with fine-grained 3D coordinates. This stage is similar to the Particle Movement, albeit with different input (the atoms in Wo) and different model parameters. Formally, the process in this stage can be represented Wr+1 fmr (W,, [mr, P; 0mr), where fmr denotes the SF(3) equivariant model with learnable parameter Omr. The term W, = { "))} represents the set of m atoms at the r-th iteration. The total number of iterations in the Molecule Refinement process is denoted by Ro.
Confidence Prediction In practical applications, it is often necessary to generate multiple molecules and select the top ones based on their binding affinities. Consequently, a reliable confidence predictor is essential for ranking these molecules. While computational simulations and experimental methods can be used to evaluate the generated molecules, these approaches are often prohibitively expensive, particularly when dealing with a large number of candidates. To enhance the usability of the proposed method and minimize additional costs associated with molecule selection, this stage aims to predict a confidence score for each generated 3D molecule. Specifically, we adopt the approach from AlphaFold2 (Jumper et al.] 2021),, utilizing the PLDDT (predicted LDDT) score as our confidence metric, cnabling the ranking and selection of the most promising molecules.
2.2 VD-Gen Training Strategies
In the VD-Gen training, we first train the 3D U-Net used for pocket cavity detection independently (refer A Appendix A.2 for details). Then, we conduct end-to-end training for the entire VD-Gen pipeline, based on the
4

---

Published in Transactions on Machine Learning Research (03/2024)
3D U-Net predicted pocket cavity. The following subsections provide detailed description of the training strategies implemented at each stage.
Training of Particle Movement
The objective of the model fpm is to move the virtual points (VPs) to the positions of the ligand molecular atoms, thereby approximating the distribution of molecular atoms. To accomplish this, a real atom can be directly assigned as the training target for each VP.
Formally, given the ground-truth aloms G {( y y/) and the initialized VPs Vo, there are n possible assignments. In accordance with the principle of least action (Feyman 1965), the assignment with the minimal moving distance is preferred. This entails optimizing Min > i-1 Mly Vd ll,, where yf denotes the initial position and a; N represents the assigned target for the i-th VP. This optimization problem can be easily solved by assigning the nearest real atom as the training target. for the i-th VP, i.e., 04 arg min 1l yý However, this nearest assignment may result in some real atoms not being assigned as training targets. To increase coverage, there are two approaches. The first involves considering atom coverage as a constraint in the aforementioned optimization problem, while the second entails using more VPs to cover the pocket cavity as extensively as possible. Although the former is more favorable from algorithmic perspective, it increases the learning difficulty of VP movement, the constrained assignment violates the principle of least action. Therefore, we adopt the latter approach, using more VPs to increase coverage.
Given the assigned targets a,, we use the following losses for the training. First, a negative log-likelihood loss used for the VPs' types. Second, a clip L2 loss used for the VPs 3D coordinates. Finally, a regularization loss is used limit the moving distances between two adjacent iterations, to stabilize the training. Combined above, the final training loss function at the r-th iteration could be denoted
4pm Σ (NLL(E, =q,,) clip(lle v., (2,7) + max([ly] -I1 6,0))
(1)
where T.. is the predicted vector of atom types of i-th VP, æg is the ground-truth atom types of i-th VP, yf (y%) is the predicted (ground-truth) coordinates of i-th VP, T is the the clip value for coordinate loss, and 8 is the threshold for moving regularization.
It is important to note that there R₁ iterations in this stage. However, training the model using multiple iterations is inefficient in terms of both computational speed and memory consumption. To reduce the training cost, we adopt the stochastic iteration approach used in AlphaFold2 (Jumper et al. 2021). ). Specifically, during the training process, the iteration r is uniformly sampled between 1 and R, where R represents the maximum iteration (R = Rı in this stage). The model is then exccuted in forward-only mode for the first r- iterations, without calculating the loss or performing gradient backpropagation. The gradient calculation and backpropagation are enabled only at the r-th iteration. During the inference phase, this sampling of iterations is not employed. The aforementioned algorithm illustrated in Algorithm !
Training of Molecule Extraction In this stage, there are two training tasks. The first task is to predict the errors, which are the distances between the VPs and their target positions. To stabilize the training, we bucket the errors into one-hot bins and convert the task into a classification problem, following the pLDDT training in AlphaFold2 (Jumper et al. 2021 ::
Lerror preat
NL,
(2)
where s; represents the one-hot vector of the bucketed target error bin, and s, denotes the predicted probability vector.
The second task aims to predict which VP pairs should be merged. Ideally, VPs with the same larget atom should be merged; therefore, the training label for a VP pair with the same target atom is set to 'true'. Given n VPs and m real atoms, the ratio of the "true" class is approximately mx(n/m)² 1. As m ranges from n² m dozens to hundreds, this binary classification task is highly inbalanced. To address this issue, introduce
5

---

Published in Transactions on Machine Learning Research (03/2024)
Algorithm 1 Iterative Movement
Require: R: iterations, P: pocket atoms, Vo: random initialized VPs, f(;0): SE) equivariant model with
parameters O
r r+ uniform(1, R) training else
Sampling is only enabled training
2: disable gradient()
b Disable gradient calculation globally
for E [l...., 1) do
4: V& 4- S(VV-((VVV P:0)
updale withouu gradients Enable gradient calculation globally update with gradients Return the positions and types of particles
enable gradient()
6: V f(V,-1, P;0)
return V,
focal loss Lin ct al. 2017 to balance the classes.
Emerge +Σ FL(F.)rr))
(3)
where denotes the number of VPs after filtering, Pay represents the target merging type, and Fyy indicates the predicted probability of merging type.
Training of Molecule Refinement The objective of the model fmr is to refine the atoms in Wo to ligand molccular atoms. Similar to the Particle Movement stage, a real atom can be directly assigned as the training larget. for each atom. Since Wo is constructed from the clusters of VPs, the training target for each atom is determined by its corresponding cluster of VPs. To be specific, we first collect all the training targets of VPs belonging to particular cluster, and then select the most frequently occurring onc. Formally, for the i-th atom, its training target atom is denoted as b; = most frequent {fayli E w,),, where w; represents the indices of VPs belonging to the i-th cluster and aj is the target atom for the j-th VP. Subsequently, the same training loss functions employed in the Particle Movement stage can be used to train the model fmr-
Training of Confidence Prediction We explicitly train task to learn the confidence scores for generated molecules. In particular, following the approach in AlphaFold2 (Jumper et al. 2021 1 compute the Local Dis- tance Difference Test (LDDT) score (Mariani et al. 2013) for both the generated and ground employing a model to predict the LDDT score. Furthermore, we bucket the LDDT score into one-hot bins, effectively transforming it into classification task, mirroring the approach AlphaFold2 (Jumper et al. 2021
molecules,
1 n NLL(
fidnnede
(4)
where ē; is the predicted LDDT (PLDDT) probability distribution, and e, is the one-hot vector of the bucketed LDDT bins. Through the extraction of the expected value from the pLDDT probability distribution, scalar pLDDT value can be derived, providing confidence score to rank and prioritize molecules.
SE(3) Equivariant Model Both fpm and for necessitate the use SE(3) equivariant models that accept 3D coordinates as inputs and yield new 3D coordinates as outputs. We mainly follow the design of the efficient. SE(3) -qqiivariant Transformer proposed in Uni-Mol (Zhou et al. 2023) and Graphormer-3D (Shi al.) 2022 However, these models do not account for the interaction between the pocket and the molecule. As result, we extend the model with an additional pocket encoder and incorporate a particle-pocket attention mechanism to capture the interactions between the pocket and the molecule. Since the focus of this paper is on pocket-based 3D molecular generation rather than the SE(3) equivariant models themselves, we provide the details of the designed SE(3) equivariant model in Appendix A.1
2.3 Extending VD-Gen to 3D Molecular Optimization
Molecular optimization is a crucial aspect of practical drug design. This process involves refining an existing molecule by replacing fragment improve its binding affinity, rather than creating new molecule from scratch. Here, have extended the VD-Gen method to encompass pocket-based 3D molecular optimization.

---

Published in Transactions on Machine Learning Research (03/2024)
As depicted in Fig. the procedure commences with the random removal of fragment from the given molecule. The model is then trained to regenerate this fragment, using the pocket and remaining atoms of the molecule as conditional inputs. Although the model is not explicitly trained for molecular optimization, it acquires the ability to remove and replace fragments within a molecule, thereby rendering it applicable to oplimization lasks. Detailed results of the molecular optimization benchmarks can be found in Appendix C.6
3 Experiments
3.1 Settings
Evaluation Metrics There is no single golden metric for evaluating generated molecules; thus, we employ
multiple metrics to ensure comprehensive assessment.
1) 3D Similarity. Since pocket-based 3D generation models are trained using 3D structures of pockets and molccules, the most direct metric for assessing the models' generative capabilities to evaluate the 3D similarity between the generated molecule and the ground-truth one. We use LIGSIFT (Roy & Skolnick 2015 to calculate the overlapping ratio in 3D space between two molecules.
2) Vina. Docking scores, such as Vina (Trott & Olson 2010 are commonly used in previous pocket-based generation studies due to their ease of computation. To maintain consistency with prior works, we also use Vina as a metric. However, previous studies often relied on Vina's re-docking, which could significantly alter molecular conformation and binding pose due to docking tools. Therefore, to directly assess the 3D molecules generated by our model, we introduce an additional Vina* score that does not use re-docking.
3) MM-PRSA. Although docking scores are quick and easy to compute, they were designed to identify potential hits in large-scale virtual screening, not for ranking. Consequently, docking scores are not suitable metrics for comparing binding alfinities across different models (Cheng et al.) 2009). As result, employ the slower but more accurate MM-PBSA (Molecular Mechanics Poisson Boltzmann Surface Area) (Genheden & Ryde 2015) metric. Based on MM-PBSA, we introduce two additional metrics: MM-PBSA B.T. (MM-PBSA Better than Target), which calculates the percentage of generated molecules with belter MM-PBSA scores than the goound-truth, and MM-PBSA Rank, which computes the average rankings of different models among various complexes. Since MM-PBSA scores can vary significantly between complexes, MM-PBSA Rank offers more effective comparison of different models.
The details of these metrics are described in Appendix
2022 Peng al,, 2022) specifically the CrossDocked data. (Francocur et al. 2020 to train VD-Gen. The CrossDocked dataset originally contains 22.5 million docked protein-ligand pairs. Luo et al. (2022) filter out data points whose binding pose RMSD is greater than 14, and use MMSeqs2 (Steinegger & Söding 2017) to cluster the data, and randomly select 100,000 protein-ligand pairs for training. For the test set, we utilize 100 protein-ligand complex crystal structures from (Yang et al. 2022), on which MM-PBSA was validated to be effective. To assess the diversity of our lest sel, we employed MMSeqs2 to cluster the protein sequences, maintaining a threshold of 40% sequence identity. Our analysis yielded approximately 73 distinct clusters, indicating significant diversity within the test set. To prevent data leakage, we exclude complexes from the training data whose protein sequences bear similarity to those in the test set. Specifically, we consider two protein sequences as similar if their e-value, obtained from the BLAST search results (Camacho et al. 2009); is greater than 0.4.
Data we use the same training dataset as used in previous works (Luo et al.
Training Initially, the 3D U-Net model used for the pocket cavity detection is trained independently, requiring around 20 hours on 8 NVIDIA A100 GPUs. Subscquently, the parameters of the 3D U-Net model are frozen, and the entire VD-Gen pipeline is trained end-to-end, laking approximately 15 hours on 8 NVIDIA A100 GPUs. Detailed eeper-parameters used during training can be found in Appendix C.I
3.2 Molecule Generation Performance
Baselines We compare the VD-Gen with several previonsly proposed 3D pocket- pocket-based molecular generation models, including the 3D density grid generative model LiGAN (Ragoza et al. 2022), the auto-regressive 3D
7

---

Published in Transactions on Machine Learning Research (03/2024)
Table 1: Performance on pocket-based 3D molccular generation. We additionally computed the standard
deviations for VD-Gen's results.
3D Sim(t)
Vina(J) Vina*(L)
MM-PBSA(J)
MM-PBSA- Rank(4)
MM-PBSA-
Model
B.T.(%))
LiGAN (Ragza et al. 2022
0.356
-6.724
-5.372
-18.462
2.59
0.3
3DSBDD
Luo et al. 2022
0.365
-8.662
-7.227
-30.560
2.31
2.29
GraphBP
Liu et al.
2022
0.333
-8.710
-3,689
-5.579
4.01
Mol
Peng al.
2022
0.352
-8.332
-6.525
-8.226
3.53
0.422
-8.998
-7.404
-50.744
1.16
11.623
VD-Gen
+0.0001
=0.0009
=0.0257
10.138
#0.163
0.43
0.43
0.43
0.43
3D similarity
0.42
0.42
0.42
0.42
0.41
0.41
0.41
0.41
0.40
0.40
0.40
0.40
0.39
0.39
0.39
0.39
0.38
0.38
0.38
0.38
20 Multiple of virtual particles
8
12
16
2
3
4
8
12
16
0.95 0.96
0.97 PLDDT
0.98
Movement iterations
Movement iterations
(a) Number of VPs
(b) Particle Movement
(c) Molecule Refinement
(d) Confidence Prediction
Figure 2: Ablation studies for VD-Gen pipeline.
generative models GraphBP (Liu et al. 2022 3DSBDD (Luo al. 2022). and Pocket2Mol (Peng al. 2022 For each model, generate 500 molecules for every pocket and subsequently select 100 of these molecules for evaluation. In the cases of 3DSBDD and Pocket2Mol, beam search is used and the top 100 molecules are chosen. For VD-Gen, the selection is based on the confidence score. For LiGAN and GraphBP, 100 molecules are randomly selected, as these models do not implement beam search. Additionally, We trained three VD-Gen models with different seeds, computed the mean and standard deviation of the inference results for VD-Gen.
Results As we pay more altention to the generated molecules with high binding affinities, we report the top 5-th percentile result for Vina, Vina*, and MM-PBSA. MM-PBSA-Rank is calculated based on the top 5-th percentile MM-PBSA result. The 10-th, 25-th, and 50-th percentile results are in Appendix C.3]
From the results in Table 1 it is easy to conclude: 1) VD-Gen significantly outperforms all other baselines in all metrics, with top-1 MM-PBSA Rank, demonstrating the superior performance of the proposed VD-Gen. 2) MM-PBSA B.T shows that VD-Gen can generate more molecules with better MM-PBSA scores than the ground-truth ones, while baseline hardly can. 3) In 3D Similarity results, VD-Gen also largely outperforms baselines, indicating that VD-Gen effectively learned the pocket-based 3D molecular generation and can generalize to unseen pockets. 4) Although some baselines achieve good performance on Vina scores, like GraphBP and Pocket2Mol, their Vina* and MM-PBSA scores are very poor. We believe the re-docking Vina fixes their generated 3D structures and then good Vina score could be obtained. This result indicates that the previously widely used Vina score is not good metric for pocket-based 3D molccular generation.
To summarize, the superior results on multiple evaluation metrics explicitly demonstrate the effectiveness of the proposed VD-Gen.
3.3 Ablation Study
Number of VPs VPs are employed approximate the distribution of molccular atoms. The accuracy of this approximation is intuitively expected to improve with an increased number of VPs. Therefore, we investigate the impact of the number of VPs on the overall performance, with the findings presented in
8

---

Published in Transactions on Machine Learning Research (03/2024)
The results clearly demonstrate that performance is inffuenced by the number of VPs, with higher numbers yielding better performance. However, we observe that the performance stabilizes when the number of VPs reaches 16 times the predicted molecular atoms. This suggests that utilizing an excessive number of VPs is unnecessary, as an appropriate balance between efficiency and performance can be achioved with a more moderate quantity.
Fig. 2(a]
Number of Movement Iterations Iterative movement critical in the VD-Gen. In Fig. 2(b) and Fig. 2 we benchmark the effectiveness of different iterations in Particle Movement and Molecule Refinement. For the results in Fig. 2(b] we reduce the iterations R2 to 0.25R2 Molecule Refinement stage, to better show the gain brought by Particle Movement stage. As shown in Fig. 2(b) and Fig. 2(c) can find more iteration
iterations improve the final performance in both two stages.
Effectiveness of Molecule Refinement The Molecule Resinement stage is used to further refine the 3D molecule extracted by Molecule Extraction. To examine how Molecule Refinement affects the final performance, we benchmarked different iterations. As shown in Fig. [(c) we can find the results with more iterations are better. The result indicates the necessity of the Molecule Refinement stage.
Effectiveness of Confidence Prediction The pLDDT score, outputed at Confidence Prediction, is utilized for selecting and ranking molecules, and we aim to evaluate its effectiveness. Specifically, we compule the correlation between 3D similarity and the PLDDT for the generated molecules within a pocket (PDBID ILF2), with the results displayed in Fig. 2(d) It is evident that a higher pLDDT score corresponds to better 3D similarity. This finding suggests that the confidence score provided by VD-Gen is reliable indicator for selecting and ranking the generated molecules.
3.4 Case Study
Here, we selected 5 protein pockets from the test set to visualize the results generated by VD-Gen for pocket- based generation (asks. Fig. 3 displays three top-scoring molecules (purple, middle column) for each pocket, based on MM-PBSA scores. These molecules are presented without any structural post-processing. Green molecules represent ground truth molecules, and the rightmost column illustrates the spatial overlap between the generated and original molecules.
In the first case (PDBID: 2XBW), the protein pocket features deep pit within the protein (bottom left of the image) that can accommodate approximately one benzene ring. This task is challenging due to the small size of the pit and its considerable distance from the pocket's center. The results reveal that VD-Gen successfully generated molecules with fragments grown within the pit. While the generated molecules exhibit good 3D similarity to the original molccules and favorable MM-PBSA scores, the Vina scores of the original molecule are substantially better than those of the generated molecules. Relying solely on Vina scores for molecule selection may result in the exclusion of optimal candidates.
In the second case (PDBID: 1BHX), the protein pocket is characterized by bulky structure, necessitating the generation of protein-interacting fragments at both ends of the pocket and connecting them via molecular backbone. The original molecule is elongated and distorted, rendering this prediction task challenging. VD-Gen-generated molecules effectively replicate the shape of the original molecules, adequately filling the irregular protein pockets. All three molecules exhibit good 3D similarity and MM-PBSA scores.
In the third case (PDBID: 2BRM), the protein pocket flat, which requires that the molccular backbone of the ligand bound to it should be close to planar structure, such as composed of conjugated aromatic rings. We can see that the molecules generated by VD-Gen are the same the original molecular structures, whose molecular backbone is planar structure composed of conjugated aromatic rings, and the part toward the outside of the pocket is flexible. We can see that in this case, Vina scoring, MM-PBSA scoring, and 3D similarity all show good agreements.
In the fourth case study (PDBID: 1D3P), the pocket presents Y-shaped configuration, with the native ligand forming a branched structure from the protein's 's interior to the exterior. The entire molecular structure is rather elongated, which adds to the complexity of the generation process. The molecules generated by
9

---

Published in Transactions on Machine Learning Research (03/2024)
Reference POBID: 2XeW
Generated Molecules by VD-Gen
Spatial Overlap
Vina -11.75 PBSA; -73.01
Vina * -8.89
PBSA: -78.77
Vina --2554
PBSA: -78.44
Vina *: -3110
PBSA: -78.20
POBID: 18HX
Vin: 3D Sim.: 0.45
Vina: -9.02
30 Sim.: 0.55
Vina: -9.87
3D Sim.: 0.50
-68.89
-85A:-61:01
PDBID: 28RM
99.17
Vina -8.26
PBSA: 42.14
Vina :: 8.83
PB54: -50.87
Vina*: -8.06
PBSA: -4.877
Vina* -8.89
PBSA: -46.43
POBID: 1D3P
Wna: -8.46 3D Sim.: 0.49
-8.32
30 Sim.: 0.44
Vino:-8.81
3D Sim.: 0.46
Vina *.8.99
P85A:-64.17
Vina +5563
P8SA:-62.36 -62.36
Vin59
PBSA: 558.15
Vina + -7115
PBSA: 56.25
PDBID: 2FGI
Vina*: P85A:-64.15
Vina*:
PBSA: -63.09
Vina *: 7.882
PBSA: 58.94
Vina -55.54 PBSA: -55.37
Figure 3: Generated molecules with high 3D similarity to the reference molecule and high PBSA scores for five protein pockets. Gray surfaces are the protein pockets. Pink molecules are the ground truth molecules. Purple molecules are the molecules generated by VD-Gen. Lower Vina score, lower PBSA score and higher 3D similarity indicate higher binding affinity.
VD-Gen successfully reproduce this branched shape, and the length each branch is nearly identical to that of the original ligand.
In the fifth case study (PDBID: 2FGI), a portion of the original ligand is oriented towards the solvent environment. (on the right side of the image). This segment does not strictly require specific structural
10

---

Published in Transactions on Machine Learning Research (03/2024)
71 1=2
4=4
Particle initialization
Particle Movement
1. PLDDT=0.932
4. PLDDT=0.958
8, pLDDT=0.992
16, PLDDT=0.986
Molecule Extraction
Molecule Refinement
Figure 4: An example to demonstrate the output results of VD-Gen's different stages, 71 (r2) is the iteration number of Particle Movement (Molecule Refinement). The increasing pLDDT scores in the pipeline indicate the effectiveness of VD-Gen.
features, it does not interact with the protein. The molecules generated by VD-Gen achieve good overlay with the original ligand in the pocket's interior, while adopting diverse structures near the solvent. This indicates that VD-Gen can effectively identify pockets and produce ligands with specificity. Consequently, the 3D similarity in this group may be slightly lower.
The cases presented in Figure 3 demonstraaee VD-Gen's robust generation capabilities across various challenging molecular generation tasks. The generated molecules can fill deep pockets, conform to large pocket trends, match specifie pocket structures and adopt diverse structures in the non-binding parts (such near the solvent) they show high 3D similarity to the molecules in the original crystal structure. Additionally, the MM-PBSA scores and 3D similarity consistently evaluate the quality of the generated molecules, whereas Vina scores are less reliable in some instances. This finding suggests that selecting molecules based solely on Vina scores may inadequate.
3.5 Visualization
To better understand how VD-Gen generates the 3D molecules, we present a visualization for the various stages in VD-Gen, as illustrated in Fig. Initially, the VPs are uniformly distributed within the protein pocket. During the Particle Movement stage, with more iterations, the VPs gradually aggregate into several clusters, converging towards the positions of molecular atoms. Subsequently, in the Molecule Extraction stage, 3D molecule with fewer atoms is extracted. Finally, the Molecule Refinement stage involves further refinement of the extracted 3D molecule to achieve more accurate 3D positions with improved pLDDT scores.
4 Related Work
Ligand-Bas -ased Molecular Generation Early studies on ligand-basee based molecular generation utilized sets of molecules as training data, generating new molecules based on the learned distribution. These methods primarily represented molecules as 1D SMILES strings and 2D molecular graphs, employing techniques
such as variational autoencoders (VAFs) Kusner al. 2017 7 Dai et al. 2018 Winter et al., 2019 Griffiths
& Hernández-Lobatte 2020 Dollar al. 2021 Oliveira al. 2022 generative adversarial networks
(GANs) (Guimaraes al. 2017: Sanchez- Sanchez-Lengeling
al. 2017). flow models (Shi et al. 2020 for one-shot
generation, recurrent neural networks (RNNs) (Olivecrona et al. 2017 Bjerrum & Threlfall 2017 Segler
2018 Flam-Shepherd et al.) 2021). and reinforcement learning approaches YYou et al. 2018 Jin et al. 2020 for step-by-step generation. Some works (Li et al. 2019 Lim el al., 2020 Gónez-Bombarell et al.
ct al.
2018 aimed to preserve structural features such as molecular scaffolds or physicochemical properties like quantitative estimate of drug-likeness to improve the quality of generated molccules compared random generation. Recently, several studies (Nesterov et al.] 20200] Simm et al.) 2020 Hoogeboom et al.) 2022 Wu
et al.
2022 have investigated ligand-based 3D molecular generation using variety of techniques, including
11

---

Published in Transactions on Machine Learning Research (03/2024)
VAEs, reinforcement learning, and diffusion-like methods, in conjunction with SE(3) equivariant models. However, these methods did not directly target the binding position and affinity against specific protein pocket, resulting in generated molecules that may not perform well in real-world tasks.
Pocket- Molecular Generation Binding affinity is a critical factor in drug design, leading recent research that incorporates protein pocket information into molecular generation. Early efforts (Skalic ct al.
condition for generating molecules in SMILES strings or molecular graphs. However, since binding alfinily depends on the spatial positions of both the pocket and the molecule, subsequent studies focused on creating molecules with 3D structures.
2019 Xu et al., 2021] encoded pocket information
Some approaches (Ragoza et al., 2022), known as 3D molecular density grid generation, converted pockets and molecules into 3D density grids and employed 3D convolutional models similar to image processing techniques. However, due to the large pocket cavity, the positions of pockets and molecules in 3D density grids are coarse-grained, resulting in information loss and difficulties in generating fine-grained molecules. Furthermore, this method is not end-to-end, converting 3D density to 3D coordinates is necessary, which often leads to additional accuracy loss.
Other works (Luo et al. 2022 Liu et al. 2022 Peng et al. 2022). known as auto-regressive 3D molecular generation, sequentially sampled atoms in 3D space to form a molecule. However, this approach is inefficient. due to the vast contimuous 3D positional space, Moreover, unlike the sequential nature of text, atoms in a molecule lack scquential order, making auto-regressive generation for 3D molccules less reasonable.
Recently, shape-based generation methods, such as (Adams & Coley 2022 Long et al. 2022); ; define shapes in 3D space and train models to generate 3D molecules that fit into these shapes. Given pocket, they first detect the shape of the pocket cavity and generate 3D molecules that can fit into that shape. However, besides shapes, the interactions between the atoms of the pocket and the molecule are also critical to binding affinity.
Lastly, some concurrent works have leveraged diffusion models, such as (Schneuing et al. 2022 2023), to generate all molecular atoms in 3D space simultancously. These works are similar to the initial version of VD-Gen, which directly moved atoms iteratively without involving virtual particles. However, these studies utilize diffusion mechanisms to alleviate training difficultios, whereas our method leverages optimal transport principles, specifically minimizing atom movement distances. Specifically, diffuusion models usually employ a small-step denoising task, which results in stable and straightforward training due to the minimal distribution change at each step. For example, Diff (Guan et al. 2023) directly leverages this small-step denoising mothod and can outperform previous models. However, Target also requires a significantly larger number of steps (1000 steps) to achieve satisfactory results. VD-Gen introduces more points (via a many-to-1 mapping) to alleviate the difficulties associated with the optimal transport in 1-to-1 mapping, This many-to-1 mapping allows for more local optimization of the transport, considcring only individual positions rather than global matching. Consequently, the training process becomes more stable. However, VD-Gen requires more points, resulting in more complex pipeline that must merge the additional points. Experimental results, as detailed in Sec. C.3 demonstrate the superiority of our method in comparison to the diffusion baseline Diff
Guan et al.,
5 Conclusion
In this paper, we present VD-Gen, a novel pocket-based 3D molecular generation pipeline designed to generate fine-grained 3D molecules with high binding affinities against protein pockets in an end-to-end manner. Specifically, numerous virtual particles are initially distributed randomly within the pocket cavity, and subsequently moved iteratively to approximate the distribution of molecular atoms based on the training data. A 3D molecule is then extracted from these virtual particles using deep learning models. Following this, the atoms within the extracted molccule are further refined through itcrative movement, resulting in a high-quality 3D molecule with fine-grained coordinates. Finally, a confidence score is predicted for the generated molecule for the need of selection and ranking. Experimental results demonstrate that VD-Gen outperforms other bascline methods in generating molocules with higher binding affinitics to protein pockets and more accurate 3D binding structures. Additionally, we provide an ablation study, case study, and visualizations to further illustrate the effectiveness of VD-Gen.
12

---

Published in Transactions on Machine Learning Research (03/2024)
Reproducibility We utilized CrossDocked data for the training, consistent with carlier models (Luo et al. 2022 Peng et al. 2022). Detailed formulations of our model can be found in Appendix A.1 while the training settings are elaborated in Appendix C.1
Broader Impact Statement Our method aims to generate molecules with high binding affinity in drug discovery. However, is critical to acknowledge and address the broader implications of this research from various perspectives, including drug safety and efficacy, dual-use concerns, and equitable development.
In terms of drug safety and efficacy, this research serves valuable starting point, but it does not guarantee the safety and efficacy of the resulting drug candidates. Further extensive experimental testing, including in vitro, in vivo, and clinical trials, will be required to confirm and validate the polential therapeutic benefits of the drug candidates.
Regarding dual-use concerns, we are aware that improved generative modeling of chemicals can carry risks of misuse, possibly leading to the development of harmful substances or drugs. As responsible researchers, we are committed to promoting and upholding ethical applications of our work.
Lastly, equitable development is a crucial aspect to consider when harnessing computational advances in drug development. We advocate for collaborative efforts and partnerships between researchers, industry, and policymakers to promote equitable access to the benefits of computational drug discovery.
References
Keir Adams and Connor W Coley. Equivariant shape-conditioned generation of 3d molecules for ligand- basedd drug design. arXie preprint arXiw:2210.04.0 2022.
Esben Jannik Bjerrum and Richard Threlfall. Molecular generation with recurrent neural networks (rnns). arXiv preprint arXi:: 1cXxc:1705.00011 2017.
Christiam Camacho, George Coulouris, Vahram Avagyan, Ning Ma, Jason Papadopoulos, Kevin Bealer, and
Thomas L Madden. Blast+: architecture and applications. BMC bioinformatics, 10(1):1 9, 2009.
Tiejun Cheng, Xun Li, Yan Li, Zhihai Liu, and Renxiao Wang. Comparative assessment of scoring functions on a diverse test set. Journal of chemical information and modeling, 49(4):1079-1093, 2009.
Christophe Chipot and Andrew Pohorille. Free energy calculations, volume 86. Springer, 2007.
Clara D Christ, Alan E Mark, and Wilfred F Van Gunsteren. Basic ingredients of free energy calculations: review. Journal of computational chemistry, 31(8):1569-1555 2010.
Özgün Çiçek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox, and Olal Ronneberger. 3d u-net: Learning dense volumetric segmentation from sparse annotation. In Sébastien Ourselin, Leo Joskowicz, Mert R. Sabuncu, Gözde B. Ünal, and William M. Wells III (eds.), Medical Image Computing and Computer-Assio Assisted Intervention MICCAI 2016 - 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II, volume 9901 of Lecture Notes Computer Science, pp. 424 432, 2016. doi: (1000/978-3-31944000 8) (, 49. URL http::://dooioree /1t.:///...0007//---1-99-9999990
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoencoder for structured data. arXiv preprint arXiv:1802.081 08786, 2018
Anita de Ruiter and Chris Oostenbrink. Free energy calculations of protein ligand interactions. Current opinion in chemical biology, 1544)5547-552, 2011.
Renee L DesJarlais, Robert Sheridan, George L Seibel, J Scott Dixon, Irwin Kuntz, and R. Venkataragha- van. Using shape complementarity as an initial screen designing ligands for a receptor binding site of known turee-dimensional structure. Journal of medicinal chemistry, 31(4):722 729, 1988.
Robert S DeWitte and Eugene I Shakhnovich. Smog: de novo design method based on simple, fast, and accurate free energy estimates. methodology and supporting evidence. Journal of the American Chemical Society, 118(47):11733 11744, 1996.
13

---

Published in Transactions on Machine Learning Research (03/2024)
Robert S Detitte, Alexey V Ishchenko, and Eugene I Shakhnovich. Smog: de novo design method based on simple, fast, and accurate free energy estimates. 2. case studies in molecular design. Journal of the American Chemical Society, 119(20):4608 4617, 1997.
Orion Dollar, Nisarg Joshi, David AC Beck, and Jim Pfaendtner. Attention-based generative models for de novo molecular design. Chemical Science, 12(24):8362-88)
2021.
Yong Duan, Chun Wu, Shibasish Chowdhury, Mathew C Lee, Guoming Xiong, Wei Zhang, Rong Yang, Piotr Cieplak, Ray Luo, Taisung Lee, et al. A point-charge force field for molccular mechanics simulations of proteins based on condensed-phase quantum mechanical calculations. Journal of computational chemistry, 24(16):1999225 2012, 2003.
Richard Mansfield Dudley. The speed of mean glivenko-cantelli convergence. The Annals of Mathematical Statistics, 40(1):40-500 50, 1969.
Jerome EberhardI, Diogo Santos-Martins Andreas F Tillack, and Stefano Forli. Autodock vina 1.2. 0: New docking methods, expanded force field, and python bindings. Journal of Chemical Information and Modeling, 61 (8):38891-338) 2021.
Richard Feyman. The character of physical law. 1965.
Daniel Flam- -hem-hhepherd, Kevin Zhu, and Alán Aspuru-Guzik, Keeping it simple: Language models can learn complex molecular distributions. Xiv preprint or arXim:2112.030) 2021.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jía, Richard B Iovanisci, Ian Snyder, and David R Koes. Three-dimensional convolutional neural networks and cross-docked data set for seructure-basct drug design. Journal of Chemical Information and Modeling, 60000)4200-4215, 2020.
Samuel Genheden and Ulf Ryde. The mm/pbsa and mm/gbsa methods to estimate ligand-binding affinities.
Expert opinion drug discovery, 10(5):449 461, 2015.
Rafael Bombarlom,,,,ll,,, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-parraguuiro, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guik. Automatic chemical design using a dala-driven continuous representation of molecules. ACS central science, 4(2):268-22) 276, 2018.
Harrison Green and Jacob D Durrant. Deepfrag: An open-source browser app for deep-learning lead optimization. Jowrnal of chemical information and modeling, 61 (1))):5523-255) 2529, 2021.
Ryan-Rhys Griffiths and José Miguel Herúández-Lobato. Constrained bayesian optimization for automatic chemical design using variational autoencoders. Chemical science, (1())5577-586 2020.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. arXiv preprint XiN::2303.03543, 2023.
Gabriel Lima Guimaraes, Benjamin Sanchez- Lengleii,,,g,,ng,, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik, Objective-reiiforeed generative adversarial networks (organ) for sequence generation models. arXiv preprint ar 1rXii:1705.10843, 2017.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molocule generation in 3d. In International Conference on Machine Learning, pp. 8867-8887. PMLR, 2022.
Araz Jakalian, Bruce L Bush, David Jack, and Christopher Bayly. Fast, efficient generation of high-quality atomic charges. am1-bec model: I. method. Journal of computational chemistry, 21()):132-146, 2000.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule gencration using interpretable substructures. In International conference on machine learning, pp. 4849 4859. PMLR, 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tumyasuvuakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 589, 2021.
14

---

Published in Transactions on Machine Learning Research (03/2024)
Hugo Kubinyi. 3D QSAR in drug design: volume 1: theory methods and applications, volume 1. Springer
Science & Business Media, 1993.
Matt J Kusner, Brooks Paige, and José Miguel Lornato Grammar variational autoencoder. In International conference on machine learning, pp. 1945-1954. PMLR, 2017.
Yibo Li, Jianxing Hu, Yanxing Wang, Jiclong Zhou, Liangren Zhang, and Zhenming Liu. Decpscaffold: a comprehensive tool for scaffold-based de novo drug discovery using deep learning. Journal of chemical information and modeling, 60(1):77 91, 2019.
Jacchang Lim, Lang-Yeon Hwang, Seokhyun Moon, Scungsu Kim, and Woo Youn Kim. Scaffold-based molecular design with graph generative model. Chemical science, 11(4):1188-1164 2020.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-299099 2988,99888 2017.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding, arXiv preprint arXiv:2204.094 2022.
Siyu Long, Yi Zhou, Xinyu Dai, and Hao Zhou. Zero-shot 3d drug design by sketching and generating. Advances in Neural Information Processing Systems, 15:23894-23909 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d molecule generative model for structure-based drug design. Xivi preprint arXiv:2203.1044 10446, 2022.
Valerio Mariani, Marco Biasini, Alessandro Barbato, and Torsten Schwede. Iddt: local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics, 29(21): 2728, 2013.
Julien Michel and Jonathan W Essex. Prediction of protein-ligand binding affinity by free energy simulations: assumptions, pitfalls and expectations. Journal of computer-atin aided moleculor design, 24(8):639 658, 2010.
Vitali Nesterov, Mario Wieser, and Volker Roth. 3dmolnet: a generative network for molecular structures. arXiv preprint Xrxix:2010.000 064777, 2020.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1 14, 2017.
André F Oliveira, Juarez LF Da Silva, and Marcos G Quiles. Molecular property prediction and molccular design using supervised grammar variational autoencoder. Journal of Chemical Information and Modeling, 62(4))817-828, 2022.
Alexey Onufriev, Donald Bashford, and David Case. Exploring protein native states and large-scale conformational changes with a modified generalized born model. Proteins: Structure, Function, and Bioinformatics, 55(2):383 394, 2004.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. 2ooket2mol: Efficient molecular sampling based on 3d protein pockets. arXiv preprint 2tl:2205.072443 2022.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 1319):2701-2713, 2022.
Ambrish Roy and Jeffrey Skolnick. Ligsift: an open-source tool for ligand structural alignment and virtual screening. Bioinformatics, 31(4):539 544, 2015.
Benjamin Sanchez-Lengellngg Carlos Outeiral, Gabricl L Guimaracs, and Alan Aspuru-Guzik, Optimizing distributions over molecular space. an objective-reinforeed generative adversarial network for inverso-design chemistry (organic). 2017.
15

---

Published in Transactions on Machine Learning Research (03/2024)
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, et al. Structure-based drug design with equivariant diffusion models. Xrviv preprint arXic:2210.1366 2022.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark Waller. Generating focused molecule librarics for drug discovery with recurrent neural networks. ACS central science, 4(1):120-131, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09383 2020.
Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tic-Yan Liu. Benchmarking graphormer on large-scale molecular modeling datasets. arxiv preprint ar Xrviv:2203. 04810, 2022.
Brian K Shoichet. Virtual screening of chemical librarics. Nature, 432(7019):8622866, 2004.
Brian K Shoichet. Scrconing in a spirit haunted world. Drug discovery today, 11(188144:607-6115; 2006.
Muhammed Shuaibi, Adeesh Kolluru, Abhishek Das, Aditya Grover, Anuroop Sriram, Zachary Ulissi, and C Lawrence Zitnick. Rotation invariant graph neural networks using spin convolutions. arXiv preprint arXiv:2106.0995 2021.
Gregor Simm, Robert Pinsler, and José Miguel Hernández-Lobato. Reinforcement learning for molecular design guided by quantum mechanics. In International Conference on Machine Learning, pp. 8959 8969. PMLR, 2020.
Miha Skalic, Davide Sabbadin, Boris Sattarov, Simone Sciabola, and Gianni De Fabritiis. From target to drug: generative modeling for the multimodal structure-based ligand design. Molecular pharmaceutics, 16 (10):4282 4291, 2019.
Martin Steinegger and Johannes Söding. Mmsoqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology. 35(11):1026-100 1028, 2017.
Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2): 455-461, 2010.
W Patrick Walters, Matthew Stahl, and Mark Murcko. Virtual screening- an overview. Drug discovery today, 3(4):160 178, 1998.
Jonathan Weed and Francis Bach. Sharp asymptotic and finite-sample rates of convergence of empirical measures in wasserstein distance. 2019.
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork- Clevert. Efficient multi-objective molecular optimization in continuous latent space. Chemical science, 10(34): 8016-8024, 2019.
Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, and Qiang Liu. Diffusion-based molecule generation with informative prior bridges. Xiv preprinl ar 2209 200o:2200000 2022.
Mingyuan Xu, Ting Ran, and Hongming Chen. De novo molecule design through the molecular generative model conditioned by 3d information of protein binding sites. Journal of Chemical Information and Modeling, (1(7)3344032544 2021.
Maohua Yang, Dongdong Wang, and Hang Zheng. Uni-gbsa: An automatic workflow to perform mm/ggbppbjsa calculations for virtual sereening. ChemRriv, 2022. doi: 10.26434 10.26434/chemrsin Accemmxiiy22222 2022-gdsgf.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovee. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information, processing systems, 31, 2018.
16

---

Published in Transactions on Machine Learning Research (03/2024)
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023. URL https///penrreiee..ntt// GK2RM6vVqKu
17

---

Published in Transactions on Machine Learning Research (03/2024)
A
VD-Gen Details
Table Symbols used this paper.
Symbol
Meaning
P
the sel of atoms in the pocket the set of virtual particles (VPs) that are generated at the r-th iteration in Particle Movement the set of atoms that generated the r-th iteration in Molecule Refinement the gridded 3D cubic of pocket atoms the predicted cubic gridded cubic
V,
W
Pp
C..
C m
the ground truth label for the gridded cubic
G
set ground-truth atoms the i-th pocket atom's 'om's lype (one-hot) the i-th pocket atom's coordinate the i-th ground-truth atom's 's type (one-hot) the i-th ground-truth atom's coordinate the i-th VP's type (one-hot) the r-th iteration in Porticle Movement the i-th VP's coordinate at the r-th iteration Particle Movement predicted atom type distribution of i-th VP at the r-th iteration
y.
a
y.
x.
y
a
a
The index of assigned target atom for the i-th VP in Particle Movement The index of assigned target atom for the i-th atom Molecule Refinement the i-th atom's 's type (one-hot) at the r-th iteration in Molecule Refinement Molecule Resinement
a
i-th atom's coordinate at the r-th iteration the pair representation of VP pair pair representation
q'
pair at I-th layer the pair representation of pocket atom pair the pair representation of VP and pocket pair
the predicted distance between VP and its target which is used filter VP. predicted probability of merging type of VP pair predicted atom number
m
kep
times of the number of atom, uses in Particle Initialization
u:
The indices VPs in i-th cluster Molecule Extraction the node representation of VP
h
h'
the node representation VP at [-th layer the node representation of VP in Particle Movement the pair representation of VP pair in Particle Movement the node representation atom Molecule Refinement the pair representation of atom pair in Molecule Refinement the node representation of pocket atom the model predict the model parameter to predict atom number
h
v
w
han
number
hps
the model predict pocket cavity Particle Initialization
pc
the model to predict pocket cavity in Particle Initialization model parameter Particle Movement the model parameter in Molecule Extraction model parameter Molecule Refinement the model parameter in Confidence Prediction
pm
ms
Pnr
Ocp
fpm
the SE(3) backbone model, return types and coordinates VPs Porticle Movement the SE(3) backbone model, return types and coordinates of atoms in Molecule Refinement
Ii
the number layers
A.1 Details of the SE(3) Backbone Model
Figure illustrates the structure of the SE(3) backbone model used in VD-Gen. The abbreviations "Repr.', Attn." and "Dist." stand for 'Representation' "Attention', and 'Distance", respectively. The pocket encoder is displayed on the left side, which initially utilizes an atom-type embedding encode the pocket atom type and a Gaussian kernel for encoding the pairwise distances between pocket atom pairs. Each layer of the pocket encoder incorporates self-attention layer. On the right, the VP encoder is shown, which similarly
18

---

Published in Transactions on Machine Learning Research (03/2024)
uses atom-type embedding and Gaussian kernel to encode the particle type and the pairwise distances between VPs. To facilitate interaction with the pocket encoder, an additional Gaussian kernel is used to encode the pairwise distances between VPs and pocket atoms. Within each layer of the VP encoder, a particlo-pocker attention layer is incorporated prior to the self-attention layer, enabling interaction with the pocket encoder.
The subsequent paragraphs detail the components within the backbone model, as well as the overall pipeline For the sake of simplicity, layer normalization has been of the backbone model, presented in Algorithm 2 omitted from the equations and algorithms.
Gaussian Kernel
The pair-type-aware Gaussian kernel (Shuaibi et al. 2021 Zhou et al. 2023 represented
Pij = (g(A(d biji a, b), nh o )]k E ((d
a br:
(d=w)= 20
where G(d. (1)) 2/ Euclidean distance of atom pair ij, and tij refers to the pair type of atom pair ij. A(di by: a, b) represents the affine transformation with parameters a and b, which transforms dj according to its pair type lij.
is a Gaussian density function with parameters μ and a. di denotes the
Pair Representation Pair representation Zhou et al. 2023 is used to further enhance the 3D spatial encoding. The pair representation is updated using the nulti-head Query-Key product results in self-attention.
h h Q LA
(h) 11
1+1 = -g 4
(6)
Vd
where h² denotes the atom/non /node representation of the i-th atom at the I-th layer, dy represents the pair representation of alom pair ij in the I-th layer, H the number of attention heads, d is the dimension of hidden representations, and W (Wi,) is the projection for Query (Key) of the (-th layer h-th head.
To incorporate 3D information into the atom representation, pair representation is utilized in self-attention.
W 1. (h., W
h 111h softmax 4+ concat, (h (11))
+ 9. h W/Y
Vd
(7)
where WV is the projection of Value for the 1-th layer h-th head.
Particlo-Pocket Attention The Particle-Pockot Attention is defined as follows:
P.Q 1, (h w Pkyr C.l.h
1+1. softmax h
1.
h P W.P.P.
+ 4if
Vd
(8)
h |+1 concata (h (1.h)
(h
h [+1 = h! + 91 h 1+1 + 92 ML.(M.(
where 91 and 82 are learned parameters with initialized values of 0, hy denotes the representation of the j-th
pocket atom, Li C.,.h represents the pair representation of particle-pockel pair ij in the 1-th layer h-th head, MLP refers to a fully-connected network with one hidden layer, and W.P.P W. P,K and P.V.V are learnable projections for Query, Key, and Value, respectively.
i.h
1,b
(E(3)-Equivarian) Coordinate Head Following (Zhou al. 2023 the head can be denoted as:
2/) (y)
y +1 yi +
Zij ReLU((a)) Ui)2,
(9)
n
19

---

Published in Transactions on Machine Learning Research (03/2024)
Algorithm Backbone Update
Require: P: pocket atoms, vx: virtual particles
1: h < Alom Type_ Embeddding(P)
b Embeddings from alom types Get invariant spatial positional embedding
q P. <- Gaussian (Dist Malrix(P,P))
3: # Updale Pocket Encoder
for E [1,... I.) do
5: hd h" <- Self Attn(hFi-1,, (
Update by self attention
6: RI + -MLP(h"))
> Update by Feed- Feed-Forwary Forward-Netwool
7: h" + -
h - Atom Type_Tyeeeeeeeeeee Embedding(V;
D Embeddings from atom types Get invarient spotial positional embedding Get inveriant spatial positional embedding of particle-pocket
9º t- Gaussian (Dist Matrix(V, (V. V.))
10: q c.u + Gaussian Kerne((DistMMariixV... P))
pairs
11: # Updote Particle Encoder
12: for 1 [1,.... L) do
13:
h., h',q <- Sell Attn(h' 111
D. Update by self allention Update by
14:
h' < -MLP(h')
15:
# Only enabled at every 4-layer
16:
if mod == then
17:
h q &- Particle Pocket_ Attn(hl, ha" q
b Update by Particle-Pocee Attention
18: x= + Atom Type_Headteee Head(h")
Atom Type Prediction
19: t1 + sample("""1)
b Sample an atom type based on predicted probability Coordinate update
20: 4I SE(3) Heaady"",4")
21: return V,41 = (x++) 3 " h²" 9"
where n denotes the total number of atoms, L represents the number of layers in the model, yf E R is the input coordinate of the i-th atom, and y - R the output coordinate of the i-th atom. The projection matrices U1 "fx and U2 R "x1 are utilized convert the pair representation to scalar, should be noted that the predicted coordinates also used to calculate the distance between VPs the predicted distance di in Equation
Atom Type Prediction Head A non-linear head with two layers is used to predict the atom type basod on the atom representation in the last layer of the particle encoder:
H; = MLP(h.)
(10)
where h represents the atom representation, and L is the number of layers of the particle encoder.
A.2 Pocket Cavity Detection by 3D U-Net
We use a 3D U-Net model Çişek et al. (2016) to predict the cavity space within a protein pocket. To do this, we first define the pocket's bounding box (cube) and construct a 3D grid with cubic cells of length 2A. Given the atoms of a pocket P. we create a 3D grid cubic with binary voxel values (where '1' indicates the presence of pocket atoms) to serve as the input for the 3D U-Net model. For the model's 's training target (the cavity space), utilize the atoms in the ground-truth molecules, treating grids containing atoms cavities.
Formally, denote this process as Cm = Mpx(Crpi Opi), where hpi represents the 3D U-Net model with learnable parameter 9pi, C, E {0, 134112222222 is the gridded 3D cubic (of size ( x le x l3) containing pocket atoms, and Cm E {0, 1 Xixxlx is the predicted cubic, with grids having a voxel value of potentially representing cavities. The training objective function is grid-wise binary classification. Additionally, we employ a focal loss Lin al. (017 to alleviate the unbalanced classification problem:
Cpt
FL( 2..1k)
(11)
11 x 1a x
Here, FL denotes the focal loss function, Uijjk is the one-hot vector of voxel labels, and U.sjjk is the predicted vector obtained from model hpi-
20

---

Published in Transactions on Machine Learning Research (03/2024)
Pocket Repr.
layer
FFN Layer
N Blocks
FFN Layer
Model Block
Attn. Layer
Self Attn. Layer
Particle
Gaussian Kernel
Gaussian
Gated
Particle Pair Repr.
Packat
Repr.
Kerne
Layer
Regr.
Particle Pocett Pair Dist.
Kernel
Pocket atom Repr.
Particle-Pocket
Dist.
Attin. Layer
Pocket Pair Dist. Coordinates
Coordinates
Linear
Linear
Linear
Partidle Pocket Pair Repr.
K
Particle Repr.
Pocket Repr.
Pocket Encoder
Virtual Particle Encoder
Figure 5: The backbone model used in VD-Gen. Details are in Appendix A.1 and Alg. 2
A.3 Training Loss in VD-Gen
we convert several regression tasks into classification tasks by binning. The training loss for classification can then be written as:
Binning Regression As described in Section 2.2
Cuioning rrgressiin
Σ NLL(E.,
(12)
where 2; represents the predicted probability distribution of bins, and z; denotes the one-hot vector of the target bin. During inference, the predicted value can be calculated from the predicted distribution over bins as follows:
Bain
(bin_valk)2 [1,)),
(13)
where is the number of samples, bin__ala represents the bin value of the k-th bin, Abin denotes the number of bins, I is the size of each bin, and [ indicates the predicted probability of the k-th bin. Notably, the bin value is not the bin boundary value; instead, it is the average of the left and right boundaries.
Focal Loss Several focal losses are also used in Section
2.2 Formally, those can be denoted as:
CPocal
log x (1 ,*
(14)
where n is the number of samples, K represents the number of types, 2, denotes the one-hot vector of the larget type, z; indicates the predicted probability of the type, and y are hyper-parameterss used to balance classes. In this paper, y set to 2.
LDDT The LDDT score is widely used in protein structure prediction MMariani et al. 2013) and can be denoted as:
LDDT
XΣ> 4
((eTT 0.5) + (erryy < 1.0)
n i=1 eqi
(15)
+ (err,
< 2.0) + (err,y) < 4.0),
where erry = L1 [1| 30 fte
3 1z 11 " 12),
(16)
Lh represents the predicted coordinate of the i-th particle after Molecule Refinement, and gf denotes its ground truth coordinate.
21

---

Published in Transactions on Machine Learning Research (03/2024)
Algorithm VD-Gen Inference Pipeline
Require:
Ri, R2: iterations in Porticle Movement and Molecule Refinement, kup: times of the number of atom, P: pocket atoms with types and positions, Cy: the gridded 3D cubic of pocket atoms, han: the model predict atom number, Pan the model parameter to predict atom number, Apc: the model to predict pocket cavity in Porticle Initialization, Op: the model parameter to prodict pocket cavily Particle Inilialization, 0pm, Ome: Omrs Oepsi model parameters in Particle Movement, Molecular Estraction, Molecular Refinement and Confedence Prediction
# Iterative Movement.
3: def ITER MOVE(R, Vo, P. P) for k [1, ... R] do
5: Vi,h", q² + Backbone Backbone_Updatee V Vx-1, P:0)
> backbone model update as in Alg.
6: output VR, ha,
7: # Particle Initialization
8: Cm = hp. p.. Ppt)
D use 3D U-net model to predict the gridded cubic Predict atom number
= Pan(P, Oan)
10: for i in (ilko. do
11: x. one (MAAS))
The types of VP' initialized meaningless [MASK] type 12: - ""Unnffom(CCC CC [Cm == 1I) b the grids in the predicted gridded cubic with voxel value are taken as cavity and the initial coordinates VPs uniformly sampled from the cavity space.
13: Vo ((x], ((){))))) 1
14:
15: # Purticle Movement
16: VR,, h .q" +- ther_Move(( R1, Vo, P,0
Predict coordinates and types with iterative movement
17:
18: # Molecule Extraction
19: 8 + Predict Distance(h* (me)
Predict the distance between VPs and the targets to filter particles Predict merging matric
20: F + Predict Merge(q" ; (0me)
21: Wo < Molecule Extraction V Rt n, 8)
22:
Filler and Merge VPs in Alg. using predicted merging matris, alom namber and predicted dislance
23:
24: # Molecule Refinement
25: WRz, h" q" x Iter _oor__Move[ftt Wo, Pm
Refine the coordinates and types
26: # Confidence Prediction
27: Pred LDDT < Predict__ (Confidenceettt" W; Ocp)
Predict LDDT Return final positions and types, and confidence score
28: return Rg, DDDT
LDDT
A.4 VD-Gen Overall Algorithm
We summarize the overall inference pipeline of VD-Gen in Algorithm 3 First, 3D U-Net model is used to predict the pocket cavity. Subsequently, the pocket encoder with a prediction head estimates the number of atoms based on the pocket representation. The main algorithm primarily relies on the "Iter Move' function, which iteratively moves the VPs/atoms. Both Particle Movement and Molecule Refinement use the "Tter Move" function for iterative refinement. In Molecule Extruction, two heads are used to predict the filtered probability and the pair merging probability, respectively, basod on the node representation h and the pair representation q¹. Based on these predictions, the 'Molecule Extraction" function constructs molecule from VPs as described in Algorithm
The training pipclinc is very similar, except for the following differences. Firstly, to enhance efficiency, both Ri and R2 are sampled from the range of 1 to 4, and gradient backward is only activated during the final iteration. Secondly, the Molecule Extraction, a teacher-forcing merging strategy (excluding binary search) is used for training. This entails utilizing the ground truth values for pair-wisc merging probabilitics and atom numbers, rather than generating predictions. Lastly, loss functions are enabled to obtain gradients essential for the training process.
22

---

Published in Transactions on Machine Learning Research (03/2024)
Algorithm Molecule Extraction Algorithm
Require: Va, = {(,," R1 W/ ",""""") ) virtual particles at Ri iterations, ne: the predicted alom num, {(t)))
the predicted merging probability matrix, s = (3:) the predicted distance between VPs and larget positions,
Ç: the filtering threshold
1: # Set the merge type between particles with the particle to be filtered to 0
2: for i in [1,.., n] do
# Filter the VPs according to predicted distance
4: if 5, > then
1 0 : 0
Set [ftg]]=1 to b Set (f5,))=1 to 0
6:
high- max ((t)))
low <- min( (=
mid low+high
10: # using the binary search to find the threshold
11: while low high do
12:
Fij mid
13:
Wo -
14:
# Greedy merge based random order
15:
for in random_ 1a1toom_permm11111 do
16:
< for j in [I....,n] n| do
17:
18:
if ri True then r[:.1] +- False wi.add(j)
19:
to the merged particle will not be merged again add the porticle indices into the i-th cluster
21:
len(w,, > 0 then 20 Uniform( (f (x w.})
22:
Sample atom type from the merging list 9 + Mean(1y Re [ wiH) & the weighted overage position according to the predicted distance is atom
23:
position after merging
24:
Wo.ddd((#n. (lm))
b add the alom to the set Count the number of clusters
25:
m m1
# find the threshold
27:
if m m then
28:
break
29:
else
30:
m < m then low mid
31:
too mony particles merged, the threshold needs to be increased
32:
else
33:
high mid
Too few particles merged, the threshold needs to be lowered Return particle set after merging
return Wo
A.5 Molecule Extraction Algorithm
The process of merging VPs into atomss is presented Algorithm 4 where a binary search technique used identify a merging threshold. To reduce training costs, teacher-forcing is utilized, thereby climinating the need for binary search during training. Instead of predicting pair-wise merging probabilities and the number of atoms, their ground truth values are directly used. During inference, the binary search technique is applied. Furthermore, to account for potential errors in atom mumber prediction, a range (=1 of atom numbers is explored, with the final selection based on confidence scores.
B The Failed Attempt
In the initial version of VD-Gen, virtual particles were not incorporated into the pipeline. Specifically, there were no Particle Movement and Molecule Extraction stages, and the number of initial particles was equal the predicted atom number m. Furthermore, to ensure that each particle had target atom, we employed
23

---

Published in Transactions on Machine Learning Research (03/2024)
0.40
3D similarity
0.35
0.30
0.25 VD-Gen 1:1
VD-Gen
Figure 6: Comparison for different training frameworks. An initial attempt, called 'VD-Gen 1:1", involves employing an equal number of particles and atoms, maintaining a 1:1 particle-atom bijective mapping, and iteratively moving particles without the VP-to-atom extraction process. Results demonstrate the effectiveness of the proposed VD-Gen framework.
the Wasserstein distance obtain an optimal bijective mapping between particles and target atoms (solvod by Hungarian algorithm). While this approach appeared more straight.orward than the current VD-Gen, it proved to be less effective. More specifically, A closer examination revealed that when randomly positioning the m virtual particles, some were close to the ground-truth positions, while others were far from their intended destinations. This discrepancy led to training instability: setting a long movement iteration caused some "near' particles to fail to learn to stay, while a short movement iteration prevented 'far" particles from reaching their destinations. This issue is closely related to the high variance and slow convergence of the Wasscrstein distance with respect to the number of samples, discussed in (Dudley 1969 Weed & Bach 2019).
observed unstable training (i.e., divergence) in this setting.
To address this problem, we introduced virtual particles(VPss, ensuring that there were more VPs than atoms. By employing more VPs and the nearest target assignment, each target atom was highly likely to have several VPs in close proximity. These VPs could then be moved to the target atom using fewer and similar movement iterations. Once the VPs were moved, they clustered around the positions of the target atoms. A clustering algorithm could then be utilized to extract aloms from the VPs. Although some VPs might fail, this would not impact the clustering results, leading stable training.
We conducted an additional experiment to elucidate the superior performance attained by utilizing VPs. Figure f illustrates that the performance of the initial version of VD-Gen (referred to as "VD-Gen 1:1") is considerably inferior to that of the current version.
C Experiment Details and More Results
C.1 Training Details
The detailed configurations of VD-Gen are listed in Table Table While a multitude of hyperparameters are introduced in our approach, only a select. lew are of paramount importance. As depicted in Table 4 beyond standard hyperparameters such as layer count, hidden dimensions, and learning rate we introduce additional parameters: six loss coefficients, three stability thresholds denoted by т, 6, and Ç, parameters R and Rg representing iterations in Particle Movement and Molecule Refinement respectively, and kep which indicates multiplicative factor. Among these, Rí, R2, and kep especially important and were carefully tuned shown in our Ablation Study (Section 3.3 For the other hyperparameters, we chose values based on previous experience since their impact model performance seems small. Notably, the model's training stability was maintained even without extensive tuning of these parameters. Our approach's 's effectiveness is further supported by empirical evidence: VD-Gen consistently outperforms previous benchmarks, showing its relative insensitivity to hyperparameter changes.
The codes of the 3D U-net model implemented based http::/ssthaaccmm///////yyvvva/v
24

---

Published in Transactions on Machine Learning Research (03/2024)
Table 3: Settings for the 3D U-Net models in VD-Gen.
Name
Value
Number of U-Net encoders
5
Number of U-Net decoders
5
Output channels in each encoder
16,
32, 64, 128, 256
Convolution kernel size
3
Pooling kernel size
2
Batch size
16
Max training steps
500k
Warmup steps
20K
Peak learning rate
2e-1
Adams é
1e-6
Adams(31, (2))
(0.9,0.99)
Gradient clip norm
0.5
C.2 Evaluation Metrics
3D similarity. We use LIGSIFT (Roy & Skolnick 2015 for calculating 3D similarity. By default, LIGSIFT aligus the input molecules before computing 3D similarity. However, we aim to assess the generated 3D structure directly to examine the end-to-end performance. Consequently, we remove the alignment feulure in LIGSIFT.
Vina. We utilize AutoDock Vinal.2 (Eberhardt et al. 2021 to obtain Vina scores. Specifically, re-docking is applied, meaning that the binding pose and the conformation of the ligand molecule generated by the model are disregarded, and a new binding pose and molecular conformation are re-calculated by AutoDock Vinal.2 2. We argue that re-docking in Vina may not accurately reflect the actual performance of bocket-based 3D molecular generation. However, for consistency with previous work, we retain as one of the metrics.
Vina*. Vina* refers to Vina without re-docking. We use the built-in energy optimization process based on the Vina scoring function in AutoDock Vinal.2 (Eberhardt el al., 2021 minimize the energy of the binding pose of generated molecules. Subsequently, we use the Vina scoring function score the energy meeryy-inimizzd binding pose, resulting in the Vina* score.
MM-PBSA. We adopt the default scttings of parameters (i.c., solvation mode: GB-2(Onufriev al. 2004] protein forcefield: amber03(Duan et al., 2003), ligand charge method: bec(Jakalian al. 2000),, dielectric constant: 4.0) and workflow (i.e., force field building, structure optimization by energy minimization, MMMGBBPBBSAA caleulation) of @Yang et al. 2022 to compute MM-PBSA scores. As the crystal structure indicates the preferred binding pose against specific target, we filter the generated molecules based on their 3D similarity to the molecule in the crystal structure and consider molecules with a 3D similarity score above 0.4 as effective. We only calculate the MM-PBSA score for these effective molecules. In Table G we present MM-PBSA S.R. (success ratc), which calculates the proportion of effective MM-PBSA scores of the generated molecules. For MM-PBSA B.T. and MM-PBSA Rank we have:
[g M./MMMPBSA(g) MM-PBSA(T)))
MM-PRSA B.T.
np 1=1
[M
MM-PBSA Rank =
rank,,
where Up denotes the number of proteins in the test set, M, represents the generaled molecular set of the i-th protein, T, denotes the ground-truth molecule in the crystal structure of the i-th protein, and rank, represents the ranking index of the current model among all compared models under the i-th protcin, ranked by MM-PBSA.
Metric for ablation studics. For ablation studies, we use the 3D similarity between the generated molecules and the ground truth the metric, since reflects the generative ability based on the pocket structure, and Table 1 shows strong correlation between 3D similarity and binding affinity.
25

---

Published in Transactions on Machine Learning Research (03/2024)
Table 4: Settings for SE(3) models in VD-Gen.
Name
Value
Training
Particle encoder layers
12
Pocket encoder layers
15
Particle-Packet Attention layers
3
Peak learning rate
5e-5
Batch size
32
Max training sleps
100k
Warmup steps
10K
Attention heads
64
FFN dropout
0.1
Attention dropout
0.1
Embedding dropout
0.1
Weight decay
le-4
Embedding dim
512
FFN hidden dim
2048
Gaussian kernel channels
128
Activation function
GELU
Learning rate decay
Linear
Adams
1e-6
Adams(33, $2)
(0.9,0.99)
Gradient clip norm
1.0
Loss weight of fan in Particle Insitialization
1.0
Loss weight Particle Movement
1.0
Loss weight of LMerge in Molecule Extraction
10
Loss weight of Lervor_pree in Molecule Extruction
0.01
Loss weight of Molecule Refinement
1.0
Loss weight. for Confidence Prediction
0.01
T, the clip value for coordinate loss
2.0
8, the threshold for coordinate regularization
1.0
Ç, the filtering threshold in Molecule Extraction
2.0
R1, Iterations in Porticle Movement
sampled from [1, 4]
Ra, Iterations in Molecule Refinemend
sampled from [1, 4]
kep times of the number of alom, uses in Particle Inilialization
sampled from [16.0, 18.0
Inference
R1, Iterations in Porticle Movement.
4
Ra, Iterations in Molecule Refinement
16
C.3 More Results
In Table 5 report more percentile results for Vina, Vina*. In Table we report more percentile MM-PBSA results and MM-PBSA S.R. scores. The MM-PBSA S.R. scores in many baselines are very low. Thus, there are not enough effective MM-PBSA results to calculate percentile results in some basclines. Therefore, each pocket, we replace the failed MM-PRSA result with the worst one generated by that baseline. And we calculated the percentile results after the replacement.
In Table 7 we compare VD-Gen with diffusion-based based model, TargetDiff Guan et al. 2023 ). From the result,
find that proposed VD-Gen is better than Tafget We also present some cases in Fig. 7 to demonstrate the difference of generated molecules between VD-Gen and TifgetDiff. It can be observed that, under similar Vina scores, the molecules generated by VD-Gen exhibit higher 3D similarity to the original molecules, better MM-PBSA results, and topological structures more closely related to the original molocules compared those generated by Target Furthermore, some unrcasonable structures are present in the molecules generated by TifgetDiff. For instance, the leftmost molecule forms rare 15-membered ring structure in drug molecules, which is predominantly composed of aliphatic bonds, implying high degree of
26

---

Published in Transactions on Machine Learning Research (03/2024)
flexibility. Such molecule is unlikely to stably bind with the target protein. Additionally, the presence of two consecutive carbon-carbon double bonds in the molecule represents an unreasonable chemical structure.
Table 5: More results on Vina and Vina*. We additionally computed the standard deviations for VD-Gen's results.
5-th
10-th
25-th
50-th
Model
Vina(4)
Vina (()
Vina(L)
Vina*
Vina (4)
Vina (()
Vina(L)
Vina (L)
LiGAN (Ragoza et
2022
-6.724
-5.372
-6.324
-4.922
-5.740
-4.215
-5.065
-3.49
3DSBDD
Luo
2022
-8,662
-7.227
-8,296
-6.664
-7.557
-5.633
-6.474
-4.078
GraphBP
Liu et al.
2022
-8.710
-3.689
-7.832
-2.774
-6.765
-1.169
-5.625
-1.2
Pocket2Mol
Peng et
2022
-8.332
-6.525
-8.015
-5.399
-7.467
-3.513
-6.837
-1.808
-8.998
-7.404
-8.570
-6.737
-7.891
-5.732
-7.207
-4.549
VD-Gen
+0.0009
+000257
+0.0024
#0.0013
#0.0017
+0.016
+.00038
+ 0.015
Table 6: More MM-PBSA results. We additionally computed the standard deviations for VD-Gen's results.
5-th
10-th
25-th
50-th
MM-PBSA-
Model
MM-PBSA(L)
MM-PBSA(L)
MM-PBSA(L)
MM-PBSA (L)
S.R.(%t)
LiGAN [Ragoza et al.
2022
-18.462
-13.774
-8.775
-7.418
11.9
3DSBDD
Luo el
2022
-30.560
-236633
-13.544
-7.739
12.9
GraphBP
Liu et al.
2022
-5.579
-4.894
-4.894
-4.894
0.2
Pocket2Mol
Peng et
2022
-8.226
-5.945
-5.398
-5.398
1.8
-50.744
-44.268
-35.178
-25.191
43.106
VD-Gen
#0.138
10.186
#0.168
#0.137
+ 0.015
Table 7: Comparison with ilffussion-bsed models. We additionally computed the standard deviations for
VD-Gen's 's results.
3D Sim(t)
Vina(I)
Vina
MM-PBSA- MM-PBSA(L)
Model
B.T.(%+)
Diff Guan et al, 2023
0.368
-8.710
-7.025
-35.46
4.3
0.422
-8.998
-7.404
-50.744
11.623
VD-Gen
+-0.0001
++0.0009
+0002557
+-0.188
+00.63
C.4 Diversity Results
In Tabel report the diversity of molecules generated by VD-Gen and other baselines. Specifically, we report the proportions of unique SMILES and scaffolds present in the generated molecules. The findings reveal that the molecules produced by VD-Gen exhibit greater diversity compared to those generated by other baselines, particularly in terms of scaffold diversity.
27

---

Published in Transactions on Machine Learning Research (03/2024)
Table 8: More results on Diversity. We additionally computed the standard deviations for VD-Gen's results.
Model
LiGAN
3DSBDD
CraphBP
Pocket2Mol
TargetDiff
VD-Gen
SMILES Diversity (t)
92.3
83.98
54.17 50.82
61.44 36.35
96.2
95.36 0.033
Scaffold Diversity (1)
54.87
55.95
61.6
79.46 0.169
Target PDBID: 2XBW
Molecules generated by TargetDiff
Vina: -11.021
3D Sim: 0.403
Vina: -9.82
3D 0.420
Vina: -8.986
Sim: 0.401
Vino -6.562
PBSA: -62.88
Vina*: -4.538
PBSA: -56.67
Vina*: -52286
PBSA: -58.11
Molecules generated by VD-Gen
Vina: -10.72 3D Sim.: 0.54
Vina: -9.72
Sim.: 0.57
Vina: -8.21
Sim.: 0.49
Vino f: -2.54
PBSA: 77.744
Vina -8.89
PBSA: -78.97
Vina* -3.10
PBSA: -78.20
Figure 7: More examples for the generated molecules. We show the molecular graphs of the generated molecules of VD-Gen and TifgetDiff with high 3D similarity to the reference molecule for the target (PDBID 2XBW) along with Vina score and MM-PBSA.
28

---

Published in Transactions on Machine Learning Research (03/2024)
Table 9: Inference Efficiency.
Model
3DSBDD
GraphBP
Pocket2Mol
Target
VD-Gen
(L)
14.153
1.660
3.476
30
3.678
C.5 Inference Efficiency
The experimental results demonstrate the effectiveness of the proposed VD-Gen, and we also evaluate its efficiency. Specifically, we compare the inference speed of generating a single molecule for 3DSDBB, GraphBP, Pockot2Mol, and our VD-Gen. The results are summarized in Table TargetDiff (Guan et al. 2023 exhibits the slowest performance, primarily due to its diffusion sampling times being set at 1000. 3DSDBB is the slowest AR method due to its inefficient. MCMC sampling. Although GraphBP the fastest, generates the lowest-quality molecules. VD-Gen and 2ocket2Mol exhibit similar efficiency levels, but VD-Gen significantly outperforms Pocket2Mol in terms of effectiveness. Given the large number of VPs and multiple movement iterations, it is expected that VD-Gen will not be the fastest method. We leave the efficiency improvement to future work.
C.6 Molecular Optimization Results
Differences in Training Molecular Optimization Models Training the molecular optimization model involves several modifications. The specific pipeline for this task is illustrated in Figure Key changes include:
Instead of predicting the entire molecule, in this task, our model aims to predict only a portion of it. To achieve this, 25% to 40% of atoms are removed from the original molecule, with the remaining atomss serving as input for prediction.
During training, the number of VPs is significantly reduced, amounting to only 8 times the number of actual atoms.
VPs are not uniformly distributed within the pocket cavity; instcad, they are distributed around the removed atoms.
Experiment We compare model with traditional molecular fragments optimization model Deep- Frag (Green & Durrant, 2021). DeepFrag can replace molecular fragments based SMILES, which is a 1D model without pocket information. The results are shown in Table I0 and Table I1 From them, it is clear that VD-Gen can outperform the bascline in molccular optimization.
Pocket Condition
Particle Initialization
Particle Movement
Molecule Extraction
Molecule Refinement
Confidence Prediction
x R
Rg
Figure 8: Extending VD-Gen to molecular optimization.
29

---

Published in Transactions on Machine Learning Research (03/2024)
Table 10: Full percentile results on Vina and Vina*, in molecular optimization tasks. We additionally computed the standard deviations for VD-Gen's results.
5-th
10-th
25-th
50-th
Model
Vina(U)
VI)
Vina(J)
VIna*(I)
Vina(L)
VIna*())
Vina(I)
Vina*(1)
DeepFrag
-8.357
-8.132
-7.775
-7.372
-8.870
-7.484
-8.592
-7.095
-8.149
-6.339
-7.677
-5.334
VD-Gen
#0.002
10.0119
10.0022
10.0009
10.0019
10.0049
10.0033
10.0067
Table 11: Full percentile results on MM-PBSA, molecular optimization tasks. We additionally computed the standard deviations for VD-Gen's results.
5-th MM-PBSA(L)
10-th MM-PBSA(L)
25-th
50-th
Model
MM-PBSA(L)
MM-PBSA (L)
MM-PBSA B.T.(f)
DeepFrag
-51.783
-48.959
-39.786
-34.485
23.9
-53.270 #0.123
-50.387
-45.403
-37.226
29.987
VD-Gen
+0.255
#0.241
10.169
10.181
30


Reviews:
[{"review_id": "KzxgSbjqsx", "reviewer": "Reviewer_wMDA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The paper addresses an important problem in drug discovery - generating novel 3D molecules that bind to target protein pockets.\nThe proposed VD-Gen pipeline is novel and represents a clear advancement over previous methods by enabling non-autoregressive generation.\n- The virtual particle concept provides an intuitive way to approximate atom distributions for 3D generation.\n\n- The comprehensive set of experiments thoroughly validates the effectiveness of VD-Gen across different metrics.\nAblation studies provide useful insights and validate design decisions like using virtual particles.Extending VD-Gen to optimization tasks demonstrates flexibility of the approach.\n\nWeaknesses:  \n- Only one dataset was used for evaluation - testing on more diverse pockets could further validate generalizability.\n- Hyperparameters were not extensively tuned - optimal settings may further boost performance.\nEfficiency and scalability could be better, especially for real-world applications.\n- Applicability to very large novel pockets remains uncertain. More rigorous evaluation of generated molecular properties beyond binding affinity could be beneficial", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- Proposes VD-Gen, a novel end-to-end pipeline for generating 3D molecules conditioned on protein pockets. The pipeline contains 5 stages: virtual particle initialization, iterative movement, molecule extraction, iterative refinement, and confidence prediction.\n\n- Introduces the concept of using virtual particles to approximate the distribution of molecular atoms within the protein pocket cavity. This allows generating all atoms simultaneously in a non-autoregressive manner, avoiding issues with determining atom generation order. \n\n- Comprehensive experiments demonstrate VD-Gen can generate high quality 3D molecules exhibiting significantly better binding affinities than previous baselines across various metrics. Ablation studies validating the importance of key components like virtual particles, movement iterations, and confidence prediction. \n\n- Case studies and visualizations offer insights into how VD-Gen operates and generates reasonable 3D molecules tailored to pocket structures.\n\n- Extends the pipeline to 3D molecular optimization tasks, still achieving strong performance compared to fragment-based baselines.\n\n- The introduced model offers a promising solution for generating novel drug-like molecules against target pockets, addressing limitations of traditional virtual screening and previous generative approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Perform additional experiments on more diverse protein pockets to better validate general applicability. Two are examined in the case study. \n- Provide more details on the training data and model architecture to enhance reproducibility. The current model structure and parameter settings are not clearly claimed in the experiment setup, and so are the training data.\n- Perform hyperparameter tuning to determine if performance can be further improved. The current experiment can be further  analysed  to explore the method's potential and upper bound.", "broader_impact_concerns": "- Drug safety/efficacy: While the method aims to improve binding affinity, evaluating full safety and efficacy requires significant further testing. The limitations of computational screening should be clearly stated to avoid overstating capabilities.\n\n- Dual use concerns: While focused on drug discovery, improved generative modeling of chemicals does carry risks of misuse. The authors could affirm their commitment to ethical applications.\n\n- Equitable development: Ensuring equity in who benefits from computational advances in drug development could be considered.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vYdZy50HiJ", "reviewer": "Reviewer_Qyn7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strengths and weaknesses of this paper are listed as follows:\n\n**strengths:**\n\n1.  As far as I know, approximating the molecular atom distribution through the iterative movement of virtual particles has not been studied before. Therefore, the proposed method should be novel to some extent.\n\n2. The proposed method performs pretty well. It achieves the highest performance on all the metrics.\n\n**weaknesses:**\n\n1. Some parts of the paper are inconsistent. For example, the author mentioned they used the LDDT score to do the final selection and ranking. However, in experiments, they reported the pLDDT scores instead of LDDT scores.\n\n2. The writing of this paper is bad, which makes this paper hard to follow. For example, when the author merged the VPs into atoms, what information did the author use to identify particles belonging to the same atoms? The 3D coordinates or the atom type? In equation 4, as the LDDT score is a float value, why didn't the author directly use Euclidean distance? After clustering the VPs, $\\hat{x}^0$ is randomly sampled from the particles in the cluster while $\\hat{y}^0$ is set to the mean position of all particles, which seems not aligned. why didn't the author directly set y to the corresponding coordinate of the sampled x.\n\n3. There are so many hyper-parameters in this paper, such as number of particle movement iteration, the number of initial VPs, VP error threshold, number of molecule refinement iteration, clip value for coordinate loss, threshold for moving regularization... I feel training the model  might be hard.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new 3D molecular generation pipeline based on binding pockets of proteins in an end-to-end manner. In particular, numerous virtual particles are randomly sampled within the pocket cavity first, which will be iteratively moved to approximate the distribution of molecular atoms. From these moved virtual particles, a 3D molecule will be extracted and further refined through another iterative process. Finally, candidates will be ranked according to LDDT scores and the one with the highest score will be selected. As far as I know, approximating the molecular atom distribution through the iterative movement of virtual particles has not been studied before. The experimental results show that the proposed model achieves the highest performance on all the metrics.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The writting of this paper really needs to be refined, especially the design intuition of some important components.\n\n2. The author may need to provide a figure illustrating the overall architecture of the proposed method.\n\n3. The author mentioned in Introduction that their model can generate diverse drug-like molecules. However, I have't seen any diversity metric about the designed molecules. The author may need to provide additional diversity and novelty about the designed molecules.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2R27NhiCxf", "reviewer": "Reviewer_XPHM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- paper is well-written and well-organized. I appreciate that it is under 12 pages.\n- targets and interesting and relevant real-world problem\n- good experimental results\n- includes interesting ablation studies and visualization. I think this gives good insight into the model's behaviour, and is definitely better than most standard ML papers that just report average error metrics\n\nWeaknesses:\n- In my opinion, the pipeline is very complex and has lots of hyperparameters. It seems like the behaviour of the algorithm would be very sensitive to this. In particular, the merging procedure (step 3) seems \"hacky\". This is a subjective weakness though.\n- The work seems very similar to diffusion models in that it evolves a set of particles over time, but the relationship to diffusion models is only discussed very briefly. Diffusion models have the advantage of having a principled end-to-end training procedure and nice probabilistic interpretation. It would be good to get the author's perspective of what pros/cons VD-gen has with respect to diffusion models. This could also include more experimental comparison.\n- Tables have no error bars", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes an algorithm called VD-gen to generate small molecules in protein binding pockets. The model has several stages:\n\n- First, a neural network predicts the location of the binding site and initializes the virtual particles\n- Second, another neural network evolves the locations of the virtual particles by using a neural network to update their positions for several time steps\n- Third, a post-processing step is done to convert the virtual particles into atoms using a hierarchical clustering procedure\n\nThe authors evaluate their method on a test dataset from a past paper (Yang et al 2022), showing favourable performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In general I think the paper is good, so most of these changes are suggestions rather than requirements for me to recommend acceptance.\n\n**Important**\n\nTo me the most important change is discussing / relating the work to diffusion models more because of the apparent similarity with VD-gen. I think it would be useful to both discuss it qualitatively, but also add representative SOTA diffusion models as baselines to Table 1 (although if the authors really don't want to run extra baselines then I won't insist on it).\n\n**Suggested changes**\n\n- Clarify function definitions more. I found the input/output spaces of some functions (e.g. $M$ in section 2) to be unclear. I suggest the authors use the $\\mapsto$ notation to define functions: e.g. $M:\\mathcal{X}\\mapsto\\mathcal{Y}$.\n- Double check citations: I saw Feynman (2017) cited, but Feynman died in 1988\n- Add error bars or standard deviations to Table 1: since these methods have randomness it is important to characterize the scale of the randomness vs the scale of the difference between the performance of different methods. If the scales are comparable then I think a statistical test is needed to make claims about performance differences.\n- Could explain \"Wasserstein distance\" approach in paragraph 2 of section 2 a bit more. I think I got it but it might not be clear to other readers. E.g. \"One approach to train M would be to use supervised learning to minimize the error between the produced atoms and the ground truth atoms. However, this requires assigning each produced atom to an at\nom in the ground truth set. While this could be done by choosing an assignment with minimal Wasserstein distance, ...\"\n- Remove the phrase \"due to space constraints\". TMLR does not have space constraints.\n- Does SE(3) model mean SE(3) _equivariant_ model? If so then maybe change \"SE(3) model\" to \"SE(3) equivariant model\"\n\nIf you do make changes, I think myself and the other reviewers would appreciate you highlighting these changes in the revised pdf, e.g. by using `{\\color{red}}` to make the text red.\n\n**Questions**\n\n- In section 2.2, when assigning virtual particles, do you ensure that every ground truth atom has at least 1 assigned virtual particle? The description in the text made me think this is not the case\n- Why do you bucket things and turn them into classification tasks instead of leaving as regression? (e.g. equations 2,4)\n- Why does the magnitude of \"MM-PBSA\" vary so much between methods in Table 1?\n- Case study in 3.4: were these proteins in the training set (or similar proteins)? How similar are these examples to examples in the training set? In general it is easy for models to regurgitate things from the training set.\n- How are the confidence scores used by the algorithm? I could not find this in the text.\n- Why is $M$ a deterministic function and not a distribution?", "broader_impact_concerns": "no concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree with the novel contribution of paper, the technique of using virtual particles to approximate atom distribution, and the strong experimental results. 

Reviewers have concerns about the complexity of the method -- too many hyper-parameters. Some parts of the paper could be revised with a clearer description (e.g. section 2), the claim about diffusion models, and the analysis on variance. 

The author may want to revise according the detailed comments of three reviewers. 

Overall, the paper made adequate contribution and could be included in TMLR with small modification. I recommend Accept with minor revision.
Decision: Accept (poster)