Structure-based Drug Design with Equivariant Diffusion Models
Authors: Arne Schneuing
Yuanqi Du
Charles Harris
Arian Rokkum Jamasb
Ilia Igashov
weitao Du
Tom Leon Blundell
Pietro Lio
Carla P Gomes
Max Welling
Michael M. Bronstein
Bruno Correia
Venue: Submitted to ICLR 2023 (2023)
Keywords: Diffusion Models
Equivariant Neural Networks
Structure-based Drug Design
Molecule Generation
Conditional Generation

Abstract: Structure-based drug design (SBDD) aims to design small-molecule ligands that bind with high affinity and specificity to pre-determined protein targets. Traditional SBDD pipelines start with large-scale docking of compound libraries from public databases, thus limiting the exploration of chemical space to existent previously studied regions.
Recent machine learning methods approached this problem using an atom-by-atom generation approach, which is computationally expensive. 
In this paper, we formulate SBDD as a 3D-conditional generation problem and present DiffSBDD, an E(3)-equivariant 3D-conditional diffusion model that generates novel ligands conditioned on protein pockets. 
Furthermore, we curate a new dataset of experimentally determined binding complex data from Binding MOAD to provide realistic binding scenario rather than the synthetic CrossDocked dataset. Comprehensive in silico experiments demonstrate the efficiency of DiffSBDD in generating novel and diverse drug-like ligands that engage protein pockets with high binding energies as predicted by in silico docking.

Under review as a conference paper at ICLR 2023
STRUCTURE-BASED DRUG DESIGN WITH EQUIVARIANT DIFFUSION MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Structure- based drug design (SBDD) aims to design small-molecule ligands that bind with high affinity and specificity pre-determined protein targets. Tradi- tional SBDD pipelines start with large-scale docking of compound libraries from public databases, thus limiting the exploration of chemical space existent pre- viously studied regions. Recent machine learning methods approached this prob- lem using an atom-by-atom generation approach. which is computationally expen- sive. In this paper, we formulate SBDD as 3D-conditional generation problem and present Dif an -qquvaaaui 3ID-conditional diffusion model that generates novel ligands conditioned on protein pockets. Furthermore, we curate a new dataset of experimentally determined binding complex data from Bind- ing MOAD to provide realistic binding scenario that complements the synthetic CrossDocked dataset. Comprehensive in silico experiments demonstrate the effi- ciency of Dif in generating novel and diverse drug-like like ligands that engage protein pockets with high binding energies as predicted by in silico docking.
INTRODUCTION
The rational design of molecular compounds to act as drugs remains an oustanding challenge in biopharmaceutical research. Towards supporting such efforts, structure-based drug design (SBDD) aims to generate mooll-molecule ligands that bind to specific 3D protein structure with high affin- ity and specificity (Anderson,) 2003). However, SBDD remains very challenging and with important limitations. A traditional SBDD campaign starts with the identification and validation of target of interest and its subsequent structural characterization using experimental structural determina- tion methods. The first step in this process is the identification of the binding pocket; cavity in which ligands may bind the target to elicit the desired therapeutic effect. This can be achieved via experimental means plethora of computational approaches (Pérot et al. 2010). Once a binding site is identified, the goal is to discover lead compounds that exhibit the desired biological activity. Importantly, transition from leads to promising candidates the compounds need to be evaluated regarding other drug development constraints that are also hard to predict (toxicity, absorption, etc.).
Traditionally, SBDD is handled either by high-throughput experimental virtual screening (Lyne) 2002 [Shoichet] 2004) of large chemical databases. Not only is this expensive and time consuming but it also limits the exploration of chemical space to the historical knowledge of previously studied molecules, with a further emphasis usually placed on commercial availability (Irwin & Shoichet 2005). Moreover, the optimization of initial lead molecules is often biased process, with heavy reliance on human intuition (Ferreira et al. 2015).
Recent advances in geometric deep learning, especially in modeling geometric structures of biomolecules (Bronstein et al. [202]] Atz et al. [02I], provide promising direction for structure- based drug design (Gaudelet et al.) (021)) Even though utilizing deep learning as surrogate docking models has achieved remarkable progress (Lu et al., 2022, Stärk et al., 2022), deep learning-bard based design of ligands that bind to target proteins is still an open problem. Early attempts have been made to represent molecules atomic density maps, and variational auto-encoders were utilized to generate new atomic density maps corresponding to novel molecules (Ragoza et al.] 2022). However, it is nontrivial to map atomic density maps back to molecules, necessitating subsequent atom-fitte fittiini stage. Follow-up work addressed this limitation by representing molecules as 3D graphs with atomic coordinates and types which circumvents the unnecessary post-processing

---

Under review as a conference paper at ICLR 2023
2(44")=2000
L deto
Figure I: DiffSBDD protein-conditioned scenario. We first simulate the forward diffusion pro- cess gain trajectory of progressively noised samples over T timesteps. We then train model Pe to reverse or denoise this process that conditional on the target structure. Once trained, we are able to sample coordinates are diffused throughout the process. Ligands (z(4)) are represented as fully-connected graphs during the diffusion process (edges not shown for clarity) and covalent bonds are added to the resultant point cloud at the end of generation. The protein (2 (P) is represented as a graph but is shown surface here for clarity.
drug candidates from Gaussian distribution N (0, I). Both atom features and
steps. Li et (202I) proposed an autoregressive generative model sample ligands given the protein pocket as conditioning constraint. |eng et al. (2022) improved this method by using an -(3)-equivariant graph neural network which respects rotation and translation symmetries in 3D space. Similarly, (roita et al. (2021); Liu et al. (2022) used autoregressive models to generate atoms sequentially and incorporate angles during the generation process. Li et al. (2021) formulated generation process as reinforcement learning problem and connected the generator with Monte Carlo Tree Search for protein pocket-conditioned ligand generation. However, the main premise of sequential generation methods may not hold in real scenarios, since there is no ordering of the generation process and, as result, the global context of the generated ligands may be lost. In addition, sequential methods pose
computational complexities that make the model inference inefficient (Luo et al. 2021 Peng et al. 2022).
An alternative is one-shot generation strategy that samples the atomic coordinates and types of all the atoms at once (Du et al.) 2022b). In this work, we develop an equivariant diffusion model for structure-based drug design (DiffBDDD which, the best of knowledge, is the first of its kind. Specifically. we formulate SBDD as a 3D-conditioned generation problem where we aim to generate diverse ligands with high binding affinity for specific protein targets. We pro- pose an (equ)vrruv 3D-conditional diffusion model that respects translation, rotation, and permutation equivariance. We introduce two strategies, protein-conditioned generation and ligand- inpainting generation producing new ligands conditioned on protein pockets. Specifically, protein- conditioned generation considers the protein as a fixed context, while ligand-inpainting models the joint distribution of the protein-ligand complex and new ligands are inpainted at inference time. We also demonstrate that our model can be used for out-of-the-box for molecular optimization. We further curate experimentally determined binding dataset derived from Binding MOAD (Hu et al., 2005). which supplements the commonly used synthetic Cross Docked (Francoeur et al. 2020 dataset validate model performance under realistic binding scenarios. The experi- mental results demonstrate that DiftSBDD is capable of generating novel, diverse and drug-like ligands with predicted high binding affinities given protein pockets. The code is available at AF75/
Sstps://anonymern 4open.
BACKGROUND
Denoising diffusion probabilistic models (DDPMs) (Sohl-Dickstein et al. 2015 Ho al. 2020) class of generative models in-
Denoising Diffuusio Probabilistic
Models
2

---

Under review as a conference paper at ICLR 2023
spired by non-equilibrium thermodynamiis.. Briefly, they define Markovian chain of random diffusion steps by slowly adding noise to sample data and then learning the reverse of this process (typically via a neural network) to reconstruct data samples from noise.
In this work, we closely follow the framework developed by Hoogeboom et al. (022). In our setting, data samples are atomic point clouds Edata [ac, h]] with 3D geometric coordinates RR-x33 and categorical features E RNxx xxd where N is the number of atoms. A fixed noise process
q(z (dan) = (Z of
(1)
adds noise to the data Zdata and produces a latent noised representation 2, for 0...., T. a2 con- trols the Agnal-to-noise ratio (t a. / and follows either learned or pre-defined schedule from ag to Or (Kingma et al., (02T). We also choose variance- suriance-rreeeving noising
process (ong et al. 2020 with a2 = /1 o.
Since the noising process is Markovian, can write the denoising transition from time step & to s < t closed form as
q(z, Edaaa: 21) =N(xx
Orl:002 of
r/s
Adata:
oi
21 and o/ of of following the notation of Hoogeboom et al. 2022). This true denoising process depends on the data sample Zdata- which is not available when using the model for generating new samples. Instead, neural network de is used to approximate the sample Edata More specifically, we can reparameterize Equation ( as 2; = at Zdata to,E with e ~ (0, I) and directly predict the Gaussian noise éσ 6o (2,, t). Thus, Edata is simply given Edma 02 2t
with 0tjs a,
de és
ai
The neural network is trained to maximise the likelihood of observed data by optimising variational lower bound on the data, which is equivalent to the simplified training objective (Ho et al. [2020] Kingma et al. 2 Crain alle Φe Φz l22 up to scale factor (see Appendix A for details).
E(n)--quivvriint Graph Neural Networks A function / X' + ) is said to be equivariant w.r.t. the group G if f(g.x) fxx( where g. denotes the action of the group element g G on X' and y (Serre et al. 1977). Graph Neural Networks (GNNs) are learnable functions that process graph-structured data in a permutation-equuvarrinn way, making them particularly useful for molecular systems where nodes do not have an intrinsic order. Permutation invariance means that GNN(IIX) II GNN(X) where II E En, is n permutation matrix acting on the node feature matrix. Since the nodes of the molecular graph represent the 3D coordinates of atoms, we are interested in additional equivariance w.r.t. the Euclidean group E(3) or rigid transformations. An -euuivrrra GNN (EGNN) satisfies EGNN + b) II EGGNN(A) + b for an orthogonal 3 matrix A T A I and some translation vector b added row-wise.
In our case, since the nodes have both geometric atomic coordinates æ as well as atomic type features h, we can use simple implementation of EGNN proposed by [Satorras et al. (202T). in which the updates for features h and coordinates x of node i at layer 1 are computed as follows:
mij = (e.h,, h/, di azj); Zig = (m.j)
(3)
h/t = on (h' Σ m.y
(4)
jpf
x On (t),
at a! +
C.., ass)
(5)
j
where de. dun on and on are learnable Multi-layer Perceptrons (MLPs) and di and arj are the relative distances and edge features between nodes i and respectively.
3 EQUIVARIANT DIFFUSION MODELS FOR SBDD
We utilize an equivariant DDPM to generate molecules and binding conformations jointly with re- spect to specific protein target. We represent protein and ligand point clouds as fully fully-connected graphs that are further processed by EGNNs (Satorras et al.] 2021). We consider two distinct ap- proaches to 3D pocket conditioning: (1) conditional DDPM that receives fixed pocket represen- tation as context in each denoising step, and (2) a model that approximates the joint distribution of ligand-pocket pairs combined with inpainting at inference time.
3

---

Under review as a conference paper at ICLR 2023
Conditional generation
100 plac (
()
Inpainting
Training
Inference
Mask
Combine
Figure 2: Comparison between the conditional generation and inpainting approaches. The condi- tional model learns to denoise molecules (4) in the fixed context of protein pockets = In the inpainting scenario, the model first learns to approximate the joint distribution of ligand and pocket nodes data with forward diffused representation of the pocket in each denoising step.
(L.P) For sampling, context provided by combining the latent representation of the ligand
3.1 POCKET-CONDITTONED SMALL MOLECULE GENERATION
In the conditional molecule generation setup, we provide fixed dice-dimensional context in each step of the denoising process. To this end, supplement the ligand node point cloud 2!44, denoted by superscript J., with protein pocket nodes z(P)) denoted by superseript P, that remain unchanged throughout the reverse diffusion process (Figure 2
2(P) () with an EGNN (attorass et al. 2021 Hoogeboom et al.] 2022). To process ligand and pocket nodes with single GNN, atom types and residue types are first embedded in a joint node embedding space by separate learnable MLPs. We employ the same cassage-passing scheme outlined in Equations 3-(5)) however, following [gashov et al.] 2022) we do not update the coordinates nodes that belong to the pocket to ensure the three-dimensional protein context remains fixed throughout the EGNN layers.
We parameterize the noise predictor é = ))((2 =112
Equivariance In the probabilistic setting with 3DD-conditioning, we would like to ensure SF((3) cquivariance in the following sense. This definition explicitly excludes reflections which are con- nected with chirality and can alter biomolecule's 's properties
Evaluating the likelihood of a molecule (L) E R33NN, given the three-dimensional rep- resentation of protein pocket !!!)) E R3xNr should not depend on global SE(3)- transformations of the system, i.e. p (L) Ra p for orthogonal R 3x3 with Rd R I, (d) 1 and E g added column-wise
At the same time, it should be possible generate samples (L) (2x from this conditional probability distribution that equivalently transformed ligands (x( +
We transpose the node feature matrices hereafter so that the matrix multiplication resembles application of group action. We also ignore node type features, which transform invariantly, for simpler notation.
4

---

Under review as a conference paper at ICLR 2023
are sampled with the same probability if the input pocket is rotated and translated and sample from p( Rx(()) t. Rx(() P) + t).
Equivariance the orthogonal group ((3) (comprising rotations and reflections) is achieved be- cause we model both prior and transition probabilities with isotropic Gaussians where the mean vector transforms equivariantly w.r.t. rotations of the context (see [Hoogeboom al. (2022) and Appendix[E] E.. Ensuring translation eguivariance, however, is not as easy hecause the transition prob- abilities p(z)-1(z() are not inherently translation-ccuiiarrant.
order to circumvent this issue, we follow previous works (Köhler et al. 2020 Xu et al. 2022 Hoogeboom et al. 2022) by limiting the whole sampling process to a linear subspace where the center of mass (CoM) of the system is zero. In practice, this is achieved by subtracting the center of mass of the system before performing likeli- hood computations or denoising steps. Since equivariance of the transition probabilities depends on the parameterization of the noise predictor ég. We can make the model sensitive to reflections with simple additive term in the EGNN's 's coordinate update:
1+1 2. = a +
(h 1 dly. asz)+ +
(ar)
x
(ar)
2 (h) hJy
by aty),
dij
11(=)
- x')
x
+
j
(6)
using the cross product which changes sign under reflection. Here, .e denotes the center of mass of all nodes at layer I. 2% is an additional MLP. This modification is discussed in more detail in Appendix F
3.2
JOINT DISTRIBUTION WITH INPAINTING
As an extension to the conditional approach described above, we also present a ligand-inpainting approach. Originally introduced as a technique for completing masked parts of images (Song al., 2020 Lugmayr et al.) 2022). inpainting has been adopted in other domains, including biomolecular structures (Wang et al. 2022 Here, we extend this idea to three-dimensional point cloud data.
We first train an unconditional DDPM to approximate the joint distribution of ligand and pocket (L)
(P)) This allows us sample new pairs without additional context. To condition on a target protein pocket, we then need to inject context into the sampling process by modifying
nodes p( daata
the probabilistic transition steps. The combined latent representation = (I.P) of protein pocket and ligand at diffusion step & - 1 is assembled from a forward noised version of the pocket that is
combined with ligand nodes predicted by the DDPM based on the previous latent representation at step t
1 known p(z data
(P)
(L.P)
-2 (L.P) (L,P)
(L,P)
(8)
1-1, L,unknown
(L.P) =
[ 12 lunknoown' 2t_1,known]
(P)
(9)
In this manner, we traverse the Markov chain in reverse order from t = T to 0, replacing the predicted pocket nodes with their forward noised counterparts in each step. Equation (8) conditions the generative process the given protein pocket. Thanks the noise schedule, which decreases the variance of the noising process to almost zero at t = 0 (Equation (T), the final sample is guaranteed contain an unperturbed representation of the protein pocket.
Since the model is trained to approximate the unconditional joint distribution of ligand-pocket pairs, the training procedure is identical the unconditional molecule generation procedure developed by Hoogeboom et al. (2022) aside from the dully-connecter neural networks that embed protein and ligand node features in a common space as described in Section 3.I The conditioning on known protein pockets is entirely delegated to the sampling algorithm, which means this approach is not limited to ligand-inpainting but, in principle, allows us to mask and replace arbitrary parts of the ligand-pocket system without retraining.
Trippe et al. 2022 show that this simple replacement method inevitably introduces approximation error that can lead to inconsistent inpainted regions. In our experiments, we observe that the in-
We use notations zLLPP) and (P)) interchangeably to describe the combined system of ligand and
pocket nodes.
5

---

Under review as a conference paper at ICLR 2023
Table 1: Evaluation of generated molecules for targets from the CrossDocked test set. denotes that we re-evaluate the generated ligands provided by the authors. The inference times are taken from their papers. Note that these results have been produced with the Eq3)-qquivariant version of our model and will be updated.
Vina Score (kcal/mol, U)
QED (1)
SA
Lipinski (1)
Diversity (1)
Time (s. J)
Test sct
-6,871 + 2.32
0.276
0.728
+ 0.14
4.300 = 1.14
3D SBDD
2021
5,868 1.91
0.502
0.875
0.14
0.51
19650 14704
Pocket2Mol (Png at 2014
-7,058 1: 2,80
0.572
0.752
2: 0.12
4,936 = 0.27
0.735 4: 0.15
2501 2207
DifSBDDI-cond (C.,)
-5.40 1.57
0.460
0.357
0.09
4.821
0.45
0.815 + 0.06
324 188
DinSBDD-inpaist (C,)
-5.735 1.80
0.127 0.11
0.313 2: 0.09
1.789
= 0.19
0.807 + 0.07
329 177
DinSBDD-cood
-6.584 + 2.06
0.495 =.13
0,336 0.09
4.795 = 0.49
0,730 D.11
1631 = 769
painting solution sometimes generates dislocated molecules that are not properly positioned in the target pocket. [Trippe et al. (2022) propose to address this limitation with a particle filtering scheme that upweights more consistent samples in each denoising step. We, however, choose to adopt the conceptually simpler idea of resampling (Lugmayr et al. 2022), where each latent representation is repeatedly diffused back and forth before advancing the next time step. This enables the model to harmonize its prediction for the unknown region and the noisy sample from the known region (Eq. which does not include any information about the generated part. We choose r 10 resamplings per denoising step based on empirical results discussed in Appendix C.
Equivariance Similar desiderata as in the conditional case apply to the joint probability model, where we desire (33naanaeee that can be obtained from invariant priors via equivariant flows (Köhler et al., 2020). The main complications compared to the previous approach are the missing reference frame and impossibility of defining valid iansaation-nnvariantt prior noise distri- bution p(zr) as such a distribution cannot integrate to one. Consequently, it is necessary to restrict the probabilistic model to a CoM-free subspace as described in previous works (Köhler al. 2020 Xu et al.) 2022 Hoogeboom et al. 2022). While the reverse diffusion process is defined for CoM- free system, substituting the predicted pocket node coordinates with a new diffused version of the known pocket as described in Equations 7 (9) can lead to non-zero CoM. To prevent this, we
translate the known pocket representation so that its center of mass coincides with the predicted rep-
resentation:
/ { 1.koonn
(P)
-1t-1 .unknown
2- before creating the new combined representation
(P)
(L.P) (L)
(P))
(P)
|0 (P)
(P)
26-1
#t-1.,nnmown
1,oo.m with 2 known
Jnnwn h ft- 1.noww]
4 EXPERIMENTS
4.1 DATASETS
CrossDocked We use the CrossDocked dataset (Francoeur et al. 2020) and follow the same fil-
tering and splitting strategies as in previous work (Luo et al. 202 I Peng et al. 2022). This results in 100,000 high-quality protein-ligand pairs for the training set and 100 proteins for the test set. The split is done by 30% sequence identity using MMseqs2 (Steinegger & Söding 2017).
Binding MOAD We also evaluate our method on experimentally determined protein-ligam com- plexes found in Binding MOAD (Hu et al. 2005) which are filtered and split based on the proteins' enzyme commission number described in Appendix [)] This results in 40,354 protein-ligand pairs for training and 130 pairs for testing.
4.2 EVALUATION
For every experiment, we evaluated all combinations of all-atom and Ca level graphs with condi- tional and inpainting- based approaches respectively (with the exception of the all-atom inpainting approach due to computational limitations). Full details of model architecture and hyperparameters are given in Appendix C We sampled 100 valid molecules for each target pocket and removed all atoms that are not bonded to the largest connected fragment. Ligand sizes were sampled from
Due to occasional processing issues the actual number of available molecules is slightly lower on average (see Appendix G.T.

---

Under review as a conference paper at ICLR 2023
Conditional (2jjg)
Inpainting-Ca (2l1g)
Reference (2jig)
Vina-6.5 Sim: 027 Vina: -B.7 sim: 0.24 Vina:-6. Sim: Vinar-B.5 Sim: 0.7 Vina:4: Sim: 19 Vina:-Ba Sim: o
Vina: -5.9 Sim: 1
5A:043 GED:
CED: 0.44
DED: 0.35
Conditional (3kc1)
Inpainting-Ca (3kc1)
Reference (3kc1)
Vina: 8.1 Sim: 0.44 Vina:-72 Sim: 0.50 Vina: M.a:-8.5 Sim: 0.40
0.m:
Vinar -8.9 Sim: 0.40 Vina: Vina:66.9 Sim: 0.32 Vina:-BA Sim: 0
Vina:-5 4.5 8im: 1
QED:
Figure 3: DifiSBDD models trained CrossDocked and evaluated against aminotransferase (top, PDB: 2jjg) and hydrolase (bottom, PDB: 3kc1). Conditional and inpainting approaches are compared (using all-atom and Ca level protein presentations respectively) and three high affinity molecules from each model are presented. 'Sim' is the Tanimoto similarity between the generated and reference ligand.
the training distribution described in Appendix C This procedure yields significantly smaller molecules than the reference ligands from the test set. We therefore increase the mean ligand size by 15 atoms sample approximately equally sized molecules. This correction improves the observed docking scores which are highly correlated with the ligand size (see Figure
We employ widely-used metrics to assess the quality our generated molecules (Peng al. 2022 Li et al. (021): (1) Vina Score is physics-based estimation of binding affinity between small molecules and their target pocket; (2) QED simple quantitative estimation of likennis combining several desirable molecular properties; (3) SA (synthetic accessibility) is measure es- timating the difficulty of synthesis; (4) Lipinski measures how many rules the Lipinski rule of five (Lipinski et al. 2012), which is loose rule of thumb to assess the trug-likeness of molecules, are satisfied; (5) Diversity is computed as the average pairwise dissimilarity (1 Tanimoto simi- larity) between all generated molecules for each pocket; (6) Inference Time is the average time to sample 100 molecules for one pocket across all targets. All docking scores and chemical properties calculated with Quick Vina2 (Alhossary et al. 2015) and RDKit (Landrum al. 2016))
4.3 BASELINES
We compare with two recent deep learning methods for structure- drug design. 3D-SBDD (Luo al.] (202l) and Pocket2Mol (Peng al. (2022) auto-regressive schemes relying graph repre- sentations the protein pocket and previously placed atoms to predict probabilities based on which atoms added. 3D)SBDD heuristics to infer bonds from generated atomie point clouds while Pocker2Mol directly predicts them during the sequential generation process.
4.4 RESULTS
CrossDocked Overall, the experimental results in Table D suggest that DiftSBDD can generate diverse small-molecule compounds with predicted high binding affinity, matching state-of-the-ae performance. We do not see significant differences between the conditional model and the inpainting approach. The diversity score is arguably the most interesting, as this suggests our model is able to sample greater amounts of chemical space when compared previous methods, while maintaining

---

Under review as a conference paper at ICLR 2023
Table 2: Evaluation of generated molecules for target pockets from the Binding MOAD test set.
Vina Score (keal/mmll 4)
QED
SA (t)
Lipinski
Diversity (1)
Time (s. L)
Tost sct
-8.328 2.05
0.602 + 0.15
0.336 + 0.08
1.838 + 0.37
DinSBD cund (C.))
6.281
0.17
0.313
4.637
0.04
44.022 1 8.98
DiASBDD DainpunDiinpuittt (Ct)
6.406 5.
0.512 0.17
0.308 + 0.09
4.681 + 0.58
0.621 0.16
98.439 30.44
DiffSBDD-cond
6.726 1.00
0.470 + 0.18
0.331 + 0.08
4.666 + 0.62
0.711 0.08
194,800 + 49.09
Conditional-Ca (6c0b)
Inpainting-Ca (6c0b)
Reference (6c0b)
Mna:-124 -1.8 as -11.9
5. os
-12.4 Sim: 0.07 Vina-12..3 Vina:-12.2 Sim: 0.12
vina:-8.40
GED: 0.74 SA: 0.45 QED: 0.68
SA: 0.25 QED: 0.68 SA: 0.25
QED 0.78 SA 0.24 GED: 0.85 0 QED: 0.83 SA: 0.34
QED: 0.38 SA: 0.89
Figure 4: DiftSBDD models trained on Binding MOAD evaluated against a human receptor protein (PDB: 6c0b). Conditional and inpainting approaches are compared (Ca for both) and the three highest affinity molecules from each model are presented. Further details of the molecules shown here are explained in Appendix G.I
high binding performance, one of the most important requirements in early-stage, structure-based lead discovery. Specifically, DiffSBDD aims to generate ligands that bind to protein pockets and learn the probability density of ligands interacting with protein pockets. While it does not optimize for other molecular properties, such as QED and Lipinski, it generates molecules similar to the test set distributions. Only SA scores are significantly lower average. While it is unclear why the model fails to approximate the distribution of synthetic accessibility scores successfully, simple techniques can be used for downstream optimization of this property once promising candidates are found (Section 4.5). Generally, presenting the full atomic context to the model constrains the space of outputs considerably, leading to higher Vina scores but lower diversity compared the C. -nlo models. The all-atom model consistently beats Ca-based models on a per target basis (Appendix Figure 13p.
A representative selection of molecules for two targets (2iig and 3kc/) are presented (Figure 3 This set is curated to representative of our high scoring molecules, with both realistic and non-realistic motifs shown. It is noteworthy that the second molecule generated for 3kc1 has similar tricyclic motif the same pocket location as the reference ligand which was designed by traditional SBDD methods to maximise the hydrophobic interactions via shape complementarity of the ring system Tsukada et al. 2010). However, a number of irregularities are present in even the highest scoring of generated molecules. For example, the high number of triangles in the molecules targeting 2jjg (from Inpainting-C, and the large rings for 3kcl would prove difficult to synthesise. Random selections of generated molecules made by all methods evaluated are presented in Figure
All docking scores reported in Table are within one standard deviation of each other, which poses challenges for the discrimination of the best models. To verify successful pocket-conditioning, we therefore discuss the agreement of generated molecular conformations with poses after docking in Appendix G.5] This experiment showcases the success of our method to model protein-drup inter- actions at the atomic level and clearly highlights the benefits of the all-atom pocket representation.
Results for the Binding MOAD dataset with experimentally determined binding complex data are reported in Table 100 valid ligands have been generated for each of the 130 test pockets resulting in 13 000 molecules in tota f DiftSBDD generates highly diverse molecules but on average docking scores are lower than corresponding reference ligands from this dataset.
Binding MOAD
*The Quick Vina score could not be computed for 49 0.4%) molecules from DifSBDD-cond.
8

---

Under review as a conference paper at ICLR 2023
Generated molecules for representative target are shown in Figure 4 The target (PDB; 6c0b) is a human receptor which is involved in microbial infection (Chen et al. 2018) and possibly tumor suppression (Ding al.] 2016). The reference molecule. long fatty acid (see Figure 4 that aids receptor binding (Chen et al. 2018), has too high a number of rotatable bonds and low a number of hydrogen bond donors/acceptors
be considered suitable drug (QED of 0.36). Our model however, generates drug-like (QED between 0.63-0.85) and suitably sized molecules by adding aromatic rings connected by a small number of rotatable bonds, which allows the molecules to adopt complementary binding geometry and is entropically favourable (by reducing the degrees of freedom), classic technique medicinal chemistry (Ritchie & Macdonald] 2009). random selection of generated molecules in presented in Figure 12
4.5
MOLECULE OPTIMIZATION
Oprimization
10
Aaforenc
Refurence
oplimized
35
QED: 0.54
S% 0.31
CA.Soore
BA optimized
aptimized
0.2
OED: 0.22
QED: 0.35
SA: 1.00
CC
SA: 6.17
Vine .
Vina: -12.2
Sm: 0.82
9im: 0.56
3
Figure 5: (a) Optimizing for various properties. (b) Examples of optimized molecules.
We use our model to optimize exciting candidate molecules, a common task in drug discovery called lead optimization. This is when take a compound found have high binding affinity and optimize it for better 'drug-like' properties. We first noise the atom features and coordinates for t steps (where is small) using the forward diffusion process. From this partially noised sample, can then denoise the appropriate number of steps with the reverse process until t = 0. This allows us to sample new candidates of various properties whilst staying in the same region of active chemical space, assuming is small (Appendix Figure 3 This approach is inspired by (Luo et al. 2022)) but note this does not allow for direct optimization of specific properties, rather directed exploration around the local chemical space according to what was learnt from the training distribution.
We extend this idea by combining the partial noising/denoising procedure with a simple evolutionary algorithm that optimizes for specific molecular properties. We find that our model performs well at this task out-of-the-box without any additional fine-tuning. As a showcase, we optimize a molecule in the test set targeting PDB:5ndu cancer therapeutic (Barone et al. 2020), which has low SA and QED scores, 0.31 and 0.35 respectively, but high binding affimity. Over a number of rounds of observe significant increases in QED (from 0.35 mean of 0.43) whilst still maintaining high similarity to the original molecule (Figure 5Jp). We can also rescue the low syn- thetic accessibility score of the seed molecule by producing a battery of highly accessible molecules when selecting for SA. Finally, we observe that we can perform significant optimization of binding affinity after only a view rounds of optimization. Figure 55p shows 3 representative molecules with substantially optimized scores (QED, SA or Vina) whilst maintaining comparable binding affinity and globally similar structures. Full details are provided in Appendix 4.5
optimization,
CONCLUSION
In this work, we propose DifSSDD, an -qquiiaaui 3D-conditional diffusion model for structure-based drug design. We demonstrate the effectiveness and efficiency of DifISBDD in gen- crating novel and diverse ligands with predicted high-affinity for given protein pockets on both a synthetic benchmark and a new dataset of experimentally determined protein-ligand complexes. We demonstrate that an inpainting- based approach can achieve competitive results to direct conditioning on wide range of molecular metrics. Extending this more versatile strategy an all atom pocket representation therefore holds promise to solve a variety of other structure-bases drug design tasks, such lead optimization linker design, and binding site design without retraining.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Keir Adams, Lagnajit Pattanaik, and Connor W Coley. Learning 3d representations of molecular chirality with invariance to bond rotations. arXiv preprint arX.v:2/ 2021.
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):211422216, 2015.
Namrata Anand and Tudor Achim. Protein structure and sequence generation with equivariant de- noising diffusion probabilistic models. arXiv preprint arXiv:2205. 2022.
Amy C Anderson. The process of structure-based drug design. Chemistry & biology, 10(9):787- 797, 2003.
Kenneth Atz, Francesca Grisoni, and Gisbert Schneider. Geometric deep learning on molecular representations. Nature Machine Intelligence, 3(12):1023-1032 2021.
Matthias Barone, Matthias Müller, Slim Chiha, Jiang Ren, Dominik Albat, Arne Soicke, Stephan Dohmen, Marco Klein, Judith Bruns, Maarten van Dinther, et al. Designed nanomolar small- molecule inhibitors of ena/vasp evhl interaction impair invasion and extravasation of breast can- cer cells. Proceedings of the National Academy of Sciences, 11 (1(47)):96842269) 2020.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan Mailoa, Mordechai Ko- rnbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1): 1-11, 2022.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104. 13478, 2021.
Peng Chen, Liang Tao, Tianyu Wang, Jie Zhang, Aina He, Kwok-ho Lam, Zheng Liu, Xi He, Kay Perry. Min Dong, et al. Structural basis for recognition of frizzled proteins by clostridium difficile toxin b. Science, 360(6389):664669) 2018.
Lin-Can Ding, Xiao-Yu Huang. Fei-Fei Zheng. Jian Xie, Lin She, Yan Feng, Bo-Hua Su, Da-Li Zheng, and You-Guang Lu. Fzd2 inhibits the cell growth and migration of salivary adenoid cystic carcinomas. Oncology Reports, 35(2): (506):100-11012 2016.
Pavol Drotár, Arian Rokkum Jamasb, Ben Day, Cätälina Cangea, and Pictro Liò. Structure-aware generation drug-like molecules. arXiv preprint arXiv:2111.04177,000 2021.
Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie- Vie-Yan Liu. Se (3) equivariant graph neural networks with complete local frames. In International Conference Machine Learning, pp. 3583-5608 PMLR, 2022a.
Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. Molgensurvey: systematic survey in machine learning models for molecule design. arXiv preprint arXiv:2203.144 14500, 2022b.
Yuanqi Du, Xian Liu, Nilay Shah, Shengchao Liu, Jieyu Zhang, and Bolei Zhou. Chemspace: Interpretable and interactive chemical space exploration. 2022c.
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru- Guzik, and Ryan Adams. Convolutional networks on graphs for learning molecular fingerprints. Advances in neural information processing systems, 28, 2015.
Leonardo G Ferreira, Ricardo Dos Santos, Glaucius Oliva, and Adriano Andricopulo. Moleeu- 2015.
lar docking and structure-based drug design strategies. Molecules,
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. vinee-dimensioonl convolutional neural networks and cross-docked data set for cauuttre--asedd drug design. Journal of Chemical Information and Modeling. 60(9):4200- 4215, 2020.
10

---

Under review as a conference paper at ICLR 2023
Thomas Gaudelet, Ben Day, Arian R Jamasb, Jyothish Soman, Cristian Regep, Gertrude Liu, Jeremy B Hayter, Richard Vickers. Charles Roberts, Jian Tang, David Roblin, Tom Blundell, Michael M Bronstein, and Jake P Taylor-King. Utilizing graph machine learning within drug discovery and development. Briefings in Bioinformatics, 22(6), May 2021. doi: 10.1093/bib/bbabb153 UtLhttpss:/// http:///oo..rr//////// 1tw.1.119///bbbbbbbbbb..
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp- 1263-1272. PMLR, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840 6851, 2020.
Lars Holdijk, Yuanqi Du, Ferry Hooft, Priyank Jaini, Bernd Ensing, and Max Welling, Path integral stochastic optimal control for sampling transition paths. arXiv preprint arXiv:2207.021499 2022.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant dif- fusion for molecule generation in 3d. In International Conference Machine Learning, pp- 8867-8887. PMLR, 2022.
Liegi Hu, Mark L Benson, Richard D Smith, Michael G Lerner, and Heather A Carlson. Binding moad (mother all databases). Proteins: Structure, Function, and Bioinformatics, 60(3):333- 340, 2005.
Ilia Igashov, Hannes Stärk, Clément Vignac, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia. Equivariant 3d-conditional diffusion models for molecular linker design. arXiv preprint arXiv:2210.05274, 2022.
John J Irwin and Brian K Shoichet. Zinc- free database of commercially available compounds for virtual screening. Journal of chemical information and modeling, (5(1):177-182 2005.
Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffu- sion for molecular conformer generation. arXiv preprint arXiv:2206.0172) 2022.
Subha Kalyaanamoorthy and Yi-Ping Phoebe Chen. Structure- drug design augment hit discovery. Drug discovery today, 16(17-18):831-5 8839, 2011.
Lawrence A Kelley, Stefans Mezulis, Christopher M Yates, Mark N Wass, and Michael JE Sternberg. The phyre2 web portal for protein modeling, prediction and analysis. Nature protocols, 10(6); 845-858, 2015.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances in neural information processing systems, 34:21696 21707, 2021.
Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molec- ular graphs. arXiv preprint 2xXiv:2003.03/2. 2020.
Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: exact likelihood generative learning for symmetric densities. In International conference on machine learning, pp. 5361-5370. PMLR, 2020.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference Learning Representations, 2021.
Greg Landrum et al. Rdkit: Open-source cheminformatics software. 2016.
Kostiantyn Lapchevskyi, Benjamin Miller, Mario Geiger, and Tess Smidt. Euclidean neural net- works (e3nn) v1. O. Technical report, Lawrence Berkeley National Lab(LLBNL), Berkeley, CA (United States), 2020.
Yibo Li. Jianfeng Pei, and Luhua Lai. Structure- de novo drug design using 3d deep generative models. Chemical science, 12(41):13664-1333 2021.

---

Under review as a conference paper at ICLR 2023
Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and develop- ment settings. Advanced drug delivery reviews, 64:4-17, 2012.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. arXiv preprint arXiv:2204.09440 2022.
Wei Lu, Qifeng Wu, Jixian Zhang, Jiahua Rao, Chengtao Li, and Shuangjia Zheng. Tankbind: Trigonometr- neural networks for trug-protein binding structure prediction. bioRxiv, 2022.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool.
Repaint: Inpainting using denoising diffusion probabilisticc models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 11461-11471, 2022.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2837-2845, 2021.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure- based drug design. Advances in Neural Information Processing Systems, 34:6229-6239, 2021.
Shitong Luo, Yufeng Su, Xingang Peng. Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen-specific antibody design and optimization with diffusion- based generative models. bioRxiv, 2022.
Paul D Lyne. Structure-based virtual screening: overview. Drug discovery today, 7(20):1047- 1055, 2002.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuuson probabilistic models. In International Conference on Machine Learning, pp. 8162-8171. PMLR, 2021.
Noel M O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geof- frey R Hutchison. Open babel: An open chemical toolbox. Journal of cheminformatics, 3(1): 1-14, 2011.
Xingang Peng, Shitong Luo. Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. arXiv preprint urXiv:2205.07249 2022.
Stéphanie Pérot, Olivier Sperandio, Maria A Miteva, Anne-Claude Camproux, and Bruno O Vill- outreix. Druggable pockets and binding site centric chemical space: a paradigm shift in drug discovery. Drug discovery today, 15(15-16):656 667, 2010.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 13(9):2001-2713, 2022.
Timothy J Ritchie and Simon JF Macdonald. The impact of aromatic ring count on compound developabllity-are too many aromatic rings liability in drug design? Drug discovery loday, 14 (21-22):0011-1020 2009.
Victor Gareia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural net- works. In International conference on machine learning, pp. 9323-9329990 2232.933. PMLR, 2021.
Kristof T Schütt, Huziel Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(2))224122, 2018.
Jean-Pierre Serre al. Linear representations finite groups, volume 42. Springer, 1977.
Brian K Shoichet. Virtual screening of chemical libraries. Nature, 432(7011))862-8655 2004. 2004.
Jascha Sohl-Dickstein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Leam- ing, pp. 2256-2265. PMLR, 2015.
12

---

Under review as a conference paper at ICLR 2023
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456. 2020.
Hannes Stärk, Octavian Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International Conference on Machine Learning. pp. 20503-205211 PMLR. 2022.
Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature Biotechnology. 35(11) 1026- October 2017. doi: 10.10388nbt.3988.
1038/1100000 htt.//nt..3. 3988
Brian Trippe, Jason Yim, Doug Tischer, Tamara Broderick, David Baker, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. arXiv preprint arXiv:2206. 00000.04100 19, 2022.
Tomoharu Tsukada, Mizuki Takahashi, Toshiyasu Takemoto, Osamu Kanno, Takahiro Yamane, Sayako Kawamura, and Takahide Nishi. Structure- drug design of tricyclic 8h-indeno [1, 2-d][1, 3] thiazoles potent fbpase inhibitors. Bioorganic & medicinal chemistry letters, 20(3): 1004-1007, 2010.
Jue Wang, Sidney Lisanza, David Juergens, Doug Tischer, Joseph Watson, Karla Castro, Robert Ragotte, Amijai Saragovi, Lukas F Milles, Minkyung Back, et al. Scaffolding protein functional sites using deep learning. Science, 37766604):387-394 2022.
Scott A Wildman and Gordon M Crippen. Prediction of physicochemical parameters by atomic contributions. Journal of chemical information and computer sciences, 39(5):868-873, 1999.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: geo- metric diffusion model for molecular conformation generation. arXiv preprint 23X2v:2203.02923, 2022.
13

---

Under review as a conference paper at ICLR 2023
Appendix for "Structure-based Drug Design with Equivariant Diffusion Models"
A VARIATIONAL LOWER BOUND
To maximise the likelihood of our training data, we aim at optimising the variational lower bound (VLB) (Kingma et al. 2021 Hoogeboom et al. 2022)
log pl (data)
S DKL (q(zT) Adata) =dma)) (p((=))) Bgo=l Fam))
log (datta [00
+
C
(10)
prior loss Eyrice
reconstruction loss Lo
diffusion loss
with
L. = DKL. (al((a(z-11 (((ze-1/Zdata 2x)11p((zz E zc))
(11)
SNR(t - 1-- ( 60]13]
= -Ee-N(0.))
(12)
SNR(t)
during training. The prior loss should always be close to zero and can be computed exactly in closed form while the reconstruction loss must be estimated described [Hoogeboom al. (2022). In
practice, however, we simply minimise the mean squared error Liain = = ěll² while randomly
sampling time steps (O........... T), which equivalent up to multiplicative factor.
B NOTE ON EQUIVARIANCE OF THE CONDITIONAL MODEL
The 3D-conditional model can achieve equivariance without the usual "subspace-trick"" The coordi- nates of pocket nodes provide reference frame for all samples that can be used to translate them to unique location (e.g. such that the pocket is centered at the origin: Σ, x 0). By doing this for all training data, translation equivariance becomes irrelevant and the CoM-free subspace approach obsolete. To evaluate the likelihood of translated samples at inference time, we can first subtract the pocket's center of mass from the whole system and compute the likelihood after this mapping. Sim- ilarly, for sampling molecules can first generate ligand in CoM-free version of the pocket and move the whole system back to the original location of the pocket nodes to restore translation equiv- ariance. As long as the mean of our Gaussian noise distribution p ((=)>=(()) (N(( ((())) o² I) depends equivariantly
the pocket node coordinates (P)) -uuieqquiveeiance is satisfied well (Appendix H). Since this change did not seem to affect the performance of the conditional model in our experiments, we decided keep sampling in the linear subspace ensure that the implementa- tion is as similar as possible to the joint model, for which the subspace approach is necessary.
C IMPLEMENTATION DETAILS
Molecule size As part of a sample's s overall likelihood, we compute the empirical joint distribution of ligand and pocket nodes p( N(N,,P observed in the training set and smooth it with a Gaussian filter (a = 1). the conditional generation scenario, we derive the distribution p(Nr.[Np)) and use it for likelihood computations.
For sampling, we can either fix molccule sizes manually or sample the number of ligand nodes from the same distribution given the number of nodes in the target pocket:
Nł p(Nr(NP).
(13)
For the experiments discussed in the main text, we increase the mean size of sampled molecules by 15 atoms to approximately match the sizes of molecules found the test set. This modification makes the reported Quick Vina scores more comparable as the in silico docking score is highly cor- related with the molecular size, which is demonstrated in Figure With the correction, the average size of generated molecules is 26.5, 28.6, 6, and 26 respectively for DiffSBDD-cond (C.), DiffSBDD- inpaint (C,.) and DiftSBDD-com (full atom). Test set molecules from the Binding MOAD data set composed of 28 atoms average.
14

---

Under review as a conference paper at ICLR 2023
A
B
ligentibe
Figure 6: Correlation between ligand size and Quick score for reference molecules from the Binding MOAD (A) and CrossDocked (B) test sets.
Preprocessing All molecules are expressed graphs. For the C, only model the node features for the protein are set as the one hot encoding of the amino acid type. The full atom model uses the same hot encoding of atom types for ligand and protein nodes. We refrain from adding categorical feature for distinguishing between protein and ligand atoms in this case and continue using two separate MLPs for embedding the node features instead.
Noise schedule al. 2022):
We use the pre-defined polynomial noise schedule introduced in HHogeeoom
âc = - t= 0,...,,
(14)
Following (Nichol & Dhariwal 2021
Hoogeboom et al. 2022), values of a? = (aa; are clipped between 0.001 and for numerical stability near / 7, and a recomputed as
ā II 0+/r-1- T=0
(15)
A tiny offset 10-5 5 is used to avoid numerical problems at 0 defining the final noise schedule:
af = (1 26) à +
(16)
Feature scaling We scale the node type features h by factor of 0.25 relative the coordinates x which was empirically found to improve model perfomance in previous work (Hoogeboom et al., 2022).
Hyperparameters Hyperparameters for all presented models are summarized in Table Training takes about 2.5 h/3 (cond/inpaint) for Binding MOAD in the C,, scenario and 12.5 h with full atom pocket representation on a single NVIDIA V100. For CrossDocked, 100 training epochs take approximately 24 h on an NVIDIA V100 GPU in the C. case and 96 h per 100 epochs on a single NVIDIA A100 GPU with all atom pocket representation.
Postprocessing For postprocessing of generated molecules, we use similar procedure as in (Luo 2021). Given list of atom types and coordinates, bonds are first added using OpenBa- bel (0 Boyle et al. (011). We then use RDKit to sanitise molecules, filter for the largest molecular fragment and finally remove steric clashes with 200 steps of force-field relaxation.
C.1 EFFECT OF THE NUMBER OF RESAMPLING STEPS
In this section we discuss the number of resampling steps required for satisfactory results. To this end, we generated ligands for all test pockets with r = 1, r = 5, and r = 10 resampling steps, respectively. Because the resampling strategy slows down sampling approximately by factor of r, we used the striding technique proposed by Nichol & Dhariwal (202T) and reduced the number of denoising steps proportionally to Nichol & Dhariwal (2021) showed that this approach reduces
15

---

Under review as a conference paper at ICLR 2023
Table 3: DifiSBDD hyperparameters.
CrossDocked
Binding MOAD
Cond
Cond (C.)
Inpaint (C.)
Cond
Cond (C,)
Inpaint (C.)
No. layers
5
Joint embedding dim.
32
32
32
32
32
32
Hidden dim.
256
256
256
128
128
128
Leurning rate
10
10-4
10-4
5 10
5 5-10-1
5 5-10-=
Weight decay
10
10
10 1000
10 12
10-12 12
10-12 12
Diffusion steps
1000
1000
500
500
500
Edges (ligand-ligg (ligand-ligand)
7
fully connected
fully connected fully connected fully connected
fully connected A
fully connected &
fully connected
Edges (iggnd-pocket)
7
fully connected
A
Edges (pocket-pocket)
< <7A
fully connected 1000
5
<8
A
Epoccs
1000
1000
800
800
800
Table 4: Evaluation of generated molecules for target pockets from the Binding MOAD test set with the inpainting approach and C. pocket representation for varying numbers of resampling steps and denoising steps T.
T
Vina Score (kcal/mol, 1)
QED (f)
SA (1)
Lipinski (t)
Diversity (1)
Time (s. 4)
500
5.601 +: 1.92
0.495 + 0.12
0.358 1 0.09
4.91040.33
0.850
0.04
40.298 13.52
5
100
-..663 + 1.93
0.541 + 0.13
0.365 + 0.09
4.946 + 0.24
0.853
0.05
45.074 + 21.14
10 50
6.080 1 2.14
0.554 1 0.13
0.367 1 0.09
4.957 0.21
0.855 0.05
41.490 1 14.32
the number of sampling steps significantly without sacrificing sample quality. In our case, it allows us to retain sampling speed while increasing the number of resampling steps.
To gauge the effect of resampling for molecule generation we show the distribution of RMSD val- ues between the center of mass of reference molecules and generated molccules in Figure 7 The unmodified replacement method (r = 1) produces molecules that are clearly farther away from the presumed pocket center than the conditional model. Increasing r moves the mean distance closer to the average displacement of molecules from the conditional method. This effect seems to saturate at = 10 which is in line with the results obtained for images (Lugmayr et al. 2022).
Table shows that neither the additional resampling steps nor the shortened denoising trajectory de- grade the performance on the reported molecular metrics. The average docking scores even improve slightly which might reflect better positioning of generated ligands in the pockets prior to docking. The same model trained with 7 = 500 diffusion steps was used in all three cases. These experiments have been conducted without the adjustment of molecule sizes described in Section
D BINDING MOAD DATASET
We curate a dataset of experimentally determined complexed protein-ligand structures from Bind- ing MOAD Hu et al. 2005). We keep pockets with valid and moderately 'drug-like' lig- ands with QED score > 0.3. We further discard small molecules that contain atom types (C,N, O, S, B, Br, CI, F. as well as binding pockets with non-standard amino acids. We the set of residues that have any atom within 8 Á of any ligand atom.
define binding pockets Ligand redundancy is reduced by randomly sampling at most 50 molecules with the same chemical component identifier (3-letter-code). After removing corrupted entries that could not be processed, 40 354 training pairs and 130 testing pairs remain. A validation set of size 246 is used to monitor estimated ligliiiishoods during training. The split is made ensure different sets do not contain proteins from the same Enzyme Commission Number (EC Number) main class.
E PROOFS
In the following proofs do consider categorical node features h as only the positions subject to equivariance constraints. Furthermore, we do not distinguish between the zeroth latent
defined in http: hwtp///ww//// bi ngmoad. org/
16

---

Under review as a conference paper at ICLR 2023
12
10
CoM
conditional
inpainting (r=1)
Inpainting (re5)
inpainting (r=10)
Figure 7: RMSD between reference molecules' center of mass and generated molecules' center of mass for the conditional model and inpaining model with varying numbers of resampling steps r. The pocket representation is C₂ in all cases.
representation and data domain representations for ease of notation, and simply drop the subscripts.
E.1 O(3) EQUIVARAIRE OF THE PRIOR PROBABILITY
The isotropic Gaussian prior p(a/2) (M)
(N)(((()) I²1) equivariant rotations and
reflections represented by an orthogonal matrix R 13x3 as long μ (P) Ru(xx()) because:
pl Rox (1) Rx (P)
μ u(RR((P))1
exp
(2T) NoT2
20
V22 2
exp
Roy
(2 (11))
IR( |R(1 (L)
(x 12)
exp
((2 No.2 2r
|| (D)
(2(x(())113)
V22 p(alf (P()))
cxp
202
Here we used
|R lz = ||>|| for orthogonal R.
E.2 0(3) OIUUVIQUUV OF THE TRANSITION PROBABILITIES
The denoising transition probabilities from time step t to < are defined as isotropic normal distributions:
Pa (2
2 (L), (P)) = ((o (1) (L)) x (P))), oZ,,
(1)
(17)
Therefore, Po (a | (1) (()), x (P)) is (3))_qquvvariant by a similar argument to Section E.
Ht-s computed equivariantly from the three-dimensional context.
17

---

Under review as a conference paper at ICLR 2023
o1l (L)
of al + a.oii x(L,, we can prove its equivariance as of
Recalling the definition of p4->5 follows:
+ Og t/a (1) (Ra) Ra!P))
P4-ss (Ra) (4)
Rx(P))
Ray
(L)
of
of
C 2
0,
Rwy
(L)
+
Ri (x) ,x())))))))))))))))))) (P)))) (1)
(equivariance of ((())
of
of
R
(L)
alL) (2) xx(P1)
0f
of
Rip-ss
(1) x (P)).
where a(4) defined as a(4)
2
(L)
a é is equivariant because:
(()) Ra (L)
Rx (P))
(L)
0: é Roo
(L) Rx (P)
a
Ray
dt Ré(w
(L)
x (P),) (é predicted by equivariant neural network)
R
1i
(L)
Tt é(
(L)
xx (P)),))
0t
0t
(L( (P))).
(T.
E.3 O(3)-EQUTVRR OF LEARNED LIKELIHOOD
Let R E 3 be an orthogonal matrix representing an element g from the general orthogonal group
033). We obtain the marginal probability density of the Markovian denoising process as follows
(a
(L) [e (11)
p(07 (1) 100
(P)
(p))) )po(oor |0 x (P) dx
(L)
T II
= p(z) (1)
x (P)
pa po(
lx (L) x (P) da
and the sample's 's likelihood -1)3))-quvvariant po Rad (T.) (1:
f pl (Rny (1) Ra (P))
II po
( RR!!"
Ra (1)
(1) Rx (P)dir
daa
T
= / p(
(L) (P)
II
po Rx (L)
Ray (L) Rx
(equivariant prior)
= f plaal (P)
II
T
(a
po
(L)
|o (
)
(equivariant transition probabilities)
po( (P())
F EEQUVARANA GRAPH NEURAL NETWORK
Chiral molecules cannot be superimposed by combination of rotations and translations. Instead they are mirrored along stereocenter, axis, or plane. As chirality can significantly alter molccule's 's chemical properties, we use variant of the E(3)-equivariant graph neural networks (Satorras et al. 2021] presented in Equations We change the coordinate update equation, Equ. (5), of standard EGNNs in the following way
5 that sensitive to reflections and hence -Eu()--quivariant.
x x
(a) x²) x (x) x')
1+ a.
ad.
Σ j
(1) 1 dl., a1y))
(h), h diy. any)
duy
[[x] x') (x) x()"
(18)
18

---

Under review as a conference paper at ICLR 2023
where z' denotes the center of mass of all nodes layer 1. This modification makes the EGNN layer sensitive to reflections while staying close to the original formalism. Since the resulting graph neural networks are only equivariant to the (3) group, we will hereafter call them SEGNNs for short.
F.1 DISCUSSION OF EQUIVARIANCE
Here study how the suggested change in the coordinate update equation breaks reflection symme- try while preserving equivariance to rotations. Messages and scalar feature updates (Eqqaation (3) and (4)) remain 3i33-invariant as the original model and are therefore not considered this sec-
tion. We analyze transformations composed of translation by R and rotation/reflection by
an orthogonal matrix R E 3R333 with Rd R I. The output at layer / + given the transformed input Rx; + at layer I is calculated as:
Rx + t (Rr', + t) (1+++ +
Ra +t- -(Ra' + t)) x (Ræ) +t (Ræ' + t))
Ra, + ++
Σ
02 (+)
dis +1
2 +1
(19)
R(a! 24)) x)(-) (Ræ) Raw) (Re) Re
Re + t+ Σ
dis
2iz +
(20)
R(ad arj) 0.(+)+ det det(R)
x²) ) x (a)) - x²))
Ro + +t+ E
(-)
du +1
Ziz +
(21)
R( (((
x²) x
(x)
x
Ro 1+1 +t+ (det(R)-
Σ 3yyt
Z +1
(22)
This result shows that the output coordinates are only equivariantly transformed if R is orientation preserving, i.e. det R)) = 1. If R a reflection (det R) = -1), coordinates will be updated with an additional summand that breaks the symmetry.
The learnable coefficients og (h', f di aij) and 6z (h,, h1 dig. ax)) only depend on relative dis- tances and are therefore E(3)-invariant. Their arguments are represented with the "." symbol for brevity. Likewise, the normalization factor [ x') x (ar) x')l is abbreviated as Z Al- ready in the first line we used the fact that the mean transforms equivariantly. Furthermore,
Ra x Rb = det R)RL R(a x b) in the second step, which can be derived as follows:
x² T Raa x Rb)- = det((n Ra, Rb))
(23)
ERIx3
det (R x. a. bl)
(24)
det(R) det R x, a, bl)
(25)
dee( 22 R(a R(a b))
(26)
x = (det(R) bi)
(27)
The stated property of the cross product follows because this derivation is true for all x E R
19

---

Under review as a conference paper at ICLR 2023
F.2 EMPIRICAL RESULTS
To show the effectiveness of this architecture simple toy example, repeat the classifi-
cation experiment by [Adams et al. (202T) who train neural networks classity tetrahedral chi- ral centers as right-handed (rectus, 'R') ) or left- handed (sinister, 'S'). We closely follow their data split and experimental set-up and only re- place the classifier with EGNN and SEGNNs, respectively. The results in Table 5 clearly demonstrate that the F33) EGNN capable of solving this task (without any hy- perparameter optimization) whereas the (3)- equivariant version does not do better than dom guessing.
Table 5: Accuracy on the R/S classification task. Results in the first section are taken from (Adams
et al. 202T and included for reference.
Model
R/S Accuracy (%)
ChiRo
98.5
SchNet
54.4
DimeNet++
65.7
SphereNet
98.2
EGNN
50.4
SEGNN
83.4
G EXTENDED RESULTS
G.1 ADDITIONAL. EXPERIMENTAL DETAILS
The numbers of available molecules differ slightly between different methods due to computational issues missing molecules in the available baseline sets. More precisely, on average 93.5, 92.8, and 98.3 molecules have been evaluated per pocket for DiftSBDD-cond, DiftSBDD-inpaint (C,), and DiftSBD- The set of 3D-SBDD molecules does not contain generated ligands for two test pockets. For the remaining 98 pockets, 89.9 molecules are available on average.
(Ca), respectively. For Pocket2Mol, 98. molecules are available per pocket.
All Figures show molecules generated where the starting number of nodes equals the number of nodes in the reference ligands, with the exception of Figure 4 which employs the sampling strategy outlined in Appendix C
G.2 ADDITIONAL MOLECULAR METRICS
In addition to the molecular properties discussed in Section 4 we assess the models' ability to produce novel and valid molecules using four simple metries: validity, connectivity, uniqueness, and novelty. Validity measures the proportion of generated molecules that pass basic tests by RDKit-mostly ensuring correct valencies. Connectivity the proportion of valid molecules that do not contain any disconnected fragments. We convert every valid and connected molecule from graph into canonical SMILES string representation, count the number unique occurrences in the set of generated molecules and compare those to the training set SMILES to compute uniqueness and novelty respectively.
Table 6 shows that only a small fraction of all generated molecules is invalid and must be discarded for downstream processing. The DiffSBDD models trained CrossDocked with C. pocket rep- resentation generate fragmented molecules about 50% of the time. Since we can simply select and process the largest fragments in these cases, low connectivity does necessarily affect the effi- ciency of the generative process. Moreover, all models produce diverse sets of molecules unseen in the training set.
G.3 OCTANOL- WATER PARTITION COEFFICIENT
The octanol-water partition coefficient (log P) is measure of lipophilicity and is commonly ported for potential drug candidates (Wildman & Crippen 1999). We summarize this property for generated molecules in Table
20

---

Under review as a conference paper at ICLR 2023
Table 6: Basic molecular metrics for generated small molecules given C. and full atom represen- lation of the protein pocket.
Model
Validity Connectivity
Uniqueness
Novelty
CrossDocked Training data
100%
100%
DifSSBDD-cond (Ca)
97.75%
48.02%
96.95%
100%
DifSSBDD-inpaint (Ca)
91.62%
51.38%
98.64%
100%
DifSBDD-cond
93.23%
83.46%
97.46%
100%
Binding MOAD Training data
96.38%
100%
DifiSBDD-cond (C.)
92.51%
52.13%
100.00%
100.00%
DiftSBDDiinpaint (C.)
90.28%
73.19%
100.00%
100.00%
DifSBDD-cond
95.39%
39.58%
100.00%
100.00%
Table 7: LogP values generated molecules.
CrossDocked
Binding MOAD
Test
0.894 + 2.73
0.456 + 1.15
3D-SBDD (AR) (Luo et al. 2021
0.273 + 2.01
Pocker2Mol (Png et al. 2022)
1.720 + 1.97
Difd (C,)
-0.184 + 1.01
0.110 + 1.03
Dif'SBDD-inpaint (C.)
-.5519 + 1.09
0.574 + 1.39
DifiSBDD-cond
-0.328 + 1.18
0.845 + 1.61
G.4 OPTIMIZATION
We demonstrate the effect the number of noising/denoising steps (t) has on various molecular prop- erties in Figure g We test all values of intervals of 10 steps and 200 molecules sampled at every timestep. Note this does not allow for explicit optimization any particular property unless combined with the evolutionary algorithm.
During the evolutionary algorithm, at the end of every generation the top 10 docking molecules are used to seed the next population. Every seed molecule is elaborated into 20 new candidates with a randomly chosen & between 10 and 150. To make the first population, we start with the single reference molecule and sample 200 molecules with f chosen above.
Similarity scores
QED scores
SA scores
LogP scores
reference molecule
0.5
Reference
0.8
1.0
0.4
ccorc
0.6
Logs scora
Sinnlarly score
0.0
0.3
DED
0.4
0.2
00.000000000
No. noisingtdencising steps
Figure 8: Effect number of noising/denoising steps on molecule properties.
G.S AGREEMENT OF GENERATED AND DOCKED CONFORMATIONS
Here discuss alternative way of using Quick Vina for assessing the quality of the conditional generation procedure besides its in silico docking score. We compare the generated raw confor- mations (before force-field relaxation) the best scoring Quick Vina docking pose and plot the
21

---

Under review as a conference paper at ICLR 2023
A
B
coderren
00-
E10
3 1100
135
08
6.0
Figure 9: RMSD between original and docked conformations for the CrossDocked dataset. (A) DifSSBDD-cond, sample size 8804. (B) DiffSBDD-cond (Ca), sample size 9611. (C) DiftSBDD- inpaint (C.), sample size 8641.
A
B
C
00-
1.0
Figure 10: RMSD between original and docked conformations for the Binding MOAD dataset. (A) DiffSBDD-cond, sample size 12 520. (B) DifISBDD-cond (C&), sample size 12 241. (C) DiffSBDD-inpaint (Ca), sample size 902.
distribution of resulting RMSD values in Figures and 10 As baseline, the procedure is repeated for RDKit conformers of the same molecules with identical center of mass. For a large percentage of molecules generated by the all-atom CrossDocked model, Quick agrees with the predicted bound conformations, leaving them almost unchanged (RMSD below 2 A). This demonstrates suc- cessful conditioning on the given protein pockets.
For the C.-only models results are less convincing. They produce poses that only slightly improve upon conformers lacking pocket-context. Likely, this caused by atomic clashes with the proteins' side chains that Quick Vina needs to resolve.
G.6 RANDOM GENERATED MOLECULES
Randomly selected molecules generated with our method and 3 baseline methods (LiGAN, SBDD- 3D and Pocket2Mol) when trained with CrossDocked are presented in Figure Randomly selected molecules generated by our method when trained with Binding MOAD are show in Figure 12
G.7 DISTRIBUTION OF DOCKING SCORES BY TARGET
We present extensive evaluation of the docking scores for our generated molecules in Figure 1 We evaluate all models trained with a given dataset first against all targets (Figure [3A+C) and TO ran- domly chosen targets (Figure 13 B+D). We note that the all-atom model trained using CrossDocked data outperforms all other methods. Unsurprisingly, model performance is highly target dependent, likely varying with properties like pocket geometry, size, charge, and hydrophbicity, which would affect the propensity of generating high affinity molecules.
H RELATED WORK
Diffusion Models for Molecules Inspired by non-equilibrium thermodynamics, diffusion models have been proposed to lear data distributions by modeling a denoising (reverse diffusion) process and have achieved remarkable success in variety of tasks such as image, audio synthesis and point
22

---

Under review as a conference paper at ICLR 2023
Target
LIGAN
SBDD-3D (AR)
pocket2mol
DifISBDD-cond
2llg
RC
LAx
g
4
-0%
X0%
3pnm
1afs
y0n
RR
9
9
PA
9
14gs
YC
16
1
3
4tos
90
ppppp
3li4
*b
9
%
Go
80
54
xt
4yhj
àgo
%
Agyz
3pnm
+
6
*
88
do
2pc8
86
At
8
9
Figure 11: Generated molecules for 10 randomly chosen targets in the CrossDocked test set. For each target, 3 randomly selected generated molecules from 4 models shown.
23

---

Under review as a conference paper at ICLR 2023
Target
DiffSBDD-cond
DiffSBDD-inpaint
2fky
Fmm
54
col
3zjx
Apoun
b
3gt9
967
8
5ndu
Ame
P
<
8
4
8
A+
2v18
*
x
x
1j78
ANC
sid
Av
Xm
3eks
mukg
mm
Xmo
90x
8,4,00
92-4
5zzb
Duyy
m
XY
wa
1fd7
ADC
Ap
2a5x
ArC
Figure 12: Generated molecules for 10 randomly chosen targets in the Binding MOAD set. For each target-model pair, randomly selected generated molecules are shown. C, level proteins were used for both models.
24

---

Under review as a conference paper at ICLR 2023
A
Docking Scares for Tergedx
Docking Scores Random Targets
CrossDocked
-10
Condit ona-s
2019
20m
1/9
Binding MOAD
Iepending
Decany Seae
-12
Figure 13: Docking scores of generated molecules for various methods trained on the CrossDocked (A-B) and Binding MOAD (C-D) datasets. (A) Violin plot of docking scores for all 3 methods trained using CrossDocked. (B) Same as before but for 10 randomly chosen targets sorted by mean score. (C) Violin plot of docking scores for all 2 methods trained using Binding MOAD, (D) Same as before but for 10 randomly chosen targets sorted by mean score.
cloud generation (Kingma al. 2021 Kong al. 202 Luo & Hu 2021 Recently, efforts have been made to utilize diffusion models for molecule design (Du et al. 2022b). Specifically, Hooge- boom et al. (2022) propose diffusion model with equivariant network that operates both on continuous atomic coordinates and categorical atom types to generate new molecules 3D space. Torsional Diffusion (Ting et al., 2022 focuses conditional setting where molecular conforma- tions (atomic coordinates) are generated from molecular graphs (atom types and bonds). Similarly, 3D diffusion models have been applied to generative design of larger biomolecular structures, such as antibodies (Luo et al. 2022) and other proteins (Anand & Achim 2022
Trippe et al. 2022).
Strucfure-based Drug Design Structure- based Drug Design (SBDD) (Ferreira et al. 2015 An-
derson 2003) relies on the knowledge of the 3D structure of the biological target obtained either through experimental methods or high-confidence predictions using homology modelling (Kelley
al. 2015). Candidate molecules then designed to bind with high affinity and specificity the
larget using interactive software (Kalyaanamoorthy & Chen 2011) and often human- intuition (Ferreira et al. 2015)). Recent advances in deep generative models have brought new wave of search that model the conditional distribution of ligands given biological targets and thus enable de novo structure-bassee based drug design. Most of recent work consider this task sequential generation problem and design variety of generative methods including autoregressive models, reinforcement learning, etc., to generate ligands inside protein pockets atom by atom (Drotár et al. 2021 Luo al., 202
Li al. 2021 Peng al. 2022).
Geometric Deep Learning for Drug Discovery Geometric deep learning refers to incorporating geometric priors in neural architecture design that respects symmetry and invariance, thus reduces sample complexity and eliminates the need for data augmentation (Bronstein et al.] [02TT. It has been prevailing variety of drug discovery tasks from virtual sereening de novo drug design as symmetry widely exists in the representation of drugs. One line of work introduces graph and geometry priors and designs message passing neural networks and equivariant neural networks that permutation- and translation-. rotation-, reflection-equue reflection-equivariintt respectively (Duvenaud et al. 2013 Gilmer et al., 2017 Satorras et al. 2021: Lapchevskyi et al. 2020; Du et al.22 and has been widely used in representing biomolecules from small molecules proteins (Atz et al. 2021
and solving downstream tasks such as molecular property prediction (Schütt et 2018 Kleppra
al. 2020), binding pose prediction (Stärk al. 2022) molecular dynamics (Batzner al. 2022
25

---

Under review as a conference paper at ICLR 2023
Holdijk et al. 2022). Another line of work focuses generative design of new molecules (Du et al. 2022b c). Specifically, they formulate molecule design as a graph or geometry generation problem and there are two strategies: one-shot generation that generates graphs (atom and bond features) in one step and sequential generation that generates them sequence of steps.
26


Reviews:
[{"review_id": "tLEU2PFvZR4", "reviewer": "Reviewer_bBKg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studied structure-based protein design, which means generating drugs specifically for a given protein target. The author proposed a conditional diffusion model for this task, which is an extension of the previous equivariant diffusion model [1] for 3D molecule generation. The author considered both conditional generation and inpainting generation tasks. Experiments show that the model can match the performance of current other methods.\n\n[1] Emiel Hoogeboom, Vıctor Garcia Satorras, Clement Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pp. 8867–8887. PMLR, 2022.\n", "summary_of_the_review": "The idea of diffusion models for structure-based drug design (SBDD) is good, but both the SBDD task and the equivariant diffusion method are not supervising things for ML community. Besides, the adoption of the \"inpainting\" technique from image generation is not clearly natural and useful for this problem. Besides, the empirical performance is also not significant, and some important baseline is missed.", "strength_and_weaknesses": "Strength:\n1. The paper is clearly written. Both the task and the method are clear and easy to follow.\n2. For the specific task of \"structure-based drug design\", the method is the first diffusion-based model implemented.\n3. The author nicely collected a new dataset for benchmarking this task.\n\nWeakness:\n1. Though the first diffusion for this specific task, the method itself of equivariant diffusion is not new. Besides, the task has also been well formulated in previous ML papers. Thus the papers seem to be a little incremental for ML conferences, with limited technical contributions.\n2. For the proposed method, the author also discussed the \"inpainting\" scenario. However, it's very unclear to me how this could be useful? It feels like a direct and unnatural adoption of diffusion models on images, but in my mind perturbing the protein targets is not very meaningful in the biological/chemical domain.\n3. Furthermore, as shown in Tab. 1 and Fig. 3, the empirical performance seems not significant compared with baselines, and many generated drugs are not realistic molecules. The major improvement is in diversity metric. However, I would say higher diversity with lower quality actually indicates lower generative modeling capacity. I suggest the authors to further improve the model's empirical performance.\n4. Some important baseline is missing [1].\n\n[1] Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. ICML 2022\nfor target protein binding.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ozuzzE1BsM", "reviewer": "Reviewer_Czkk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a diffusion-based model for generating a small molecule complete with 3D atomic coordinates in a protein pocket.  The performance of the model on a prior benchmark is well below that of pocket2mol (a related model) on the CrossDocket test set. The paper also presents a new dataset for the same task, which the authors call \"binding MOAD\" (from Mother Of All Databases).", "summary_of_the_review": "This method is valid, but not strong enough for ICLR2023.  The paper uses space on unnecessary items and avoids trying and to understand some of the results, which could have strengthened the presentation or lead to an improvement.  The current version of this paper is not at the level of ICLR2023.\n", "strength_and_weaknesses": "\nThe paper has a couple of strengths: the task that it attacks is of high importance for human health and the ideas that it discusses represent modern and powerful developments in machine learning.  The paper sets out to test inpainting in the context of small molecule generation in a pocket, a technique that worked extremely well for protein generation in David Baker's lab.  Importantly, the paper also adds a new protein-ligand dataset for generative small-molecule tasks.\n\nThe weaknesses of this work overwhelm the positives, unfortunately.  Although the particular task of generating desirable ligands that match a pocket is noble, there is no clear argument why the direct generation would be desirable in principle (for example, it is possible that a more practical approach is to simply learn a surrogate function.) Of course, this would not be important if the results were overwhelming; however, they are not. The model performs significantly worse than pocket2mol.  Importantly, the authors do not even evaluate pocket2mol on the new dataset that they generated (even though the code for pocket2mol is publically available).\n\nAlso the paper spends a lot of time discussing parts unrelated to this work (the paper does not use eq (3), (4), (5); instead it predicts the noise, as per eq (6)). Similar in page 5, the paper spends a lot of space discussing things that are not used.  Perhaps some of these things could move to an appendix or a different publication.  In page (4) what is the reference to Anonymous. ICLR2023 submission. 2022?\n\nImportantly, the authors discuss an E3 architecture but then mention rigid body transformations, which is SE3 only (E3 without reflections).  If they indeed use E3 (as appears to be the case, though I didn't check the code), then it is possible that update (10) might perhaps break due to a reflection symmetry of E3.  Did the authors check that indeed there is no mixup of stereo chemistries in the generated molecules (the distinction is important because a reflection could turn a therapeutic drug to a killing toxin.)\n\nWhy did the inpainting idea not work as well as expected based on the prior work by Baker's lab?  Also, why did the authors suggest that the inpainting was competitive to the conditional generation (it was possibly better on only 1 of 12 subtasks in tables 1, 2).  Why does the model perform so much worse than pocket2mol on the common CrossDocket set?  Why didn't the authors also run pocket2mol (code is available) on their new MOAD binding dataset?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NA9Vkj_4ABt", "reviewer": "Reviewer_WRoN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a diffusion model that conditionally generates 3D molecules. The proposed model modifies the existing E(n)-equivariant Graph Neural Networks with a new coordinate update step considering the characteristics of pocket coordinates. They frame pocket conditioning in two ways; pocket-conditioned small molecule generation and joint distribution with inpainting. Besides the proposed model, they also curate a new benchmark reflecting more realistic binding scenarios. Experiments show that the proposed method is better than other baselines in terms of diversity.\n\n", "summary_of_the_review": "Since it does not outperform others and the novelty is marginal it's straightforward to score it as rejected.", "strength_and_weaknesses": "\n[Strength]\n* For the first time, they approach the 3D molecule generation problem as an inpainting problem, which is novel.\n* They evaluate 3D molecule generation methods on a new benchmark to reflect more realistic binding scenarios.\n* The paper is easy to read and well organized.\n* Domain-specific case studies show various aspects of the model. \n\n\n[Weaknesses]\n* Although their approach is novel, the core model relies on the existing model EGNN with minor modifications.\n* The primary goal of molecule generation is to achieve high affinities (low VINA scores). However, the proposed method underperforms not only other baselines but also the test set. Therefore, the motivation of the proposed methods is not convincing.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "DFR0oYTsb1", "reviewer": "Reviewer_qoqG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose DiffSBDD, an approach to structure-based drug design (SBDD) that aims to generate molecules in 3D that are conditioned on a protein binding interface. The approach is E(3)-equivariant and leverages the denoising diffusion probabilistic model (DDPM) formulation inspired by non-equilibrium thermodynamics.\n\nTwo distinct approaches to 3D pocket conditioning are proposed. The first keeps the pocket constrained in each denoising step (conditional) and the second approximates the joint distribution of ligand-pocket pairs enabling sampling of both ligand and protein positions (inpainting). \n\nTo demonstrate the applicability of the proposed model, the authors create a new dataset of experimentally derived protein-ligand complex data from the publically available database Binding MOAD in addition to a synthetic cross-docked dataset. \nDe-novo molecule generation was evaluated for the conditional approach in the context of both all-atom and Cα (for protein) level graphs. For the inpainting approach, only Cα level graphs could be evaluated due to computational limitations. In silico metrics were calculated for generated molecules and compared with two recent deep-learning approaches to SBDD (3D-SBDD and Pocket2Mol). **DiffSBDD did not perform better**. ", "summary_of_the_review": "The authors propose DiffSBDD, an E(3)-equivariant 3D-conditional diffusion model for SBDD which they evaluate in the molecular generation of novel/diverse ligands that maximize predicted affinity. While they demonstrate that inpainting can achieve competitive results to other deep learning approaches to SBDD, it was not superior, and no null model is presented to demonstrate the significance of the results. In addition, the curation and preparation of the protein and ligand structures were not reported/neglected calling into question the validity of the results. Lastly, the proposed DDPM formulation does not appear to be a novel contribution which limits the significance of the manuscript contributions. Therefore, my recommendation is that the manuscript is premature for acceptance to ICML.", "strength_and_weaknesses": "**Strengths**:\n\nThe paper takes a recently published DDPM formulation for modeling protein-ligand interactions and introduces what appears to be a novel approach to 3D pocket conditioning (conditional and inpainting). They also use an experimental database (Binding MOAD) for training the model instead of synthetic data that often has no grounding in reality.\n\n**Weaknesses**:\n\nA fundamental flaw in this work appears to be a misunderstanding by the authors revealed in the second sentence of the abstract. SBDD pipelines start with ligand and protein preparation which is a prerequisite for large-scale docking. This study excludes this extremely important step as ionization/tautomeric states for ligands and H-bond optimization/protonation for proteins are neglected. SBDD is only as good as the structures you put into it (garbage in~garbage out). \n\nThe novelty of the proposed DDPM formulation is lacking as this work closely follows the framework developed by Hoogeboom et al. 2022. In addition, the paper does not include results for the inpainting experiments using all-atom protein level graphs. The poor results achieved using only C*α* atoms for the protein are not surprising as SBDD requires explicit representation of rotamers for amino acids as the orientation/protonation of these residues can dramatically alter what interactions can be made and ultimately whether a molecule will bind with sufficient potency to warrant detection in an in vitro assay.\n\nSome of the molecules generated by this approach are not physically possible to produce and would be incredibly unstable. The paper uses a reference to Ertl & Schuffenhauer 2009 to justify that the low SA score may be because it is based on historical knowledge of previously synthesized molecules, but this is precisely the knowledge needed to understand what is possible on planet earth at 1 ATM with the tools we have at our disposal to synthesize real molecules that we can test experimentally to evaluate these predictions. I take a similar issue with the use of the CrossDocked dataset for training as these are in silico docking predictions that have not been experimentally validated and therefore should not be expected to generate results that are grounded in reality.\n\nThe paper does not propose a null model to evaluate the significance of the reported results.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject