3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction
Authors: Jiaqi Guan
Wesley Wei Qian
Xingang Peng
Yufeng Su
Jian Peng
Jianzhu Ma
Venue: ICLR 2023 poster (2023)

Abstract: Rich data and powerful machine learning models allow us to design drugs for a specific protein target <em>in silico</em>. Recently, the inclusion of 3D structures during targeted drug design shows superior performance to other target-free models as the atomic interaction in the 3D space is explicitly modeled. However, current 3D target-aware models either rely on the voxelized atom densities or the autoregressive sampling process, which are not equivariant to rotation or easily violate geometric constraints resulting in unrealistic structures. In this work, we develop a 3D equivariant diffusion model to solve the above challenges. To achieve target-aware molecule design, our method learns a joint generative process of both continuous atom coordinates and categorical atom types with a SE(3)-equivariant network. Moreover, we show that our model can serve as an unsupervised feature extractor to estimate the binding affinity under proper parameterization, which provides an effective way for drug screening. To evaluate our model, we propose a comprehensive framework to evaluate the quality of sampled molecules from different dimensions. Empirical studies show our model could generate molecules with more realistic 3D structures and better affinities towards the protein targets, and improve binding affinity ranking and prediction without retraining.

Published as a conference paper at ICLR 2023
3D EQUIVARIANT DIFFUSION FOR TARGET-AWARE MOLECULE GENERATION AND AFFINITY PREDICTION
Jiaqi Guan Wesley Wei Qian Xingang Peng?, Yufeng Su' Jian Jianzhu Ma
1 Department of Computer Science, University of Illinois Urbana- Urbana-Champaa Champaign
2 School of Intelligence Science and Technology, Peking University
3 Institute for AI industry Research, Tsinghua University
{jiagi, weiqian3, ji jianpeng] 1 Linoisi..... .edu, ma jajianzhueair..o weeiiiirssiinggi cginngua..nnno.. edu. cn
ABSTRACT
Rich data and powerful machine learning models allow us to design drugs for a specific protein target in silico. Recently, the inclusion of 3D structures during targeted drug design shows superior performance to other target-free models as the atomic interaction in the 3D space is explicitly modeled. However, current 3D target-aware models either rely on the voxelized atom densities or the autoregres- sive sampling process, which are not equivariant to rotation or easily violate geo- metric constraints resulting in unrealistic structures. In this work, we develop 3D equivariant diffusion model to solve the above challenges. To achieve target-aware molecule design, our method learns a joint generative process of both continuous atom coordinates and categorical atom types with Squia-equivariant network. Moreover, we show that our model can serve as an unsupervised feature extractor estimate the binding affinity under proper parameterization, which provides an effective way for drug screening. To evaluate our model, we propose a compre- hensive framework evaluate the quality of sampled molecules from different dimensions. Empirical studies show our model could generate molecules with more realistic 3D structures and better affinities towards the protein targets, and improve binding affinity ranking and prediction without retraining.
INTRODUCTION
Rational drug design against a known protein binding pocket is efficient and economical approach for finding lead molecules (Anderson 2003 Batool et al., 2019) and has attracted growing attention from the research community. However, it remains challe challenging and computationally intensive due to the large synthetically feasible space (Ragoza al. (022). and high degrees of freedom for binding poses (Hawkins) 2017). Previous prevailed molecular generative models are based on either molecular string representation (Bjerrum and Threlfall 2017: Kusner et al. 2017] Segler et al.) 2018) or graph representation et al. 2018: Liu et al. 2018: Jin et al. 2018 Shi et al., (220), but both representations do not take the 3D spatial interaction into account and therefore not well suited for target-aware molecule generation. With recent development in structural biology and protein structure prediction (Jumper al.) 2021), more structural data become available (Francoeur al.) 2020) and unlock new opportunities for machine learning algorithms to directly design drugs inside 3D binding complex (Gebauer al. 2019 Simm et al. 2020a b).
Recently, new generation of generative models are proposed specifically for the target-aware molecule generation task (Luo al., 2021] Ragoza al., 2022 Tan et al. 2022 Liu et al. 2022 Peng et al. 2022). However, existing approaches suffer from several drawbacks. For instance, Tan al. (2022) does not explicitly model the interactions between atoms of molecules and proteins in the 3D space, but only considers the target as intermediate conditional embeddings. For those that do consider the atom interactions in the 3D space, [Ragoza et al. (2022) represents the space as voxelized grids and model the proteins and molecules using 3D Convolutional Neural Networks (CNN). However, this model is not rotational equivariant and cannot fully capture the 3D inductive
"qqual Contribution

---

Published as a conference paper at ICLR 2023
biases. In addition, the voxelization operation will lead to poor scalability since the number of vox- els increases at a cubic rate to the pocket size. Advanced approaches achieve SE(3) -quivvarre through different modeling techniques (Luo et al., [021 Liu et al. 2022; Peng et al. 2022). How- ever, these methods adopt autoregressive sampling. where atoms are generated one by one based on the learned probability density of atom types and atom coordinates. These approaches suffer from several limitations: First, the mismatch between training and sampling incurs exposure bias. Secondly, the model assigns an unnatural generation order during sampling and cannot consider the probability of the entire 3D structure. For instance, it would be easy for the model to correctly place the n-th atom to form a benzene ring the n 1-th carbon atoms have already been placed in the same plane. However, it would be difficult for the model to place the first several atoms ac- curately since there is limited context information available, which yields unrealistic fragments as consequence. Moreover, the sampling scheme does not scale well when generating large binding molecules is necessary. Finally, current autoregressive models could not estimate the quality of gen- erated molecules. One has to rely other tools based on physical-chemical energy functions such as Dock (Trott and Olson 2010) to select the drug candidates.
To address these problems, we propose TargetDiff, a 3D full-atom diffusion model that generates target-aware molecules in a non-autoregressivve fashion. Thanks to recent progress in probabilistic diffusion models (Ho et al.
2020; Hoogeboom al. (02T) and equivariant neural networks (Fuchs) ett al. 2020; [Satorras et al.
2021b). our proposed model can generate molecules in continuous 3D) space based on the context provided by protein atoms, and have the invariant likelihood w.r.t global translation and rotation of the binding complex. Specifically, we represent the protein binding pock- ets and small molecules as atom point sets in the 3D space where each atom is associated with 3D Cartesian coordinate. We define a diffusion process for both continuous atom coordinates and discrete atom types where noise is gradually added, and learn the joint generative process with a SE(3)-equivariant graph neural network which alternately updates the atom hidden embedding and atom coordinates of molecules. Under certain parameterization, we can extract representative fea- tures from the model by forward passing the input molecules once without retraining. We find these features provide strong signals to estimate the binding affinity between the sampled molecule and target protein, which can then be used for ranking drug candidates and improving other supervised learning frameworks for binding affinity prediction. An empirical study on the Cross dataset (Francoeur et al., 2020) shows that TirgetDiff generates molecules with more realistic 3D structures and better binding energies towards the protein binding sites compared to the baselines.
Our main contributions can be summarized as follows:
An end-to-end framework for generating molecules conditioned on protein target, which explicitly considers the physical interaction between proteins and molecules in 3D space.
So far as we know, this is the first prohabilistic diffusion formulation for target-aware drug design, where training and sampling procedures are aligned in a non-autoregressive as well as SE(3)-equivarian fashion thanks to a shifting center operation and equivariant GNN.
Several new evaluation metrics and additional insights that allow us to evaluate the model generated molecules in many different dimensions. The empirical results demonstrate the superiority of our model over two other representative baselines.
Propose an effective way to evaluate the quality of generated molecules based on our frame- work, where the model can be served as either a scoring function to help ranking or an unsupervised feature extractor to improve binding affinity prediction.
RELATED WORK
Molecule Generation with Different Representations Based on different levels of representa- tions, existing molecular generative models can be roughly divided into three categories string- based, graph-based, and 3D-structure-based The most common molecular string representation is SMILES [Weininger 1988), where many existing language models such as RNN can be re-purposed for the molecule generation task (Bjerrum and Threlfal] 2017 Boonnzz-lBmarellll et al. 2018 Kus- ner et al.) 2017 Segler et al., 2018). However, SMILES representation is not optimal choice since it fails to capture molecular similarities and suffers from the validity issue during the genera- tion phase (Jin et al. 2018). Thus, many graph-based methods are proposed to operate directly on
2

---

Published as a conference paper at ICLR 2023
graphs (Liu al. 2018: Shi et al. 2020 Jin et al. 2018 2020 You al. 2018 Zhou al. 2019
On the other hand, these methods are very limited in modeling the spatial information of molecules that is crucial for determining molecular properties and functions. Therefore, recent work (Gebauer 2019; Skalic et al., 2019a Ragoza et al.] 2020; (Simm et al. 2020a|b) focus on generating molecules 31D space. More recently, flow-based and diffusion-based generative models (Satorras
et al.
et al.
202 Hoogeboom et al. 2022) are developed to leverage E(n)-Equivarian GNN [Satorras
al. b02lb and achieve SE(3 eequ)evraaaannne in molecule generation.
Target-Aware Molecule Generation As more structural data become available, various genera- tive models are proposed to solve the targtt-aware molecule generation task. For example, Skalic al. (2019b); Xu al. (2021) generate SMILES based on protein contexts. Tan al. (022) pro- pose a flow-based model to generate molecular graphs conditional on a protein target as a sequence embedding. Ragoza et (2022) try generate 3D molecules by voxelizing molecules in atomic density grids in a conditional VAE framework. Li et al. (2021) leverage Monte-Carle Tree Search and a policy network to optimize molecules in 3D space. Luo al. (202T): Liu al. (2022); Peng et al. (2022) develop autoregressive models to generate molecules atom by atom in 3D space with GNNs. Despite the progress made in this direction, the models still suffer from several issues,
including separately encoding the small molecules and protein pockets (Skalic et al., 2019b Xu ct al. 2021; Tan et al., 2022 Ragoza et al.) 2022), relying on voxelization and non non-equivarinnc, networks (Skalic al. 2019b Xu et al. 2021 Ragoza et al., 2022). and autoregressive sampling (Luo et al. 2021 Liu et 2022 Peng et al. 2022). Different from all these models, our equivariant
model explicitly considers the interaction between proteins and molecules in 3D and can perform non-autoregressive sampling, which better aligns the training and sampling procedures.
Diffusion Models Diffusion models (Sohl- et al., 2015) are a new family of latent vari- able generative models. [o et al. (2020) propose denoising diffusion probabilistic models (DDPM) which establishes a connection between diffusion models and denoising score-based models (ong and Ermon 2019). The diffusion models have shown remarkable success in generating image data (Ho al.[[202)] | Nichol and Dhariwal] 202T) and discrete data such as text (Hoogeboom et al. 2021 Austin et al. [202]). Recently, it has also been applied in the domain of molecules. For example, GeoDiff (Xu et al. (022) generates molecular conformations given 2D molecular graphs. EDM (Hoogeboom al., 2022) generates 3D molecules. However, the unawareness to potential targets
make it hard to be utilized by biologists in real scenarios.
METHODS
3.1 PROBLEM DEFINITION
A protein binding site is represented a set of atoms P ( where Np is the number of protein atoms, E R represents the 3D coordinates of the atom, and up E R represents protein atom features such as element types and amino acid types. Our goal is to generate binding molecules M = {(x()) 2z
(z) } conditioned on the protein target. For brevity, we denote molecules as M = xx, v). where [ -] is the concatenation operator and x E RM3 and v E M denote atom Cartesian coordinates and one-hot atom types respectively.
3.2 OVERVIEW OF TARGETDIFF
As discussed in Sec. and Sec. 2 we hope develop a non-autoregressive model to bypass the drawbacks raised autoregressive sampling models. In addition, we also require the model to represent the pootein-ligand complex in continuous 3D space to avoid the voxelization operation. Last but not least, the model will also need to be SEquivaaui to global translation and rotation.
We therefore develop TargetDiff, molecule generation based on the DDPM framework Ho et al. (2020). TargetDiff is a latent variable (Mo (P)ddP))MM where M,,
equivariant non-autoregressive method for target-aware
model of the form Pa PP) latent variables with the same dimensionality as the data Mo ~ (()))P). As shown in Fig. [ the approach includes a forward diffusion process and reverse generative process, both defined as Markov chains. The diffusion process gradually injects noise to data, and the generative process
Mr is sequence of
3

---

Published as a conference paper at ICLR 2023
g M,
Diflusion Process
MT
M
Mi-1
M
Mo
Intiialization
Pa J.. P)
Generative Process
Figure 1:
Overview of TargetDiff. The diffusion process gradually injects noise to the data, and the generative process Iearns recover the data distribution from the noise distribution with network parameterized by 0.
learns to recover data distribution from the noise distribution with network parameterized by 0:
g(M1. Me, P) II.= (M,M,(M)
ps |M P) II2 (M [M, P) (1)
Since our goal to generate 3D molecules based on given protein binding site, the model needs to generate both continuous atom coordinates and discrete atom types, while keeping SE(3)-equivarian during the entire generative process. In the following section, we will elaborate on how we construct the diffusion process, parameterize the generative process, and eventually train the model.
3.3 MOLECULAR DIFFUSION PROCESS
Following recent progress learning continuous distributions Ho et al. (2020) and discrete distri- butions Hoogeboom al. (2021) with diffusion models, use Gaussian distribution / to model continuous atom coordinates x and a categorical distribution C to model discrete atom types v. The atom types are constructed as one-hot vector containing information such element types and membership in an aromatic ring. We formulate the molecular distribution as a product atom coordinate distribution and atom type distribution. At each time step small Gaussian noise and uniform noise across all categories are added to atom coordinates and atom types separately, according a Markov chain with fixed variance schedules 31,...,37:
q( (M M P) = Nxx V 1, B,I) - B./K).
(2)
We note that the schedules can be different in practice, but we still denote them with the same symbol for conciseness. Here, we decompose the joint molecule distribution as the product of two independent distributions of atom coordinates and atom types during diffusion, because the independent distributions have concise mathematical formulations and we can efficiently draw noisy samples from them. In the next section, we will see the dependencies between atom coordinates and atom types are considered by the model the generative process.
Denoting 1- 3r and ar = II Ora, desirable property of the diffusion process is to calculate noisy data distribution q( [M, Mo) of any time step in closed-form:
((x (xx; Xo (1 I)
<(v vo) C(v.lavvo + - (t)KK)
Using Bayes theorem, the normal posterior of atom coordinates and categorical posterior of atom types can both be computed in closed-form:
(xx,-1 xo)
(x 15 ph (x,, xo), 3.1)
q(ve-1/Ve vo) -(v vo)). (4)
where (x,, xo) -ol T&
1-at Xo
vorl--au 1-or
Xx, 3r
1 1- 3r. I-lie
and (vi,, Vo) c² EK ot and c¹ (Vi, vo) (arve + - oz) 1o (1-
3.4 PARAMETERIZATION OF EQUIVARIANT MOLECULAR GENERATIVE PROCESS
The generative process, on reverse, will recover the ground truth molecule Mo from the initial noise Mr, and approximate the reverse distribution with neural network parameterized by 0:
PM Mc, P) -X oo ([x,, (l,,,P),o C(v.-1 ((x vz), (,))
(5)
4

---

Published as a conference paper at ICLR 2023
One desired property of the generative process is that the likelihood po (M should be invariant to translation and rotation of the protein-ligand complex, which is a critical inductive bias for gen- erating 3D objects such molecules (Köhler al. 2020 Satorras et al.] 2021a Xu et al. 2022 Hoogeboom et al., 2022). One important piece of evidence for such achievement is that an invariant distribution composed with an equivariant transition function will result in invariant distribution. Leveraging this evidence, we have the following proposition in the setting of target-aware molecule
Proposition 1. Denoting the srans-rionooonnaaioon as Ty. we could achieve invariant likelihood w.r.t T, on the protein-ligand complex: po(T, (MolP)) (a(M)) if we shift the Center of Mass (CoM) of protein atoms to zero and parameterize the Markov transition p(xt-1 xx,x)) Xp) with an SE(3) eqqivvaraa network.
A slight abuse of notation in the following is that we use xt(L 1,..., T) to denote ligand atom coordinates and Xp to denote protein atom coordinates. We analyze the operation of shifting CoM in Appendix B and prove the invariant likelihood Appendix C
There are different ways to parameterize [ Vel. P) and ce (x.. P). Here, we choose let the neural network predict [x., Vo] and feed it through equation | to obtain pág and eo which define the posterior distributions. Inspired from recent progress in equivariant neural networks (Thomas et al. 2018 Fuchs et al., 2020; /aatorass al. 2021b; Guan et al.) 2022), we model the teraction between the ligand molecule atoms and the protein atoms with SE(3) Eqqivvaaaa GNN:
[ vol Φe [Me, t, P) de 60([x,, v)),
At the (-th layer, the atom hidden embedding h and coordinates x updated alternately as follows:
h net1
h +
Σ
fu h4. hnj, cij; 0%)
JEV
x X
Σ JEV
(x}- (x
h nit eiy; m
where dij ||x; x, is the euclidean distance between two atoms and j and eij is an additional feature indicating the connection between protein atoms, ligand atoms or protein atom and ligand atom. I mot is the ligand molecule mask since we do not want update protein atom coordinates. The initial atom hidden embedding hd is obtained by an embedding layer that encodes the atom information. The final atom hidden embedding h is fed into a multi-layer perceptron and softmax function to obtain Vo. Since &o is rotation equivariant to X; and it is easy to see X is rotation equivariant to Xg according to equation we achieve the desired equivariance for Markov transition. The complete proof can be found in Appendix A
3.5 TRAINING OBJECTIVE
The combination of q and is variational auto-encoder (Kingma and Welling 2013). The model can be trained by optimizing the variational bound on negative log likelihood. For the atom coordi- nate loss, since (Xx111 xx,,6) and (x xx) are both Gaussian distributions, the KL-divergence can be written in closed form:
(x)
|ir (x), vol, P)I C == =ellxo - x011 C
(8)
207
a and C' is a constant. practice, training the model with an unweighted where "/ 20
MSE loss (set 7/ = 1) could also achieve better performance as Ho et al.) (2020) suggested. For the atom type loss, we can directly compute KL-divergence of categorical distributions follows:
4.21 = Eclve vols log c(ve. volk e(ve, Vo)k
The final loss is a weighted sum of atom coordinate loss and atom type loss: L We summarize the overall training and sampling procedure of TargetDiff in Appendix
1( + -L
5

---

Published as a conference paper at ICLR 2023
3.6 AFFINITY RANKING AND PREDICTION AS UNSUPERVISED LEARNER
Generative models are unsupervised learners. However, in the area of target-aware molecule gen- eration, nobody has established the connection between the generative model and binding affinity, which is an important indicator for evaluating generated molecules. Existing generative models can not (accurately) estimate the quality of generated molecules. Especially for models relying on auto- regressive sampling. they have to assign an unnatural order when performing likelihood estimation (if possible) and cannot capture the global context as a whole.
We first establish the connection between unsupervised generative models and binding affinity rank- ing / prediction. Under our parameterization, the network predicts the denoised [Xg. Vol. Given the protein-ligand complex, we can feed the with [xo. vol while freezing the x-update branch (i.e. only atom hidden embedding updated), and we could finally obtain h and Vn:
h I+
hy
E fuld h/, h1y, Czy; 0n) JEV
Vo softmax(ML((h*))
(10)
Our assumption is that if the ligand molecule has a good binding affinity to protein, the flexibility of atom types should be low, which could be reflected in the entropy of Vo. Therefore, it can be used as a scoring function to help ranking, whose effectiveness is justified the experiments. In addition, h also includes useful global information. We found the binding affinity ranking performance can be greatly improved by utilizing this feature with a simple linear transformation.
4 EXPERIMENTS
4.1 SETUP
Data We CrossDocked2020 (Francoeur al., 2020) to train and evaluate TargetDiff. Similar to Luo et al. (222)) we further refined the 22.5 million docked protein binding complexes by only selecting the poses with low (< 1Å) and sequence identity less than 30%. In the end, we have 100,000 complexes for training and 100 novel complexes as references for testing.
Baseline For benchmarking, we compare with various baselines: liGAN (Ragoza et al. 2022 AR (Luo et al. 2021), Pocket2Mol (Peng et al. 2022). and GraphBP (Liu al. 2022). liGAN is 3D CNN-based method that generates 3D voxelized molecular images following conditional VAE scheme. AR. Pocket2Mol and GraphBP are all GNN-based methods that generate 3D molecules by sequentially placing atoms into protein binding pocket. We choose AR and Pocket2Mol as representative baselines with autoregressive sampling scheme because of their good empirical per- formance. All baselines are considered in Table 3 for a comprehensive comparison.
TifgetDiff Our model contains 9 equivariant layers described in equation 7 where fn and fa are specifically implemented as graph attention layers with 16 attention heads and 128 hidden features. We first decide on the number of atoms for sampling by drawing prior distribution estimated from training complexes with similar binding pocket sizes. After the model finishes the generative process, we then use OpenBabel OO'Boyle et al. 201T) to construct the molecule from individual atom coordinates done in AR and liGAN. Please Appendix F for the full details.
4.2 TARGET-AWARE MOLECULE GENERATION
We propose a comprehensive evaluation framework for target-aware molecule generation to justify the performance of our model and baselines from the following perspectives: molecular structures, target binding affinity and molecular properties.
Molecular Structures First, we plot the empirical distributions of all-atom distances and carbon- carbon bond distances in Figure 2 then compare them against the same empirical distributions for reference molecules. For overall atom distances, TargetDiff captures the overall distribution very well, while AR and Pocket2Mol has overreepresentation for small atom distances. Due to its limited voxelized resolution, liGAN can only capture the overall shape but not specify modes. Simi- larly, different carbon-carbon bonds form two representative distance modes in reference molecular

---

accurate 3D representation of Published as a conference paper at ICLR 2023
Bond
liGAN
PocketZMol
TargeDii
C-C
0.601
0.609
0.496
0.369
C-C
C
0.665
0.620
0.561
0.505
C-N
0.634
0.474
0.416
0.363
C-N N
0.749
0.635
0.629
0,550
C-O
0.656
0.492
0.454
0,421
C=0
0.661
0.558
0.516
0.461
C.C
0.497
0.451
0.416
0.263
C:N
0.638
0.552
0.487
0.235
Cisance carten bondja)
olartion bandud)
Dirance band
Jensen-Shannol divergence be- tween the distributions of bond distance for reference vs. generated molecules. "-", "=", and "." represent single, double, and aro- matic bonds, respectively. A lower value better.
Table 1:
Comparing the distribution for distances of all- atom (top row) and carbon-carbon pairs (bottom row) for reference molecules in the test (gray) and model gener- ated molecules (color). Jensen-Shannon divergence (ISD) between two distributions is reported.
Figure 2:
Ring Size
Ref.
liGAN
AR
Pocker2Mol
TargeiDidf
PocketzMlo
Median RVSD (4)
1.7%
28.1%
29.9%
0.1%
0.0%
4
0.0%
15.7%
0.0%
0.0%
2.8%
5
30.2%
29.8%
16.0%
16.4%
30.8%
6
67.4%
22.7%
51.2%
80.4%
50.7%
2.6%
2.6%
8
0.0%
0.8%
0.7%
0.3%
2.7%
9
0.0%
0.3%
0.5%
0.1%
0.9%
Fragment Size
Figure 3: Median RMSD for rigid fragment be- fore and after the force-field optimization.
Table 2: Percentage of different ring sizes for refer- ence and model generated molecules.
structures. While we can still the two modes in TargetDiff generated structures, only a single mode is observed for ones generated by liGAN, AR, and Pocket2Mol. In Table [ we further eval- uated how well different generated molecular structures capture the empirical distributions of bond distances in reference molecules, measured by -insen-Shannon divergence (JSD) Lin (991).. We found that TargetDiff outperforms other methods with clear margin across all major bond types.
Secondly, we investigate whether TargetDiff' can generate rigid sub-structure / fragment in a consis- tent fashion (e.g., all carbons benzene ring are in the same plane). To measure such consistency, we optimize the generated structure with Merck Molecular Force Field (MMFF) Haaggenn (1996) and calculate the RMSD between pre-and pos- MMFT-optimized coordinates for different rigid fragments that do not contain any rotatable bonds. As shown in Figure 3 TargetDiff is able gen- crate more consistent rigid fragments. In a further analysis, we discover that liGAN and AR tend to generate large amount of 3- and 4- member rings (Table 2). While TargetDiff shows larger proportion of 7-memher-ring, we believe this represents a limitation in the reconstruction algorithm and could be an interesting future direction to replace such post-hoc operation with bond generation.
These results suggest that TargetDiff can produce more realistic molecular structures throughout the process compared to existing baselines and therefore perceive the molecular dynamics leading to better protein binders.
Target Binding Affinity Figure 4 shows the median Vina energy (computed by AutoDock Vina (Eberhardt al. (021i) of all generated molecules for each binding pocket. Based the Vina energy. generated molecules from TargetDiff show the best binding affinity in 57% of the targets, while the ones from liGAN, AR and Pocket2Mol In terms of high-affinit binder, we find that on average 58.1% of the TargetDiff molecules show better binding affinity than the reference molecule, which clearly better than other baselines (See
only best for 4%, 13% and 26% of all targets.

---

Published as a conference paper at ICLR 2023
Median Vina Energy
-10
target 0
target 10
target 20
target 30
Target 40
target 50
target 60
target 70
target 80
target 90
lIGAN (lowest in 4%)
AR (lowest in 13%)
Packet2Mall (lowest 26%)
TargetDiff (lowest in 57%%)
Figure 4: Median Vina energy for different generated molecules (liGAN vs. AR vs. TifgetDiff) across 100
testing binding targets. Binding targets sorted by the median Vina energy of TargetDiff generated molecules. Lower Vina energy means a higher estimated hinding affinity.
Median Shift Center Mass
Reference
AR AR(0ng M.ngg.m1.700 7971
4YKJAA
You Smip
TargetDin
aggrt
4YH2_A
4GLK,A
Figure a) Binding poses two poor-AR R--biniig pockets. b) Estimating the shift binding poses hetween generated and reference molecules by calculating the distance hetween their center of mass (CoM). Each point represents the median CoM distance for target.
Table 3).
We further compute Vina Score and Vina Min in Table 3 directly computed or locally optimized without re-docking. They directly reflect the quality of model generated 3D molecules and similarly, our model outperforms all other baselines.
where the Vina score function
To better understand the differences in generated molecules, we sample generated molecule from each model for two pockets where TirgetDiff outperforms AR. As shown in Figure 5h, while Tar- getDiff can generate molccules occupying the entire pocket, AR is only able to generate a molecule that covers part of the space and potentially loses its specificity for the desired target and cause off- target effects. Let us consider the AR-generated molecules for 4QLK_A. Despite having a similar number of atoms as the TargetDiff molecule (27 vs. 29), the frontier network in AR keeps placing molecules deep inside the pocket instead of considering the global structure, and trying to cover the entire binding pocket results poor binding affinity. To further quantify such effects, we measure the distance between the center of mass (CoM) for reference molecules and the CoM for generated molecules. As shown in Figure 5p. the sequential generation nature of AR results in larger shift in CoM (1.79Á vs. 1.45A) and presents sub-optimal binding poses with poorer binding affinity.
Molecular Properties Besides binding affinity, we further investigate other molecular properties for generated molecules, including drug likeliness QED (Bickerton et al. 2012). synthesizability
SA (Ertl and Schuffenhauer 2009 You al. 2018), and diversity computed as the average pairwise
Metric
Vina Score (1)
Vina Min (L)
Vina Dock (I)
High Affinity (1)
QED
(f)
Diversity (*)
Model
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Mod.
liGAN
11.1%
0.39
0.57
GraphBP
-4,00
-4.70
14.2%
6.7%
0.43
0.45
0.49
0.48
0.79
0.78
37.9%
Pocket2Moll
.5.14
4.70
-6.22
-5.82
-7.15
-6.79
48.4%
51.0%
0.56
0.57
0.74
0.75
0.69
0.71
TargetDiff
Reference
6.36
6.46
6.71
6..99
77.5
.7.26
0.48
0.47
0.73
0.74
Table 3: Summary of different properties of reference molccules and molecules generated by our model and other baselines. For liGAN and GraphBP, AutoDock Vina could not parse generated atom types and thus we use QVina (Alhossary et al. 2015) perform docking, additional evaluation results Appendix
8

---

Published as a conference paper at ICLR 2023
0.42
-0.35
Metric
RMSE
Pearson
1 Spearman MAE
Model
TranCPI
1.741
0.576
0,540
1.404
MONN
1.438
0.624
0.589
1.443
vind
v_ent
IGN
1.433
0.698
0.641
1,9
14
0.46
0.65
HOLOPROT
1.208
STAMP-DPI
1.658
0,545
0.411
1.325
15
EGNN
1.445
0.648
0.598
1.141
EGNN + ours
1.374
0.680
0.637
1.118
combined
hioden_emb
Figure 6: Binding affinity ranking results on CrossDocked2220, Spearman's 's rank correlation coefficients between different indicators and ex- perimentally measured hinding affinity are shown.
Table 4: Binding affinity prediction results PDBbind v2020. EGNN augmented with our unsupervised features achieves best results on all four metrics.
Tanimoto distances (Bajusz et al. 2015 Tanimoto 1958). As shown in Table TargetDiff can gen- erate more high-affinity binders compared to liGAN, AR, and GraphBP while maintaining similar other 2D metrics. The metrics TirgetDiff does fall behind Pocket2Mol are the QED and SA scores. However, we put less emphasis on them because in the context of drug discovery, QED and SA are used as rough filter and would be fine as long as they are in a reasonable range. Therefore, they might not be the metrics we want to optimize against. We believe future investigation around prediction on bonds and fragment-bass baased (instead of atom-based) generation could lead to improvement.
4.3 BINDING AFFINITY RANKING AND PREDICTION
To justify that our model can serve as an unsupervised learner improve the binding affinity ranking and prediction, we first check Spearman's 's rank correlation CrossDocked2220. AutoDock Vina score (i.e. vina) and negative bog-transformed experimentally measured binding affinity pK are provided along with the dataset. As shown in Figure found: (1) The entropy of denoised atom type Vo (i.e. v_ent) has a reasonable correlation with pK, indicating unsupervised learning can provide certain degree of information for binding affinity ranking. (2) The entropy score provides some complementary information to traditional chemical/ physical-basee score function like Vina, since the combination of them (i.e. comb i ned) achieves better correlation. (3) When provided with labeled data, the final hidden embedding h" (i.e, hi hidder.emb) with a simple linear transformation could improve the correlation
large extent.
We further demonstrate that our unsupervised learned features could improve supervised affinity prediction on PDBBind v2020 dataset (Liu al. 2015). We perform more difficult time split as Stärk et al. (202)) in which the test set consists of structures deposited after 2019, and the training and validation set consist of earlier structures. We augment EGNN (Satorras et al., 2021b) with the unsupervised features h² provided by our model, and compare it with two ofate-of-the-arr sequence- based models TransCPI (Chen et al. 2020) and MONN (Li et al. 2020), one complex model IGN (Tang al. (2021), two structure-based model HOLOPROT (Somnath al. 2021) and STAMP- DPI (Wang et al. 2022) and finally the base EGNN model. As shown Table 4 the augmented EGNN clearly improves the vanilla EGNN and achieves the best results among baselines.
CONCLUSION
This paper proposes TargetDiff, a 3D equivariant diffusion model for target-aware molecule gener- ation and enhancing binding affinity prediction. In terms of future work, it would be interesting to incorporate bond generation as part of the diffusion process such that we can skip the bond inference algorithm. In addition to bond inference, another interesting future direction would be incorporating some of the techniques in fragment- based molecule generation Podda et al. (2020) and generating molecules with common and more synthesizable molecular sub-structures.

---

Published as a conference paper at ICLR 2023
Reproducibility Statements The model implementation, experimental data and model check- points can be found here:
com/ quan
Acknowledgement We thank all the reviewers for their feedbacks through out the review ey- cles of the manuscript. This work was supported by National Key R&D Program of China No. 2021 YFF12011000 U.S. National Science Foundation under grant no. 2019897 and U.S. Depart- ment of Energy award DE-SC0018420
REFERENCES
Amr Alhossary, Stephamus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):2214-2216, 2015.
Amy C Anderson. The process of structure-based drug design. Chemistry & biology, 10(9):787- 797. 2003.
Jacob Austin, Daniel Johnson, Jomathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems,
2021.
Dávid Bajusz, Anita Rácz, and Károly Héberger. Why is tanimoto index an appropriate choice for fingerprint- based similarity calculations? Journal of cheminformatics, 7(1):1-13, 2015.
Maria Batool, Bilal Ahmad, and Sangdun Choi. A structure-based drug discovery paradigm. Inter- national journal of molecular sciences, 20(1 20(11):2783, 2019.
Richard Bickerton, Gaia Paolini, Jérémy Besnard, Sorel Muresan, and Andrew Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
Esben Jannik Bjerrum and Richard Threlfall. Molecular generation with recurrent neural networks (mnss). arXiv preprint
Lifan Chen, Xiaoqin Tan, Dingyan Wang, Feisheng Zhong, Xiaohong Liu, Tianbiao Yang, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, and Mingyue Zheng. Transformercpi: improving compound- protein interaction prediction by sequence-basee deep learning with aff-attention mechanism and label reversal experiments. Bioinformatics, 36(16):4406 4414, 2020.
Jerome Eberhardt, Diogo Martos-Martins, Andreas Tillack, and Stefano Forli. 1.2. O: New docking methods, expanded force field, and python bindings. Journal of Chemical Information and Modeling, 68)8)338113388 2021.
Autodock vina
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug- like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1(1):1-11, 2009.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. Three-dimeesioonl convolutional neural networks and a cross-docked data set for structure-based drug design. Journal of Chemical Information and Modeling, 60(9):4200- 4215, 2020.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-transformers:: 3d roto- translation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970-1981, 2020.
Niklas Gebauer, Michael Gastegger, and Kristof Schütt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. Advances in Neural Information Processing Systems, 32, 2019.
Rafael KömezzBoombared Jennifer Wei, David Duvenaud, José Miguel Hernández-Lobate Benjamín Sánchez-Lengeling Dennis Sheberla, Jorge cauullrra-parraguing Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Gude Guzik. Automatic chemical design using a data-driven contin- representation of molecules. ACS central science, 4(2):268-22 276, 2018.
10

---

Published as a conference paper at ICLR 2023
Wei-Ying Jiaqi Guan, Wesley Wei Qian, Qiang Liu, Wei- Ying Ma, Jianzhu Ma, and Jian Peng. Energy- inspired molecular conformation optimization. In International Conference on Learning Representations, 2022.
Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and per-
formance of mmf194, Journal of computational chemistry, 17(5-6):490-519, 1996.
Paul CD Hawkins. Conformation generation: the state of the art. Journal of Chemical Information and Modeling, 57(9)117771175, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34, 2021.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation 3d. arXiv preprint arXiv:2203. 17000, 2022.
Dejun Jiang, Chang- Yu Hsieh, Zhenxing Wu, Kang, Jike Wang. Ercheng Wang. Ben Liao, Chao Shen, Lei Xu, Jian Wu, et al. Interactiongraphnee: A novel and efficient deep graph representation learning framework for accurate protein-ligand interaction predictions. Journal of medicinal chemistry, 64(24):18209-18222,, 2021.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. International Conference on Machine Learning, pages 2323- 2332. PMLR, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Composing molecules with multiple property constraints. arXiv preprint 2xXiv:2002.03244, 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera- Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with Al- phaFold. Nature, 596(7873);583-58) July 2021. doi: 10. 10.1088841588--21-0338999 URL htttps://dii org. / 10. 00000/44411660010081199999
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, arXiv preprint
1rXiv:1412.6900 2014.
Diederik P Kingma and Max Welling. 1312.66 14, 2013.
Auto-encoding variational bayes. arXiv preprint
Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: for symmetric densities. In International Conference on Machine Learning, pages 5361-5370. PMLR, 2020.
exact likelihood generative learning
Matt Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational autoen- coder. In International Conference on Machine Learning, pages 1945-1954. PMLR, 2017.
Shuya Li, Fangping Wan, Hantao Shu, Tao Jiang. Dan Zhao, and Jianyang Zeng. Monn: multi- objective neural network for predicting compound-protein interactions and affinities. Cell Sys- tems, 10(4):308-33 3222 2020.
Yibo Li, Jianfeng Pei, and Luhua Lai. Structure-based de novo drug design using 3d deep generative models. Chemical science, ((41):13664-13675 2021.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascamu, and Peter Battaglia. Learning deep generative 2018.
models of graphs. arXiv preprint

---

Published as a conference paper at ICLR 2023
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions Information theory, 37(1):145-151, 1991.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. International Conference on Muchine Learning, 2022.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph varia- tional autoencoders for molecule design. Advances in neural information processing systems, 31, 2018.
Zhihai Liu, Yan Li, Li Han, Jie Li, Jie Liu, Zhixiong Zhao, Wei Nie, Yuchen Liu, and Renxiao Wang. Pdb-wide collection of binding data: current status of the pdbbind database. Bioinformatics, (3):4054422, 2015.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162 8171. PMLR, 2021.
Noel M O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geof- frey R Hutchison. Open babel: An open chemical toolbox. Journal of cheminformaties, 3(1): 1-14, 2011.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. arXiv preprint urXiv:2205.0724 2022.
Marco Podda, Davide Bacciu, and Alessio Micheli. A deep generative model for fragment-based molecule generation. In International Conference on Artificial Intelligence and Statistics, pages 2240-2250. PMLR, 2020.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Learning a continuous representation of 3d molecular structures with deep generative models. arXiv preprint arXiv:2010.0865) 2020.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3D molecules conditional receptor binding sites with deep generative models. Chem Sci, 13:2701-2713, Feb 2022. doi:
10.1039/D1S005976..
Victor Garcia Satorras, Emiel Hoogeboom, Fabian B Fuchs, Ingmar Posner, and Max Welling. E (n) equivariant normalizing flows. arXiv preprint arXiv:2105.090) 2021a.
Viéctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International Conference on Machine Learning, pages 9323-9332. PMLR, 2021b.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2018
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382. 2020.
Gregor Simm, Robert Pinsler, and José Miguel Hernández-Lobato Reinforcement learning for molecular design guided by quantum mechanics. In International Conference on Machine Learn ing, pages 9959-8966, PMLR, 2020a.
Gregor NC Simm, Robert Pinsler, Gábor Csányi, and José Miguel Hernández-Lobato. Symmetry- aware actor-critic for 3d molecular design. In International Conference on Learning Representa- tions, 2020b.
Miha Skalic, José Jiménez, Davide Sabbadin, and Gianni De Fabritiis. Shape-based generative modeling for de novo drug design. Journal of chemical information and modeling. 59(3): 1205- 1214, 2019a.
12

---

Published as a conference paper at ICLR 2023
Miha Skalic, Davide Sabbadin, Boris Sattarov, Simone Sciabola, and Gianni De Fabritiis. From target to drug: generative modeling for the multimodal structure- based ligand design. Molecular pharmaceutics, 16(10):4282 42914 2019b.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learn- PMLR, 2015.
ing. pages
Vignesh Ram Somnath, Charlotte Bunne, and Andreas Krause. Multi-scale representation learning on proteins. Advances in Neural Information Processing Systems, 34:25244-25255. 2021.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Hannes Stärk, Octavian Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International Conference on Machine Learning, pages 20503-20521, PMLR, 2022.
Cheng Tan, Zhangyang Gao, and Stan Zii. Target-aware molecular graph generation. arXiv preprint 2:Xi::2202.04829, 2022.
Taffee Tanimoto. Elementary mathematical theory classification and prediction. 1958.
Nathamiel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivaaiann neural networks for 3d point clouds. arXiv preprint arXiv:1802.082/59 2018.
Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with a new scoring functtinn, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):455-461 2010.
Penglei Wang, Shuangjia Zheng, Yize Jiang, Chengtao Li, Junhong Liu, Chang Wen, Atanas Pa- tronov, Dahong Qian, Hongming Chen, and Yuedong Yang. Structure-aware multimodal deep learning for drug peot-piteiin interaction prediction. Journal of chemical information and modeling. 1308-1133 2022.
David Weininger. Smiles, chemical language and information system. 1. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31 11:31-33,,, 1988.
Mingyuan Xu, Ting Ran, and Hongming Chen. De novo molecule design through the molecular generative model conditioned by 3d information protein binding sites. Journal of Chemical Information and Modeling, 61(7):3240 32544444--,24, 2021.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: geo- metric diffuusion model for molecular conformation generation. arXiv preprint arXiv2203.009233, 2022.
Jiaxuan You, Bowen Liu, Rex Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. arXiv preprint arXiv: 1806.02473, 2018.
Zhenpeng Zhou, Steven Kearne,, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9():11100, 2019.
13

---

Published as a conference paper at ICLR 2023
A PROOF OF SE(3)-EQUIVARIANC OF GENERATIVE MARKOV TRANSITION
One crucial property the model needs to satisfy is that rotating or translating the protein-ligani complex P) will not change the estimated likelihood po (P)) Leveraging the conclusion from Köhler et al. (2000;; Xu et al. (2022). it requires the initial density of our generative process p( PM is SE(3)-inva -invariant and the Markov transition pe [M M,, P) is SE(3)-qqiivarian, Since during the generative process, we only need to consider the atom coordinates. More concretely, the model needs satisfy:
atom types are always invariant to
p(xT, xp) = (XT,((X)XP)
p(X.- Xr, xp) (x)
where Tg is the group of transorranfooonatiionn x; and XP denote the atom coordinates of ligand molecule and protein separately. (x) can also be written explicitly as (x(x) = Rx + b, where R. E 3x3 is the rotation matrix and b E TR3 the translation vector.
In Sec. 3.4
provide way implement -qquivariae of the generative Markov transition. In this section, we will prove the SEu3--equivariance of our design.
Recall the equation follows:
update the atom hidden embedding h and coordinates alternately as 7
h +++1 h
Σ fu h/, hn', eiy; 0n)
JEViiSj
X
Σ
(xf
-x) b ( mol
jEV.
First, it is easy to see di does not change with the 3D roto- tato-translation Tg:
dy [TT - Tx(x,)11 = 11| [| + b) - (Rx, + b)]l = Rx: - Rx211
(x; x;) R R(x; xx,) = (x: x xi) ||x: 1x1--1112- db
Since h; hy, Gij are initially obtained from atom and edge features, which are invariant to SE(3)- transformation, we have hy is SE(3)-invariant for any == -,....,1. L.
Then, prove that updated from equation 7 is SE(3)--quivariant follows:
(T(x)) T(x)
Σ
(T(x)) T(x (x) h .n/ (egi02)) 1
jEV,
Rx + +
Σ
R(x x))fa( ofth hit! .ey: 0z) mmo
jEV,ttj
(11)
R x
Σ
R(x - xf) b/4t h1tt eiji
Oz))00) 0z) 1 +
jEViiti
Rx,+1 + b
=(oo(xx))
Under our parameterization, the neural network predicts [Ko, vo). By stacking L such equivari- ant layers together, we can draw the conclusion that the output of neural network Xo SE(3)- equivariant w.r.f the input X,. Finally, we can obtain the mean of posterior x,-1 from equation 4 x:-1 var-i& 1-a:: &o + (I
I-ãe x,. The last thing the model needs to satisfy that x,- is
SE(3)-equivariant w.r.t x,. However, we can see the translation vector will be changed under this formula:
1- āi 1 T(x))
Var(1 ai- T(x:)
(T(()x)))
āi R(x;)
(12)
18 āi
var(l ā1-1)
R(Xo) +
1 ā
14

---

Published as a conference paper at ICLR 2023
where b =
( l-a; +
y/ul yauil1-a--1 1 l-as
As the Sec. discusssed, we can move CoM of the protein atoms to zero invariance in the whole generative process, which is same to how we achieve the invariant initial density. Thus, only need consider rotation equivariance in the Markov transition, which is straightforward to see that it can be achieved from equation 12 when b is ignored: 140 [R(x),, t) = R( (x,,
achieve translation
B ANALYSIS OF INVARIANT INITIAL DENSITY
We assume when the timestep T of the diffusion process is sufficiently large, ((xT(xp) would be a Gaussian distribution whose mean is the center protein and standard deviation is one, i.e. (xx|Xx) ~ (Cp @ IN,, II.Nn). ). where Cp Np Exp, denotes the kronecker product, Ig denotes the l k iddettity matrix and 1 denotes the k-dimensional vector filled with one.
To achieve the -Evaa-in initial density, we move the center of protein to zero, i.c. Cp = O. One can also define the initial density other CoM-free systems such as the ligand or complex CoM-free system. We choose protein CoM-free system here since only one step of shifting center operation is needed at the beginning of generative or diffusion process (protein context is fixed input). Formally, it can be considered as a linear transformation: &p = Qxp, where Q = 13 (IN- N I I) It has several benefits in simplifying the formula: In the diffusion process, ( (P)) would be a standard Gaussian when T is sufficiently large; Accordingly. in the generative process, we can sample Xy from p(XT(Xp). which also a standard Gaussian distribution.
For evaluating a complex position (xT,Xp), we can firstly translate the complex to achieve zero CoM on protein positions, which can also considered as linear transformation:
IM
N 1 IN
(Kr, xp) = Q(xT xp),, where Q = Is 2
(13)
N 1N1
Then, we can evaluate the density p(xT Xp) with the standard normal distribution. We denote ý as the density function on this protein zero CoM subspace: ((xT(xp)= = p(Q((x( xp))
It can be seen that for any rigid transformation (x(x) Rx + b, we have Q (T(Xx, Q R(XT, xp). Since Q is a symmetric projection operator and rotation matrix R is a orthogonal matrix, we have ||Q Rx| [2 = ||x||_ Given p is an isotropic normal distribution, we can casily have P(T, (XT, xP)) = p(xr, Xp). which means an SE(3)-invariant density.
C PROOF OF INVARIANT LIKELIHOOD
In Sec. 3.4 we argue that an invariant initial density composed with an equivariant transition func- tion will result in an invariant distribution. In this section, we will provide the proof of it.
The two conditions to guarantee an invariant likelihood
are as follows:
p(xr, xp) = (x),
Invariant Prior)
p(x--1Xx,, xp) (x. (T,
Equivariant Transition)
We can obtain the conclusion follows:
T pe
Po (T xx)) = / P(T,(XX, Xp)
T Σ
p(xT, xp)
po (x
Apply !
-/ ppxxr, Xp)
Σ po (x, (x|-1/Xx,,)) xp)
AAplly 2
PX xp)
15

---

Published as a conference paper at ICLR 2023
D DERIVATION OF ATOM TYPES DIFFUSION PROCESS
According to Bayes theorem, we have:
((Ve-ilVe, vo) v(vr)
Σ v11/)/v..1.N)(((V-1110 Vi-1
(14)
((vv)vv-1)9(Vv-IMM)
Σ v(v_/vv-1)
According to Eq. and
((V. Ve- and (V can calculated as:
((v, ve-1) C(v. + (1 dx,/K) g(v=-1lvo) ivo) = - + -0--)/1)))))))
(15)
Note that when computing C(v:la.ve-1 // the value is az + (1 /,) V and (1 o4)/A K otherwise, which leads to symmetry of this function Hoogehoom et al.
2021 i.e.,
C(velavVi-1 + k) C(vi-1lavv, + a))
Let (V,,,, vo) denotes the numerator of Eq.
14
Then can be computed as:
c" (v2.. vo) (ovvlVv=1)9
(16)
[avV; (1 / K| @ lāv- + +11 (1--111//11
and therefore the posterior of atom types derived as:
qVi:-llv, vo) = C(v.-1ē((v,,vo))
(17)
where VvV,,, vo) /
E c
E OVERALL TRAINING AND SAMPLING PROCEDURES
In this section, summarize the overall training and sampling procedures of TargetDiff as Algo- rithm and Algorithm respectively.
Algorithm 1 Training Procedure of TargetDiff'
Input: Protein-ligand binding dataset (P,M) neural network dy
1: while de not converge do
2: Sample diffusion time E (d.....,
3: Move the complex make CoM of protein atoms
4: Perturb Xo to obtain Xx: X4 = / Vo,x0 + (1 - oz)c, where E N00 T)
5:
Perturb Vo obtain Vz:
log c = log (āeVo - ((1-a))/))
Vi one hot (arg max; [g: + log c;l), where g Gumbel 1)
6:
Predict [&p, vul from (x,, v. with dæ: Xu,, vol de (xx(,,
7:
Compute the posterior atom types c(v, vo) and c(V,, vn) according to equation 4
8:
Compute the unweighted MSE loss on atom coordinates and the KI. loss on posterior atom types: T, Xx - xoll² KI(Cev,, vo) c(v,, Vo))
9:
Update 0 by minimizing L
10:
end while
F EXPERIMENT DETAILS
F.1 FEATURIZATION
At the 1-th layer, we dynamically construct the protein-ligand complex as a k-nearest neighbors (knn) graph based known protein atom coordinates and current ligand atom coordinates, which
16

---

Published as a conference paper at ICLR 2023
Algorithm 2 Sampling Procedure of TargetDiff
Input: The protein binding site P, the learned model dg.
Output:
Generated ligand molecule M that binds to the protein pocket.
1: Sample the number of atoms in M based prior distribution conditioned the pocket size
2: Move CoM of protein atoms to zero
3: Sample initial molecular atom coordinates Xy and atom types Vy:
Xr (N,
VI = one_htt (arg max; 9t), where g Gumbel(O,1 1)
4: for in - 1,....... do
5: Predict [8o; Vol from Xx, vzl with de: [Xo, Vol ([x;; Vel, t, P)
6:
Sample X from the posterior pe (x X4; Xo) according to equation
7: Sample v from the posterior Po(V Vt. Vo) according to equation
8:
end for
is the output of the / - 1-th layer. We choose k = 32 in our experiments. The protein atom features include chemical elements, amino acid types and whether the atoms backbone atoms. The ligand atom types are one-hot vectors consisting of the chemical element types and aromatic information. The edge features are the outer products of distance embedding and bond types, where we expand the distance with radial basis functions located at 20 centers between 0 Á and 10 Å and the bond type is a 4-dim one-hot vector indicating the connection is between protein atoms, ligand atoms, lottin-ligand atoms ligand- ligand-protein atoms.
F.2 MODEL PARAMETERIZATION
Our model consists of equivariant layers as equation shows, and each layer is Transformer with hidden.dimn dit.mdiim22212 12 and n. 6. The key/value embedding and attention scores are generated through 2-layer MLP with LayerNorm and ReLU activation. We choose to sigmoid 3 schedule with 31 1e-7 and Br 2e-3 for atom coordinates, and cosine / schedule suggested in Nichol and Dhariwal (021) with 0.01 for atom types. We set the number of diffusion steps as 1000.
F.3 TRAINING DETAILS
The model is trained via gradient descent method Adam Kingma and Ba (2014) with init.learning.rated/ 001, betas= (0. 95, 0. 999). batch.sia sze and clwp.gradient.noree norm=8.. We multiply factor a = 100 on the atom type loss to bal- the scales of two losses. During the training phase, standard deviation of 0.1 to protein atom coordinates as data augmentation. We also schedule to decay the learning rate exponentially with factor of 0.6 and minimum learning rate of le-6. The learning rate is decayed if there is no improvement for the validation loss in 10 consecutive evaluations. The evaluation performed for every 2000 training steps.
add small Gaussian noise with
We trained our model on one NVIDIA GeForce GTX 3090 GPU, and it could converge within 24 hours and 200k steps.
F.4 DETERMINE THE NUMBER OF LIGAND ATOMS
Pocket Size Estimation We compute the top 10 farthest pairwise distances of protein atoms, and select the median of it as the pocket size for robustness.
Prior Distribution of Number Ligand Atoms We compute 10 quantiles of training pocket sizes and estimate the prior distribution of number of ligand atoms for each bin. Specifically, take the histogram number of atoms in the training set as the prior distribution. The relationship between estimated prior distribution and actual training testing number of atom distribution is shown in Figure[ST] We can see that the number of ligand atoms has a clear positive correlation with pocket sizes and the prior distribution estimated from the training set also be generalized to the test set.
17

---

Published as a conference paper at ICLR 2023
15.5025.54| 12] 10000
15.0-26.5A
26.5-27.94 30003
126.5
0.8
0U8
120
0:44
6:2
0.0
train 27.9 28.88 8A 19 19900
3058 27.9 28.84 19.1
3ram 28.8-25.74
25.74 21 10005
10.2.
225.0 29.
7A
(2 1.
poleson
polsson normel ait.u
poisten normel a/0uol
polsson
normall
actual
0150
actual
0:44
0.08
6.00
train 29, 2-30.44 19995
[ee 29.2-3044 23 16
[304-33.24]
10000
32.4-5..2
polsson
polsson
0.0
polsson
polsson
normal actual
normel
normel
normall
0.08
6.4
train 33..
30000
test 32.04 27)
trann 33.0--3.04
10000
test:
0A 28
pa
polsson normal actual
3.5
normal adiua
0.0
normal actual
actual
6.88
0.2
0.08
Irs 63.034..6 30 10000
1est 33.0-34 33.0-36.61 | 30 11
tman 34.6-inth 33 10000
test (3.6-inAA 33] 166
passson
2013
poisson normal act.un
possor normal actus
poisson
normal actual
normal
actual
3350
0.08
0.03
6:00
0.3
Figure $1: Prior distributions of the number of ligand atoms. For each prior distribution within specific range of pocket size, train/test pocket size range median number of ligand atoms number of data points within current bin are shown as titles.
During the training phase, we provide the model with the number of atoms of reference molecules since we use them to perform training. During the generation phase, we do not have to require the generated molecules has the same number of atoms as the reference molecule, and the numbers are randomly sampled from these prior distributions computed based on training data.
G ADDITIONAL EVALUATION RESULTS
In the main text, we provided the evaluation results in Table. 3 where the Vina score computed with AutoDock Vina (Eberhardt et al.) 2021). Here, provided the evaluation results in Table [S] where Vina scores are computed with Q (Alhossary et al. 2015),, a faster but less accurate dock- ing tool (following what AR and Pocket2Mol used). We can observe a similar trend that molecules generated by TafgetDiff could achieve SOTA binding affinity.
Upon further investigation, we also discover strong negative correlation between SA score and molecular size as shown in Figure S2 (Pearson R=-0.56, p < 10 10-80), and the difference in SA score between these model generated molecules could be the artifact of their size differences.
18

---

Published as a conference paper at ICLR 2023
AR
Metric
High Affinity (t)
QED (f)
SA (1)
Diversity (f)
08
Model
08
Med.
Med.
Med.
Avg.
Med.
lIGAN
21.1%
11.1%
0.39
0.39
0.39
0.57
0.66
0.67
33.7%
24.2%
0.51
0.50
0.63
0.63
0.70
0.70
05
GraphBP
6.7%
0.48
0.79
0.78
03
Mocket2Mal
49.8%
52.0%
0.56
0.57
0.74
0.75
0.69
0.71
20
30
50
TargetDifl 51.3%
50.0%
0.48
0.48
0.58
0.58
0.72
0.71
Reference
0.74
Figure S2: scatter plot compares the SA and the number of atoms for given molecule.
Table S1: Summary of different properties of reference molecules molecules generated by our model and other baselines.
H SAMPLING TIME ANALYSIS
One major advantage of TirgetDiff over auto-regressive based model such AR is that TirgetDiff' scales better against the size of the molecules. While AR required to run additional steps to generate larger molecules, the diffusion model can operate additional atoms in parallel fashion without sacrificing a lot of inference time.
To better demonstrate such effect, randomly select 5 binding pockets targets and record the time spent in generating 100 molecules for each pocket using both autoregessive models (includ- ing AR, Pocket2Mol and GraphBP) and TargetDiff. Since these models have different numbers of parameters and sampling schemes, we first compare the inference time ratio against generating a 10- atom molecule for these models, instead of simply comparing their wall time. As shown in Figure S3. as we start to generate larger and larger molecules, the wall time for AR grows almost linearly along with the molecule size, while the wall time for TargetDiff stays relatively flat.
In terms of wall clock time, AR, Pocket2Mol and GraphBP use 7785s, 2544s and 105s for generating 100 valid molecules on average separately, and takes 3428s on average for TargetDiff. GraphBP has the fastest sampling time but the quality of generated molecules is lower than other models (See Table 3). TifgetDiff has the moderate sampling efficiency compared AR and Pocket2Mol.
14
AR
Median Size for Reference Molecules
Pocket2Mol
GraphBP
Time Ratio
10.
TargetDiff
10
20
30
40
Molecule Size
Figure S3: Inference time growth as a function of molecule size for AR, Pocket2Mol, GraphBP and Tiffet
I MORE EXAMPLES OF GENERATED RESULTS
Please see next page.
19

---

Published as a conference paper at ICLR 2023
1H36_A (Target in Figure 4)
Reference
AR
TargetDiff
vina = -12.8
vina -10.6
vina = -13.3
1DX0 A (Target 49 in Figure 4)
Reference
AR
TargetDiff
OH
vina -5.2
vina -3.9
vina -7.0
2GNS A (Target 99 in Figure 4)
Reference
AR
TargetDiff
OH
OH
vina -4.8
vina = -3.9
vina -3.3
Figure S4: More examples of binding poses for generated molecules. present of fair overview of model performance, we specifically select the best (1H36 A), median ( (IDXO A), and worst (2GNS A) targets shown in Figure 4 for visualization along with the generated molecules and calculated Vina energy (kcal/mol). 20

---

Published as a conference paper at ICLR 2023
J EXAMPLES WHERE AR OUTPERFORMS TARGETDIFF
Please see followings for two binding targets where AR outperforms TargetDiff in terms of Vina estimated binding affinity.
vina = -10.5
vina = -9.5
vina -8.9
AR
vina -8.2
vina = -7.9
vina = -633
Ours
Figure Example binding pocket (4KCQ A. target Figure 4) where outperforms Targe terms of Vina estimated binding affinity. Three examples of AR generated molecules are shown in the top half, and three examples from TafgetDiff are shown in the bottom half.
21

---

Published as a conference paper at ICLR 2023
vina -9.0
vina = -9.8
vina -911
AR
HD
vina = -633
vina = -6.8
vina = -7.7
Ours
Figure S6: Example binding pocket I (IRIHLA, target 61 in Figure 4) where AR outperforms TargetDiff in terms of Vina estimated binding affinity. Three examples of AR generated molecules are shown the top half, three examples from Tifget are shown the bottom half.
22


Reviews:
[{"review_id": "li13P8ck7G", "reviewer": "Reviewer_uzYP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles target-aware small molecule generation. Specifically, small synthetic ligand molecules are generated using a deep generative model, such that the molecules best fit into the binding site of a larger protein. To this end, the paper jointly models the ligand and the protein using a generative diffusion model, conditioning the generation of the ligand on the protein. Technically, in contrast to some previous papers, the work explicitly models 3D atom coordinates and also uses appropriate equivariant neural networks. Diffusion processes over both atom coordinates and also atom types and features are used. The paper also shows the entropy of the generated atom feature distribution can serve as an approximate measure for binding affinity. Empirically, the paper compares mostly to two simple baselines, a 3D voxel-based molecule generative model and an autoregressive model, and outperforms them on various molecule generation tasks.", "summary_of_the_review": "In summary, this is an okay paper with thorough experiments. Nevertheless, I do not think it meets the bar at this point, mainly due to two concerns: (a) the methodological novelty is not very significant, as discussed above. (b) This would be okay, if the experimental results would be very strong. However, it seems the paper is somewhat overclaiming in that regard, as it mostly compares to only two baselines that do not seem to represent the strongest existing methods. In particular, the method is motivated as an approach that can generally outperform autoregressive methods. Yet, only a simple autoregressive model (\"AR\") seems to be considered in most experiments, but not methods such as Pocket2Mol. In conclusion, I do not think the paper is ready yet for publication. I would recommend the authors to more thoroughly compare to baselines. Furthermore, it could be interesting to investigate even further how the learnt features can be used for property prediction, beyond just using the entropy as a measure for binding affinity.", "strength_and_weaknesses": "**Strengths:**\n\n- The particular combination of using a mixed continuous-discrete diffusion model, together with an equivariant architecture, to model atom coordinates and features in a protein ligand complex seems to be new (although it's a straight-forward combination, see below).\n\n- The work's quantitative results look okay and outperform the two chosen baselines liGAN and AR, although I am not sure the baseline comparisons are sufficient.\n\n- It is interesting that the entropy of the distribution of generated atom features correlates with binding affinity.\n\n**Weaknesses:**\n\n- Methodologically, the method is a straight-forward combination of existing techniques. The Gaussian and categorical diffusion processes used are well established. The equivariant neural network architectures seem to be standard. And jointly modeling the ligand-protein complex with a generative model is also not new. Merely this exact combination of techniques seems to be new.\n\n- The paper makes claims that the equivariant diffusion model approach is better than autoregressive techniques in its introduction and motivation. Yet, the method is outperformed by Pocket2Mol -- an autoregressive method -- with regards to the molecular properties (Table 3, approximately similar performance on affinity and diversity, significantly worse on SA and QED). In the other experiments, the Pocket2Mol baseline is not even considered. However, I believe it would be very appropriate to compare to this work more rigorously, as it also uses modern equivariant architectures, considers similar molecule generation tasks with the same data, and also has code available.\n\n- The model only predicts 3D atom coordinates and atom types and features, but no bonds.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gkYg1R5f18", "reviewer": "Reviewer_GWjC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, TargetDiff, a new 3D diffusion model, is introduced. The model generates molecules in a non-autoregressive fashion. The generated molecules are conditioned on the binding pocket, and the generation is equivariant to rotations and translations thanks to the equivariant GNNs used. Additionally, an unsupervised entropy-based method for ranking molecules is proposed. The model is evaluated against multiple recent generative models: liGAN, AR, GraphBP, and Pocket2Mol. The results indicate the strong performance of the proposed method.", "summary_of_the_review": "Based on the above comments, I am leaning towards the acceptance of this paper.", "strength_and_weaknesses": "Strengths:\n- Non-autoregressive sampling helps to better fill the available space in the binding pocket.\n- The related work contains a set of very recent publications demonstrating the advancements in the fields of target-based molecule generation and diffusion models.\n- SE(3)-equivariant networks ensure the strong performance independent of the protein orientation.\n- Both continuous and discrete diffusion models are used to model coordinates and atom types, respectively.\n- The main claims of the paper are formally proven.\n- The authors discover a correlation between binding affinity and the entropy of the predicted atom representation when atom coordinates are frozen. Thus, this entropy can be treated as an affinity predictor trained in an unsupervised fashion.\n\nWeaknesses:\n- Currently, the code is not available, but it should be open-sourced upon publication.\n- In Tables 3 and 4, confidence intervals could be added to make these results more convincing.\n\nQuestions:\n- How do you combine v entropy and vina scores in Section 4.3?\n- Coordinates and atom types are modeled independently in Equation 2. Do you think this could have a negative impact on the generated structures?\n\nMinor points:\n- Typos: “more structural data become available and unlock new opportunities”, “protien”->”protein”\n- The full name “Center of Mass (CoM)” should be introduced at its first occurrence.\n- Figure 7 reference is missing.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cr5dHYdw8lK", "reviewer": "Reviewer_3Z2H", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles the problem of generating molecules for a specific protein target. With diffusion models and SE(3)-equivariant networks, this paper learns a joint generative process of both continuous atom coordinates and categorical atom types. Specifically, this paper defines a diffusion process for the continuous atom coordinates and discrete atom types, gradually adding noise, and learns the joint generative process for atom types and coordinates of molecules. \nBesides, this paper evaluates the quality of generated molecules based on the features outputted by the model, which can be used as a scoring function for ranking or binding affinity prediction. \n", "summary_of_the_review": "Given the incremental novelty and the marginal improvement in empirical experiments, I recommend the rejection.", "strength_and_weaknesses": "This paper tackles the problem of structure-based drug design using diffusion models and SE(3)-equivariant networks. However, there are some concerns about the method and the experiments:\n- This method seems to be a combination of diffusion models, equivariant networks, and structure-based molecular generation tasks. Compared with [1], which combines diffusion models and equivariant networks, the innovation of this work is not obvious. It seems the only difference is the atom information in the protein target is used. The authors can highlight more about its innovation compared with previous works.\n- The number of atoms is sampled by drawing a prior distribution estimated from training complexes with similar binding pocket sizes. As determining the number of atoms in advance is crucial to the generation of the diffusion model, I wonder if the authors can show the difference between the sampled number of atoms and the number of atoms of reference molecules. What will happen if the error is too large to cover the reference molecules?\n- The details of defining \"similar binding pocket sizes\" is not provided in the paper. \n- The detail of the data split is not clear. To avoid leakage, it is important to exclude similar pockets in training data.\n- It seems the test set is from the CrossDocked2020, not the experimental crystal protein-ligand complex. \n- Vina will first \"re-docking\" the molecules and then calculate the docking score. Since re-docking can largely change the molecular conformation and binding pose, I wonder if the authors have considered this problem when evaluating the binding affinity?\n    - I would suggest providing the Vina score without re-docking, to demonstrate the end-to-end 3D generation performance, rather than post-fixed by docking tools.\n- In many results, only AR is used as baselines. And From table 3, there are several baselines better than AR. I think these baselines should be included in most tables/figures. \n- In Table 3, TargetDiff seems only marginally better (and worse in some metrics) than other baselines, especially compared with Pocket2Mol.\n- The quality of generated molecules. From Figure 5, S2, and S3, It seems TargetDiff tends to generate the molecule with large flexibility, which is unstable in the real world.\n- Did you use any tools, like rdkit/openbabel, to post-fix and filter the generated molecules?\n- This paper compares the efficiency of TargetDiff and AR in the appendix. Since GraphBP and Pocket2Mol in Table 3 are also auto-regressive generative models especially Pocket2Mol adopts a more efficient sampling method than MCMC sampling in AR, I wonder how efficient TargetDiff is when compared with Pocket2Mol and GraphBP.\n- Can you show the wall clock time, rather than the ratio, in the efficiency comparison?\n- The model architecture is not clear. In Sec. 3.4, it says \"we model the interaction between the ligand molecule atoms and the protein atoms with a SE(3)-Equivariant GNN\". But in Sec. 4.1, it says \"where fh and fx are specifically implemented as transformers with 16 attention heads and 128 hidden features.\" I am confused about the statements, is the model a GNN or a transformer? \n\n\n[1] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pages 8867–8887. PMLR, 2022. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster