Protein-Ligand Interaction Prior for Binding-aware 3D Molecule Diffusion Models
Authors: Zhilin Huang
Ling Yang
Xiangxin Zhou
Zhilong Zhang
Wentao Zhang
Xiawu Zheng
Jie Chen
Yu Wang
Bin CUI
Wenming Yang
Venue: ICLR 2024 poster (2024)
Keywords: Diffusion Models
Structure-Based Drug Design
Molecule Generation
Primary area: generative models

Abstract: Generating 3D ligand molecules that bind to specific protein targets via diffusion models has shown great promise for structure-based drug design. The key idea is to disrupt molecules into noise through a fixed forward process and learn its reverse process to generate molecules from noise in a denoising way. However, existing diffusion models primarily focus on incorporating protein-ligand interaction information solely in the reverse process, and neglect the interactions in the forward process. The inconsistency between forward and reverse processes may impair the binding affinity of generated molecules towards target protein. In this paper, we propose a novel Interaction Prior-guided Diffusion model (IPDiff) for the protein-specific 3D molecular generation by introducing geometric protein-ligand interactions into both diffusion and sampling process. Specifically, we begin by pretraining a protein-ligand interaction prior network (IPNet) by utilizing the binding affinity signals as supervision. Subsequently, we leverage the pretrained prior network to (1) integrate interactions between the target protein and the molecular ligand into the forward process for adapting the molecule diffusion trajectories (prior-shifting), and (2) enhance the binding-aware molecule sampling process (prior-conditioning). Empirical studies on CrossDocked2020 dataset show IPDiff can generate molecules with more realistic 3D structures and state-of-the-art binding affinities towards the protein targets, with up to -6.42 Avg. Vina Score, while maintaining proper molecular properties. https://github.com/YangLing0818/IPDiff

Published as a conference paper at ICLR 2024
PROTEIN-LIGAND INTERACTION PRIOR FOR BINDING- AWARE 3D MOLECULE DIFFUSION MODELS
Zhilin Huang 1.2 Ling Yang Xiangxin Zhou Zhilong Zhang Wentao Zhang
Xiawu Zheng?. Jie Chen 2.3 Yu Wang²1 Cui Bin Wenming Yang! 1.21
1 Shenzhen International Graduate School, Tsinghua University 2 Peng Cheng Laboratory 3 Peking University University of Chinese Academy of Sciences Xiamen University tzerinhwang03 zz1201 18mth, wentao. zhang, bin. cui @d Lyangling08 8, yange 1 wm) @163. com, zhouxiangxinl 9 98 @emmgmmall com zhengxi awu xmu edu. en, {chenj, wangy20| @pcc.p....o appclaa.... ac. en
edu. cn,
ABSTRACT
Generating 3D ligand molecules that bind to specific protein targets via diffusion models has shown great promise for structure-based drug design. The key idea disrupt molecules into noise through fixed forward process and learn its reverse process to generate molecules from noise in a denoising way. However, existing diffusion models primarily focus on incorporating protein-ligand interaction infor- mation solely in the reverse process, and neglect the interactions in the forward process. The inconsistency between forward and reverse processes may impair the binding affinity of generated molecules towards target protein. In this paper, we propose a novel Interaction Prior-guided Diffusion model (IPDIFF) for the protein-specific 3D molecular generation by introducing geometric protein-ligand interactions into both diffusion and sampling process. Specifically, we begin by pretraining a protein-ligand interaction prior network (IPNET) by utilizing the binding affinity signals as supervision. Subsequently. we leverage the pretrained prior network to (1) integrate interactions between the target protein and the molec- ular ligand into the forward process for adapting the molecule diffusion trajectories (prior-shifting). and (2) enhance the binding- -awre molecule sampling process (prior-conditioning) Empirical studies on CrossDocked20200 dataset show IPDIFF can generate molecules with more realistic 3D structures and state-of-the-art bind- ing affinities towards the protein targets, with up to -6.42 Avg. Vina Score, while maintaining proper molecular properties. http://ituub..omm/angggiiii 8/IPDiff
INTRODUCTION
Structure-baset drug design (SBDD) (Anderson, 2003) plays an important role drug discovery. Given target protein and its 3D structure. need to design ligand molecules in silico with desired properties, such as high binding affinity to the target. This problem can be formulated as a conditional generation task. Powerful deep generative models have already achieved promising results in SBDD tasks. For example, Luo et al. (2021); Liu et al. (2022); Peng et al. (2022) propose to generate atoms (and bonds) in autoregressive way form ligand molecule, and Zhang al. (2023) propose to generate 3D molecules fragment by fragment. Particularly, recent diffusion models (Sohl-Dickstein al., 2015; Ho al., 2020) exhibit remarkable abilities synthesize realistic ligand molecules with high binding affinity to target proteins (Guan et al., 2023a; Lin et al., 2022; Schneuing et al., 2022; Guan et al., 2023b). They usually perturb atom types and positions of ligand molecules (in the forward process), and train an SE(3)-equivariant neural network to denoise the atom positions (resp. types) considering the interactions with target protein (in the reverse process) for fitting the reversal of the forward process.
We find that there is a discrepancy between the forward process and the reverse process regarding the utilization of interactions between the target protein and the generated molecular ligands, which
Equal Contribution
Corresponding Author

---

Published as a conference paper at ICLR 2024
may limit the performance of the diffusion models for SBDD tasks. In the forward process, the ways of injecting noises are the same for all training samples with different target proteins. In other words, the differences of pocket binding sites between different training samples neglected and all the molecules are perturbed in the same way during the forward process. However, the reverse process, generate ligand molecules that bind specific receptors, the differences in pocket binding sites are considered. Such discrepancy introduces a bias that hinders the diffusion models from fully capturing the interaction between pockets and ligand molecules, while such intermolecular interaction is the essence of ligaed binding.
Mu
Po
Forward Process
q(M Mo, P
Interaction
Mo
Mt
M1+1
MT
(PM, ,F")
Poi (M, Me Po F Focc ole) Reverse Process
IPNet
IPDiff
Figure 1: An illustration of forward (Equation (8)) and reverse diffusion steps (Equation (11)) incorporated with the interaction prior in IPDIFF, where M. indicates the molecular ligand at time step Po indicates given protein pocket and F indicates the modeled interaction prior modeled by IPNET as described in Sec. 4.1. The difference between previous diffusion- -based SBDD methods and IPDIFF highlighted in red.
To eliminate this discrepancy, we propose novel Interaction Prior-guided Diffusion model (IPDIFF) to adaptively adjust the diffusion trajectories based on the specification of pocket-ligand interaction prior as illustrated Figure Specifically, the pocket-ligand interaction prior is captured with a pretrained neural network, named IPNET, which is supervised by binding affinity signals. Then we design an effective learnable adapter to explicitly incorporate such bocket-ligand interactions into all the timesteps in both forward and reverse processes for binding-aware trajectory adaptation (prior- shifting). Thus the two processes are jointly optimized during training, and IPDIFF theoretically able to achieve better likelihood compared to previous molecular diffusion models. Besides, propose to enhance the denoising by explicit conditioning on the estimated protein-ligand complex the previous step in the reverse process pprioo-conditionnng)), which also powered by our learned interaction prior network. To demonstrate the efficacy of our IPDIFF, we conduct extensive experiments on 200 dataset. Empirical results show that our IPDIFF can generate ligands that not only bind tightly to target pockets but also maintain proper molecular properties, outperforming existing diffusion-hasee hased molecular diffusion models.
We highlight our main contributions as follows:
We propose a novel 3D molecular diffusion model (IPDIFF) for SBDD where the pocket- ligand interaction is explicitly considered in both forward and reverse processes.
We propose prior-shifting that shifts the diffusion trajectories of the forward process based the interactions between pocket-binding sites and corresponding ligand molecules.
We design prior-conditioning to enhance the reverse process by conditioning the denoising of ligand molecules on the previously estimated potein-ligandd interactions.
IPDIFF achieves SOTA performance CrossDocked2201 benchmark, and it generate the molecules with -6.42 Avg. Vina Score while maintaining proper molecular properties.
RELATED WORK
Structure-based Drug Design Structure-baser drug design (Anderson, 2003) aims to generate ligand molecules that bind to given target protein and plays critical role in the process of drug discovery. Skalic et al. (2019); Xu et al. (2021) proposed to generate ligand molecules in the format of SMILES conditioned on protein contexts. Tan et al. (2022) designed a flow model that can generate validated molecular drugs the format of 2D graph conditioned on sequence embedding of specific targets. Ragoza et al. (2022b) voxelized molecules in atomic density grids and utilized VAE to generate 3D ligand molecules on receptor binding sites. Luo et al. (2021): Liu et al. (2022); Peng al. (2022) proposed to generate atoms (and bonds) in 3D Fuclidean space in autoregressive
2

---

Published as a conference paper at ICLR 2024
way. Zhang al. (2023) proposed to generate 3D ligand molecules fragment by fragment. Recently, diffusion models have been applied to SBDD and achieved promising performance, which will be introduced in more detail in the following paragraph. Our work focuses improving 3D molecular diffusion models for SBDD.
Diffusion Models for SBDD Guan et al. (2023a); Lin et al. (2022); Schneuing et al. (2022) employ diffusion models to first generate atom types and positions, and then define the bonds as post-processing. They propose to utilize SEui)-eqiivarian neural network to denoise the ligand molecules in the context of the protein-ligand complex, where the protein pocket is fixed. Schneuing et al. (2022) additionally tries to generate compounds (i.c. the protein-ligand complex) by inpainting conditioned the pocket binding site. Guan et al. (2023b) further introduces decomposed priors into diffusion models for SBDD inspired by traditional drug discovery and achieves high binding affinity average. Binding affinity, critical evaluation metric in the process of drug discovery, measures whether drugs bind their target proteins selectively and specifically. Though DecompDiff (Guan al., 2023b) has achieved exciting results, it heavily relies on external computation tools, which employs AlphaSpace2 (Rooklin et al., 2015) to extract subpockets and produce pocket priors when generating ligand molecules for new pockets. Differently from all the above, our molecular diffusion model for the first time considers pockets, ligand molecules, and their interactions in both forward and reverse processes. Moreover, our IPDIFF has no dependency on any external tools and achieves higher binding affinities than all other methods.
3 PRELIMINARY
The SBDD task from the perspective of generative models can be defined as generating ligand molecules which can bind to a given protein binding site. The target (protein) and ligand molecule can be represented as (resp. Nar) refers the number of atoms of the protein P (resp. the ligand molecule M). R²3 and v E RK denote the position and type of the atom respectively. In the sequel, matrices are denoted by uppercase boldface. For a matrix X, x; denotes the vector its i-th row, and X1:v denotes the submatrix comprising its 1-st to N-th rows. For brevity, the ligand molecule is denoted as M X.M. V.M] where X RNx and V.M R NN x and the protein is denoted as P= V where X E Nrx3 and VP E N The task can be formulated as modeling the conditional distribution p(M[P). Recently, diffusion models (Ho et al., 2020; Rombach al., 2022: Song et al.. 2020) have achieved promising performance in SBDD tasks (Guan et al., 2023a; Schneuing et al., 2022: Lin al., 2022). The types and positions of the ligand molecular atoms are modeled by DDPMs (Ho et al., 2020), while the number of atoms NM is usually sampled from an empirical distribution (Hoogeboom et al., 2022; Guan et al., 2023a) or predicted by a neural network (Lin et al., 2022), and the chemical bonds are generated by the post-processing programs. And we define 3. (L = ,,..., ?) as fixed variance schedules, Oa = 1 - 3r, Or II Oss B, = 1 ä More detailed molecular diffusion processes can be found in Appendix A.
{(x.. of ));r and M = {(x:M my )) respectively. Here Np
4 METHODS
As discussed in previous sections, we aim to incorporate protein-ligand interaction prior into 3D molecular diffusion model for generating ligand molecules binding tightly to the given pockets. We therefore propose IPDIFF, novel diffusion- We first design a prior network IPNET to capture the interactions between pockets and ligands from the perspective of both 3d structures and chemical properties, and pretrain it by binding affinity signals (Sec. 1).. Then, we take the pretrained IPNET interaction prior facilitate the binding-aware ligand diffusion process. Two mechanisms, prior-conditioning and prior-shifting (Sec. 4.2), are proposed in IPD1FF to fully utilize the protein- molecule interactions in both forward and reverse processes of our diffusion framework.
model for binding-aware 3D molecule generation.
4.1 LEARNING PROTEIN-LIGAND INTERACTION PRIOR WITH IPNET
IPNET consists of (Borgeaud et al., 2022; Hou 2019). Two shallow fully-connected SE())-equivarian neural
neural networks (Satorras et al., 2021) and cross-attentie layers
3

---

Published as a conference paper at ICLR 2024
Pou 1Mp PMMMMMPPPeeee Poreou
Fage
iPNet
Prior- Shifting
Prior-
Conditionin (P.d (p.8
Reverse Process
Ground truth Vina Score
IPNet
Forward Process
(P.M,)
Loss
Conditioning
Shittig
IPNet
Per
Prodicted Vina Score
Helhoot
(.FFF)
(a) IPNet
(b) IPDiff
Enciigg Deaabars
The overall schematic diagram of IPDIFF. The pretrained IPNET are frozen during both training and sampling process for providing interaction priors. The molecule Mo and Muje are utilized for extracting interaction prior in forward and reverse process, respectively. The Maje the estimated molecule at the time step of the sampling process due to the inaccessibility of Mo, F denotes the interactive representations and S denotes the position shifts of molecules.
Figure 2:
networks are applied on the graphs of the protein Gd and ligand molecule G-M, respectively, to learn the intramolecular interactions. And another shallow fully-connected SE3)ieiqivvriian neural network is applied on the graphs of the complexes graph G g. u G which is constructed by g. and g", in order to modeling the intermolecular interactions. Given a ligand graph g.,, the 1-th SE(3)-equivariant layer works follows:
h MJ+1 h:m.
Σ JENu((
M. ()x -x. (e.,,ngi) nngh))..
M1+1
x
M.l
Σ jENu(()
(x - -x. soad (fixten xtill hi ny n.MHH))...
(2)
where h; M,1+1
E Rdd and x: M,1+1 E Rd are the SE(3)-invariant and SE(3)- hidden states of the atom of the ligand after the I-th equivvaiaa layer, respectively. NM (i) stands for the set neighbors of atom i on g.M, and the initial hidden state h M.U obtained by an embedding p.I and a complex graph ge,
layer that encodes atom information. Given protein graph GP, h," x h C.I. C.l can be derived in the same way. X;
In IPNET, an atom-wise cross-attention layer is introduced to model both intramolecular and inter- molecular interactions of protein-ligand pairs, which essentially accounts for the binding affinity. The SE(3)-invarilg features H.M. E R Nuxx and HP. E IR Nyxx are firstly concatenated along with the first dimension, and then concatenated with the SE(3)-invariant features HC,L e NN) along with channel dimension and finally fed into MIPP
F.M. P MLP M.L. H.. ..P.I HH..I,HH.II))) H(.)))
where [-] denotes concatenation along the first dimension. Then, F. and FP are delivered the cross-attention layer for extracting interactions:
F.M = Attn Attn(Query (FMM), (ey(FP)) Value
(4)
F Atta(Query (P)) (e)())))) Value(FMM)
(5)
where the functions Attn, Query, Key, and Value compute the cross attention weights and the query, key, and value matrices, respectively. The interactive representations of the ligand molecule F.M and protein F are further aggregated into a global feature to predict their binding affinity: (An M) IPNET(P, describe how to utilize the prior network IPNET to facilitate the 3D molecular diffusion generation with protein-molecute interactions in both the forward and reverse processes.
Please refer Appendix D.1 for pre-training details. Next, will
4

---

Published as a conference paper at ICLR 2024
4.2 PRIOR-GUIDED 3D MOLECULAR DIFFUSION MODEL
4.2. 1 PRIOR-SHIFTING
We propose prior-shifting to shift the molecular diffusion trajectories on the positions of the ligand molecules based on the protein-molecule molecule interactions modeled by IPNET. Firstly, IPDIFF feeds the ligand molecule M and the given pocket P into the pretrained prior network IPNET to extract the interactive representations of protein F and molecule F.M, Then, a learnable neural network v/o2() is introduced for producing the iiteraction-basee shift for adapting the positions of ligand molecules. In practice, ())2(-) is I-layer MLP which maps the updated molecule features with time step t to the cumulative mean shift S.M E R. x3 of the positions of ligand molecule:
S.M = r7- hi 102 (Fo2,FM..
where [-] denotes concatenation along the first dimension, k is shift-mode coefficient and 1) is shift-scale coefficient. We set the k to vā (1 vā). Since the shifts S.M are the cumulative values, this setting can keep the start point and the end point of the diffusion trajectories unchanged. is introduced control the shift scales of diffusion trajectories, and we set r) = 1 in our experiments.
More concretely, the forward process, we adapt the molecular diffusion trajectories by injecting the learned binding-relasd related molecule representation as a mean shift at each time step (the differences from the original molecular diffusion in Appendix A are highlighted in blue):
(MM P, )
II NX / Bixo.! + s B:t) C(voN((1 - Bo) + 3./K),
(7)
NM
F
[x /I Bixe + s VI B.,
(8)
C(v (11 M, + B.KK,,
where A and stand for the Gaussian and categorical distribution, respectively. And sM denotes
the vector on the i-th row of S.M. We shall prove these two definitions are consistent in Appendix B. It is worth noting that, to fully exploit the interaction prior without introducing misleading noises, we utilize the interactive representations F.M in Equations (6) to (8) in reacher-forcing fashion, where F/M is obtained by feeding the ground truth grotein-ligand pair [X,M X.1. [VM, Vill into the pretrained IPNET:
FFM M F. I IV..VVV Vil
(9)
Therefore, the corresponding shifted posterior can be analytically derived as follows:
Nas
q(M- Mo. P.F
II ( (x(x x (.1),,
(10)
C(viM cc(v vol)))
where fx(x;,1 M (/,1) = X0,07
var M
(o.(I- (xx - s(,1) + s1-1, Br = T-
M.M
1-ā,-1 3r.
I-se 0
T-
a² = 1 Br, 04. = II ai, č(Ve,,s Vo.i) [ā.-1Vo,s + (1-a(-1)KK)
and ee (Vr,s, Vo.s) = [aiVe,s + (1 - -aa))/K Ek-1
Σ
In the reverse process, since the ground-truth molecule X.M and V.M are inaccessible the time step f, we utilize the molecule Møj+11 = [X V.M o estimated in the previous time step + 1 to substitute the Mo and feed it with P into the pretrained IPNET for obtaining the interactive repre- sentation F.M 1 And then we calculate the S, from F.M
0jr-1 according to the Equation (6). Similarly, we calculate the S from F.M. Therefore, the reverse transition kernel can be approximated with oje-
predicted atom types Voje.; and atom positions Xoje. as follows:
NM
Pei [M.1|M,, P. F.M
FM))= = II (xx (xxM xMM. PM fu P. (M..), (M,.),,TT)
01
(11)
i=1
M, voM)))
5

---

Published as a conference paper at ICLR 2024
where (x(x X 6ofe+1,5 folt.)
f.M
I- x + الالعمم ( - + 1 In this way, I-āe
our IPDIFF can align molecular diffusion process with molecular sampling process regarding the information utilization of target protein, and optimize the diffusion trajectories according to the protein-molecule interaction.
4.2.2 PRIOR-CONDITIONING
In order to maximize the exploitation of protein protein-ligand interaction prior in the pretrained IPNET, we propose prior-conditioning to condition the molecular sampling process previously estimated protein-molecule complex for facilitating the binding-aware molecular generation. Specifically, we leverage the ligand atom embedding F.M 0/++1 and protein atom embedding F
oft+1 extracted by the
(NNET(Mojat
P) to enhance the embeddings at the time step t:
H H.M. Concat .F H Concat (H, F.P FF
(12)
where H,M (HF)) denotes the initial hidden states of the ligand (protein) at the time stpe / to be fed into the first -equivaraa layer of the diffusion model. Actually, the molecule M estimated at time step ++ 1 is supposed to be a candidate ligand with high binding affinity towards target protein, especially when + is large (i.e., the generative process nearly ends). Due to the inaccessibilityy of Mojt+1 during the training phase, we directly utilize the ground truth molecule M substitute it in a teacher-forcing fashion.
3D Equivariant Molecular Diffusion We then apply a neural network with L2 layers on the k-nn graph of the -igand complex enhanced by our prior-conditioning (denoted as C [M, where [-I] denotes concatenation along the first dimension) to learn the geometric interactions between the ligand atoms and the protein atoms. The hidden state H. and positions X at the time step t updated as follows:
CJ+1 h
Σ
f
Ci x
x
h miy ett
(13)
jENo(0)
c1+1 C.I. Xii x1, +
Σ jENc(()
(xe1 - x
Sed
(1x5f x
hett c.411
o 1
(14)
where Ni((i) stands for the set of k-nearest neighbors of atom on the protein-ligand complex graph, e,, indicates the atom i and atom are both protein atoms both ligand atoms one protein atom and one ligand atom, and mot is the ligand atom mask since the protein atom coordinates are known and thus supposed to remain unchanged during this update. We let H. = [H. ,H,'01 as the representations incorporate the interaction information through the prior-conditioning mechanism as described in Equation (12). Finally, we use Vo = (eatmma((MP((H() C.L. o. and X = X
M.0
as the final prediction. We leave the details about the loss function and summarize the training and sampling procedures of IPDIFF in Appendix C.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS
Datasets and Baseline Methods For fully modeling protein-ligand interactions, we utilize the protein-ligand pairs (complexes) in PDBbind v2016 dataset Liu et al. (2015) with binding affinity signals to pretrain the IPNET. The PDBbind v2016 dataset consists 3767 training complexes and 290 testing complexes, and it is commonly employed in binding-affinity prediction tasks. For molecular generation, following the previous work Luo et al. (2021); Peng et al. (2022); Guan et al. (2023a), train and evaluate IPDIFF on the CrossDockedd2200 dataset (Francoeur al., 2020). The same data preparation and splitting as Luo et al. (2021) are employed, where the 22.5 million refined to high-quality docking poses (RMSD between the docked pose and the ground truth < 1Å) and diverse proteins (sequence identity 30%). Specifically, 100, 000 protein- ligand pairs are utilized for training and 100 proteins for testing. In our study, we conduct comparative analysis of our model with five recent representative methods for SBDD. LiGAN
docked binding complexes

---

Published as a conference paper at ICLR 2024
(Ragoza et al., 2022a) is CVAE model trained on an atomic density grid representation of protein- ligand structures. AR (Luo et al., 2021) and Pocket2Mol (Peng et al., 2022) generate 3D molecules atoms conditioned on the protein pocket and previous generated atoms in an autoregressive manner. DifgetDiff (Guan et al., 2023a) and DecomposeDiff (Guan et al., 2023b) are recent state-of-thh-ar diffusion methods which generate atom coordinates and atom types in non-autoregrrssive way.
Pocket2Nol, 1350-8.14
Band
BIAAN
Pocket2
Target Diff
Decnmp
Diff
C-C
0.601
0.609
0.496
0.369
0.359
0.386
Decomphatr
0.665
0.505
Ours
1350:0.60)
C-N
0.634
0.474
0.466
0.363
0.344
0.298
C-N
0.749
0.635
0.629
0.550
0.584
0.238
C-O
0.656
0.492
0.454
0.421
0.3%
0.366
C=O
0.661
0.558
0.516
0.461
0.374
0.353
0.497
0.263
0.638
0.235
10
12
Distance of all atom pairs (A)
Table 1: Censen-Shannon divergence be-
Comparing the distribution for distances of all-atom for reference molecules in the test set (blue) and model generated molecules (color). Jensen- Shannon divergence (JSD) between two distributions is reported.
Figure 3:
tween bond distance distributions of the reference molecules and the generated molecules, and lower values indicate better performances. 4_", "=", and "." represent single, double, and aromatic bonds, respec- tively.
Evaluation Three perspectives: molecular structures, target binding affinity and molecular properties are considered for evaluating generated molecular ligands. The Jensen-Shannon di- vergences (ISD) in empirical distributions of atom/bond distances between the generated and the reference molecules are calculated for evaluating the generated molecules in terms of molecular structures. Following previous work Luo et al. (2021); Ragoza et al. (2022b); Guan et al. (2023a), AutoDock Vina (Eberhardt et al., 2021) is utilized to compute the mean and median of binding-related metrics, including Vina Score, Vina Min, Vina Dock and High Affinity. Vina Score directly estimates the binding affinity based on the generated 3D molecules; Vina Min performs a local structure minimization before estimation; Vina Dock involves additional re-docking process and reflects the best possible binding affinity; High affinity measures the ratio of how many generated molecules binds better than the reference molecule per test protein. And we utilize the QED, SA, Diversity as metrics following Luo et al. (2021); Ragoza et al. (2022a) for evaluating molecular properties. QED a simple quantitative estimation of mikgllikeness combining several desirable molecular properties; SA (synthesize accessibility) is a measure estimation of the difficulty of synthesizing the ligands; Diversity is computed as average pairwise dissimilarity between all ligands generated by the given pocket. All sampling and evaluation procedures are following Guan et al. (2023a) for fair comparison.
5.2
MAIN RESULTS
Generated Molecular Structures We compare the molecular structures of molecules generated by our IPDIFF and the other representative methods. The all-atom pairwise distance distribution of the generated molecules are plotted in Figure 3. And Tab. 1 presents the bond distributions of the molecules generated by different methods compared against the corresponding reference empirical distributions. And our IPDIFF achieves superior performance other methods, which demonstrating the ability of IPDIFF in generating stable molecular structures.
major bond types compared to all
Target Binding Affinity and Molecule Properties We evaluate the effectiveness of IPDIFF by comparing with two types of SBDD methods: non-diffusion methods and diffusion- methods in Tab. 2. Our IPDIFF significantly outperforms non- non-diffusion baselines in binding-related metrics. Notably, IPDIFF also surpasses strong autoregressive method Pocket2Mol by large margin of 24.9 %, 16.0% and 19.9% in Avg. Vina Score, Vina Min and Vina Dock, respectively. Compared with the state-of-the-art diffusion-based method DecompDiff, IPDIFF not only increases the binding-related metrics Avg. Vina Score, Vina Min and Vina Dock by 13.2% % 5.8% and 2.1 %. but also increases the property-related metric Avg. QED and Avg. Diversity by 15.6% % and 8.8%. In terms of high-affinity binder, we find that on average 69.5% % of the IPDIFF molecules show better binding affinity than the reference molecule, which is significantly better than other baselines. These gains demonstrate
7

---

Published as a conference paper at ICLR 2024
that the proposed IPDIFF effectively utilize protein-ligante interaction priors from IPNET to enable generating molecules with improved target binding affinity and molecular property.
Table 2:
Summary of different properties of reference molecules and molecules generated by our model and other non-diffusion (Non-Dift.) and diffusion- (Diff.) baselines. (t) (4) denotes
larger smaller number is better. Top 2 results are highlighted with bold text and underlined text, respectively.
Vina Score (1)
Vina Min (})
Vina Dock (1)
High Allinity (T)
QED (f)
SA (T)
Diversity (f)
Methods
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Mod.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Reference
6.36
66.4
-6.71
-6.49
-7.45
-7.26
0.48
0.47
0.73
0.74
LiGAN
-6.33
-6.20
21.1%
11.1%
0.39
0.39
0.59
0.57
0.66
0.67
Comp. with
GraphBP
4,80
-4.70
14.2%
6.7%
0.43
0.45
0.49
0.48
0.79
0.78
Non-
AR
-5.55
-5.64
-6.18
-5.88
-6.75
-6.62
37.9%
31.0%
0.51
0.50
0.63
0.63
0.70
0.70
Dilr.
Pocket2Mol
4.70
.6.42
-5.82
-7.15
-6.79
48.45)
51.0%
0.56
0.57
0.74
0.75
0.69
0.71
IPDIFF
-6.42
-7.01
-7.45
-7.48
-8.51
69.5%
75.5%
0.52
0.53
0.61
0.59
0.74
0.73
Comp. TargeDiif
-6.30
-6.64
-6.83
-7.80
-7.91
58.15)
59.1%%
0.48
0.48
0.58
0.58
0.72
0.71
with
DecompDift
-5.67
-6.04
-7.04
-7.09
-8.39
-8.43
64.4%
71.0%
0.45
0.43
0.61
0.60
0.68
0.68
Diff.
IPDIFF
-6.42
-7.01
-7.45
-7.48
-8.57
-8.51
69.5%)
75.5%
0.52
0.53
0.61
0.59
0.74
0.73
Achieving Better Trade-off' From Tab. 2, we can see trade-off between binding-related metrics and property-related metrics QED in previous methods. DecompDiff performs better than AR and Pocket2Mol in binding- related metrics, but falls behind them in QED scores. In contrast, our IPDIFF not only achieves the state-of-the-art binding-related scores but also maintains proper QED score which is comparable to Pocker2Mol, achieving better trade-off than DecompDiff. Nevertheless, we put less emphasis on QED and SA because they are often applied rough screening metries in real drug discovery scenarios, and it would be acceptable as long as they are within a reasonable range. Figure 4 shows some examples of generated ligand molecules and their properties. The molecules generated by our model have valid structures and reasonable binding poses to the target, which are supposed to be promising candidate ligands. More ablation studies, experimental results and visualized examples of generated molecules are present in the Appendices E and G.
5.3 MODEL ANALYSIS
Effect of Pretrained IPNET In IPDIFF, we leverage the llogedn interaction prior in pre- trained IPNET to facilitate the molecular diffusion. Here we conduct ablation study on the effect of our IPNET. We designed two types of IPNET: (1) IPNET, simultaneously considers the geometry and sequence information modeling interaction priors by simply stacking SEui)-Equivariant layer as described in Sec. 4.1, and (2) IPNET-Seq, replaces SE(3)-equivarian layers in IPNET with graph attention layers (Velickovic et al., 2017) and only considers the sequence information by taking [V-M V as inputs, and then applied these two types of IPNET into the training and sampling process of IPDIFF. The results are present in Tab. 3. We can observe that even if only the sequence-level interaction prior is utilized to guide the generation process of sequences and geometric structures of IPDIFF, IPDIFF can achieve superior performance than the baseline model, especially in the property-related metrics. It reveals that our IPDIFF does not rely heavily on the elaborate designs of the IPNET. Moreover, we jointly train the IPNET with IPDIFF from scratch without pre-training. We found that training whole model from scratch converges slowly on the same device that used for all the experiments. And the final performance is consistently worse than that with pre- pre-training. The main reason is that training generative models from scratch lacks explicit supervision (i.e., binding affinity) for IPNET to model the accurate interaction prior. This demonstrates that the modeled interaction prior through pretraining IPNET plays an important role in our method.
Effectiveness of Prior-Conditioning and Prior-Shifting Our primary hypothesis is that intro- ducing the 3D prolein-molecule interaction prior into both forward and reverse process benefits the training and sampling efficiency, and thus improving the molecular generation performance in both binding- and property-related metrics. To verify it, we conducting a set of experiments to showcase the effectiveness of mrior-conditioning and prior-shifting All results are present in Tab. 4. We observe that self-conditionig mechanism proposed by Chen et al. (2023) can not improve the generation performance, because the estimated molecules from previous time step does not include the protein- ligand interaction information for self refinement. In contrast, our prior-conditioning mechanism
8

---

Published as a conference paper at ICLR 2024
Referanco
Dicorpoit
PCW
Rebrerca
Becompär
IPDI
4A4 GED: B.M.
BR VIa -18), DED: GAT
41.20
314
Refurance
DecompOOt
Oecorppee
Figure 4: Examples of generated ligands for protein pockets (4yhj. 2v3r, 4keu and 3li4). Carbon atoms in reference ligands, ligands generated by DecompDiff (Guan al., 2023b) and IPDIFF are visualized in green, blue and orange, respectively. Vina Score, QED and SA are reported.
significantly boosts both binding-related and property-related metries by introducing informative protein-ligand interaction knowledge from our pretrained IPNET. Besides, our prior-shifting also has a notable improvement over baseline binding-related metrics which reveals that prior-shifting can effectively assist IPDIFF to generate ligand molecules binding tightly the given protein pockets. Kindly note that prior-shifting does significantly contribute to property-related metrics. This because the prior-shifting is only equipped position of molecular atoms, while property-rekee related metrics are less dependent to the geometry of the protein-ligane ligand pair. Furthermore, simultaneously utilizing condiiiiiii and prior-shifting IPDIFF yields the best performance in both binding-relatee and property- property-related related metrics, which demonstrates the effectiveness of two mechanisms.
Table 3: Effect of the pretrained IPNET. (?) / ()) denotes a larger smaller number is better. Top results highlighted with bold text and underlined text, respectively.
Vina Score
Vina Min
Vina Dock ())
High Affinity
QED
SA)
Methods
Avg.
Med.
Med.
Avg.
Med.
Med.
Med.
Med.
baselinc
-5.04
-3.75
-6.38
-6.22
-7.55
-7.72
54.2%
54.1%
0.46
0.46
0.57
0.57
IPNeT wio Prestraining
.3.88
-4.88
-5.38
-5.80
-6.96
-7.20
43.2%
34.0%
0.39
0.39
0.58
0.57
IPNeT-Seg
55.78
6.95
-7.11
-7.53
.8.34
-8.56
69.0%
76.4%
0,57
0,58
0.55
0.54
IPNET
-6.42
-7.01
-7.45
-7.88
-8.57
.8.51
69.5%
75.5%
0.52
0.53
0.61
0.59
Table 4: The effect prior-conditionig and prior-shifting mechanism. (Q) denotes a larger / smaller number is better. Top results are highlighted with bold text and underlined text, respectively.
Vina Score (i)
Vina Min (1)
Vina Dock (
High Affinity (1)
QED (f)
SA
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Mod.
haseline
-5044
-3.5
-6.38
-6.52
-7.55
-7.72
54,25
54.1%
0.46
0.46
0.57
0.57
baseline seff-conditioning
-6.06
-6.18
-6.46
-7.43
-7.55
53.3%
51.7%
0.49
0.56
baseline- priee-conitioning
60.6%
baseline prier-shiftng
-5.11
-6.39
-6.87
-7.09
-8.06
-8.24
61..%%
61.0%
0.18
0.48
0.56
0.56
IPDIFF
-.42
-7.01
-7.45
-7.48
-8.57
-8.51
69.5%
75.5%
0.52
0.53
0.61
0.59
6 CONCLUSION
In this paper, we for the first time introduce target protein into both diffusion and sampling process, and propose novel Interaction Prior-guided Diffusion model (IPDIFF) for the protein-specific 3D> molecular generation. We design IPNET to learn protein-ligand interaction prior with the supervision of binding affinity signals, which further utilized to facilitate the binding-aware 3D molecular diffusion generation with our proposed prior-shifting and prior-conditioning Empirical studies on CrossDocked2020 dataset show IPDIFF can generate molecules with more realistic 3D structures and state-of-Ne hht binding affinities towards the protein targets, with up -6.42 Avg. Vina Score, while maintaining proper molecular properties. Moreover, we conduct extensive analysis experiments to demonstrate the effectiveness and superiority of the proposed model.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
The work was partly supported by the National Natural Science Foundation of China (No.62171251), the Major Key Project of GZL under Grant SRPG22-001, the Major Key Research Project PCL under Grant PCL2023A08 and PCL2023A09.
REFERENCES
Amy Anderson. The process of structure-based drug design. Chemistry & biology, 10(9)787-7777 2003.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean- lean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning. pp. 2206-22044 PMLR, 2022.
Martin Buttenschoen, Garrett M Morris, and Charlotte M Deane. Posebusters: Ai-based docking methods fail to generate physically valid poses or generalise to novel sequences. arXiv preprint S0Xiv:2308.05777, 2023.
Ting Chen, Ruixiang ZHANG, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with coldiiondiion,ng. The Eleventh International Conference on Learning Representations, 2023, URL https: /oppnrevieeyy net forum iddiiiiii jR9QXEu.
Jerome Eberhardt, Diogo Santos- Santos-Martins, Andreas F Tillack, and Stefano Forli. Autodock vina 1.2. O: New docking methods, expanded force field, and python bindings. Journal of Chemical Information and Modeling, 61(8):3891-3899 2021.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B Iovanisci, Ian Snyder, and David R Koes. Three ditre-dimensional convolutional neural networks and cross-docked data set for structure-based drug design. Journal of Chemical Information and Modeling, 60(9):4200-422 2020.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In International Conference on Learning Representations, 2023a.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang. and Quanquan Gu. DecompDiff: Diffusion models with decomposed priors for structure-bast based drug design. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 1187--11866. PMLR, 23-29 Jul 2023b. URL https: : http//peeceee...../oo/p mlr. prwsss /asss/vv22 wwtps//////////// guan22aa html.
Charles Harris, Kieran Didi, Arian Jamash, Chaitanya Joshi, Simon Mathis, Pietro Lio, and Tom Blundell. Posecheck: Generative models for 3d structure-based drug design produce unrealistic poses. In NeurlPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International Conference Machine Learning, pp. 8867-8887. PMLR, 2022.
Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. Advances in Neural Information Processing Systems, 32, 2019.
Yibo Li. Jianfeng Pei, and Luhua Lai. Structure- de novo drug design using 3d deep generative models. Chemical science, 1244)):36644-136) 2021.
10

---

Published as a conference paper at ICLR 2024
Jaechang Lim, Seongok Ryu, Kyubyong Park, Yo Joong Choe, Jiyeon Ham, and Woo Youn Kim. Predicting interaction using a novel graph neural network with 3d structure-embedder graph representation. Journal of chemical information and modeling, 59(9):988 2019.
Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Li. Diffbp: Generative diffusion of 3d molecules for target protein binding. arXiv preprint arXiv:2211.11214 2022.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. In International Conference on Machine Learning, 2022.
Zhihai Liu, Yan Li, Li Han, Jie Li, Jie Liu, Zhixiong Zhao, Wei Nie, Yuchen Liu, and Renxiao Wang. Pdb-wide collection of binding data: current status of the pdbbind database. Bioinformatics, 31(3): 405-412, 2015.
Calvin Luo. Understanding diffusion models: A unified perspective. arXiv preprint arXiv:2208 11970, 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229- -2399, 2021.
Thin Nguyen, Hang Le, Thomas P Quinn, Tri Nguyen, Thuc Duy Le, and Svetha Venkatesh. Graphdta: predicting drug-targed binding affinity with graph neural networks. Bioinformatics, 37 (8):1140-1147, 2021.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng. and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. arXiv preprint arXiv:2205.0734 2022.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3D molecules conditional on receptor binding sites with deep generative models. Chem Sci, 13:2701-2713, Feb 2022a. doi:
(..1099/DSCC05910
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 13(9):2701-2713, 2022b.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In IEEE Conference Computer Vision and Puttern Recognition, pp. 10684-10695, 2022.
David Rooklin, Cheng Wang, Joseph Katigbak, Paramjit S Arora, and Yingkai Zhang. Alphaspace: fragment-centrie topographical mapping to target -aeeeinn interaction interfaces. Journal of chemical information and modeling. (5(8))1585511) 2015.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323-9332. PMLR, 2021.
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, et al. Structure- hased drug design with equivariant diffusion models. arXiv preprint 10 13695, 2022.
Miha Skalic, José Jiménez, Davide Sabbadin, and Gianni De Fabritiis. Shape-based generative modeling for de novo drug design. Journal of chemical information and modeling, 59(3): 1205- 1214, 2019.
Jascha Sohl Sohl-Dicke Diak--isktein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
Cheng Tan, Zhangyang Gao, and Stan Z Li. Harget-aware molecular graph generation. arXiv preprint arXiv:2202. 04829, 2022.
11

---

Published as a conference paper at ICLR 2024
Petar Velickovie, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. stat, (050(20):10-488555 2017.
Mingyuan Xu, Ting Ran, and Hongming Chen. De novo molecule design through the molecular generative model conditioned by 3d information of protein binding sites. Journal of Chemical Information and Modeling, 61(7):3240-3254, 2021.
Zaixi Zhang, Yaosen Min, Shuxin Zheng, and Qi Liu. Molecule generation for target protein binding with structural motifs. The Eleventh International Conference Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2024
A CONVENTIONAL PROTEIN-AWARD 3D MOLECULAR DIFFUSION PROCESS
In the forward diffusion process, a small Gaussian noise is gradually injected into data as a Markov chain. Because noises are only added on ligand molecules but not proteins in the diffusion process, we denote the atom positions and types of the ligand molecule at time step as X,M and V.M. The diffusion transition kernel can be defined as follows:
NM
[MM|MM P)
M.
/1 8 Be1) C(v2 -,) + B./KK) (15)
1=1
Nar
MM, P)-
M. NV ui
3,x 3.1) (1- + B1/K), /
(16)
where NV and C stand for the Gaussian and categorical distribution respectively, 8, is defined by fixed variance schedules. The corresponding posterior can be analytically derived as follows:
NM
q(M,- Mo, P)
II
N [xx
(x xx.),, B,1) C(v vod)), (17)
where (xxx x5,1) = Ihc- M
l-a: 0i, + (1, M Br 1-ū-- Be, 02 = 1- a. = IT Ugs
1-û: 1,5
1-a:
and c² (v vor) = Jos M + +(1 (at)///KK) @ -ā.- M + +(1 - 11- K]. C(v, vol1 Σ oz
In the approximated reverse process, also known as the generative process, neural network param- eterized by 01 learns to recover data by iteratively denoising. The reverse transition kernel can be approximated with predicted atom types Vil,s and atom positions & at time step f follows:
P01 (M. P)
N M(x SM..). B.D) C(v Clv vol.)) (18)
B PROOFS
B. DERIVATION OF FORWARD DIFFUSION KERNELS OF OUR IPDIFF
Since our IPDIFF is designed better generate molecular ligands that bind tightly to the given protein pockets, the prior-shifting is considered on the diffusion process of the molecular atoms' positions only. For brevity, X denotes the molecular atom positions, and Fg denotes the ground-truth interactive representations utilized in the forward process as described in Sec. 4.2.2.
Firstly, we have the marginal Gaussian for X,- and X, as described in Equation (7):
q(x, Xo. P. Fo) (X /or Xo + S. (1 i)-1))
(19)
X Xo, P. F) (X v + St (1 ) 2)
(20)
S = 17 (Fu, - 1), S, = ki vo2 (Fo. t),
(21)
we can assume that:
X, X, Xo, Fo) N(X,; AX, + bbLL'),
(22)
then, we can derive the marginal Gaussian for X, according to Equations (19) and (22), for all <> 1:
XX, = A(Var +, (()-a"))AAA")
(23)
= N(x; X, - S,, E),
(24)
therefore, we can derive that:
A VarI.
(25)
b = S. VayS:-1,
(26)
-- [| ā₁ ozl - - (1 E.))E
(27)
13

---

Published as a conference paper at ICLR 2024
and
= ( V -
(28)
Particularly, according to Equations (19) and (28), we have:
(X1 Xo. P, Fu) = N(X V + +1 VanSo. {1 X X1 P, Fo) N V + VanSi 32
(29)
(30)
q ( P. Fo) = (X X, + S2, (1 ā2 Σ
(31)
from Equations (30) and (31), we can derive that:
x Xo, Po) = NX1 an X + 31 E); For making Equation (29) and Equation (32) matched, we set Sg = O.
(32)
B.2 DERIVATION OF THE POSTERIOR DISTRIBUTIONS OF THE SHIFTED FORWARD PROCESS
Following Luo (2022), For all / according to the Bayes' rule:
(Xx,-11 X, x. P.F.)
(33)
_ X X ( P.FF)
(34)
(X, Xu, P,Fu)
(X(i 04 1 + S.
Vo 3/E) N( 1-1 0111X + S (1 xx-1)
(35)
N(Ki V 2, + S, -,,
1 (X, ,,,X,-1- - S
S112 (X,-1
10-11 S2-1)
xexpl
2
(1 (1-
(1 11- -1-1)
(36)
(X,
Va,X, - -s,)2 }}
(1-ār)
1 0x.2111 1-1 2 2r, X + 2 X, X1
=exp(-
2
(37)
X
1} + CC(oo X,) 2 X1--XX-1XX, Xo 22
+
--a=1
xexpl
an
1
x2
1-1 + +
Vaixi + VarS a1
2
04
1 a4-1
1 01
(38)
Xo + S
X|-11}}
1
1- a1
=exp
[x3
(1 at)) 01-1)
/ + vii -a,8
a4-
(39)
+2(
xx,11}}
1-d;
(I-as)(1-o) (1-01-1)
1 a₁
=exp(
|
2 (1 (1 a(-1)
(40)
/ /a,Xi Va:S S, ()(1)))1111p) -ā- (VVrrIXo- + S. ( at)
+
x,-1|}
1- δz
1
=expf
x
2
[T
[1
[-m=1)
(41)
Cr Br
var (1 - ar-1)
1 - āc-1
2(
X +
X, Vo: 04
S. + (.-
ai
1- 14
Vin-1 13:
vai(1 a1-1)
āt-l
01-1 (1-01-1)
<N(x,11
X, +
X, - Vot
S, + S
3,E)
1 1- āi
1 - δ
1 - σ
1 a,
(42)
14

---

Published as a conference paper at ICLR 2024
As mentioned in Sec. 4.2.1, since the ground-truth molecule X and VaM are inaccessible the time step t of the reverse process, utilize the molecule M0 [ V.M estimated in the previous time step + to substitute the Mo and feed it with P into the pretrained IPNET
0je+11 ofr+
for obtaining the interactive representation FM 0fo+11 And then we calculate the S, from FM 0jr+1
Oje" according to the Equation (6). Similarly, we calculate the S from F.M. In IPDIFF, we employ a
model Φe (X,, 1) for predicting Xo directly. Then we can get the predicted posterior distributions parameterized by #1:
P8l 1 Fol,, FoJ++1. FojO) =N y a
Var āt-1) X1 āe
081 (X,,t)
1 ai
(43)
(1 S. S
(1 (4--1) 1)
Var
B1E).
ae
B.3 DERIVATION OF THE TRAINING ORJECTIVES
According to Equation (42), the training objective can be represented as:
=gg looopoo (Xol X1, Foje+1: FOje) + (xx((((x (xo Fo)l (Xr))
T
(44)
Σ D X, Xo, (o) F Foje41, FOjI)]}
1=2
For the first and the second terms, we can derive them as constants and discard them in the objective function. For the third term, we can derive it by Gaussian Keullback-Leibler divergence:
DKL (q( ( X, Xo. Fo) (X,- X,, F 1. Foj:)]
v a 9
100 (X.,) Xol + Va ā.
(S. - S.) + -S:-11-
10851-SS--00 -S--)) at-1 (1E)-1
1-a;
(45)
Assuming that S, Lipschitz continuous w.r.t Xa. then we can simplify the Gaussian Keullback- Leibler divergence:
1
- 0=- (S. (S:) + +S S 1)l (T -)))
100) (X.,)) Xoo + Vat
2 at
1 at
Scilloo01 (X,, -Xo] + va ā1-1 ($, - S,) + ($.-1 - -S-11)l
04
- 01-1 (St S.) + ||($-1 - -S-1))
Sa((ll001(X (X,,1) - Xoo1 + |ll
1 5t
Stellomm (X,,t) -X,]l,
(46)
by the lipschitz continuity of Mahalanobis Distances and S,. Here Ct2 "t are scaling factors. Finally, the training objective of atom position at time step t - 1 are defined as follows:
1
NM
NM
iū(x,,,, Xo fo.s) (x & fo foje.) = 7
xoo:
Ro,ell: (47)
2
where Xo and Vo are predicted from X, and Vr, where % is scaling factor. And we use the same objective function of atom type at time step t - 1 as Guan et al. (2023a):
NM K
o(ve, Vo.:) log ē(Ve,i Vo.s)s
(48)
c(ve,,, Vo,s
Kindly recall that Xt.i, Ve.i. N0.s. Vo.s. fo.., fojet1.; and foje. correspond to the i-th row of X,, Vt- Xo. Vo. Fo,s, Fo and Fo respectively. The final loss combines the above two losses with hyperparameter À as: L [(*) XL We summarize the training procedure of IPDIFF in Algorithm and highlight the differences from its counterpart, TargetDiff (Guan et al., 2023a), in blue.
15

---

Published as a conference paper at ICLR 2024
B.4 THE BETTER LIKELIHOOD
We shall show that IPDIFF is theoretically able to achieve better likelihood compared to previous diffusion models. As the exact likelihood is intractable, aim compare the optimal variational bounds for negative log likelihoods (NLL). The objective function of IPDIFF at time step is
Egg D (40, (xt- 2o, Fo, P)lp (1z-1 Foje+1 Poje, P))
(49)
and its optimal solution
min Eggg (ar zo, Fo, Pp (xt-1 1 aee Folt+1+ Foje, 0,,02
(50)
=mine, (mino, Equ Dx (1o, (2 122, To. For P) P)ppe (21-1 Fo Foje: P))
(51)
- minoe E, ( 20, P) Ipp, (3t 1z P));
(52)
where ming, ExDxz(gle (z,-1 | zo. plpo, (x 1.,, P)) is the optimal loss of previous diffusion models that do not use interactive representations F in the forward process. Similar inequality can be obtained for t=1:
min Eyg loogge Pe, (2o) 21 Foj1: P) 01.02
(53)
S min Eg - log (30 P)
(54)
As result, have the following inequality by summing up the objectives at all time steps:
Eggzal P) log po, (zop P)
(55)
S ming, 82
Σ Egs DKL (01 To, Fo, P)l[po,
(x1-1 Fojet1, Foje: P))
1>1
(56)
+ Ego log pe, (xo 11 1o11 P) C
<ming, Σ
EaD vo, P)llpo (11-1/21, P)) + Eg - log pe, (zo æ1 P) + C (57)
where a constant defined by var. Henee, IPDIFF has tighter bound for the NLL, and thus theoretically capable of achieving better likelihood, compared with the previous diffusion models.
C TRAINING AND SAMPLING PROCEDURE
We summarize the training and sampling procedure as Algorithms and 2.
D IMPLEMENTATION DETAILS
D.1 DETAILS OF IPNET
Initialization of Inputs Following (Guan et al., 2023a), we use one-hot element indicator (H,
C, N, O, S, Se) and one-hot amino acid type indicator (20 types) to represent each protein atom. Similarly, each ligand atom repsented with one-hot element indicator (C, N, O, F, P, S, CIJ. And an additional one-dimensional flag indicating whether the atoms belong to the protein or ligand introduced. Two 1-layer MLPs are used to map the input protein and ligand into 128 128-dim latent spaces respectively.
The IPNET is designed model the complex intramolecular and intermolecular 3D interactions between the atoms of proteins-ligand pairs. To achieve this, we use three shallow SE(3)-equivarian neural networks for geometric message passing on the fully-connected graphs of the protein, ligand and complexes (consists of protein and ligand), respectively. We then apply cross attention layer to the paired protein-ligand graph for learning the inter-molecule interactions. Finally, use sum-poote aum-pooling layer to extract global representation of the protein- ligand pair by pooling all atom nodes. And a two-layer MLP is introduced to predict the binding affinity SAT- More details about the model architecture are provided in Tab. 6.
Architectures
16

---

Published as a conference paper at ICLR 2024
Algorithm 1 Training Procedure of IPDIFF
Input: Protein-ligand binding dataset (P,M) i=1 learnable diffusion denoising model d0i, learn-
able neural network 4o2 and pretrained interaction prior network IPNET
1: while d91 and 2/2 not converge do
2:
[X, X IVo Vol (P.M)
3:
Lo,,.......
4:
Move the complex to make CoM of protein atoms
5: Obtain interactive features [FM
M..M F from IPNET:
[FA F IPNET( ( X xI [V. VoI)
6: Perturb x to obtain X with shifts St
<~ (0.1)
S.M = 1- k2 U02 (F t)
(Equation (6))
X, V
+ S.M
+ v1 dec
7: Perturb V.M to obtain V.M.
g Gumbel 1)
log M log(dū, VoM + (1 (a/K)))
V.M = omehol (arg max; (g: + log &.
Embed VM into H. M.0 and embed V? into H."
H (H. P.0 =... HP
8:
Obtain features [H; M,0 H..'11 through prioo-conditioning:
9:
[HMM. H HPP11 = ( H1 [FM FF
(Equation (12))
10:
Predict (X.M Vol) from Φ01:
ojer
X.M V.M = de ([ X H HD Ol8 0ft
(Equations (13) and (14))
11:
Compute loss with (X.M. Vol) and (X& VOM
(Equations (47) and (48))
ojr Jointly update 01 and 02 by minimizing L
12:
end while
Training Details During the training, we use the Mean Squared Error (MSE) loss with respect to the difference between the predicted and ground truth binding affinity scores as the optimization objective. The binding affinity values of protein-ligand pairs range from 2.0 to 11.92. For avoiding information leakage, we filter the training set by calculating the Tanimoto similarity with the molecules in the testing set of CrossDocked2220. and the similarity threshold was set to 0.1. As result, there 23 complexes filtered out from the training set. We train IPNET on single NVIDIA V100 GPU, and the Adam The experiments are conducted on PDBBind v2016 dataset as mentioned Sec. 5.1.
optimizer with learning rate 0.001, betas (0.95, 0.999), batch size 16.
Evaluation of IPNET Following Li et al. (2021), select Root Mean Square Error (RMSE),
Mean Absolute Error (MAE), Pearson's 's correlation coefficient (R) and the standard deviation (SD) in regression measure the prediction error. Meanwhile, we use these metrics select the pretrained IPNET utilized in IPDIFF because we believe that the ability to predict the binding affinity is highly related to the interaction modeling. The testing results of IPNET (in Tabs. 3 and 8) in binding affinity prediction are present in Tab. 5. And we introduce two GNN-based based binding affinity prediction methods: GraphDTA Nguyen et (2021) and GNN-DTI (Lim et al., 2019) make comparisons, indicating the rationality of our model design.
Table 5: Performance of IPNET in binding affinity prediction. (1) / (() denotes a larger smaller number is better. Top results are highlighted with bold text and underlined text, respectively.
Methods
RMSE (1)
MAE (L)
SD (4)
R (1)
GraphDTTA
1.562
1.11
1.558
0.697
GNN-DTI
1.492
1.192
1.471
0.736
IPNET-Sen Seq.
1.566
1.263
1.547
0.704
IPNET
1.612
1.317
1.589
0.683
IPNET (Ø 0.1)
1.544
1.239
1.516
0.718
IPNET (a 0.5)
1.499
1.140
1.386
0.771
IPNET (o
1.641
1.332
1.590
0.683
17

---

Published as a conference paper at ICLR 2024
Algorithm 2 Sampling Procedure of IPDIFF
Input:
The protein binding site P. the learned diffusion denoising model des, the learned neural network v/w2 and pretrained interaction prior network IPNET
Output: Generated ligand molecule M that binds to the protein pocket P
1: Sample the number of atoms NM of the ligand molecule M as described in Sec.
2: Move CoM of protein atoms to zero
3: Sample initial ligand atom coordinates X and atom types V
4: Let [FM F
+1 O, S =0
5: for in ufr+1
......1 do
6: Embed V:M into H M.o and embed Vo into H.P P.P (Ho = HP
7: Obtain features [H; M,0 HH through prior-conditionng::
[HAM, H "1 = c[ H [FM F 1oj++1)
ojt+1
(Equation (12))
8: Predict (X.M. 0(1) from Φ01:
oip V.M den ([X,, X [ M
(Equations (13) and (14))
Dje
9: Sample XM from the shifted posterior por (X M I.M. X. Fo
(Equation (11))
where 100,1)
M, = 17 ki-1 V/02
F.MM
1)
(Equation (6)
0fe-1:
x 1-1
vār-ife XM 1-6e Ujt
(arl1-ur- (X. - +
B
I-a:
Gee
Sample VMM from the posterior a( 2-11 V.M V.M, V
1-1
(V
ole>
(Equation (18)
11: Obtain features [F
ojer F., through IPNET:
F.M FP olte 0lt = IPNET 1 I
0f2 V!D
12: end for
D.2 DETAILS OUR PRIOR-GUIDED DIFFUSION MODEL
Initialization of Inputs
The representation of the atoms in the proteins and molecules is the same as the representation used in IPNET Appendix D.1. A 4-dim one-hot vector indicating four bond types: bond between protein atoms, ligand atoms, protein-ligane atoms or ligand-protedd atoms is introduced to representing the connection between atoms. And we introduce distance embeddings by using the distance with radial basis functions located 20 centers between 0 Á and 10 Å. Finally we calculate the outer products of distance embedding and bond types to obtain the edge features.
Architectures At the I-th layer, dynamically construct the protein-ligand complex with k- nearest neighbors (knn) graph based on coordinates of the given protein and the ligand from previous layer. In practice, we the number of neighbors kn, 32. As mentioned in Sec. 4.1, we apply an SE(3)-equivariant neural network for message passing. The 9-layer equivariant neural network consists of Transformer layers with 128-dim hidden layer and attention heads. Following Guan et al. (2023a), in the diffusion process, we select the fixed sigmoid 3 schedule with 31 le-7 and 3T = 2e-3 as variance schedule for atom coordinates, and the cosine schedule with s 0.01 for atom types. The number of diffusion steps are set to 1000. We denote PCM and PSM as prior- conditioning and prior-shifiig mechanisms, repecitively. More details about the model architecture provided in Tab. 6.
Training Details Following Guan et al. (2023a), we use the Adam as our optimizer with learning rate 0,001, betas = (0.95, 0.999), batch size 4 and clipped gradient norm 8. We balance the atom type loss and atom position loss by multiplying a scaling factor = 100 on the atom type loss. During the training phase, add small Gaussian noise with a standard deviation of 0.1 to protein atom coordinates as data augmentation. We train the parameterized diffusion denoising model of our IPDIFF single NVIDIA V100 GPU, and it could converge within 200k steps.
18

---

Published as a conference paper at ICLR 2024
Table 6: Details of both IPNET and Prior-Guided Diffusion Model in our IPDIFF
Network
Module
Backbone
Inpui Dimensions
Output Dimensions
Blocks
Protein Encoder
BGNN
Np 128
Np 128
2
IPNET
Ligand Frendeer
EGNN
128
128
Complex Encoder
EGNN
(Np Nar) 128
(Np Naz) 128
Interaction Layer
Graph Attention Layer
(Np Naz) 128
(Np Naz) 128
Pooling
Sum pooling
(Np + Nur) 128
Protein Encoder
Graph Attention Layer
Np 128
Np 128
IPNeT-Seq.
Ligand Encoder
Graph Attention Layer
Nar x 128
Nas x 128
2
Complex Encoder
Graph Attention Layer
(Np + NNx) x 128
(Np + Nar) x 128
2
Interaction Layer
Graph Attention Layer
(Np + NNr) x 128
(Np Nar) 128
Pooling
Sum-pooling
(Np + Nar) x 128
1 128
Position Dynamics
Transformer
(Np + NN)) x
(Np (Np++Ni)) x
9
Diffusion
Atom Type Dynamics
Transformer
(Np + Nur) x 128
(Np +.r) 128
Denoising Network
PCM Fusion Layer
MLP
(Np + Nas) (128 + 128)
(Np Naz) 128
PSM Adapter Layer
MLP
(128 + +1
Nar x
Exy
Eggl
Namce cilckkeee
Nomud cilloiaas
Figure 5:
Ablation study on diffusion step number. We compare 5 experiments described in Sec. 5.3 in terms of best validation loss, QED, SA and Vina Score under different diffusion steps settings.
E MORE ABLATION STUDIES
Effect of Diffusion Steps We conduct a set of experiments: (1) Expl: the baseline model which is obtained by removing prior-conditioning and prior-shifting mechanisms from IPDIFF; (2) Exp2: the baseline model equipped with the self-conditioning mechanism which can be regarded as our prior-conditioning mechanism without IPNET, (3) Exp3: the baseline model equipped with the crior-conditioning mechanism as mentioned in Sec. 4.2.2. (4) Exp4: the bascline model equipped with the mhir-shifting mechanism as mentioned in Sec. 4.2.1, (5) Exp5: our IPD1FF, for comparing the performance of Expl-5 under different number of diffusion steps. We firstly train the models with 200, 400, 600, 800, 1000 diffusion steps, and then evaluate the generated molecules by sampling the same number of steps as training. As shown in Figure 5, the baseline model equipped with prior-conditionin and prior-shifting mechanism can achieve better validation loss under each setting of diffusion steps. Under the same setting, the validation loss can be viewed as surrogate of negative Evidence Lower Bound (ELBO) and lower validation loss means the model can better approximate the data distribution (Guan et al., 2023b). The fact that the model trained with fewer diffusion steps achieves lower validation loss is because it fits noises better at fewer time steps with limited model capacity. Moreover, the comparison results demonstrate that model equipped with wrior-conditioning and prior-shifting mechanism is able to generate high-quality ligand molecules (high QED and SA. low Vina) even with fewer sampling steps. And we can find that the number of diffusion steps increases, the baseline model equipped with prior-conditioning and prior-shifting mechanism improves more significantly on the Vina Score and SA.
Effect of the Shifting Scales As mentioned in Equation (6), the shift S, at the time step t consists of a coefficient k; and a 3-dim vector generated by a learnable neural network ((.), where the coefficient
k = 1 vā (1 v@) and y is a hyper- parameter to adjust the scale the shifts in the diffusion hyper-parameter
trajectory. To inverstigate the effect of different shifting scales,
set the 77 to 5 values: (1) 17 0.
(2) 7 = 0.1, (3) 7 1, (4) 7 = 10, (5) 1 = Vā (1 - Vā) and present the results in the Tab. It worth noting that 7) 0 indicates the prior-shifting mechanism is removed from IPDIFF. We
19

---

Published as a conference paper at ICLR 2024
found that reducing the shifting scales in the whole diffusion trajectories (n 0.1) can not bring gains to IPDIFF, and enlarging the shifting scales (17 = 10) even hurts the performance of IPDIFF in property-related metrics. While highlighting the shifting scales in the middle diffusion trajectories (1) v (1 vā)) will further improve the performance of IPDIFF on High Affinity. In practice, set 1) =
Table 7: The effect of the different shifting scales on binding-related metrics. (1) / (4) denotes larger smaller number is better. Top results are highlighted with bold text and underlined text, respectively.
Vina Score (J)
Vina Min (L)
Vina Dack ()) Med.
High Affinity (1)
Methods
Avg.
Med.
Avg.
Med.
Avg.
Avg.
Med.
baseline
-5.04
-5.75
-6,38
-6.52
-7.55
-7.72
54.2%
54.1%
7 =
-5.30
-6.17
-6.61
-6.81
-7.94
-8.01
60.6%
66.9%
0.1
-5.10
-5.93
-6,51
-6.63
-7.78
-7.91
62.7%
66.1%
7
-6.42
-7.01
-7.45
-7.48
-8.57
-8.51
69.5%
75.5%
10
-5.84
-6.41
-6.87
-6.90
-7.91
-8.07
60.9%
61.1%
7 Va v))
-6.03
-6.71
-7.26
-7.31
-8.48
-8.51
70.3%
74.0%
Reducing The Gap between Training and Sampling Processes As described in Secs. 4.2.1 and 4.2.2, the interaction prior utilized in the training and sampling processes of IPDIFF obtained in different manners. Specifically, in the training process of IPDIFF, the ground-trutt molecular protein-ligand pairs are fed into the pretrained IPNET obtain the interaction prior, while in the sampling process of IPDIFF, the molecules predicted in the previous time step and given condition pockets are utilized to obtain the interaction prior. In order to reduce the gap between the interaction prior in training and sampling processes, especially when the molecules cannot be accurately estimated with large noise during sampling process, we add Gaussian noises z AO, I²I) the position of each atom in molecules during the training of IPNET, and control the scale of noises through a. The experimental results are present in Tab. We observed that when no noises are introduced and the introduced noises too large or too small lead to limited improvements. In practice, we set the a 0.5.
Table 8: Effect of the noise scales in training process of IPNET. (t) / (C) denotes larger smaller number is better. Top results are highlighted with bold text and underlined text, respectively.
Vina Score (L) Avg.
Vina Min (L)
Vina Dock (4)
High Affinity (1)
Methods
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
baseline
-5.04
-5.75
-6.38
-6.52
-7.55
-7.72
54.2%
54. 14.1% %
0
-5.37
-6.47
-6.69
-6.81
-7.82
-7.90
60.1 %
58.2%
0.1
-5.63
-6.80
-7.10
-7.36
-8.17
-8.35
64.7%
67.0%
0.5
-6.42
-7.01
-7.45
-7.48
-8.57
-8.51
69.5%
75.5%
-3.89
-6.05
-6.20
-6.75
-7.84
-7.97
60.0%
58.4%
Performance of Generated Molecules with Different Number Rotatable Bonds The Figure presents the performance of generated molecules with the different number of rotatable bonds. And we can observed that our method can achieve superior performance compared with TargetDiff, even for the molecules with the large number of rotatable bonds (> 10),
Time Complexity For investigating the sampling efficiency, we report the inference time of our model and other baselines for generating 100 valid molecules on average. Pocket2Mol, TargetDiff' and DecompDiff use 2037s, 1987s and 3218s, and our IPDIFF takes 3063s.
F FUTURE WORK
In this paper, we introduce paradigm that combines a pretrained protein-ligand interaction model with the diffusion models for SBDD. In Tab. 3, we have present that our IPNET can effectively serve the downstream diffusion model even when it is solely pre-trained on sequential data. This is
20

---

Published as a conference paper at ICLR 2024
Engettel
Tergeita
Tugester ITTLE
IFYLE
STONT
1P94T
Mi
77744
Figure 6: The performance of generated molecules with different number of rotatable bonds.
significant because precise geometric data is often scaree. while sequence data is abundant. This finding suggests that we can further explore the integration of self-supervised or weakly-supervised learning techniques in the training IPNET solely relying on abundant sequential data, for extracting general interaction-relate information and effectively serving the downstream diffusion model. These aspects make our model scalable and open up new possibilities for leveraging large-scale sequence data in SBDD tasks. Moreover, It is necessary to evaluate our methods from different evaluation perspectives. We will provide comprehensive experiments and analysis on the other metrics, such PoseCheck (Harris et al., 2023) and PoseBusters (Buttenschoen al., 2023).
G
MORE RESULTS
We provide the visualization of more ligand molecules generated IPDIFF, comparing to both reference and TargetDiff (Guan et al., 2023a), as shown in Figure 7.
We provide the source files containing generated molecules and the evaluation code that reproduce the results Tab. 2 in the supplementary material.
We committed to open source the code of training and inference well the pretrained model upon paper acceptance.
21

---

Published as a conference paper at ICLR 2024
Zazy
Reference
DecompDiff
IPDIH
Vina: 8.54,
Vina: -.10, DED: 9.31,
Vina: -222 GED: 0.48,
Vinr: -11 0.74, SA:
Vinar -10.77 GED: 0.77, SA:
1phk
Reference
DecompDiff
IPDI!!
m.4.66.4.00 23,
4zfa
Reference
DicampDiff
IPDIt
1h0i
Reference
DecompDIff
IPDIt
6.58,
Vine: GED:
6.69,
Reference
DecompDiff
IPDIT!
Vina: 9.62, GED. 678, SA. 6:88
Vina: .7.77 GED 6.68. SA. 6.46
Vina: 2929, GED: 9.6, SAI
Vina: 8.66 GED: 0.69. 8A: 0.58
Viaa: -0 OED: 882, BA 0.58
Figure 7: More examples generated ligands for protein pockets Carbon atoms in reference ligands, ligands generated by DecompDiff (Guan al., 2023b) and IPDIFF are visualized in green, blue and orange, respectively. We report Vina Score, QED and SA for each molecule.
22


Reviews:
[{"review_id": "QJoVAxp6Ub", "reviewer": "Reviewer_ssPe", "summary": "The authors present IPDiff for structure based drug design - wherein they propose to factor receptor-ligand pocket interactions into the noising and de-noising of the diffusion models. They do so by training IPNet which models the receptor and ligand as graphs - and is trained to predict the binding affinity. The trained IPNet is then used towards what the authors term as prior shifting - where the diffusion trajectories of forward process are influenced by the interactions. The authors then design a prior-conditioning step to enhance the reverse process by conditioning the denoising of ligand molecules on the previously estimated protein-ligand interactions.", "questions": "Please address the concerns in the weaknesses section.\n\nAdditionally, in Section 3/ 4.1 - How are the receptor and ligand graphs constructed - is it based on atom connectivity or distances between atoms?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The ideas to perform prior shifting and prior-conditioning for structure based drug design are novel to the best of my knowledge.\n2. The proposed mechanism achieves SOTA performance on CrossDocked2020 benchmark, and is also able to generate\nthe molecules with -6.42 Avg. Vina Score (in comparison to -5.67 from prior baselines) while maintaining proper molecular properties. The authors also present the impact of prior shifting and conditioning in their model analysis section.", "weaknesses": "The clarity of the paper could be improved. Specifically\n1. The paper cites Sattoras et al - and says its an SE(3) Equivariant neural network - whereas it is E(N) GNN. This is important to consider in the context of protein molecules as chirality is an important property of protein molecules.\n2. The paper explicitly uses message passing only over the neighboring nodes as a part of IPNet. This is in contrast to E(N) GNN wherein, messages are passed between all pairs of nodes. Again this is crucial is this what ensures it is a rigid body and allows for E(3) equivariance/ invariances. The authors haven't show their version is SE(3) equivariant.\n3. In section 4.2.1 it is unclear how it S is trained to ensure the molecule is not distorted to impossible molecules - as its not a simple translation matrix but positions for every atom - and can lead to arbitrary invalid deformations.\n4. Using the pocket structure for the diffusion process is counter intuitive. When we start with the receptor for an unknown ligand (small molecules which we wish to discover) - the pocket or the holo structure of the receptor is not a single conformation (but an ensemble). Moreover, without the presence of the ligand - we only have access to the apo structure of the receptor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b7TCCWlL0X", "reviewer": "Reviewer_qHa3", "summary": "This work is focused on improving the quality of the generated molecular pose within the protein pocket by3D-diffusion protein-ligand models. Authors argue that the different utilisation of the protein-ligand interactions by the forward and reverse process impacts the quality of the  generated structures.  In particular this is because the differences of the pocket binding sites in training samples are neglected by the forward process while leveraged during the reverse process. Authors explored this problem by proposing a solution where they adapt the trajectories such that these contain information about the protein-ligand interactions into the forward process by altering the drift of the diffusion trajectory based on the interactions while conditioning the reverse diffusion trajectory on the estimates of the protein-ligand interaction. \n\nAuthors introduce atom-wise cross attention layer to model intra- and intermolecular interactions of protein ligand pairs which they use  foir pre-training and use the pretrained representations in the diffusion model. \nAuthors further  provide comparisons with various methods to demonstrate the strengths of their proposed approach.", "questions": "1. What is the difference in the chemical diversity of the datasets used for training IPNET and IPDfiff? e.g. different scaffolds, molecules with large flexibility, etc. Were the poses used in testing and evaluation of  IPDiff excluded from the training set for IPNet? \n\n2. How does the model perform when generating the molecules with larger numbers of rotatable bonds? How does the model generalises into unseen targets and novel pockets? \n\n3. What is the percentage of the  novel molecules generated by the IPDiff? \n\n4. IPDiff generates molecules in average 31s per molecule. How fast is the docking algorithm on the same hardware and similar sized structures?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is very clearly written, the authors identified an interesting problem for which they proposed a solution and demonstrated by well designed experiments, that their proposed solution works well. \n- The derivations of actions on the forward and backward diffusion kernels  and training objectives are provided in the Appendix, very clearly written and easy to follow. \n- The ablation study exploring the role of the scaller within the drift correction  of the forward process is provided.\n- The experiments are sufficiently described and most likely reproducible.", "weaknesses": "- There are minor typos on various places of the manustript. \n- The edge cases such as molecules with high number of rotable bonds are not investigated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9zWqcUAr2r", "reviewer": "Reviewer_AJPX", "summary": "In this paper, the authors propose to augment standard diffusion-based models for pocked-conditioned 3D ligand generation. The proposed approach, called IPDiff, leverages protein-ligand interactions in both the forward and reverse diffusion processes. The protein-ligand interaction is learned with an auxiliary network (called IPNet) and trained separately on a PDBBind. The authors show good results (on standard metrics) on CrossDocked2020 dataset.", "questions": "- Please see the comments on \"Weaknesses\" above.\n- What is the objective used to train IPNet? Was it mean square errors? What is the ground-truth label and what is its range? I feel some details about the training of IPNet are missing.\n- In the last paragraph of the Introduction, the authors say that “IPDiff is theoretically able to achieve better likelihood compared to previous diffusion models”. Can the authors elaborate on this? \n- Between equation 10 and 11, the authors mention that they need to approximate X_0^M and V_0^M (since we dont have access to ground truth during generation). How bad is this approximation? How much performance is lost because of this? Would it be possible to do some experiment to analyse it?\n- Based on the results on the tables of this paper, the proposed method has better metrics than the reference. This seems a bit strange to me and points to the fact that the metrics used are probably not very informative/useful. What does this mean in the opinion of the authors?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The main idea of the paper—to leverage protein-ligand interactions to improve generation of 3D ligands—is novel and well motivated.\n- The paper shows that ligand generation conditioned on pocket can be improved (at least in the case of diffusion models) if one can successfully leverage protein-ligand interactions.\n- The proposed approach achieves good results on benchmark metrics/datasets (however, this method requires labeled data and one extra trained network, so comparisons aren’t really apples-to-apples)", "weaknesses": "- The proposed method improves over the baseline (diffusion model conditioned on pocket) by relying on supervised learning on PDBbind. This makes the model more complicated to train/use and, more importantly, more difficult to scale. These points needs to be mentioned in the main paper. \n- IPNet is trained on PDBBind and then it is used as a feature extractor to the training of the diffusion model. What if PDBBind contains samples that are similar (or identical) to those on the test set of CrossDocked? If this is the case, the good performance could be due to some kind of “leaking” of information from the test set to the training set. The authors did not mention anything about this on the manuscript.\n- The authors use the same metrics reported on previous work. Although this is good for benchmarking, it is well now by the community that these metrics are not great. For example, recent work (eg, PoseCheck and PoseBusters) propose other metrics that can better assess quality of generated molecules. It would be very nice the authors would have shown results on these metrics. \n- Since the authors leverage protein-ligand interactions, it would be nice to compare how good the generated molecules are in terms of protein-ligand interactions with the pockets. Posecheck (mentioned above) proposes a metric to measure to compute this, but many other could be imagined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AJ1X1ZsZka", "reviewer": "Reviewer_F898", "summary": "This paper introduces a novel approach to 3D molecule generation for structure-based drug design using diffusion models. Recognizing the inconsistency in existing diffusion models that incorporate protein-ligand interaction information predominantly in the reverse process, the authors propose the Interaction Prior-guided Diffusion model (IPDiff). IPDiff integrates geometric protein-ligand interactions into both the diffusion and sampling processes. The methodology involves pretraining an interaction prior network (IPNet) using binding affinity signals, followed by leveraging this network to adjust molecule diffusion trajectories (termed \"prior-shifting\") and enhance the binding-aware molecule sampling process (referred to as \"prior-conditioning\"). Empirical studies on the CrossDocked2020 dataset demonstrate that IPDiff can effectively generate molecules with realistic 3D structures that exhibit superior binding affinities towards target proteins, while preserving essential molecular properties.", "questions": "See Weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The concept of \"Prior-Shifting\" is innovative. Adjusting the forward process based on specific data can indeed enhance the efficiency of reverse diffusion.\n\n2. The paper's presentation is well-structured and coherent.", "weaknesses": "1. The author's evaluation seems limited to the CrossDocked2020 benchmark. For a more comprehensive assessment, it would be beneficial to test the method across multiple benchmarks.\n\n2. “In the forward process, the ways of injecting noises are the same for all training samples with different target proteins.” While the statement mentions that the noise injection methods are consistent for all training samples with varying target proteins, it's essential to recognize that the forward diffusion process might differ. Even if the noise injection methods remain the same, the original samples' distinctiveness ensures varied forward diffusion outcomes.\n\n3. It's noteworthy that there have been prior studies that learned the beta parameters for forward diffusion. A comparative analysis with those works would add depth to this study.\n\n4. Based on\"Analog bits: Generating discrete data using diffusion models with self-conditioning,\" the novelty of the prior conditioning contribution in this paper appears less significant.\n\n5. I am also quite curious whether this prior-shifting idea can be used in class-conditional image generation as different classes are very different.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)