Paper: Protein-Ligand Interaction Prior for Binding-aware 3D Molecule Diffusion Models (ICLR-2024)

Reviewer comment (Experiments):
The author's evaluation seems limited to the CrossDocked2020 benchmark. For a more comprehensive assessment, it would be beneficial to test the method across multiple benchmarks.

Author response:
A1: We added a comparison to BindingMOAD dataset which is used in DiffSBDD [1], and the other SBDD methods only conducted comparisons on the CrossDocked2020. We preprocessed the data following DiffSBDD. The results are shown in the following table. The results demonstrate that our IPDiff can achieve the best performance by comparing with DiffSBDD and TargetDiff [2].

| Methods    | Vina Score (-) | QED (+)  | SA (+)   | Diversity (+) |
| ---------- | -------------- | -------- | -------- | ------------- |
| DiffSBDD   | -5.20          | 0.47     | 0.57     | **0.91**      |
| TargetDiff | -5.49          | 0.46     | 0.56     | 0.89          |
| IPDiff     | **-6.11**      | **0.50** | **0.61** | 0.88          |

Reviewer comment (Theory):
“In the forward process, the ways of injecting noises are the same for all training samples with different target proteins.” While the statement mentions that the noise injection methods are consistent for all training samples with varying target proteins, it's essential to recognize that the forward diffusion process might differ. Even if the noise injection methods remain the same, the original samples' distinctiveness ensures varied forward diffusion outcomes.

Author response:
A2: We want to convey that in previous diffusion models, the noise added to the atom of ligand is sampled from a normal distribution that is independent of the given pocket. In contrast, our method takes into account the interaction information between the protein and ligand during the noise addition process.

Reviewer comment (Experiments):
It's noteworthy that there have been prior studies that learned the beta parameters for forward diffusion. A comparative analysis with those works would add depth to this study.

Author response:
A3: We conduct a experiment that makes the beta parameter in TargetDiff learnable. The performance comparison between TargetDiff, TargetDiff with learnable beta and our IPDiff are present below. The results demonstrate that simply make beta parameters learnable for the forward diffusion can not improve the performance of diffusion models for SBDD.


| Methods                     | Vina Score (-) | QED (+)  | SA (+) |
| --------------------------- | -------------- | -------- | ------ |
| TargetDiff                  | -5.47          | 0.48     | 0.58   |
| TargetDiff + learnable beta | -5.33          | 0.49     | 0.58   |
| IPDiff                      | **-6.42**      | **0.52** | **0.61**   |

Reviewer comment (Novelty):
Based on "Analog bits: Generating discrete data using diffusion models with self-conditioning," the novelty of the prior conditioning contribution in this paper appears less significant.

Author response:
A4: The novel technical contributions of our proposed prior-conditioning mechanism are: (1) It incorporates protein-ligand interaction information modeled in the pretrained prior network into the molecular diffusion model while self-condition can only consider self-prediction molecular information and is unable to model the critical protein-molecule geometric interactions. 
(2) It introduces a new paradigm that combines the protein-ligand pretraining with protein-conditioned molucular diffusion model, fundamentally different from previous diffusion-only methods. 
And we have conducted a quantitative comparison between our prior-conditioning and self-conditioning. The results in **Table 4** of the manuscript demonstrate that self-conditioning is not effective in SBDD tasks since it can not provide the interaction information between ligands and proteins.

Reviewer comment (Experiments):
I am also quite curious whether this prior-shifting idea can be used in class-conditional image generation as different classes are very different.

Author response:
A5: From a broader perspective, IPDiff represents a cross-modal conditional diffusion model, which is designed to emphasize the interaction between different modalities in the diffusion process and utilize it as a condition. We conduct a preliminary experiment on CIFAR-10 where we incorporate class-image semantic interactions (we obtain the shifts from the combination of class embeddings and image embeddings) into the image diffusion process, the results are in the table below. We can find that in class-conditional image generation, our model can still improve the baseline diffusion model, demonstrating the satisfying generalization ability of our alrorithm.

| Methods                               | FID $\downarrow$ |
| ------------------------------------- | -------- |
| DDPM                                  | 3.17     |
| DDPM + class-image correlation (Ours) | **2.95** |

Reviewer comment (Experiments):
The proposed method improves over the baseline (diffusion model conditioned on pocket) by relying on supervised learning on PDBbind. This makes the model more complicated to train/use and, more importantly, more difficult to scale. These points needs to be mentioned in the main paper.

Author response:
A2: Indeed, we utilize the supervision of binding affinity in PDBBind during training. However, we do not think that this will make our model difficult to use or train. It is important to note that the binding affinity is only used as a supervision in the training of IPNet, and the diffusion model in the second stage does not rely on binding affinity. The purpose of IPNet is to mine some general interaction clues, and we can arbitrarily design the diffusion model without changing the IPNet. **Additionally, our method introduces a paradigm that combines a pretrained protein-ligand interaction model with diffusion models for SBDD, which has a boarder impact on AI-for-science community.** We have conducted experiments to demonstrate that our IPNet can effectively serve the downstream diffusion model even when it is solely pre-trained on sequential data (please refer to **Table 3** in the manuscript). This is significant because precise geometric data is often scarce, while sequential data is abundant. This finding suggests that we can further explore the integration of Self-Supervised Learning or Weakly-Supervised Learning techniques in the training of IPNet solely relying on abundant sequential data, for extracting general interaction-related information and effectively serving the downstream diffusion model. These aspects make our model scalable and open up new possibilities for leveraging large-scale sequential data in SBDD. We are grateful for your insightful suggestion, and we thoroughly explained and discussed it in the **Appendix-F** revised manuscripted as part of our future work.

Reviewer comment (Experiments):
IPNet is trained on PDBBind and then it is used as a feature extractor to the training of the diffusion model. What if PDBBind contains samples that are similar (or identical) to those on the test set of CrossDocked? If this is the case, the good performance could be due to some kind of “leaking” of information from the test set to the training set. The authors did not mention anything about this on the manuscript.

Author response:
A3: We have **excluded the molecules used in testing and evaluation of IPDiff from the PDBBind v2016 dataset for IPNet** according to the calculated Tanimoto similarity between two molecules. After that, the average Tanomoto similarity between these datasets is only **0.12\%**. Additionally, we have also calculated the Tanimoto similarity between the molecules generated by IPDiff and the molecules in the training of IPNet, and the average Tanimoto similarity is only **0.079\%**. Based on these, we believe that there is no issue of information leakage in our method.

Reviewer comment (Evaluation):
Since the authors leverage protein-ligand interactions, it would be nice to compare how good the generated molecules are in terms of protein-ligand interactions with the pockets. Posecheck (mentioned above) proposes a metric to measure to compute this, but many other could be imagined.

Author response:
A2: We report the mean value of Clashes and Interactions (No. Acceptor, No. Donors, vdWs and Hydrophobic interactions), and the median value for Strain Energy due to its large ranges as mentioned in PoseCheck [8]. The complete comparisons between IPDiff, TargetDiff, DecompDiff, and reference are presented as follows:

For the generated conformations of ligands:

| PoseCheck  | Clashes (-) | Strain Energy  (-) | No. Acceptor (+) | No. Donors (+) | vdWs (+) | Hydrophobic interactions (+) |
| ---------- | :----: | :----: | :----: | :----: | :----: | :----: |
| TargetDiff | 10.7| 1404.8 | 0.87 | 0.33 | 7.7| 5.5 |
| DecompDiff | 8.3 | 1204.1| 0.83  | 0.36| 7.2| 4.2|
| IPDiff     | **8.1**| **1105.9** | **0.95**| **0.42** | **8.0**  | **5.8**|

For the redocked comformations of ligands:

| PoseCheck  | Clashes (-) | No. Acceptor (+) | No. Donors (+) | vdWs (+) | Hydrophobic interactions (+) |
| ---------- | :----:| :----: | :----: | :----: | :----: |
| Reference  | 5.3         | **2.12**         | **1.31**       | 6.7      | 4.7  |
| TargetDiff | 4.8         | 0.96             | 0.44           | 6.9      | 5.3 |
| DecompDiff | 3.7         | 0.71             | 0.28           | 5.3      | 4.8  |
| IPDiff     | **3.4**     | 0.99             | 0.48           | **7.4**  | **5.7** |


Our method outperforms its counterparts, TargetDiff and DecompDiff, in almost all metrics, which has demonstrated the effectiveness of our method. Notably, the evaluation metrics used in our paper and in the rebuttal assess the quality of the generated molecules from different aspects, and reveal the consistent superiority of our method over previous works.

Reviewer comment (Reproducibility):
What is the objective used to train IPNet? Was it mean square errors? What is the ground-truth label and what is its range? I feel some details about the training of IPNet are missing.

Author response:
A5: We have reported more details about the training of IPNet in the **Appendix-D.1 of the updated manuscript marked in red**, please refer to it.

Reviewer comment (Experiments):
Between equation 10 and 11, the authors mention that they need to approximate X_0^M and V_0^M (since we dont have access to ground truth during generation). How bad is this approximation? How much performance is lost because of this? Would it be possible to do some experiment to analyse it?

Author response:
A7: Approximating $X_0^M$ and $V_0^M$ is a necessary step in the original diffusion model, and the closer the time step t is to T, **the closer the $X_0^M$ approximated by the model at time step t is to the noise**. In order to further assess how much performance lost in our IPDiff brought by the inaccurate approximation and interaction prior, we conducted two experiments: (1) Exp1: we use reference ligands to extract the interaction prior in the first 200 time steps of the sampling process, and the remaining steps were consistent with the original sampling process in IPDiff; (2) Exp2: we use randomly generated atomic positions and types to extract interaction prior for IPDiff in the first time 200 steps of the time step. The performance are presented below. We can observe that the vina score of Exp1 is slightly improved, but the diversity is simultaneously decreased. And the performance of Exp2 is largely degraded. 
In conlusion, although the interaction prior introduced by the approximated $X_0^M$ and $V_0^M$ is not optimal, such approximation can still bring effective interaction clues for facilitating protein-based molecular generation.

| Methods | Vina Score (-) | Diversity (+) |
| ------- | -------------- | ------------- |
| IPDiff  | -6.42          | **0.74**      |
| Exp1    | **-6.45**      | 0.71          |
| Exp2    | -4.30          | **0.74**      |

Reviewer comment (Evaluation):
Based on the results on the tables of this paper, the proposed method has better metrics than the reference. This seems a bit strange to me and points to the fact that the metrics used are probably not very informative/useful. What does this mean in the opinion of the authors?

Author response:
A8: The reference ligands in CrossDocked are obtained by recombining protein-ligand pairs from PDBBind and then redocking them. However, some of these reference ligands do not bind perfectly with proteins, indicating that their performance does not represent the upper bound. And the similar comparsion results between the generated molecules with the references are also presented in previous methods [1-3].

[1] E(n) Equivariant Graph Neural Networks, ICML, 2021.

[2] 3d Equivariant Diffusion for Target-aware Molecule Generation and Affinity Prediction, ICLR, 2023a.

[3] DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design, ICML, 2023.

[4] PoseCheck: Generative Models for 3D Structure-based Drug Design Produce Unrealistic Poses, NeurIPS Workshop, 2023.

Reviewer comment (Experiments):
What is the difference in the chemical diversity of the datasets used for training IPNET and IPDfiff? e.g. different scaffolds, molecules with large flexibility, etc. Were the poses used in testing and evaluation of  IPDiff excluded from the training set for IPNet?

Author response:
**Q1: What is the difference in the chemical diversity of the datasets used for training IPNET and IPDfiff? Were the poses used in testing and evaluation of IPDiff excluded from the training set for IPNet?**

A1: CrossDocked2020 (the dataset used by IPDiff) is an expanded dataset obtained by recombining protein-ligand pairs from PDBBind (the dataset used by IPNet) and then redocking them. **Therefore, the chemical diversity of these two datasets is consistent.
And we excluded the molecules used in testing and evaluation of IPDiff from the PDBBind v2016 dataset for IPNet** according to the calculated Tanimoto similarity between two molecules. After that, the average Tanomoto similarity between these datasets is only 0.12\%. Additionally, we have also calculated the Tanimoto similarity between the molecules generated by IPDiff and the molecules in the training of IPNet, and the average Tanimoto similarity is only **0.079\%**. Based on these, we believe that there is no issue of information leakage in our method.

Reviewer comment (Evaluation):
What is the percentage of the  novel molecules generated by the IPDiff?

Author response:
**Q3: What is the percentage of novel molecules generated by the IPDiff?**

A3: The percentage of novel molecules generated by the IPDiff is 99.999971\%.

Reviewer comment (Experiments):
IPDiff generates molecules in average 31s per molecule. How fast is the docking algorithm on the same hardware and similar sized structures?

Author response:
**Q4: How fast is the docking algorithm on the same hardware and similar sized structures?**

A4: The average time consumed for docking a molecule to the given pocket using the Vina Dock algorithm on the same hardware is 27.9 seconds.

Reviewer comment (Theory):
The paper cites Sattoras et al - and says its an SE(3) Equivariant neural network - whereas it is E(N) GNN. This is important to consider in the context of protein molecules as chirality is an important property of protein molecules.

Author response:
A1: Thank you for your feedback. The architecture design of networks is not our primary focus of our paper. The backbone of networks utilized in our method just following with previous work [1-3]. And we acknowledge that EGNN [1] we used is an E(3)-equivariant neural network and chirality is also an important property of protein molecules. However, the input of the EGNN is a protein-ligand complex. Thus, for example, given a protein pocket (which is not symmetric in general), **a pair of stereoisomers will not share the same representation. This also follows TargetDiff [2]**. We appreciate your suggestion and will consider it in our future work.

Reviewer comment (Theory):
The paper explicitly uses message passing only over the neighboring nodes as a part of IPNet. This is in contrast to E(N) GNN wherein, messages are passed between all pairs of nodes. Again this is crucial is this what ensures it is a rigid body and allows for E(3) equivariance/ invariances. The authors haven't show their version is SE(3) equivariant.

Author response:
A2: The graph in our IPNet is fully-connected, as detailed in the **Appendix-D.1**. 
However, even using the KNN graph, IPNet still maintains SE(3) equivariance. The proofs are provided as below:
Since atom types are always invariant to SE(3)-transformation during the generative process, we only need to consider the atom coordinates.
Recall the **equation 13 in the manuscript**, and we denoted $x_{t,i}^{C,l}- x_{t,j}^{C,l}$ as $d^{C, l}_{t,ij}$ for simplification. The atom hidden embeddings $h$ and coordinates $x$ can be updated as follow:

$$\begin{aligned}
h_{t,i}^{C,l+1}&=h_{t,i}^{C,l}+\sum_{j\in \mathcal N_C(i)}f_{h}^{C,l}(d_{t,ij}^{C,l},h_{t,i}^{C,l},h_{t,j}^{C,l},e_{ij}^{C}) \\\\
x_{t,i}^{C,l+1}&=x_{t,i}^{C,l}+\sum_{j\in \mathcal N_C(i)}(x_{t,i}^{C,l}-x_{t,j}^{C,l})f_{x}^{C,l}(d_{t,ij}^{C,l},h_{t,i}^{C,l+1},h_{t,j}^{C,l+1},e_{ij}^{C})\cdot\mathbb 1_{mol}
\end{aligned}$$

First, it is easy to see $d_{ij}$ does not change with the 3D roto-translation $T_g$:
$$\begin{aligned}
\hat{d_{ij}^2} &= \left\Vert T_g(x_i) - T_g(x_j) \right\Vert^2 = \left\Vert (\mathbf R x_i + \mathbf b) - (\mathbf R x_j + \mathbf b) \right\Vert^2 = \left\Vert \mathbf R x_i - \mathbf R x_j \right\Vert^2 \\\\
&=(x_i - x_j)^T \mathbf R^T \mathbf R(x_i - x_j) =(x_i - x_j)^T\mathbf I(x_i - x_j) = \left\Vert x_i - x_j \right\Vert^2 = d^2_{ij}
\end{aligned}$$
where $T_g$ is the group of SE(3)-transformation, which can be written explicitly as $T_g(x) = \mathbf R x + \mathbf b$, $\mathbf R \in \mathbb R^{3\times3}$ is the rotation matrix and $\mathbf b \in \mathbb R^3$ is the translation vector.
Since $h_i$, $h_j$, $e_{ij}$ are initially obtained from atom and edge features, which are invariant to SE(3) transformation, we have $h_i^l$ is SE(3)-invariant for any $l=1,...,L$.

Then, we can prove that $x$ updated from **equation 13** is SE(3)-equivariant as follows:
$$\begin{aligned}
\phi_{\theta 1}(T(x_t^{C,l})) &= T(x_{t,i}^{C,l}) + \sum_{j \in \mathcal N_C(i)}\left(T(x_{t,i}^{C,l})-T(x_{t,j}^{C,l})\right)f_{x}^{C,l}\left(d_{t,ij}^{C,l},h_{t,i}^{C,l+1},h_{t,j}^{C,l+1},e_{ij}^{C}\right)\cdot \mathbb 1_{mol} \\\\
&=\mathbf R x_{t,i}^l + \mathbf b + \sum_{j \in \mathcal N_C(i)}\mathbf R(x_{t,i}^{C,l}-x_{t,j}^{C,l})f_{x}^{C,l}\left(d_{t,ij}^{C,l},h_{t,i}^{C,l+1},h_{t,j}^{C,l+1},e_{ij}^{C}\right)\cdot \mathbb 1_{mol} \\\\
&=\mathbf R (x_{t,i}^l + \sum_{j \in \mathcal N_C(i)}\mathbf R(x_{t,i}^{C,l}-x_{t,j}^{C,l})f_{x}^{C,l}\left(d_{t,ij}^{C,l},h_{t,i}^{C,l+1},h_{t,j}^{C,l+1},e_{ij}^{C}\right)\cdot \mathbb 1_{mol}) + \mathbf b \\\\
&= \mathbf R(x_{t,i}^{C, l}) + \mathbf b \\\\
&= T(\phi_{\theta 1}(x_t^{C, l}))
\end{aligned}$$
where $\phi_{\theta 1}$ is our IPNet.

Reviewer comment (Reproducibility):
In section 4.2.1 it is unclear how it S is trained to ensure the molecule is not distorted to impossible molecules - as its not a simple translation matrix but positions for every atom - and can lead to arbitrary invalid deformations.

Author response:
A3: The original diffusion process inherently introduces noise and denoises to the position of each atom, which can be treated as a shifting process to position of each atom in the previous step. Current molecular diffusion models [1-3] designs reconstruction-based training objective to maximally ensure that the molecule is not distorted to impossible molecules. We follow this and further enhance this shifting process by incorporating protein-ligand interaction prior obtained from a pre-trained IPNet model. These interaction prior preserves informative geometric relationships between intra-molecules and inter-molecules, thus facilitating the 3D molecular generation process. Theoretical proofs are provided in the **revised Appendix-B.4**, which demonstrate the potential of our method to achieve a better likelihood compared with previous diffusion-based methods. Moreover, the $S_t$ in our method represents the cumulative shift as time step $t$, which is 0 at the beginning and end of the diffusion process by setting the coefficient $k_t$ of $S_t$ as $\sqrt{\bar{\alpha}_t} \cdot (1-\sqrt{\bar{\alpha}_t})$. It means that our method specifically focuses on affecting the positions of each atom in the intermediate steps of the diffusion process without changing the initial and target distributions.

Reviewer comment (Theory):
Using the pocket structure for the diffusion process is counter intuitive. When we start with the receptor for an unknown ligand (small molecules which we wish to discover) - the pocket or the holo structure of the receptor is not a single conformation (but an ensemble). Moreover, without the presence of the ligand - we only have access to the apo structure of the receptor.

Author response:
A4: Indeed, as you rightly point out, it is necessary and practical to consider apo/holo structures. However, to the best of our knowledge, there are no existing methods for SBDD take apo structures into consideration. The main reasons are that these data is difficult to obtain, and its distribution is very unique, making this task extremely challenging. Therefore, we just follow the task definition and experimental setup as existing computational methods for SBDD [2-8], only considering holo structures. We really appreciate your suggestions and will consider them for future work.

Reviewer comment (Reproducibility):
In Section 3/ 4.1 - How are the receptor and ligand graphs constructed - is it based on atom connectivity or distances between atoms?

Author response:
A5: Yes, in our paper, we follow previous methods like TargetDiff and DecompDiff, representing each atom as a node and encoding the inter-atom distances as edge attributes in the construction of Protein-Ligand graphs.