Paper: Molecular Conformation Generation via Shifting Scores (ICLR-2024)

Reviewer comment (Novelty):
*Mischaracterization of prior works*. I do not understand the benefit of diffusion on interatomic distances. The authors claim GeoDiff assumes distances follows a gaussian distribution. Having read GeoDiff, I do not see this assumption. GeoDiff applies diffusion directly on atomic positions. More so, GeoDiff achieves SE(3) equivariance.

Author response:
In GeoDiff, authors stated that under the common assumption that $d$ also follows a Gaussian distribution (p7 Chain-rule approach before Sec 4.4). GeoDiff injects the noise on the coordinates and then transforms the distribution of the coordinates into the distribution of the distance. By such a transformation, GeoDiff attains the SE(3) equivalence. We suggest reviewer go through the official code of GeoDiff to have a better understanding of GeoDiff's implementation.

Reviewer comment (Experiments):
*Missing baseline*. Torsional diffusion has been out for more than a year and achieves state-of-the-art results on the baselines considered in this work. There is no mention of this paper in the related works and it is missing from the baselines. This is a major red flag.

Author response:
We strongly suggest reviewers go through the **whole** paper more carefully. We mentioned Torsional diffusion in Introduction (paragraph 2, line 3-4) and Related work (line 6). Even in Appendix F, **we use a full page to discuss our work and Torsional diffusion**. Our modeling method is quite different from Torsional diffusion. Our model is purely data-driven while Torsional diffusion uses strong chemical prior. We think that chemical prior can greatly enhance the model's performance. In [1], the authors claim that using the RDkit and a simple clustering algorithm can surpass most of the deep learning models. Hence, we think that including Torsional diffusion is not fair and did not include modeling methods that use strong prior in the comparison.

Reviewer comment (Experiments):
*Unconvincing results*. The authors claim a new state-of-the-art results. However, they leave out a important baseline and the improvements are extremely small in Table 1. The benefits of SDDiff compared to GeoDiff are within noise. The improvement is at most 0.08 on metrics... Furthermore more difficult benchmarks have been released since [2] almost a year ago. Evaluating against GeoDiff when GeoDiff already achieves 95% seems like the wrong problem to be focusing on.

Author response:
The main contribution of this work is to propose a new shifting distribution for a more accurate approximation of the distance distribution, we did not greatly change the modeling method (i.e., model architecture, modeling manifold) for a fair comparison. Hence, the improvement may not seem to be very large. About the benchmarks released in [2], it is essentially the same as the benchmarks in our paper but with a different error tolerance. We set a toy Drugs dataset that only contains 1/10 of the original training set and the same test set as SDDiff. We trained 2 models whose distance distribution hypotheses are Gaussian and the shifting distribution, respectively. The results can be seen in the following table:
   |                       | GEOM-QM9 |        |         |        | GEOM-Drugs |        |         |        |
|:---------------------:|:--------:|:------:|:-------:|:------:|:----------:|:------:|:-------:|:------:|
|         Method        |  COV(%)↑ |        | MAT(Å)↓ |        |   COV(%)↑  |        | MAT(Å)↓ |        |
|                       |   Mean   | Median |   Mean  | Median |    Mean    | Median |   Mean  | Median |
| Gaussian distribution |   85.51  |  89.30 |  0.2495 | 0.2378 |    86.02   |  94.24 |  0.9017 | 0.8963 |
| shifting distribution |   86.86  |  90.60 |  0.2397 | 0.2365 |    88.36   |  97.06 |  0.8940 | 0.8887 |

Reviewer comment (Theory):
*Unexplained analysis*. Section 4.5 is confusing to me. The first part regarding the marginal vs. joint seems to be saying the dependence between distances can be thrown out. This is done without explanation other than a hypothesis and throwing this out makes diffusion on distances no different than diffusion on particles to me. Even in the introduction, diffusion on distances is motivated through the dependence on interatomic forces so throwing them out seems to go against the original motivation. Furthermore, the approximation to OT is very brief and I did not understand the point here.

Author response:
To clarify, in this work, **we do not apply the diffusion on distances**. We apply diffusion on coordinates, model the distribution of distances, and then transform this distribution back to the distribution of the coordinates. Since it is extremely difficult to model the dependence of the distances, hence, we hypothesis that this dependence has a limited infect on the final results. In the introduction, our motivation is that by applying diffusion on coordinates, the relative ``speed'' of atoms can be considered as the MB distribution which is the distribution of the speed of particles when particles move freely with the bare occurrence of collisions.

Reviewer comment (Presentation):
Why are there negative distances for the gaussian pdf in figure 2?

Author response:
The x-axis in Figure 2 is the values of $\tilde{d} - d$, which can be negative. We will clarify this misleading in our final version.

Reviewer comment (Novelty):
The SDDiff authors claim their method is useful in achieving SE(3) equivariance but why is this novel if GeoDiff and related works can already do so?

Author response:
The main contribution of this paper is that, while previous work achieved SE(3)-invariance by transforming the conformation distribution to distance distribution, they mainly rely on heuristic assumptions such as the distribution of the distance being Gaussian. In this work, we more carefully model the distribution of distances and prove that by using the proposed distribution, we can enhance the model's performance.

Reviewer comment (Reproducibility):
How is p_t sampled? This and the training procedure are not specified. What is the \sigma schedule?

Author response:
We stated the loss function in Eq. 9. We can train a model $s_{\boldsymbol{\theta}}$ which minimizes the Eq. 9 and sample data following Algorithm 1. The training procedure and the sampling procedure have no difference compared with the standard diffusion model, which is also similar to the diffusion introduced in GeoDiff. The novelty of this work is that the proposed diffusion distribution is not studied in previous literature. $\sigma$ schedule is a hyperparameter that controls how much noise is to be injected in the diffusion forward process at time $t$. Our selection of the $\sigma$ scheduler can be found in Appendix D. The maximum value of $\sigma$ is 12.

Reviewer comment (Evaluation):
The main weakness from my perspective is the significance of empirical comparison. The improvement over GeoDiff is not significant to me.

Author response:
SDDiff outperforms GeoDiff greatly when the dataset is small. We set a toy Drugs dataset that only contains 1/10 of the original training set and the same test set as SDDiff. We trained 2 models whose distance distribution hypotheses are Gaussian and the shifting distribution, respectively. The results can be seen in the following table:
|                       |  GEOM-QM9  |        |         |        | GEOM-Drugs |        |         |        |
|:---------------------:|:----------:|:------:|:-------:|:------:|:----------:|:------:|:-------:|:------:|
|         Method        | COV(%)↑    |        | MAT(Å)↓ |        |   COV(%)↑  |        | MAT(Å)↓ |        |
|                       |    Mean    | Median |   Mean  | Median |    Mean    | Median |   Mean  | Median |
| Gaussian distribution |    85.51   |  89.30 |  0.2495 | 0.2378 |    86.02   |  94.24 |  0.9017 | 0.8963 |
| shifting distribution |    86.86   |  90.60 |  0.2397 | 0.2365 |    88.36   |  97.06 |  0.8940 | 0.8887 |

Reviewer comment (Experiments):
Could the author provide more ablation study about the $f_\sigma$ function in Eq7, which can help to verify the importance of the proposed MB diffusion distribution.

Author response:
SDDiff outperforms GeoDiff greatly when the dataset is small. We set a toy Drugs dataset that only contains 1/10 of the original training set and the same test set as SDDiff. We trained 2 models whose distance distribution hypotheses are Gaussian and the shifting distribution, respectively. The results can be seen in the following table:
|                       |  GEOM-QM9  |        |         |        | GEOM-Drugs |        |         |        |
|:---------------------:|:----------:|:------:|:-------:|:------:|:----------:|:------:|:-------:|:------:|
|         Method        | COV(%)↑    |        | MAT(Å)↓ |        |   COV(%)↑  |        | MAT(Å)↓ |        |
|                       |    Mean    | Median |   Mean  | Median |    Mean    | Median |   Mean  | Median |
| Gaussian distribution |    85.51   |  89.30 |  0.2495 | 0.2378 |    86.02   |  94.24 |  0.9017 | 0.8963 |
| shifting distribution |    86.86   |  90.60 |  0.2397 | 0.2365 |    88.36   |  97.06 |  0.8940 | 0.8887 |

Reviewer comment (Theory):
I feel the direct perturbation in distances will also potentially result in infeasible geometry.

Author response:
When the noise is injected into the coordinates, if the noise is small, then the distances follow a Gaussian distribution as proposed by GeoDiff. However, if the noise is large enough, the distribution of the distances follows the MB distribution. That's why we proposed a shifting distance distribution between Gaussian and MB distribution for a more accurate approximation of the exact distribution. Directly injecting noise on the distance results in infeasible geometry. This method was proposed in ConfGF, which was mentioned in related works. From the experimental results, SDDiff and GeoDiff are better than ConfGF.

Reviewer comment (Theory):
Probably my biggest problem with the paper is the motivation for using the Maxwell-Boltzmann distribution (MBD). The MBD describes the distribution of the length of velocity vectors in an ideal gas, and to some good approximation even in a real molecule. The authors generate molecular conformations based on distances, velocities are never generated nor used (e.g., the math on page 5 after eq 7 only includes distances). It is also unclear how interatomic distances (which are by definition purely positive) or the length of velocities could ever follow a Gaussian distribution. Therefore, the appearance and meaning of velocity v in eq 5 needs to be explained clearly.

Author response:
We are not claiming that the distances between atoms follow the Gaussian distribution, we are proving that under small perturbations, the change of the distances, i.e. $\tilde{d} - d$ follows the Gaussian distribution. Also, $d$ nor $\tilde{d}$ is relative to the concept of speed. However, $\tilde{d} - d$ is relative to the speed since it measures the change of the distances.

Reviewer comment (Experiments):
Regarding Section 4.5, if a molecule were to break apart, there will be some correlation of atom-atom distances, i.e., if an atom's distance to another one which is far away in the graph increases this will almost certainly also mean that the distance of the neighboring atom to the far away atom increases. This is more correlation than causation, but it does contain information. Additionally, instead hypothesizing can the authors at least show empirically that their statement is true?

Author response:
We generate samples $\tilde{d}_1, \tilde{d}_2, d_1, d_2$, where $d_1, d_2$ are the edge lengths of two incident edges and $\tilde{d}_1, \tilde{d}_2$ are edge lengths after perturbation on coordinates. We compute the Pearson r correlation between random variables $\tilde{d}_1 - d_1$ and $\tilde{d}_2 - d_2$. The results show that under different levels of $\sigma_t$, the correlation coefficient is always round 0.24. Hence, we ignore such a correlation.

Reviewer comment (Evaluation):
Regarding the measures COV and MAT. If we assume that a molecule has, e.g., 3 major conformers which are all very close in RMSD. Wouldn’t a model that samples always only one conformer achieve a COV of 1, even though it has never generated the other 2? Also, the definition of MAT seems odd, is there sum over S_r and maybe a min() missing?

Author response:
The COV and MAT measures can cause this problem. We notice that COV-P and MAT-P can reflect the quality of all generated conformers, we will include these metrics later. And thanks for pointing out the wrong definition of MAT in our paper. It should be $\operatorname{MAT-R} =\frac{1}{|S_r|} \sum_{\mathcal{C} \in S_r} \min_{\mathcal{C}' \in S_g} \operatorname{RMSD} (\mathcal{C}, \mathcal{C}')$. we have revised it.

Reviewer comment (Evaluation):
On page 9: The authors write that it is evident that the proposed distribution matches the Gaussian closely, however, in Fog. 4 the orange distribution appears to be tri-modal. Can the authors compute the overlap of orange and blue for the two values of sigma?

Author response:
The orange distribution is not tri-modal, it is unimodal, and the distribution is very close to Gaussian.

Reviewer comment (Theory):
In Section 4.2, even though D is defined as image(d), R^(n×3)/SE(3) is not isomorphic to D: a molecule and its mirror image would have the same distance matrices, but they are not the same element of R^(n×3)/SE(3) if they are chiral. In this regard, it would be valuable for the authors to clarify how the proposed method handles the generation of conformers for stereoisomers or enantiomers.

Author response:
When we say the distance matrices, we are referring to the adjacent matrix, i.e., we consider a fully connected graph. Although two chiral molecules have the same set of bound lengths, their adjacent matrices are different. Hence, $D$ is isomorphic to $\mathbb{R}^{n \times 3} / \operatorname{SE(3)}$ if we define $D$ to be the manifold of adjacent matrices (not the manifold of bond length).

Reviewer comment (Evaluation):
In Section 5.1, the COV and MAT metrics introduced correspond to the “Recall” version (COV-R and MAT-R, without the typo mentioned below). Some of the baseline methods under comparison, such as GeoDiff (Xu et al., 2022) and Torsional Diffusion (Jing et al., 2022), also include the “Precision” version (COV-P and MAT-P) to assess the quality of the generated conformers. The authors should include the “Precision” metrics in Table 1 as well.

Author response:
Thanks for your suggestion. we will include these metrics later.

Reviewer comment (Writing):
Especially in the introduction several papers are missing the year of publication, e.g., Xu et al., Jing et al, Zhu et al., it is therefore not clear which paper is being cited and if multiple occurrences denote the same paper.

Author response:
Thanks for your pointing out it. We have revised the references.

Reviewer comment (Theory):
What do the authors mean by marginal distribution of interatomic distances? If the full set of 3N(3N-1)/2 number of distances are included, this distribution would be even higher dimensional than the 3N-dimensional Boltzmann distribution of Cartesian coordinates.

Author response:
We refer $p(d_{ij})$ as the marginal distribution of interatomic distances where $d_{ij}$ is the distance among the $i$-th node and the $j$-th node. Indeed such modeling would increase the modeling manifold but still this is the so fared best choice to avoid directly dealing with the SE(3)-invariance in Cartesian coordinates.

Reviewer comment (Theory):
It is not clear how equation 7 follows from 6 nor how it justifies to “simply” use a Gaussian kernel.

Author response:
Eq.6 gives constraints that the distribution of perturbations on distance is proportional to the normal distribution when $\sigma=\sigma_0$, and proportional to MB distribution when $\sigma=\sigma_T$ We construct $f_\sigma(\tilde{d}, d)=1-e^{-\sigma / d}$, then substitute it into $p_{\sigma_t}(\tilde{d} \mid d) \propto \tilde{d}^{2 f_\sigma(\tilde{d}, d)} e^{-\frac{(\tilde{d}-d)^2}{4 \sigma_t^2}}$ and then we can obtain the score function shown as Eq.7 (the $\nabla_{\tilde{d}} \log q_\sigma(\tilde{d} \mid d)$ should be $\nabla_{\tilde{d}} \log p_\sigma(\tilde{d} \mid d)$, we have fixed it).

The justification to "simply" use a Gaussian kernel is detailed after Eq 7 in those unnumbered equations. We perturb the coordinates with Gaussian noise, then the conditional perturbed inter-atomic distance are Gaussian and MB distributions.

Reviewer comment (Writing):
On the bottom of page 5, where the authors state that n has to be greater-equal to 5. It would be good to mention there that “Each individual atom must be associated with a minimum of four distances, in order to establish isomorphisms between …”

Author response:
Thanks for your suggestions, We are open to revisiting and refining the wording here in accordance with your suggestions.

Reviewer comment (Writing):
The implications of the “Note…” after eq 9b remain unclear.

Author response:
Here, we are trying to remind readers that the score function of conformation is not simply $-\frac{\tilde{\mathcal{C}} - \mathcal{C}}{\sigma^2}$. This is because the considered probability density function has the SE(3)-equivalence. Hence, for example, if $\tilde{\mathcal{C}} = \mathcal{C} + t$, where $t$ is a scaler indicating that we move the atoms in $\mathcal{C}$ along x, y, z axis by $t$. In the conditional states of $p(\tilde{\mathcal{C}} \mid \mathcal{C})$, $\tilde{\mathcal{C}}$ and $\mathcal{C}$ represents the same state since we are considering in a SE(3)-invariant space. Hence, the score function of $p(\tilde{\mathcal{C}} \mid \mathcal{C})$ cannot be easily computed. That's why we transform the score function of distances to the score function of conformations.

Reviewer comment (Writing):
The paragraph on optimal transport says that the authors use the regularized Wasserstein barycenter but then continue saying that it is not suited. So what exactly do the authors do?

Author response:
We are approximately the distribution between the Gaussian distribution and the MB distribution. Hence, one may consider using the OT to solve the intermediate distribution. However, we discussed that using this method has numerical stability issues. Hence, we are using the proposed methods in our paper to estimate the intermediate distribution. Discussion about OT is to save time for those trying to solve the intermediate distribution by OT method.

Reviewer comment (Presentation):
The caption of Table 1 is too short. The caption should at least explain the meaning of COV and MAT and refer the reader to their definition in the text.

Author response:
Thank you for your suggestions. We have expended it.

Reviewer comment (Theory):
Why exactly is planarity a problem? Don’t 4 neighbors define any point in 3d exactly?

Author response:
Consider the case where 4 points are all in the same plane. You may consider there are 4 points in an A4 paper. Then, given the distances between them and the other additional points, you cannot decide whether this point is on the top of the A4 paper or the downside of the A4 paper.

Reviewer comment (Evaluation):
In Section 5.1, The MAT(-R) metric should be corrected as follows:  
MAT = 1/|S_r| ∑_(C∈S_r) min_(C'∈S_g) RMSD(C,C').

Author response:
Thank you so much for pointing it. We have made corrections.