Paper: RINGER: Conformer Ensemble Generation of Macrocyclic Peptides with Sequence-Conditioned Internal Coordinate Diffusion (ICLR-2024)

Reviewer comment (Experiments):
An ablation study is essential to elucidate the respective contributions of the positional encoder and the ring closing algorithm to the overall performance.

Author response:
Thank you to reviewers 2uae and yeES for suggesting the ablation study to demonstrate the effect of the cyclic positional encoding. 

We designed our positional encoding as an inductive bias to account for the symmetries of the molecule; e.g., the sequence R.I.N.G.E.R. has no natural start position and should be represented identically as I.N.G.E.R.R. We note that standard relative positional encodings do not incorporate this geometric prior, and hence should not generalize as readily with fewer examples.

To demonstrate the increased efficiency of our positional encoding and ability to learn from less data, we trained two models on a subset of the training data for 100 epochs. Both models are identical with the exception of the positional encoding: one is trained with a standard relative positional encoding and the other uses our cyclic relative positional encoding defined in equations 1 and 2. These new results (included in the supplementary material and referenced in section 3.3) clearly demonstrate the benefit of the cyclic encoding, which obviates the need for training on additional or augmented data to learn the cyclic invariance. Furthermore, we hypothesize that the cyclic encoding is most beneficial when training on more diverse data, e.g., heterogeneous macrocycles and larger ring sizes. For a preliminary assessment of this, we also include the ablation study results split by the number of residues in each test macrocycle (4-mers, 5-mers, and 6-mers). This additional result demonstrates the benefit of the encoding is even more pronounced for larger ring sizes, which possess more possible cyclic permutations. 

We believe this demonstrates the clear benefit of our designed positional encoding and welcome your feedback.

Reviewer comment (Evaluation):
What is the information regarding the rejection rate for samples where the ring torsion fingerprint deviation exceeds 0.01 before and after optimization using Equation 3?

Author response:
To select the 0.01 rTFD threshold, we found that the rTFD(pre-opt., post-opt.) distribution was bimodal with the split between the two modes being approximately 0.01 (with most of the density <0.01). This indicates that a small fraction of optimizations were able to find Cartesian coordinates that closely satisfy the bond distance constraints at the cost of distorting the torsions to an unacceptable degree.

Reviewer comment (Experiments):
What's the rationale for not directly modeling non-rotatable side-chain groups like phenyl rings and instead generating them using RDKit?

Author response:
Non-rotatable side chains like phenyl rings have limited degrees of freedom, so they are well-modeled by pre-defined ideal geometries (not necessarily generated from RDKit). Explicitly modeling them in the diffusion process would increase complexity without much benefit.

Reviewer comment (Experiments):
Why was the training set restricted to 30 conformers per molecule with the lowest energy as opposed to a threshold based on the lowest energy?

Author response:
We limited the training data to the 30 lowest-energy conformers for simplicity of training and found good performance relative to adding additional conformers. Our results demonstrate that using 30 conformers allows us to generalize to entire ensembles of hundreds to thousands of conformers; and hence provides a parsimonious choice of training data.

Reviewer comment (Evaluation):
I’m seeking clarification on why, in section 4.4, you claim that the additional xTB optimization demonstrates the efficacy of the diffusion scheme in achieving diverse sampling. Could you elaborate on this point?

Author response:
Apologies if this was unclear – RINGER already achieves excellent recall prior to xTB optimization whereas other methods require xTB optimization to improve recall. This demonstrates that the underlying diffusion approach is capable of directly generating diverse ensembles (with recall being a measure of this diversity).

Reviewer comment (Experiments):
In the context of unconditional generation tasks, a comparison with other existing methods would be highly valuable in order to assess RINGER's performance and capabilities more comprehensively. Additionally, in the comparison of conditional generation, where rRMSD and rTFD metrics are employed, it appears that there is an absence of a method focused on backbone generation.

Author response:
**3.** i) The focus of our work is on complete, all-atom conformer generation.  We performed studies on unconditional generation to validate that our underlying approach is able to capture the underlying, complex backbone distributions for macrocycles, rather than to claim that we perform SOTA backbone generation; hence we do not believe further benchmarking for unconditional generation is warranted. ii) Similarly, the purpose of reporting detailed backbone metrics is to better understand the ability of these methods to capture accurate backbone geometries; this complements the all-atom metrics and allows us to dissect whether backbone- or side-chain conformations are driving performance. Backbone conformation is a critical determinant of macrocycle discovery and design (see refs 1-3 below by Rezai et al, Bhardwaj et al, and Hosseinzadeh et al). Finally, outside of our studies in the appendix, there currently exist no methods for backbone-specific conformer generation.

[1] Rezai et al. Conformational flexibility, internal hydrogen bonding, and passive membrane permeability: successful in silico prediction of the relative permeabilities of cyclic peptides. J Am Chem Soc, 2006

[2] Bhardwaj et al. Accurate de novo design of membrane-traversing macrocycles. Cell, 2022

[3] Hosseinzadeh et al. Comprehensive computational design of ordered peptide macrocycles. Science, 2017

Reviewer comment (Experiments):
Why the baseline excludes Torsional Diffusion [2]? I can understand that ‘Methods such as torsional diffusion only alter freely rotatable bonds and cannot sample macrocycle backbones by design.’, but I think Torsional Diffusion can be compared ‘in the context of all-atom geometries (RMSD)’ in section 4.3 if I do not misunderstand.

Author response:
1. We recognize the value of benchmarking against Torsional Diffusion for all-atom geometries, along with adding an additional baseline (DMCG) which achieves SOTA performance on the GEOM dataset. We have added these results to provide a more comprehensive comparison in addition to GeoDiff. Notably, these methods provide some improvement over heuristic baselines (RDKit and OMEGA). Interestingly, DMCG provides strong recall performance across our metrics, but with low precision. To better understand the balance between these two, we have updated the main results Table with an F1-score that quantifies the balance between recall and precision.

Reviewer comment (Reproducibility):
Why GeoDiff-Macro performs so poor? Can you provide experiment details of GeoDiff-Macro?

Author response:
2. We have included more details on the the training procedure of the deep learning baseline methods in the appendix. In brief, we follow the official GeoDiff implementation but retrain it from scratch using the CREMP dataset on the exact train-val-test split in this study. The standard GeoDiff model trained on GEOM provides poor performance, and hence we did not believe this to be a good reflection of the method. Although we have not performed extensive studies on GeoDiff, we suspect that the limited performance is due to the challenges of performing Euclidean diffusion over Cartesian coordinates.