A diffusion model on toric varieties with application to protein loop modeling

Authors: Xin Cao, Evangelos Coutsias, Ernest Glukhov, Sergei Kotelnikov, Dmytro Kozakov, Thu Nguyen
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Protein loop, Toric varieties, Jacobian, Tangent space, MHC, Nanobody

Abstract:
The conformation spaces of loop regions in proteins as well as closed kinematic linkages in robotics can be described by systems of polynomial equations, forming Toric varieties. These are real algebraic varieties, formulated as the zero sets of polynomial equations constraining the rotor angles in a linkage or macromolecular chain. These spaces are essentially stitched manifolds and contain singularities. Diffusion models have achieved spectacular success in applications in Cartesian space and smooth manifolds but have not been extended to varieties. Here we develop a diffusion model on the underlying variety by utilizing an appropriate Jacobian, whose loss of rank indicates singularities. This allows our method to explore the variety, without encountering singular or infeasible states. We demonstrated the approach on two important protein structure prediction problems: one is prediction of Major Histocompatibility Complex (MHC) peptide interactions, a critical part in the design of  neoantigen vaccines, and the other is loop prediction for nanobodies, an important class of drugs. In both, we improve upon the state of the art open source AlphaFold.

Decision: reject
Reviewer scores: 5, 5, 3, 5

Reviews and discussion:
<thread>
creation date:
1730652285372

thread_type:
official_review

<thread_object>
creator: reviewer_p9bc

<content>
summary:
The paper describes heuristic studies of random walks on spaces of chained atoms.

strengths:
Two algorithms are given with pseudocodes.

weaknesses:
Though the paper studies protein chains, they are called "molecules". In structural biology, a protein molecule can be a collection of many chains, which is called a bilogical assembly or a protein complex, while all smaller molecules (interacting with proteins) are called ligands.

Line 118: "we assume the internal conformation of side chains to be fixed"

This assumption seems too strict because side chains provide hydrogen bonds that form the most important secondary structures.

Line 120: "the space of internal protein chain conformations is the hypertorus."

Some values of torsion angles lead to clashed atoms involving side chains, hence a realizable space of torsion angles is smaller, see https://www.pnas.org/doi/abs/10.1073/pnas.1014674107

Lines 126-127: "Unlike diffusion on Euclidean spaces or smooth manifolds,
diffusion on varieties can be challenging in the vicinity of singularities."

Though singularities are explicitly mentioned here, Figure 1 and all further constructions discuss smooth manifolds. The word "variety" appears only two more times in the abstract. 

Line 131: "a loop region can move in a finite number of directions to keep the ends fixed"

Because atom coordinates are continuous, they can move in infinitely many directions. 

Lines 215: "The score model is using a similar architecture based on SE(3)-equivariant convolutional networks over point clouds"

If points are not ordered in these clouds, this case is not relevant for protein chains whose atoms are naturally ordered along the backbone. The ordered case is much easier than unordered because a complete SE(3)-invariant has a linear size.

questions:
The major weakness is the lack of a rigorous problem statement. 

Line 17: "we develop a diffusion model that performs a random walk"

What the purpose of this "random walk"?

The key concept of a "loop", which appears 80+ times in the paper including the title is never formally defined. Did the authors mean a geometric embedding of several consecutive amino acid residues or only some atoms (which?) along the backbone without side chains, which are not marked as alpha-helices or beta-strands?

Lines 114-115: "the observed variations of bond lengths and angles in experiments are relatively small"

How can this claim be justified? For protein chains, bond lengths and angles are not "observed" but are often fixed or refined to "ideal" values. Despite this artificial fixing, the Protein Data Bank has a large percentage of chains whose bond lengths and angles are outside many standard deviations from "ideal" values. 

Lines 176-177: "(the Implicit Function Theorem) guarantees that six of the variables may be expressed as differentiable functions"

This theorem holds only under strict conditions. Why are they satisfied?

Lines 198-202: "We apply R6B6 Cao et al. (2023) algorithm, which is designed to solve closure problem in the chain with fixed ends ... if the closure problem is not solvable by R6B6 which indicating an infeasible movement, the next perturbation will be tried until one solvable case is sampled successfully."

Can these tries in a continuous space continue infinitely long?

Lines 285-287: "in the diffusion on toric verieties, the basis vectors for the tangent space at one point on the manifold are dependent on the position of the point, so we need to compute the basis vectors at every step"

There is no need to consider any tangent spaces of high-dimensional tori because these manifolds are flat (locally Euclidean in terms of distances and angles), see https://en.wikipedia.org/wiki/Flat_manifold. For a simple example, a 2D torus has two angle coordinates in ranges [0,2pi). Why do you need to embed this torus in 3D?

Line 301: "Each structure was split into two parts: the loop region and the remaining part of the protein."

While a loop region wasn't formally defined, a protein structure likely contains many loop regions. Which one is chosen here?

Line 308: "we used the same AF2 model to obtain predicted Local Distance Difference Test (pLDDT) values Mariani et al. (2013)" 

Do you agree with the conclusion of Mariani et al. (2013) on page 2728: ``One disadvantage of the LDDT score is that it does not fulfill the mathematical criteria to be a metric"?

If yes, do you know that if a distance fails the triangle axiom with any positive error, then results of clustering can be pre-determined and hence not trustworthy as proved in https://ieeexplore.ieee.org/abstract/document/10574843? 

Line 310: "We then used these values to rank the structures and determine the confidence of the predictions."

How are these predictions justified in the light of the references above?

Line 320-321: "The testing results showed that closer conformations to the PDB structures exist within the ensemble, demonstrating that the model functions correctly."

The loop regions have huge uncertainty in PDB structures because the structure determination methods usually have low resolutions in these regions. Is it possible justify that the "model functions correctly" by using unreliable "ground truth"?

How large are the databases discussed in sections 4.1 and 4.2?

Lines 13-14: "Toric varieties. These are real algebraic varieties, formulated as the zero sets of polynomial equations constraining the rotor angles in a linkage or macromolecular chain."

The abstract promised to describe toric varieties of macromolecular chains by polynomial equations. In what line numbers did these equations appear if the word "polynomial" was found twice in the abstract and never again?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730673566035

thread_type:
official_review

<thread_object>
creator: reviewer_26s9

<content>
summary:
The paper introduces a diffusion model that allows to explore toric algebraic varieties. The authors demonstrate an improvement upon the state-of-the-art AlphaFold approach on two protein structure prediction data sets.

strengths:
The structure of the paper is good, and the choice of experiments (data sets, baseline method) seem to be appropriate.

weaknesses:
The paper is extremely difficult to read, I think it is inaccessible to the majority of the wide ICLR audience. This is the main reason behind my rejection, I am willing to raise my score if the main goal and contributions of the paper are much more clearly described.

For instance, after reading the Abstract and Introduction, it was still not clear to me what the problem is one is aiming to solve. Formulations such as "molecular modeling", "molecular structure generation" and "protein prediction" are used, but the abstract also mentions kinematic linkages in robotics, so what is the (precise) formulation of the general problem? In addition, so much jargon is used, such as "conformer sampling" (already in the very first paragraph, without any further context or explanation), "number of flexible torsions in the loop", "energetics", "rotors", ... Related work is using too much jargon too, and is almost unreadable for general audience. 

See Questions below for more examples, that could help to make the paper clearer and of interest to a wider community. For these reasons, it was impossible for me to properly assess the quality of this manuscript.

questions:
(Q1) What is precisely the problem you are aiming to solve? As I mention above, you use "molecular modeling", "molecular structure generation" and "protein prediction", but at a later point in the Introduction you also talk about learning a constrained high dimensional manifold. If the latter also applies, should the Related work not also discuss the manifold learning literature?

(Q2) What is the (idea behind) diffusion model and why is it useful for this problem? In Introduction, you explain this with "Since conformer sampling can be treated as a generative problem, many diffusion models have been proposed to produce molecular conformers ...", but this is not very useful if one does not yet understand the problem (Q1), nor is familiar with conformers.

(Q3) The result from Angeles (2014) that guarantees that six of the torsions can be determined once other n-6 torsions are assigned is only mentioned as such in the Introduction, but then heavily used throughout the paper. Can you at least provide some intuition behind this results, where is the number 6 coming from? Also, since this is a book of almost 600 pages, it would be useful to point to a specific result (theorem or at least a section).

(Q4) At the end of the Related work section, you mention that AF can be run several times starting from different random seeds. Until this moment, the reader does not even know that the method is not deterministic. Since you use this approach as the baseline you compare against, can you describe it briefly, how does it work? Which group of methods does it belong to, from the ones you mention early in this section? What are the crucial differences between AF and your proposed approach?

(Q5) The score model is using a similar architecture based on e3nn and tensor field network that have not been peer reviewed and have respectively around 150 and 1000 citations, why not PointNet that has over 15000 citations and is commonly used as a benchmark deep learning architecture for point clouds?

(Q6) What is Å?

(Q7) Why is AF3 used in Section 4.1, but not also in Section 4.2?

(Q8) In the Conclusion (and all of a sudden), you write that "Previous studies on protein structure prediction have maintained that relationships between residues at long distances cannot be learned using a diffusion model." What is this based on, what is the underlying logic and why did you decide for a diffusion model despite these insights?

Typos and minor comments:
- line 088: conformerss
- line 089: varierty
- line 298: Appendix Table 3 -> Appendix A Table 1
- References: The references should be properly formatted in the main text (so that they appear in brackets whenever appropriate, choose carefully between \citet and \citep). Also, in the References section, improve the formatting, e.g., capitalization of journals (Advances in neural information processing systems).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730707726647

thread_type:
official_review

<thread_object>
creator: reviewer_rd41

<content>
summary:
This paper proposes a diffusion model on the conformation space of loop regions in proteins, using tools from inverse kinematics in robotics. In particular, for a given protein loop region, they solve for the basis vectors of the manifold of torsion angles which maintain the position of the loop ends, using singular value decomposition. They then add noise to the given torsion angle along this manifold, ensuring loop closure with the R6B6 algorithm. They then seem to learn the score function w.r.t. this diffusion process. Experimental results indicate improved performance compared to AlphaFold 2 and 3. Overall the paper was reasonably well-written and proposes an interesting approach. However, I am unsure about the significance and motivation of the approach. Additionally, I felt that more elaboration is needed on some aspects, e.g. the score-matching algorithm and the method R6B6.

strengths:
1. Presents a novel approach, to the best of my knowledge.
2. Make an interesting connection between molecules and robotics.
3. Takes advantage of AlphaFolds predictions as inputs, making much more data available.

weaknesses:
1. Can be cumbersome to do an SVD of the Jacobian matrix at every step.
2. Not much elaboration on the score-matching part of the algorithm.
3. Uses an external method, R6B6, to ensure loop closures, but does not elaborate much on it.

questions:
1. What is the cost of computing the SVD of the Jacobian at every step compared to classical geometric methods?
2. How much does the change in loop conformation affect the conformation of the rest of the protein?
3. Could you please elaborate more on the use of R6B6
</content>
</thread_object>

</thread>
<thread>
creation date:
1731134554498

thread_type:
official_review

<thread_object>
creator: reviewer_q7yb

<content>
summary:
The paper focuses on the problem that the loop regions usually have lower accuracy compared to other structures in the protein prediction tasks. The paper proposes a diffusion model to generate loop conformations in toric varierty space. The performance of the method was tested using the MHC dataset and the nanobody dataset. It achieves improvement upon AlphaFold 2 by 15% and comparable performance to AlphaFold 3.

strengths:
- The paper focuses on the loop regions in the protein, which is a useful topic to improve the protein prediction results.
- The paper proposes a novel diffusion process on toric varieties to generate conformations for protein loop regions with constrained ends.

weaknesses:
- Although the paper selects "No Acknowledgement Section", there is an acknowledgement section in the paper...
- Besides, I just have some additional questions listed in the "question" section.

questions:
- How long does it take to run each inference step?
- Does the diffusion model also takes non-loop regions as input information?
- In Figure 2, the paper compares results with AF2(1) and AF2(20) and AF3. Was there a specific reason to not compare with AF3 in the similar way as AF2 (i.e., also compare the best results of 20 random seeds)?
- The method first uses AF2 to predict the protein structures and uses that as inputs. The diffusion model then improve the protein prediction results by generated conformations for the loop regions. It achieve improvement upon AlphaFold 2 by more than 15% and comparable performance to AlphaFold 3.  - If the input protein structures are generated by other methods (such as AF3) instead, will the proposed method also give the similar amount of improvement upon the initial structures?
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a diffusion generative model for looped structures by leveraging toric varieties formed from systems of polynomial equations. The applications to two protein structure prediction problems are considered. Both the reviewers and me found the approach interesting, but there are also major concerns about significance, innovation, and presentation. Recognizing the potential of the idea, I recommend the authors take reviewers' comments into consideration and submit again.