Symmetry-Driven Discovery of Dynamical Variables in Molecular Simulations

Authors: Jeet Mohapatra, Nima Dehmamy, Csaba Both, Subhro Das, Tommi Jaakkola
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: AI4Science, Molecular Dynamics, Second Order Methods, Symmetry Discovery

Abstract:
Molecular dynamics simulations are crucial for understanding complex biomolecular systems, but they are often hindered by the high dimensionality of the configurational space. This paper introduces two novel approaches for discovering effective degrees of freedom (DoF) in molecular dynamics simulations by leveraging approximate symmetries of the energy landscape. We present a scalable symmetry loss function compatible with existing force-field frameworks and a Hessian-based method efficient for smaller systems. Both approaches enable systematic exploration of conformational space by connecting structural dynamics to energy landscape symmetries. Applied to alanine dipeptide, our methods comprehensively sample the Ramachandran plot, including shallow minima. Simulations initiated from our DoF-sampled points converge to all important conformations, demonstrating the methods’ effectiveness in navigating complex energy landscapes. These approaches offer powerful tools for efficient exploration in molecular simulations, with potential applications in protein folding and drug discovery.

Decision: reject
Reviewer scores: 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1729647448710

thread_type:
official_review

<thread_object>
creator: reviewer_w5rx

<content>
summary:
In this paper, the authors develop two novel approaches for the analysis of degrees of freedom of molecular dynamics simulations. The key idea is to study the symmetry of energy landscape with Lie algebra and further connect the structure dynamics with energy landscape symmetries. Two detailed approaches are considered, including a scalable symmetry loss function and a Hessian-based method. The proposed model has been validated on alanine dipeptide molecular dynamic simulation dataset. The identified two effective degree of freedoms are the two dihedral angles over the peptide bonds, which are consistent with setting.

strengths:
The authors propose two new models for for the analysis of degrees of freedom of molecular dynamics simulations, by the consideration of symmetry of energy landscape. The method are very novel and interesting. The results on the Alanine dipeptide are consistent with the general setting.

weaknesses:
1) The method have only be validated on a "toy"-type example of Alanine dipeptide, which contains only two degrees of freedom. It is not clear the potential performance of the model on more general MD simulations, in which more degrees of freedom are standard cases. Further, the authors have not compare with other models. It is not clear what is the potential advantage of the current models over existing models. 
2) The code is not available, thus it is hard to really evaluate the algorithm. 
3) Even though this is an interesting algorithm for analyzing MD simulation data, it does not fall into the general machine/deep learning category. I would think the paper is more suitable for a computational chemistry/biology journal.  
4) The paper is badly prepared with lots of typos and mistakes. Details will be given in the question part.

questions:
1) The missing of the code and data for the paper. Thus it is impossible to check or reproduce the results in the paper.
2) Too many typos and mistakes, for instance 
    a1. Page 1, "a symmetry. loss"
    a2. Page 5, "0 The second term"
    a3. Page  7, "equation ??"
    ....
    In general, the paper is poorly prepared! 
3) The advantage of the model and the potential application of models on complicated systems are still not clear. The toy-example of Alanine dipeptide has only two very simple degree of freedom, which should be (easily) identified. What is the advantage of the current model over existing models? How is the potential of using the current model for more realistic complicated MD simulations.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730715663857

thread_type:
official_review

<thread_object>
creator: reviewer_tfvz

<content>
summary:
The others propose a method that tries to identify the effective degrees of freedom of molecular simulations. They do this 2 ways, with Hessian based methods for small systems, and with a loss function for larger ones. They suggest this can explore the conformation space better.

strengths:
They describe interesting theory that indeed is useful for molecular modeling.

weaknesses:
They do not mention existing similar work in machine learning within the last 4 years it seems. 
The figures need to be improved. Which is a? b? c? etc in Figure 1 for instance. 
The results do not seem to match their analysis. Their only experimental results is Ramachandran plots of alanine dipeptide. They mention that 'we see that almost all the Hessian-based methods recover all the major
conformations of alanine dipeptide with relatively short simulation times.'. but this is very difficult to see from their results. All the plots seem to show completely different distributions. 
They do not compare to other baselines in recent literature. (https://pubs.acs.org/doi/10.1021/acs.jctc.4c00454, https://pubs.aip.org/aip/jcp/article/160/17/174109/3287814/Deep-learning-path-like-collective-variable-for,
https://openreview.net/forum?id=TnIZfXSFJAh)
They do not show the distributions of energies of the generated molecules. It could very well be the Ramachandran plots look decent, but the energies are completely off and non-physical. Judging by the molecules they show, it looks like the energies for many of them are very high.

questions:
Can you plot the distribution of energies?
Can you share the github page?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730717619598

thread_type:
official_review

<thread_object>
creator: reviewer_jhsj

<content>
summary:
The authors propose a method to discover effective degrees of freedom in molecular systems, for the aim of fast exploration of low-energy configuration space without prior knowledge of internal coordinates or collective variables. Effective degrees of freedom are directions in configuration space which approximately preserve the energy, and are in this sense viewed as symmetries of the energy. Specifically, degrees of freedom are unit vectors of the Lie algebra of the general linear group of transformations acting on configuration space. The authors propose a symmetry loss to find effective degrees of freedom, which are those along which energy changes minimally, but where conformation changes significantly. The authors refine this symmetry loss with variants using the Hessian of the energy. The proposed method is validated on the model system of alanine dipeptide, where it recovers all important conformers but with dramatically fewer timesteps of molecular dynamics.

strengths:
I find the work exciting because the application of symmetry discovery to enhanced sampling is novel, introduces a new notion for effective degrees of freedom, and empirically appears to dramatically accelerate exploration of molecular configuration spaces. The explanation and derivation of the method is clear and well-paced. The ideas introduced by this work inspire many directions for future work.

I appreciate the extensive discussion of methods for enhanced molecular dynamics sampling.

weaknesses:
There are issues with presentation which hold this submission back from acceptance.

The major issue is that not enough analysis is given to characterize the learned degrees of freedom.
- How does the conformation qualitatively change for a given learned degree of freedom? Do these actually resemble the individual true torsions?
- It is stated that "the degrees of freedom still remain invariant across" conditions in vacuum and solvent - how can this be true if each column of Figs 2 and 4 discovers slightly different conformers?
- What does the beta-sheet conformation look like, and where does it appear on the Ramachandran plot? Where do the rediscovered conformers lie on the plot?
- In Figs 1 and 3, the twisted gridlines differ for each method, and do not appear to cover the entire phi-psi space. How should I interpret this? It is my understanding that the beta conformation should be located at the origin of these grids, but it is not clear to me where that is located.
- Many conformers shown in Fig 4 do not visually resemble the leftmost column. What is the threshold for dihedral angle similarity?

There are numerous minor errors:
- Citation style should have parentheses (\citep) when not using the authors as a noun.
- Inconsistent capitalization of "Lie algebra", "Alanine dipeptide", "DOF"
- Line 50: "symmetry, loss"
- Line 62: "WE show"
- Line 192: should be $x'=gx$
- Line 286: $H_2$ is defined incorrectly
- Line 286: "Then it also minimizes ... can be minimized"
- Line 343: "Effective-ness"
- Line 345: Missing equation number
- Line 363: "m is the number samples"
- Line 377: "Alanine Dipepetide" is misspelled
- Line 388: "direct otimization"
- Lines 754-755: duplicate statements

Experimental setup is not entirely clear. What optimization algorithm was used to minimize losses? How costly is this optimization in terms of number of energy gradient evaluations?

What are the min/max bounds on the 31x31 gridpoints?

questions:
- The method of discovering effective degrees of freedom is stochastic - would multiple trials from the same initial optimized point be expected to converge to similar effective degrees of freedom? How do the discovered degrees of freedom vary as the initial optimized point is varied?
- Given that the proposed method locally searches for directions where energy remains flat, is it correct that the method would not be able to discover new minima that are separated from the initial point by large energy barriers?
- Starting from a single optimized configuration, is it true that the method only requires calculating the Hessian once?
- What, if any, challenges are expected when applying this method to energy functions defined by neural force fields?

An overview figure visualizing the learned effective degrees of freedom as paths on an energy landscape could help to quickly explain what these effective degrees of freedom are.

Since energy calculation is usually the bottleneck in sampling, one way to quantitatively state your speedup is by listing the number of energy, gradient-energy, and Hessian evaluations required by each method.

Another metric, if just focused on sampling, could be the effective sample size (ESS) as measured for Boltzmann generators:

- Klein, L., Krämer, A., & Noé, F. (2024). Equivariant flow matching. Advances in Neural Information Processing Systems, 36.


The related work section "Identifying the DoF" could touch on more of the "collective-variable-free" literature:
- Sipka, M., Dietschreit, J. C., Grajciar, L., & Gómez-Bombarelli, R. (2023, July). Differentiable simulations for enhanced sampling of rare events. In International Conference on Machine Learning (pp. 31990-32007). PMLR.
- Holdijk, L., Du, Y., Hooft, F., Jaini, P., Ensing, B., & Welling, M. (2024). Stochastic optimal control for collective variable free sampling of molecular transition paths. Advances in Neural Information Processing Systems, 36.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732143044260

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Shared response for all reviewers

comment:
Thank you all. We would like to address some common questions prior to the detailed response. We are updating figures, adding new Figs 5 and 6 which also show the energies and distribution of conformers found using each method. We are also making new 3D visualizations of the conformers and send an update once that's ready. 

## Pros and cons of each of our algorithms
The core of our methodology consists of two parts: 1) Discover DoFs as approximate symmetries by minimizing symmetry loss; 2) Choose most important DoFs by maximize symmetry-breaking loss. However, in practice we introduce four algorithms, each with its own pros and cons. Let us detail our observations:   
   1. **Symmetry Loss direct optimization:** directly minimizes eq (3) $\\mathcal{L} \= \\nabla E \\cdot \\mathbf{L} x$. It needs a number of perturbed samples for the maximization step eq (15). Its gridlines vary significantly depending on the samples. In the new Fig 4, the grid cover all $\phi$ angles, but only a portion of the $\psi$. This method yielded all known conformers, though higher energy conformers were    rarely found. 
   2. **Hessian backbone null space (aka “slow modes”):** Generally doesn’t cover a large portion of phi-psi space and remains fairly close to initialization. Nevertheless, simulations starting from these transformed configurations do reach almost all known conformer hotspots.   
   3. **Hessian backbone fast degenerate subspace (subspace rotations):** These yield the best grids, covering most of the phi-psi space, but rare conformers are discovered less frequently than the slow subspace.   
   4. **Full Hessian (no backbone)**: The full Hessian has four indices (two spatial and two atom indices). In this method, we flatten the Hessian into a matrix by grouping a spatial and atom index together. The resulting DoF have coverage similar to direct symmetry loss.   


## Related Work
We thank the reviewers for pointing us to related works on enhanced sampling methods. Here, we give a brief comparison of our work with the referred papers, which we will add to the related works sections of the paper. One key distinction between our method and other works is that we don't require much data. More details: 

- **Collective Variables-based methods**: [1], [2] use neural networks to learn CVs derived from neural network-based embeddings of the system descriptors. To learn these embeddings, the authors **require the use of simulation end-points** (bound vs unbound states, or starting conformer and end conformer for transition paths) as well as valid simulation trajectories to learn CVs. Our approach, on the other hand, uses second-order information about the energy landscape to explore the energy landscape, and **discover new conformers without any prior knowledge** of their existence. 
- **CV-free methods**: The CV-free methods ([3], [4]) mentioned by the reviewers bias/modify the equations governing the system dynamics to more efficiently sample rare events and transition paths, respectively. In both cases, the system needs to be trained on multiple trajectories in order to learn these biasing potentials. To learn the biasing potential in [3], the bias is parametrized as a combination of small Gaussian biasing potential along the dihedral angles, which uses knowledge of the underlying DoF. Meanwhile, [4]  requires access to high-quality initial samples of transition trajectories to learn the drift for the Schrodinger bridge formulation of the problem. Our proposed method uses Hessian computation to build multiple starting configurations for exploring the CV space without requiring simulation trajectories. 

We feel our method provides an orthogonal contribution to the works mentioned above. Our main objective is to efficiently explore the CV space and discover new conformers using locally-approximated CVs computed only using the spatial co-ordinates of the atom positions and the local energy function. The knowledge of these discovered conformers can then be applied to understanding/sampling transition paths or providing better estimations of the free-energy surface using the afore-mentioned methods.

[1] Machine Learning Derived Collective Variables for the Study of Protein Homodimerization in Membrane (https://pubs.acs.org/doi/10.1021/acs.jctc.4c00454).  
[2] Deep learning path-like collective variable for enhanced sampling molecular dynamics   (https://pubs.aip.org/aip/jcp/article/160/17/174109/3287814/Deep-learning-path-like-collective-variable-for).  
[3] Differentiable simulations for enhanced sampling of rare events. In International Conference on Machine Learning (pp. 31990-32007). PMLR.  
[4] Stochastic optimal control for collective variable free sampling of molecular transition paths. Advances in Neural Information Processing Systems, 36.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This submission develops several new algorithms for calculating or, to use the parlance of the paper, "discovering" the degrees of freedom in a molecular dynamics (MD) simulation. Leveraging symmetries of the configuration space of such a simulation, the authors derive a new symmetry loss function, which is subsequently optimised.

I found this paper to exhibit the following strengths:

- Tackling a new problem domain with strong theoretical underpinnings. Coming from a mathematics background, I enjoyed the exposition of the paper.
- The proposed method does _not_ require any data but just an energy function; this is very elegant.

However, the present manuscript also suffers from shortcomings, specifically:

- The method is only evaluated on a single example (alanine dipeptide), which makes it hard to assess its overall benefits.
- In a similar vein, the method is not compared to other methods. To some extent, this is due to its special "data-free" nature, but the authors mention that, technically, their method could also address other tasks described in the literature such that some baselines would be available.

Overall, this lack of contextualisation makes it hard to assess the merits of the work. While I emphatically agree with the authors about the relevance of the method (and indeed, MD applications are one of fastest-growing areas of ML these days), the current version of the manuscript appears "out-of-scope" at _first_ glance. To counteract this impression, a more comprehensive experimental suite would be required; alternatively, more weight could be given to the optimisation strategies or—potentially—a derivation of approximation schemes based on neural networks (given the proclivity of ML to assume that NNs are great, scalable function approximators).

It is for these reasons that I suggest rejecting the paper for now. I understand that this is not the preferred outcome for the authors, but I believe that a revision of the work, in particular in light of the issues raised with respect to contextualisation, experiments, and comparison partners, would substantially improve the impact and potential audience of the work.