Title: Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression
Venue: NeurIPS 2025
Accepted: yes
Keywords: Protein Conformation

Generative Modeling

Molecular Dynamics

Diffusion Models

Language Models
Abstract: Understanding protein dynamics is critical for elucidating their biological functions. 
The increasing availability of molecular dynamics (MD) data enables the training of deep generative models to efficiently explore the conformational space of proteins.
However, existing approaches either fail to explicitly capture the temporal dependencies between conformations or do not support direct generation of time-independent samples.
To address these limitations, we introduce *ConfRover*, an autoregressive model that simultaneously learns protein conformation and dynamics from MD trajectory data, supporting both time-dependent and time-independent sampling.
At the core of our model is a modular architecture comprising: (i) an *encoding layer*, adapted from protein folding models, that embeds protein-specific information and conformation at each time frame into a latent space; (ii) a *temporal module*, a sequence model that captures conformational dynamics across frames; and (iii) an SE(3) diffusion model as the *structure decoder*, generating conformations in continuous space.
Experiments on ATLAS, a large-scale protein MD dataset of diverse structures, demonstrate the effectiveness of our model in learning conformational dynamics and supporting a wide range of downstream tasks. 
*ConfRover* is the first model to sample both protein conformations and trajectories within a single framework, offering a novel and flexible approach for learning from protein MD data.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new method for generative modelling of protein MD trajectories. An MD trajectory is considered as a sequence of 3D structures that are temporally related. Thus, the proposed model autoregressively generates/samples a trajectory frame-by-frame. The model is benchmarked for 3 sampling problems compared to other existing models for the same tasks, and tends to perform favorably.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- **Novel idea:** The idea of framing generative modelling of molecular dynamics trajectories as autoregression is exciting and new, to the best of my knowledge.\n\n- **Good results on standard benchmarks:** Empirical results look strong relative to baselines, overall. The experimental setup largely follows past published papers based on my understanding (except S.4.3 highlighted below). I will rely on other reviewers' assessment on the rigour of the experiments.\n\n- Figure 1 does a good job showing the pipeline and main contributions.\n\nWeaknesses:\n\n- **No standard deviations:** For almost all the tables, there are no standard deviations or variations reported. Only an averaged metric is reported. This is esp. important for several of the claims of outperforming other models when the results are very close (eg. Table 3). It may be the case that several models are within an error range of one-another, making it harder to make an outright claim of SOTA performance.\n     - I realise that it may be conventional to train only one big model and report the best mean metric. However, it can still be useful to readers to additionally report some appropriate measure of variance in performance over your test set.\n     - I will adjust my assessment and increase my score if the authors can address this concern in the rebuttal.\n\n- **Missing inference time metrics:** I would be interested to see some metrics on the inference time for the proposed model as well as the previous models being compared to, as well as how that compares to actually running an MD simulation. I think this is important, b/c the goal of this line of work broadly is to accelerate and augment MD simulations. So are current models actually faster than running MD?\n    - As an example, in the conformation interpolation task: Say I am a user and I know the two conformational states of my protein. Now, I can either run an MD simulation or I can use your AI method (which is slightly less accurate than MD). Thus, I will only ever run your AI method if it is much faster than MD. Would you agree with this assessment?\n\n- (Minor) **Conformation interpolation results and associated claim (S.4.3)** The model requires finetuning to perform this task. However, when I read the Abstract/Intro/Conclusion, I got the impression that the model can perform this task out of the box (e.g. statements like \"CONFROVER supports three tasks in a unified manner\"). I think this claim needs better support/more nuance.\n\n---\n\nSubjective note on presentation:  As a reader who works in closely related fields, but not deeply familiar with all aspects of this paper's field, I found that the paper's presentation of the technical content (method, experimental setup) was a bit hard to grasp. Esp. the methods were presented in what I felt was a very dense/condensed form, and required jumping back and forth to references in the appendix. I would suggest trying to improve the presentation of the methods.\n\nThe Intro/Conclusion/etc. reads very well.\n\nQUESTIONS:\nSee questions within weaknesses.\n\nLIMITATIONS:\nYes, I think some limitation are discussed.\n\nIts worth asking: Why are proteins dynamic and undergo conformational changes? There is some amount of inherent dynamics of atomic systems involved, and this paper tackles the problem well. However, another major reason proteins undergo conformational changes is to interact with other partner molecules. It may be worth mentioning that the current method is limited in modelling interactions (or at least, can only implicitly model interactions at present).\n\nIt may also be interesting to discuss relative trade-offs vs. emerging methods in machine learning interatomic potentials for organic systems including proteins. Examples include MACE OFF (https://pubs.acs.org/doi/10.1021/jacs.4c07099) and the Orb family of models, which were applied to a simulating the dynamics of a couple enzyme proteins.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a method that holistically takes into account both time-independent and time-sequence protein conformations.\nThis is done by a representation where the encoder is at the level of individual frames, and the sequence are modeled using masked auto-regression. The same model is used for conformation dynamics, time-independent conformations and conformation interpolation.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n\n- The paper is very well written and pleasant to read (though I advise thoroughly proof-reading during rebuttal for typo's, see my minor suggestions below). The authors have carefully thought about which concepts to explain, and I appreciate the space given to practical insights and discussions, and motivations for the experimental setup, while leaving prior (relatively well-known) concepts, such as the SE(3) diffusion model, for the appendix or relevant literature.\n- Experimental validation yields convincing results and the setup seems reasonable and is well motivated.\n- I appreciate the clear and thorough Limitations section.\n\nWeaknesses:\n\n- The method is relatively simple, from a theoretical perspective, though this is not necessarily a weakness.\n- Validation is a bit tailored to the problem, but this seems common in this relatively new research area, where standard benchmarks are not yet widely established (as the authors point out themselves). I've only recently switched to this application domain, so other reviewer might be better positioned to gauge this.\n\nQUESTIONS:\nQ1: From the perspective of molecular dynamics, the concept of (machine learning based) coarse grained dynamics is related to your work, I suspect. Could you please comment on this? See e.g.:\n- Arts et al., \"Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics\".\n\nQ2: You start the paper with Langevin dynamics, which is a second order differential equation, thus the full state would not only be the positions of the atoms, but also their velocities. As far as I understand, you don't model velocities, and don't take them into account in any way. Do you have any comments or thoughts on this?\n\nQ3: You explicitly do not take any temporal information into account in the FrameEncoder, because you want to model both time-independent frames, and frame sequences. Specifically for the task of frame sequences, do you think this causes a decrease in performance?\n\nMinor remarks/suggestions:\n- Fig. 1: iii): the second block has a black outline. Should this not be the last block, corresponding to $x^L$?\n- line 89: insights **in** the\n- line 129: **A** similar'\n- line 140: sequence\n- line 148: Since both the encoder ..\n- line 151: Details .. are\n- line 183: **is** invariant .. and **is** ..\n- line 219 Baselines. Can you please provide citations here for MDGen, AlphaFlow and ConfDiff?\n- line 248: compare**d**\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduces Confrover, an autoregressive framework that couples a transformer “trajectory module” with an SE(3) diffusion decoder so that one model can (i) roll out protein trajectories, (ii) draw single, time-independent conformations, and (iii) interpolate between two end states. Training is performed on the ATLAS MD data set with a hybrid loss that mixes trajectory, single-frame and (optionally) interpolation objectives. On held-out proteins the authors report better dynamical metrics than MDGen (but worse than MD) and competitive equilibrium-ensemble metrics with AlphaFlow and ConfDiff.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The motivation and objective for the paper are clear, the task is important, and the modular design of the architecture is sensible. The observation that frame autoregression subsumes the three target tasks is very sensible. \n2. The joint training for the different tasks makes sense and improves equilibrium sampling.\n3. Empirically, the paper demonstrates reasonable (albeit still not excellent) performances.\n4. Overall, the paper can be easily followed.\n\nWeakness:\n1. This paper is primarily empirical, with a few previous studies exploring similar ideas (e.g., autoregressive + diffusion has been explored in MarDini for image/video, and AlphaFolding). The architecture also reuses existing components and fuses them. That said, this should not result in the rejection of the paper.\n2. The evaluation is primarily based on RMSD, PC-L2, JSD, etc. These statistical metrics have no physical meaning. In my opinion, kinetics (e.g. implied timescale)/energies/Rama plots should be evaluated to show that the model is physically grounded. Specifically, since the model breaks peptide bonds, I suspect it is not very good at distinguishing which dihedrals are more rigid and which ones are more flexible. The interpolation task should also have physical validation (e.g. minimum free energy paths; especially since there are no baselines). \n2a.  I note the empirical performance of the paper, while better than MDGen (unclear how good it is compared to AlphaFolding or other similar methods due to closed-source code), it is still very far from MD itself. The model often breaks peptide bonds (which should never happen) and has some C-alpha steric clashes. I am open to raising my score if evaluations comparisons can be made with AlphaFolding or more SOTA methods.\n3. There's a potential data leakage problem. Presumably, the frozen OpenFold has been trained on (some) PDB structures of the ATLAS test proteins.\n\nQUESTIONS:\n1. Given deterministic encoder & time modules, how diverse are the multiple samples from identical conditioning frames?\n2. Have the authors checked energy conservation or compared implied timescales against MD?\n3. How is CA clash/peptide bond break calculated exactly? How about other clashes/bond length distribution?\n3. There are many things for which ML based MD sampling like AlphaFlow does not work, e.g. https://www.biorxiv.org/content/10.1101/2024.08.31.610605v1, does the Confrover work better?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a new model for the simulation of protein conformations that can perform multiple tasks: time-independent sampling, trajectory simulation and interpolation between two conformations. This is achieved by defining all tasks as conditioned generation, where the conditioning is generated by an encoding layer based on sequence only in the case of sampling or on one or multiple frames for the other tasks. The support for multiple frames is ensured by a specialized trajectory module working on a sequence of frame embeddings with a causal mask. The model is trained on a dataset with a large amount of proteins but with limited MD time sampling, ATLAS, and shown to achieve better results with respect to the open-source state of the art on each of the tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n* The model is based on validated components in its modular design, e.g. openfold2, confdiff. These models have been shown to work well for their respective tasks.\n* The model can generate structures in multiple settings, unifying conformation generation and molecular dynamics trajectory generation, altho it is unclear how important it is to unify these.\n* The model performs well on the ATLAS dataset, appearing to simulate convincing MD trajectories for some of the proteins outside of the training set according to visual inspection, however numerical validation is incomplete.\n* Introduction of a modern causal transformer for capturing the dependency between frames is a significant improvement for the practitioners in the field. \n\nWeaknesses: \n* Although the modular design is great, the modules are computationally very expensive: encoding layer of the model is based on a pre-trained OpenFold model, which may be limiting for large protein or multimer use case. The trajectory module is also based on the pairformer architecture, based on triangle operations, which are notoriously expensive. This leads to a rather expensive model, making it potentially unfeasible for the simulation of larger proteins. Although there are some math libraries coming up for such operations, the cost still should be discussed. \n* While the results shown indicate the ability of the model to produce more realistic trajectories than a similar approach, a more rigorous analysis of the trajectories would be needed to convince the reader of the statistical validity of the sampled states. Authors may follow analysis from the papers they cite, e.g. TimeWarp or EquiJump. Altho I 100% agree with the issue of not having open source baselines limiting the reproduction of former work and 1-to-1 comparison, there needs to be more analysis on the statistical validity of the MD trajectories. The unfiltered results in Figure 12 in Appendix appear somewhat lacking alignment with MD. \n* Not a weakness but a comment that authors may want to address in review stage for the general authorship that might share the same opinion as this reviewer - If I understand correctly, there is a strong remark about the unification of non-time aligned vs time aligned sampling, ie conformer generation vs trajectory generation, being an important contribution, and this being the first model to do so. I am not sure this is as significant as the paper puts it to be. Non autoregressive trajectory generation models cited in the paper can be easily modified to be conformer generation models as well, since they are based on simple conditioning using an interpolant scheme. If they are trained with interpolation from noise to data frame, rather than previous frame to next frame, these models would function as conformer generators. Repeating that I don’t believe a unified framework is a weakness, but just may not be the strength it is emphasised to be, especially if the conformer generation model becomes too computationally heavy in order to use the same framework of the trajectory generation model. \n\nMinor: \n* Line 187: updateing -> updating\n* Line 188: StructualUpdate -> StructuralUpdate\n* Line 189: StructralUpdate -> StructuralUpdate\n* Line 306: trajecotry -> trajectory\n\nQUESTIONS:\n* It is not very clear to me how the “MD timestep” is inserted in the model. Is it an explicit input to the model, or is the model left to infer it from the previous frames? And in this second case: are a few frames sufficient to univocally infer the desired timeframe? In either case, it would be interesting to see the trajectories generated for different timesteps and compare their statistics.\n* What is the exact setting of the multi-frames task? I.e. how many frames are presented to the model and how many frames are generated?\n* This may be a miss on my part in appendix but from the main description of the model, it does not seem to be equivariant under rotations of the protein. Can the authors comment on this point? Is any augmentation used on the training dataset?\n* For the interpolation task, the trajectories are shown to interpolate “smoothly” between start and end point. However, from the PCA plots, several trajectories seem to pass through potentially unphysical paths. Would it be possible to perform some rough energetic analysis to verify that the intermediate states retain some physical plausibility? \n* While some information on model cost is presented in appendix, it would be nice to see this section slightly expanded, especially covering a more direct comparison for specific proteins on the same hardware, since the scaling of this model to larger systems is a concern.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-07"}