ProTDyn: A Foundation Protein Language Model for Thermodynamics and Dynamics Generation
Authors: Yikai Liu
Haoyang Zheng
Lining Mao
Yanbin Wang
Ming Chen
Guang Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Transformer
Protein Language Model
Protein ensemble generation
Protein dynamics
generative model
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A unified generative model for simultaneously generating protein conformation ensemble and dynamic trajectories

Abstract: Molecular dynamics (MD) simulation has long been the principal computational tool for exploring protein conformational landscapes, but its application is limited by high computational cost. We present ProTDyn, a foundation protein language model that unifies conformational ensemble generation and multi-timescale dynamics modeling within a single framework. Unlike prior approaches that treat these tasks separately, ProTDyn allows flexible i.i.d ensemble sampling and dynamic trajectory simulation. Across diverse protein systems, ProTDyn yields thermodynamically consistent ensembles, faithfully reproduces dynamical properties over multiple timescales, and generalizes to proteins beyond its training data—offering a scalable and efficient alternative to conventional MD simulations.

Published as a conference paper at ICLR 2026
PROTDYN: A FOUNDATION PROTEIN LANGUAGE MODEL FOR THERMODYNAMICS AND DYNAMICS GEN- ERATION
Yikai Liu', Haoyang Zheng', Lining Mao", Yanbin Wang', Ming Chen', Guang Lin" Purdue University Norttwesterrn University
ABSTRACT
Molecular dynamics (MD) simulation has long been the principal computational tool for exploring protein conformational landscapes and dynamics, but its ap- plication limited by high computational cost. We present ProTDyn, a foun- dation protein language model that unifies conformational ensemble generation and multi-timescale dynamics modeling within single framework. Unlike prior approaches that treat these tasks separately. ProTDyn allows flexible indepen- dent and identically distributed (i.id.) ensemble sampling and dynamic trajectory simulation. Across diverse protein systems, ProTDyn yields thermodynamically consistent ensembles, faithfully reproduces dynamical properties over multiple timescales, and generalizes proteins beyond its training data. It offers a scal- able and efficient alternative to conventional MD simulations.
Code: hatppr//ggthbb..ommm//aaaaaxa//p
INTRODUCTION
Proteins are the fundamental building blocks of life, carrying out essential functions such as catal- ysis, signaling, and transport. Understanding their conformation flexibility and dynamic nature is crucial for uncovering the molecular mechanism of protein functions Whisstock & Lesk (2003)- Traditional computational methods, such as molecular dynamics (MD) simulations, have been op- timized and widely used for decades study biomolecular processes such protein folding and unfolding Shaw et al. (2010); Robustelli et al. (2018). However, MD remains computationally ex- pensive, since the simulation time step must be orders of magnitude smaller than the timescales of biologically relevant processes Izaguirre et al. (1999). As result, simulating ain-ttiesccale protein dynamics often infeasible.
Recent advances in machine learning, particularly deep generative models, have introduced powerful alternatives model proteins. Generative approaches have been developed to sample equilibrium conformational ensembles Lu et | (2023): Lewis et al. (2025); et al. (2024a); Noé et al. (2019);
Zheng et (0024); Wayment-Steele et al. (2024) and, separately, to learn protein dynamics Jing et al. 2222bb: Raja et al. (2025); Lelievre et al. 2003) Du et al. (2024); Fu et al. (2022). Re- cent models also incorporate the principles of statistical mechanics that correlates thermodynamics and dynamics Raja al. 2025); ; Arts et al. (2023) However, these models still rely very small timesteps to propagate molecular dynamics, and generalize poorly beyond training dataset. Thus, scalable and transferable protein emulator models that can simultaneously describe both equilib- rium conformational ensembles (thermodynamics)) and conformational transitions across multiple timescales (dynamics) still unavailable, which limits further progress such as accurately mod- eling protein biophysics, where both equilibrium ensembles and transition dynamics are essential for understanding protein-protein interaction, allosteric regulation, biocondensation, and conforma- tional heterogeneity Brandsdal & Smalás (2000); Guo & Zhou (2016).
Based on recent progress and gaps, we introduce ProTDyn, a unified framework for generative modeling of Protein Thermodyaamics and Dynamics. Unlike previous approaches that treat con- formational ensemble generation and conformational dynamics propagation separately, ProTDyn
"Corresponding author: guang? 1 innpurdueeeee

---

Published as a conference paper at ICLR 2026
unifies them within singlemulti--e architecture. ProTDyn is trained on hundreds of thousands of protein sequences and over a million of conformations, leveraging both single-structun and equi- librium MD simulation data to train the model for comprehensive understanding of protein confor- mational space. Moreover, ProTDyn can perform multi-timescale training which enables modeling conformational transitions of diverse protein systems with timescale ranging from nanoseconds to microseconds. This flexible scheduling bridges short- and long-timescale dynamics. Together, these capabilities are mutually reinforcing: accurate thermodynamic ensembles provide stable baselines for dynamic propagation, while realistic dynamics improve the diversity and fidelity of generated equilibrium ensembles.
In sum, we demonstrate three key capabilities of ProTDyn:
Thermodynamics generation: sampling independent and identically distributed (i.id.).. equilibrium protein structures from the learned ensemble distributions that consistent with Boltzmann statistics.
2. Multiscale dynamics generation: generating temporally coherent trajectories at multiple time-resolutions, capturing both fast local fluctuations and slow global transitions.
3. Dynamics inpainting: refining coarse time-resolution trajectories by recovering fine- grained time-resolution, physically plausible dynamic pathways.
By unifying thermodynamics and dynamics within single generative framework, ProTDyn enables scalable, transferable, flexible, and computationally efficient protein modeling. In particular, com- pared to existing protein ensemble and dynamies generators, ProTDDn offers greater flexibility in generating conformational ensembles across diverse settings. We validate the accuracy and transfer- ability of ProTDDy on multiple tasks, including generating conformational ensembles for proteins outside the training dataset and simulating protein dynamics beyond the training regime. Experi- mental results confirm that Pron agrees well with reference MD simulations while generalizing effectively to unseen systems.
BACKGROUND
Molecular dynamics. Molecular dynamics (MD) simulates the time evolution of molecular sys- tem by integrating Newton's equations of motion. For each particle i in a molecular configuration ( XN)= E R&N the equations are
MX. Vx, (x)
where Mj is the mass of particle and U IR + R is the potential energy that is often modeled by a force field. By construction, an MD simulation at a fixed temperature will converge to the Boltzmann distribution of the system,
P(x) -Ux)///)/b
where ki is the Boltzmann constant.
Deep generative modeling for proteins. Recent advances in deep generative modeling have opened new opportunities to simulate protein conformational ensembles. These models can generate con- formational ensembles and transitions within hours, offering an efficient alternative to conventional MD simulations. One line of work targets thermodynamics, directly learning the stationary Boltz- mann distribution P(x 8) over conformations from structural databases or equilibrium MD trajec- tories Lu et al. (2023); Lewis al. (002); Jing et al. (2024a); Noć al. (2019); Zheng et al.) (2024) y Solyment-Steckae et (2024). Such models recover equilibrium ensembles but lack dynam- addition to modeling thermodynamics, complementary approaches focus on dynamics, learning the transition density P(x++Ar x,, s) to accelerate MD Jing et al. (2024b); )): Fu et al. (2022). Although these methods can predict short-time kinetics, they are often trained on limited MD data, restricting their ability to generalize to rare or long-timescale transitions. Despite rapid progress, current approaches remain specialized in either thermodynamics or dynamics, overlooking their intrinsic connection based on statistical mechanics. A unified foundation model capable of predicting both equilibrium ensembles and transition dynamics across scales has yet be established.
ical information.
Raja et al. (2025): Lelièvre et al. (2023); Du et al. (2024);
2

---

Published as a conference paper at ICLR 2026
Thermodynamics
Dynamics
Dynamics inpainting
aee
Structure head
ProTDyn
Transformer Blocks
Cross-entropy loss
Residual positional embedding
Residual positional embedding
Temporal positional embedding
Sequence
Sturcutre c
Sturcutre &t
Sturcutre Mot
Figure I: An illustrative framework of ProTDyn. ProTDyn a protein language model that operates on discretized representations of protein sequence and structure. It leverages powerful autoregres- sive transformer architecture to simultaneously perform three tasks: (i) equilibrium conformational ensemble generation (thermodynamics), (ii) forward trajectory generation across multiple timescales (dynamics), and (iii) recovery of fine-grained trajectories from coarse trajectories (dynamics inpaint- ing).
Protein conformation representation. Although proteins are inherently three-dimensional objects, recent advances have shown that conformations can also be represented as sequences of discrete tokens, enabling the use of powerful sequence modeling techniques. In this work. we adopt the pretrained ESM3 Hayes et al. (2025) structure tokenizer to map protein conformations to tokenized sequences с E Z where each residue is assigned one the 4,096 structure tokens (plus 4 special tokens). These tokens provide compact, learned representation of the local structural neighbor- hood around each residue. The discretization is performed with a VQ-VAE encoder Van Den Oord al. (2017), while paired decoder reconstructs the generated token sequences back into three- dimensional coordinates.
METHOD
In this section, we will introduce the high-level framework of ProTDDy,, as illustrated Fig. ProTDyn is multi-task protein language model designed to perform three complementary tasks within single framework: (i) equilibrium conformation ensemble generation (thermodynamics), (ii) multi-tinescale dynamic trajectory generation (dynamics), and (iii) fine-grained trajectories covery from coarse time-resolution trajectories (dynamics inpainting). The model operates by au- toregressively predicting the structure tokens of successive residues. Its architecture builds upon the state-of-the-aa protein language model ESM3, and additionally incorporates a temporal positional embedding encode dynamic transition information.

---

Published as a conference paper at ICLR 2026
3.1 OBJECTIVE
We represent protein conformations in a discrete tokenized space and define three complementary learning objectives:
1. Thermodynamics the thermodynamics head of ProtDyn aims to learn the equilibrium dis- tribution of conformations (a s). Specifically, models the equilibrium conformation distributions autoregressively as
N-1 Palc 8) II Pa(c, c<i,
(3)
where c; denotes the structure token residue and Czj represents all preceding residues. The thermodynamic, head is learned by minimizing the cross entropy between Po and the observed protein equilibrium conformation ensemble distribution:
N N-1
Lehermo (0) = - (s(s,c)
Σ
log Po(ca Cei,
(4)
where 'D denotes the dataset of protein sequences and their equilibrium conformations.
2. Dynamics: the dynamics head aims learn the temporal correlations across multiple timescales Po C 8), where C = with time step 81, We factorize the trajectory distribution as
c.M66t) is trajectory segment of length M
M
Po s)
II Po |C*,,s) 1=0
(5)
where C'et represents all preceding conformations up to time step 1- Each conformation jōt (c)
CN CN-1) is further decomposed residue-wise as
N-1
Po(c Cst s) II i=0
Pa(c i6
ca Cst s).
(6)
The model parameters optimized by minimizing the negative log-likelihood of observed trajectory data:
M log (c
Loyn (.,,
Σ
c (s)
where D denotes the dataset of protein sequences paired with dynamic segments of length M. In this project, we aim to capture protein dynamical behaviors multiple timescales. Specifically, we let the model learn öt = 1 ns, 10 and 100 ns resolution. Additionally, we set memory kernel of M 10, which corresponds effective timescales of 10 100 ns, and 1000 ns, respectively.
3. Dynamic inpainting: The dynamics head enables trajectory generation with large timesteps, but this comes at the cost of losing fine-grat fine-grained temporal resolution. To ad- dress this, we introduce a dynamic inpainting head that reconstructs physically plausible transition sequences between metastable conformational states. Formally, the inpainting task to recover fine-resolution conformations trajectory C between two states e and e. M8t, modeled from coarse trajectories Pe P& ca. Mft ss.. Similar to dynamics head, the dynamic inpainting between two states c" and Mat is formulated as an autoregressive conditional generation problem:
M-1
PC ca, M8t
Po(ci8t C Cx,t,e M8t ;8); j=1
II
(8)
and the training objective is:
M Σ
(0) -,E
log Po (o(ct
ast ca Met 18)
4

---

Published as a conference paper at ICLR 2026
The choice of time step is ns and 10 ns and memory kernel follows the same training dynamics head.
Together, these objectives enable the model to simultancously generate equilibrium ensembles and reproduce protein dynamics at multiple timescales. During training, ProTDyn is optimized to mini- mize losses from all three heads. The losses are combined using hyperparameter weights:
LyirTDyn W1 Linemm + WyLdyn + wiy Cdyni.
(10)
where wi, w2, and w/ are hyperparameters.
3.2 TRANSFORMER ARCHITECTURE
We adopt transformer backbone following ESM3 Hayes et al. (2025). In particular, use Pre- LN instead of Post-LN, rotary positional embeddings (RoPE) Su et al. (2024) instead of absolute positional embeddings, and SwiGLU activations instead of ReLU. To encode both residue- and temporal-lever positional information, we introduce a two rotary embedding scheme. The residue- embedding layer encodes positions along the protein sequence by assigning integer indices se- quential residues (1 for the first residue, 2 for the second, and so on). desidue-embedding are applied to both sequence tokens and structure tokens. To incorporate temporal information for mod- cling multiscale dynamics, we introduce additional lemporal-emeeddee embedding is defined such that the smallest temporal unit corresponds to ns, which is the finest time resolution accessible to our model. For structure tokens, all tokens in the first structural seg- ment are assigned temporal value of O, those in the second segment are assigned òt, those in the third segment are assigned 28t, and so forth. The cemoora--ebeddiie tokens before the residue-embe sebiddde-ebedddngg layer, and is not applied to sequence tokens. The training data incorporates 5t 1, 10, 100 ns. At inference time, any integer timestep between and 100 ns will work.
layer. The temporal
layer is applied to structure
4 EXPERIMENT
In this section, we describe the details of our training and evaluation setup. Our results demon- strate that ProTDyn achieves performance equivalent to, even surpassing, state-of-the-art protein ensemble generators equilibrium ensemble generation, while additionally capable of capturing long-timescale dynamics. demonstrate the effectiveness of unifying the thermodynamics and dy- namics heads within a single model, we additionally train a model with same architecture and model parameters using only the dynamics information. We show that the unified model achieves signifi- cantly better performance on all dynamic prediction tasks, indicating that incorporating thermody- namic signals provides complementary information that improves dynamic inference.
4.1 TRAINING SETUP
Data Following BioEmu Lewis et al. (2025). combine single-structure and equilibrium MD datasets for thermodynamics training. For the single-structure dataset, we use the Swiss-Prot subset from the AlphaFold database, which contains 542,378 MD simulations, we include two sources: mdCath and BioEmu. For dataset construction and simu- lation details, encourage readers read both sources for comprehensive understanding.
structure pairs. For equilibrium
mdCath Mirarchi al. 2024)
5,398 proteins with simulation lengths up 500
BioEmu Lewis et al. 20255 prolonged simulations of diverse protein systems. The
BioEmu corpus includes several subsets:
Octapeptides Charron al. (025): : 1,100 peptides of length 8, each simulated for
5 ps.
CATH1 Charron et al. 2025
Sillitoe et al. (2021) 50 CATH domains, each simu-
lated for 100 jas.
-AATH2 Sillitoe et al. (2021) 1,100 CATH domains, each simulated for μss
-MEGAsim Tsuboyama et al. (023): extended simulations 271 wild-type proteins, together with 1 jas simulations for each of 22, 18 single-point mutants.
5

---

Published as a conference paper at ICLR 2026
For dynamics and dynamics inpainting training, we simulations from mdCath at temporal reso- lutions of ns, and from the Octapeptides, CATH2, and MEGAsim subsets at temporal resolutions of 10 and 100 ns.
Model details For better sequence and structure representation, we use the pretrained sequence and structure embedding head from ESM3 and freeze it throughout the training. The backbone of ProTDyn consists of 24 transformer blocks, which together contain 1.4 billion parameters.
Optimization We trained the model using AdamW optimizer Loshchilov & Hutter (2017) with learning rate of x 10-1 and weight decay of 1 x 10-5 We used a learning rate scheduler that reduces the learning rate by factor of 0.5 if the training loss does not improve for consecutive epochs.
4.2 INFERENCE SETUP
We conduct ablation studies with three types of sampling methods:
Thermodynamics:
independent and identically distributed (i.i.d)) sampling of protein conformational ensembles. In our experiments, we generated 2,000 ensembles for each protein.
2. Dynamics (10 ns): forward generative simulations with a time step of 10 ns. In our ex- periments, we generated 50 independent trajectories, each propagated for 100 steps, corre- sponding to a total of 50 jus of simulation time and 5,000 conformational ensembles.
3. Dynamics (100 ns): Forward generative simulations with a coarse time interval of 100 ns. To recover fine-grained details, each 100 ns interval is refined by dynamics inpainting into ten 10 ns sub-intervals. Similarly to the "Dynamics (10 ns)" sampling head, we generate 50 independent trajectories, each propagated for 10 coarse steps and refined by inpainting, also yielding 50 yrs of simulation time and 5,000 ensembles.
For the experiment of the thermodynamics head, we generate ensembles for 50 proteins from the CATH1 dataset. These proteins are included in the thermodynamic training and serve primar- ily as a benchmark against the baseline model. To further assess generalization, we also generate ensembles for 10 octapeptides that were not included in the training dataset.
For the experiment of the dynamics head, we apply both dynamic sampling heads to 50 proteins from the CATH1 dataset. Although these proteins part of the thermodynamies training set, they are excluded from dynamic and inpainting training. This setting enables direct evaluation of the generalization capacity Pro 's dynamic generation. We conduct same dynamic experiments using the model trained with only dynamic data, and use the same initial structures as starting point of dynamics during inference.
4.3 EVALUATION
Distributional similarity Our main comparison of ensemble distribution quality is against the BioEmu model Lewis et al. (2025). We selected BioEmu as our benchmark baseline because represents the most recent major advance in quantitative protein conformation ensemble modeling. In contrast, previous approaches were typically trained on very limited MD simulations of short duration. In fairness, we used a training dataset that closely matches the one used by BioEmu. We report the Jensen-Shannol divergence (JSD) between the reference MD simulations and pre- dicted conformation ensembles along various collective variables (CVs). The first set of CVs are low dimensional physical features: radius of gyration (Rg) and root mean square distance (RMSD) w.r.t the native structure predicted by AlphaFold 3 Abramson et al. (2024). The second set of CVs the top 2 independent components from time-lagged independent components analysis (TICA) Schultze & Grubmuuller (2021), representing the slowest dynamic modes of proteins. We evaluate distributional similarity ensembles generated by all three sampling heads.
Dynamical content We first evaluate the autocorrelation functions the TICA components. To fur- ther assess the accuracy of predicted dynamics, discretize the conformational space into states and construct Markov state models (MSMs) Husic & Pande (2018): Chodera & Noč (2014); Pande et al. 20100; Bowman al. (2013) estimate transition probabilities and stationary distributions.

---

Published as a conference paper at ICLR 2026
MD
Thermodynamics
Dynamics (100 ns)
Dynamics (10 ns)
Bioemu
1w0tA00
TICA
-2.
-7.5
1
3goxA01
TICA
0
18:05
12.63
10.53
8.42
4.21
Boss
2:11
0.00
2
-25
0.0
2.5
-2.5
0.0
2.5
-2.5
0.0
2.5
-2.5
0.0
2.5
3ethA03
-2
-2 TICA
-5.0
-2. 0.0
-2
-5.0
-2.5
0.0
-2.5
0.0
TICA 1
TICA
TICA
TICA
Figure 2: Free energy surface along the top two TICA components, parameterized from the back- bone torsion angles of reference MD simulations. The TICA projection then applied to conforma- tional ensembles generated by the three sampling heads of ProTDDy: (1) "Thermodynamiis" (2) "Dynamies (100 ns)", and (3) "Dynamics (10 ns)", as well as to ensembles generated by the baseline model BioEmu.
MSMs built from reference 100 μs MD trajectories serve as ground truth, while the MSMs con- structed from the ProTTDyn trajectories are used for evaluation. For direct comparison, we also construct MSMs from the first 25%. 50%, and 75% of the reference MD trajectories, corresponding to 25, 50, and 75 uas of simulation time, respectively. Following metrics from previous works Jing jet al. (2024b); Raja al. (2025), we report the Jensen-Shannon divergence (JSD) between the sta- tionary distributions of Markov states obtained from ProTDyn and those from reference MD. In addition, we report the JSD between
corresponding transition probability matrices. We also sample dynamic trajectories from MSM, and compute the mean negative log-likelihood (NLL) of dynamic trajectories. (details of MSM construction and evaluation methods in Appendix A
4.4 RESULT
Distributional similarity result We visualize the free energy surface (FES) of the TICA projection in Fig.2) [ As shown, all three ProTDDn sampling heads recover the major conformational metastable states along with qualitatively correct energy barriers that separate them.
For quantitative assessment, we report the ISD of the conformation ensembles generated by ProT- Dyn using the three sampling heads and BioEmu, average over proteins the test data in Table I Across all collective variables, ProTDyn shows good agreement with the reference 100 μss sim- ulations MD distributions. Among the three sampling heads, "Thermodynamics" sampling yields the highest-quality ensembles. This is expected, i.i.d. samples directly generated from the equilibrium distribution and while the invariant probably from dynamics head may be affected by possible error accumulation in autoregressive dynamic simulations.
Interestingly, we observe that "Dynamics (100 ns)" produces higher quality ensembles than "Dy- namics (10 ns)". We hypothesize that the "Dynamics (10 ns)" head, which requires many successive

---

Published as a conference paper at ICLR 2026
MD
Dynamics (100
Dynamics
10
1.0-
0.5
0.5
energ (10/
TICK
005
12.63
-0.
0.5
10.53
8.42
1.5
6.32
-15
-1.5
-.00
4.21
-2.5
2.11
-20
-2.0
330
0.00
TICA
TICA 1
TICA 1
Figure 3: Representative conformational metastable states and dynamic transition pathways illus- trated on the 2D TICA free energy surface (FES) for CATH1 protein system: 1b43A02.
steps to reach long timescales, is more susceptible to error accumulation and degradation of pre- dictive accuracy. In contrast, the "Dynamics (100 ns)" head provides a more stable strategy: large time steps enable robust exploration long-timesse timescale conformational transitions, while dynamic inpainting subsequently refines each interval to recover fine-grained transition details.
Another significant observation is that without training on thermodynamic information, the dynamics-only model diverges rapidly from the ground truth. It fails maintain the folded struc- ture and performs substantially much worse in capturing the distribution of collective variables. This demonstrates the importance of integrating thermodynamic signals predict the dynamic evolution.
Finally, we evaluate the generalization of the "Thermodynamics" head on 10 octapeptides unseen in the training dataset, with results reported in Table 2 We observe high level of agreement with the reference MD simulations and thermodynamic performance that is very close to the baseline BioEmu model. Note that these 10 proteins are included the BioEmu training dataset but not for ProTDyn. This demonstrates the strong generalizability of ProTTDyn's thermodynamic head beyond its training dataset.
Table 1: Distributional similarity evaluation on CATHI test dataset. Metrics are reported as Jensen-Shannon divergence (JSD) the radius of gyration (Rg), root-mean-square distance (RMSD) w.r.t the native structure, and the top two TICA components capturing slow protein dy- namics. Evaluation is conducted for ProTDyn under three different sampling heads and for the baseline method BioEmu.
Model
Rg
RMSD
TICA |
SerTDyy--Thermoyyaamme
0.023
0.012
0.155
WooTDyn-Dyyamann (10 ns)
0.052
0.032
0.278
MooTDyn-Dyamaan (100 ns)
0.030
0.018
0.206
rrdmDDyy-yynamic--ooly (100 ns)
0.077
0.142
0.315
BioEmu
0.082
0.137
0.293
Dynamical content result We visualize the dynamic pathways of a test protein in CATH1 in Fig, where systems demonstrate multiple conformation states. Both ProTDyn dynamic sampling heads can recover the conformation ensembles and the rare transition events. For qualitative analysis, we visualize the autocorrelation of the first two TICA components over 800 ns across all test proteins in Fig. The autocorrelation functions from "Dynamies (100 ns)" sampling head closely reproduce the reference results, whereas the autocorrelation functions from "Dynamics (10 ns)" head decay much faster than the the reference results. This observation is consistent with expectation that
8

---

Published as a conference paper at ICLR 2026
Table 2: Distributional similarity evaluation the Octapeptide test dataset, using the same evalua- tion metrics as in CATHI test dataset. Note that the these proteins are used as training proteins for the baseline model Bioemu but for our PronDyn model.
Model
Rg +
RMSD +
TICA
0.034
0.020
0.207
BioEmu
0.031
0.020
0.134
smaller time intervals accumulate errors more rapidly, leading to a loss of long-timescale correla- tions. Finally, construct MSM and quantitatively evaluate the JSD of stationary distributions and transition probability from MSM and the negative ligeeiilddood of dynamic paths between full reference MD simulations, Pro Prynn-n-geeeaan ence MD data (Table 3). The 50 jus trajectory generated with the "Dynamics (10 ns)" head recovers MSM stationary distributions and transition probabilities with accuracy comparable to 25 μrs of MD, while the "Dynamics (100 ns)" head achieves MSM quality similar to 50 jas of MD. These results demonstrate the strong dynamical fidelity of ProTDDy and highlight the effectiveness of combin- ing large-timestep generation with inpainting to recover fine-scale transitions. connstiioss.Finally.. evaluation of the dynamic trajectories generated by the dynamics-only model demonstrates that the absence of thermodynamic supervision results in poor dynamic performance, producing unrealistic trajectories and failing to recover realistic stationary distributions and transition probabilities. These evaluations highlight the necessity of unifying thermodynamics and dynamics head.
trajectories, and different portions of the refer-
MD
Dynamic 100ns
Dynamic 10ns
TICA1
TICA2
TICA1
TICA2
1.0
1w0zA00
3ossC00
0.5
Autocorrelation
0.5
250
500
750
250
500
750
250
500
750
250
500
750
3goxA01
3ethA03
0.5
0.5
250
500 Lagins
750
250
500
750
250
500
750
Lagins
Lagins
Lagins
Figure 4: Autocorrelation of the top two TICA components from 0 to 800 ns lag time, evaluated on four test CATHI proteins using reference MD trajectories and dynamic trajectories generated by the two dynamic sampling heads of ProTDyn.
Table 3: Dynamical contents evaluation of ProTDyn under two dynamic sampling heads. Evalua- tion metrics include Jensen-Shannon (JS) divergence of the stationary distribution and transition probability between Markov states, and mean negative log-likelihood (NLL) of 200 ns transition paths.
Model
Stationary JSD |
Transition JSD J
NLL +
ProTDyn--ynamics (10 ns)
0.042
0.105
0.806
ProiDya--ynamics (100 ns)
0.021
0.055
0,656
ProTDDya-dynamics-only (100 ns)
0.088
0.167
1.013
25 js MD
0.040
0.117
0.815
50 ps MD
0.018
0.049
0.682
75 us MD
0.007
0.019
0.639
9

---

Published as a conference paper at ICLR 2026
5 DISCUSSIONS
Limitations Similar to other generative models, ProTDDy is constrained by the availability and scale of training data, especially equilibrium MD simulation data. Its performance could be improved with access to larger and more diverse datasets. In addition, the current training procedure manually specifies the memory kernel at each timescale, without rigorous investigation of its optimal form. Recent studies have explored principled approaches to designing or learning memory kernels, and incorporating these insights could further strengthen the model. Wu al. (2024); Ge al. (2024)
Opportunities Two important capabilities of ProTDyn that we have not explored in detail are transi- tion path sampling and likelihood evaluation. Currently, ProTDyn employs transition path sampling purely as an inpainting technique to recover short-timescale details within long-timescale generative dynamics trajectories. However, rigorous transition path sampling requires specialized sampling strategies such transition interface sampling, milestoning, and string methods Van Erp & Bolhuis 2205); Pan et al. (2008); Votapka & Amaro (2015), and general-purpos MD simulations are not the most suitable training data for this purpose.
A key distinction of ProTDyn is that, through autoregressive modeling of discrete protein tokens, it provides exact likelihood evaluation for both conformational ensembles and dynamic trajectories. This contrasts with popular diffusion- or flow-based protein ensemble generators, where likelihoods only approximated and often computationally prohibitive to evaluate [Song al. (2020) Exact likelihoods create opportunities for integration with physics-base energy functions, or even the development of new top-down protein force field.
Finally, current generative models for proteins are not explicitly grounded in principles in statisti- cal mechanics, such as detailed balance Tolman (1925). With its likelihood evaluation capability, ProTDyn offers an opportunity to enforce such physical laws, thereby unifying thermodyname and dynamics. This could help overcome the limitations data scareity and enable more physically consistent generative modeling.
ETHICS STATEMENT
We have read and adhered to the ICLR Code of Ethics. Our study does not involve human subjects, sensitive personal data, applications with direct negative societal consequences. All experiments are conducted on publicly available datasets and open-source models. We are committed to faimesss transparency, and responsible reporting of results. To the best of our knowledge, this work does not present risks of misuse beyond those already inherent in large language models.
REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure reproducibility of our results. Details of the evaluation protocols are provided in the Appendix. All datasets used publicly available, and the preprocess- ing steps are clearly documented.
ACKNOWLEDGEMENTS
Guang Lin would like to thank the support of National Science Foundation 20MS-2533878, DMS- 2053746, DMS-2134209, ECCS-2328241, CBET-2347401 and OAC-231 1844), and U.S. Depart- ment of Energy (DOE) Office of Science Advanced Scientific Computing Research program DE- SC0023161, the SciDAC LEADS Institute, and DOE-Fusion Energy Science under grant number: DE-SC0024583. Ming Chen would like to thank supports from the National Science Foundation (CSEDI EAR-2246687..
REFERENCES
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans. Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630(8016):4935500 2024.
10

---

Published as a conference paper at ICLR 2026
Marloes Arts, Victor Garcia Satorras, Chin-War Huang, Daniel Zugner, Marco Federici, Cecilia Clementi, Frank Noć, Robert Pinsler, and Rianne van den Berg. Two for one: Diffusion mod- els and foree fields for coarse-grained molecular dynamics. Journal of Chemical Theory and Computation, 19(1 ((18))6151-6159 2023.
Gregory R Bowman, Vijay Pande, and Frank Noé. An introduction to Markov state models and their application to long timescale molecular simulation, volume 797. Springer Science & Busi- ness Media, 2013.
BO Brandsdal and AO Smalãs. Evaluation of protein- association energies by free energy perturbation calculations. Protein Engineering, 13(4))239- 245, 2000.
Nicholas E Charron, Klara Bonneau, Aldo S Preos- Andrea Guljas, Yaoyi Chen, Félix Musil, Jacopo Venturin, Daria Gusew, Iryna Zaporozhets, Andreas Krämer, al. Navigating protein landscapes with machine-learee transferable coarse-grainee model. Nature chemistry, pp. 1-9, 2025.
John D Chodera and Frank Noé. Markov state models of biomolecular conformational dynamics. Current opinion in structural biology, 13:-1414 2014.
Yuanqi Du, Michael Plainer, Rob Brekelmans, Chenru Duan, Frank Noe, Carla P Gomes, Alan Aspuru-Guzik, and Kirill Neklyudov. Doob's lagrangian: A sample-efficient variational approach to transition path sampling. Advances in Neural Information Processing Systems, 37:65791- 65822, 2024.
Xiang Fu, Tian Xie, Nathan J Rebello, Bradley D) Olsen, and Tommi Jaakkola. Simulate time- integrated coarse-grained molecular dynamics with multi-scale graph networks. arXiv preprint 22X1::2204.10348, 2022.
Pei Ge, Zhongqiang Zhang, and Huan Lei. Data-driven learning of the generalized langevin equation with state-dependent memory. Physical Review Letters, 133(7):077301, 2024.
Jingjing Guo and Huan-Xiana Zhou. Protein allostery and conformational dynamics. Chemical reviews, 1 116(1 1):6503-655 2016.
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of cvolution with a language model. Science, 38766736):850 858, 2025.
Moritz Hoffmann, Martin Scherer, Tim Hempel, Andreas Mardt, Brian de Silva, Brooke E Hu- sic, Stefan Klus, Hao Wu, Nathan Kutz, Steven L Brunton, et al. Deeptime: a python library for machine learning dynamical models from time series data. Muchine Learning: Science and Technology, 311)001500, 2021.
Brooke E Husic and Vijay S Pande. Markov state models: From an art to a science. Journal of the American Chemical Society, 140(7):2386-2396 2018.
Jesús A Izaguirre, Sebastian Reich, and Robert D Skeel. Longer time steps for molecular dynamics. The Journal of chemical physics, 1 110(00)9853-99)) 9864, 1999.
Bowen Jing, Bonnie Berger, and Tommi Jaakkola. Alphafold meets flow matching for generating protein ensembles. arXiv preprint arXiv:2402.04484 04845, 2024a.
Bowen Jing, Hannes Stärk, Tommi Jaakkola, and Bonnie Berger. Generative modeling of molecular dynamics trajectories. Advances in Neural Information Processing Systems, 37:40534-40564, 2024b.
Tony Lelièvre, Geneviève Robin, Innas Sekkat, Gabriel Stoltz, and Gabriel Victorino Cardoso. Gen- erative methods for sampling transition paths in molecular dynamies. ESAIM: Proceedings and Surveys, 73:238-256, 2023.
Sarah Lewis, Tim Hempel, José Jiménez-Luna, Michael Gastegger, Yu Xie, Andrew YK Foong, Victor García Satorras, Osama Abdin, Bastiaan S Veeling, Iryna Zaporozhets, et al. Scalable emulation protein equilibrium ensembles with generative deep learning. Science, pp. eadv9817, 2025.
11

---

Published as a conference paper at ICLR 2026
lly Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.0100120000 2017.
Jiarui Lu, Bozitao Zhong, Zuobai Zhang, and Jian Tang. Str2str: A score-based framework for
zero-shot protein conformation sampling. arXiv preprint arXiv:2306.03111 2023.
Antonio Mirarchi, Toni Giorgino, and Gianni De Fabritiis. mdcath: A large-scale md dataset for data-driven computational biophysies. Scientific Data, 11(1):1299, 2024.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium
states of many-body systems with deep learning. Science, 365(6457) 1147, 2019.
Albert C Pan, Deniz Sezer, and Benoît Roux. Finding transition pathways using the string method with swarms of trajectories. The journal of physical chemistry B, 11:31)33332-3444 2008
Vijay S Pande, Kyle Beauchamp, and Gregory R Bowman. Everything you wanted to know about markov state models but were afraid to ask. Methods, 52(1):99-105, 2010. 2010.
Sanjeev Raja, Martin Sipka. Michael Psenka, Tobias Kreiman, Michal Pavelka, and Aditi S Kr- ishnapriyan. oetion-minimization meets generative modeling: Efficient transition path sampling with the onsager-machlup functional. arXiv preprint arXiv:2504. 18506, 2025.
Paul Robustelli, Stefano Piana, and David E Shaw. Developing molecular dynamics force field for both folded and disordered protein states. Proceedings of the National Academy of Sciences, 115 (21):E4758--E47) 2018
Steffen Schultze and Helmut Grubmuuller. Time-lagged independent component analysis random walks and protein dynamics. Journal of Chemical Theory and Computation, 17(9):5766-5776 2021.
David E Shaw, Paul Maragakis, Kresten Lindorff-Larsen. Stefano Piana, Ron O Dror, Michael P Eastwood, Joseph A Bank, John M Jumper, John K Salmon, Yibing Shan, et al. Atomic-level characterization of the structural dynamics of proteins. Science, 330(6002):341-346, 2010.
lan Sillitoe, Nicola Bordin, Natalie Dawson, Vaishali Waman, Paul Ashford, Harry M Scholes, Camilla SM Pang. Laurel Woodridge, Clemens Rauer, Neeladri Sen, et al. Cath: increased struc- tural coverage of functional space. Nucleic acids research, D9D11):D26-D223, 2021.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13355 2020.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
Richard C Tolman. The principle of microscopic reversibility. Proceedings of the National Academy of Sciences, 11(7):436 439, 1925.
Kotaro Tsuboyama, Justas Dauparas, Jonathan Chen, Elodie Laine, Yasser Mohseni Behbahani, Jonathan Weinstein, Niall M Mangan, Sergey Ovchinnikov, and Gabriel Rocklin. Mega-scale experimental analysis of protein folding stability in biology and design. Nature, 620(7973):434- 444, 2023.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation leaming. Advances in neural information processing systems, 30, 2017.
Titus S Van Erp and Peter G Bolhuis. Elaborating transition interface sampling methods. Journal of computational Physics, 205(1):157-811, 2005.
Lane W Votapka and Rommie E Amaro. Multiscale estimation of binding kinetics using brownian dynamics, molecular dynamics and milestoning. PLoS computational biology, 11(10)c1104381, 2015.
12

---

Published as a conference paper at ICLR 2026
Hannah K Wayment-Steee Steele, Adedolapo Ojoawo, Renee Otten, Julia M Apitz, Warintra Pitsawong, Mare Hömberger, Sergey Ovchinnikov, Lucy Colwell, and Dorothee Kem. Predicting multiple conformations via sequence clustering and alphafold2. Nature, 625(7996):832 839, 2024.
James C Whisstock and Arthur M Lesk. Prediction of protein function from protein sequence and Structure. Quarterly reviews of biophysics, 36(3):307-340, 2003.
Yue Wu, Siqin Cao, Yunrui Qiu, and Xuhui Huang. Tutorial on how to build non-markovian dynamic models from molecular dynamics simulations for studying protein conformational changes. The Journal of Chemical Physics, 160(12), 2024.
Shuxin Zheng, Jiyan He, Chang Liu, Yu Shi, Ziheng Lu, Weitao Feng, Fusong Ju, Jiaxi Wang, Jianwei Zhu, Yaosen Min, et al. Predicting equilibrium distributions for molecular systems with deep learning. Nature Machine Intelligence, 6(5):558-56) 2024.
13

---

Published as a conference paper at ICLR 2026
A EXPERIMENT DETAILS
A.1 DISTRIBUTIONAL SIMILARITY EVALUATION
Radius of gyration. The radius of gyration (R2) is a fundamental descriptor of protein structure, quantifying the overall compactness of a molecule. It is defined as the souureean-squree distance of the constituent atoms from their common center of mass, thereby capturing how mass is distributed around the protein's 's centroid.
The radius of gyration is defined
N rr Fem 1
R₂
(11)
where N is the number of atoms, r, is the position vector of the i-th atom, and Tem is the position vector the center of mass. This expression measures the spatial dispersion the atomic positions relative to the center of mass. In this work, R, is computed using only the backbone atoms.
Root Mean Square Distance (RMSD). The root mean square distance (RMSD) is a standard metric in structural biology for quantifying the similarity between protein conformations. RMSD is computed by superimposing the two structures and calculating the square root of the average squared distance between corresponding atoms:
N
RMSD
min ESB(3)
(r(r,) - 1
(12)
N
where N is the number of atoms, are the atomic coordinates of the structure under comparison, r are the coordinates of the reference structure, and T, E SE(3) denotes the optimal rigid-body transformation (rotation and translation) aligning the two structures. Lower RMSD values indicate greater structural similarity.
Time-lagged Independent Component Analysis (TICA). TICA is linear method for extracting the slowest dynamical modes from time-series data, widely used in molecular dynamics. Unlike PCA. which finds directions of largest variance, TICA identifies directions with maximal autocorre- lation at lag time r. solves the generalized eigenvalue problem
C+r, CoAprs
(13)
where Co is the covariance matrix, C. is the time-lagged covariance, and X the time- autocorrelations. The resulting components capture the slow collective motions that dominate long- timescale protein dynamics. In this project, choose the feature protein backbone torsion angles, and extract the top 2 TIC components.
A.2 DYNAMICAL CONTENTS EVALUATION
Autocorrelation We define the autocorrelation of each TICA component as
[[( 11) ()) /²
(14)
where μ4, are computed from the reference MD simulation trajectory. We conduct TICA on lag times = [10, 20...., 800] ns.
Markov state models Markov state models (MSMs) provide a statistical framework for describ- ing the long-timescale dynamics of biomolecules by coarse-graining the continuous conformational space into finite set of metastable states. The dynamics are modeled chain, where the probability of transitioning between states depends only on the current state and chosen lag time Formally, the state-to-state transition probabilities are encoded in transition matrix T(7):
discrete- sicrete-timee Markov
(t(++)) = =0000 Ep((t)
(15)
14

---

Published as a conference paper at ICLR 2026
where (t()) is the probability of being in state at time t, and (ry(F) is the probability of transitioning from state to j over lag time т. At equilibrium, MSMs satisfy the stationary distribution condition
TT(),,
Σn =
(16)
where my denotes the equilibrium probability of state i.
We follow previous works Jing et al. (2024b); Raja et al. 2023 and build MSM using Deeptime Hoffmann et al. (202T). We first represent protein systems with backbone torsion angles and run TICA to obtain the top 2 Time Independent Component dimensions. We then perform k-means clustering of the reference MD simulations into 10 clusters. We then fit MSM with lag time of 10 ns. This gives us the transition probability matrix T. The stationary distribution æ can be easily obtained the left eigenvector of the transition matrix with eigenvalue 1. We use the same settings to construct MSM for reference MD simulations. dynamic trajectories generated by ProTDDy, and different portions of reference MD simulations.
We evaluate the following two metrics:
Stationary distribution Jenson-Shannon divergence We evaluate the stationary distribu- tion of each MSM state for reference and comparison trajectories, and compute the JSD between the categorical distributions.
Transition probability Jensen-Shannon divergence We evaluate the transition probabil- ity matrix between MSM states for both reference and comparison trajectories, and com- pute the JSD between the corresponding transition probability distributions.
Trajectory negative log likelihood We use the stationary distribution of reference MSM to generate 1000 initial states: 80 Tref and then propagate dynamic path of length L = 20 from each iniitial state for each comparison MSM using its transition proability matrxi: St+1 Te. The probability of path can thus be written P(s
1 Ele log Test We report the mean negative log likelihood over all trajectories.
B ADDITIONAL EXPERIMENT RESULTS
15

---

Published as a conference paper at ICLR 2026
MC
Themedmunia
Dynamia 1300 rs
Bmamica (22 -sl
Boemm
1b43A02
1bi0A02
1c75A00
2
-05
-10
-25
2.0
1lz6A02
35
05
-25
-25
-1.
Figure 5: Additional results on test CATHI proteins: free energy surface along the top two TICA components and dynamic transition pathways. Note that the thermodynamics head of ProTDDn and bascline model Bioemu are both i.id sampler and thus does not have dynamic trajectory.
16

---

Published as a conference paper at ICLR 2026
Dysamics
ImpgA00
1wfxA01
2bh1xd0
Figure 6: Additional results another 5 test CATH1 proteins.
17

---

Published as a conference paper at ICLR 2026
Themodynamics
1.0
0.5
Octa0006
0.0
0.0
005
25
-1.0
-15
Octa0008
3.0
3.0
2.5
2.5
Octa0013
2.0
1.0
0.5
0.0
0.0
005
-1.0
Octa0022
0
-1
Octa0024
TICA 1
TICA
TICA 1
Figure 7: Additional results on test Octapeptides: free energy surface along the top TICA components.
18

---

Published as a conference paper at ICLR 2026
Thermadynamics
2.5
1.5
2.0
Octa
1.0
10-
0.5
0
0.0
-05
-1.0
0
3
-2
2
Octa0034
Octa0039
05
-05
Octa0040
5
-1
2.5
2.5
2.0
2.0
Octa0049
15
3.5
3.0
0.5
0.
0.5
0.0
-0.5
-05
1.0
TICA
TICA 1
TICA
Figure 8: Additional results another 5 test Octapeptides: free energy surface along the top two TICA components.
19


Reviews:
[{"review_id": "8vdzoupe0L", "reviewer": "Reviewer_87Fq", "summary": "The paper proposes a protein foundation model based on ESM3 that unifies equilibrium distribution modelling and dynamics generation for protein structures. Training of the model optimizes the loss for distribution modeling, dynamic trajectory prediction, and dynamic trajectory inpainting at the same time. To enable the modeling of dynamics, ProTDyn proposes two-layer rotary embedding scheme to combine time rotary embedding. Experiment results show that ProTDyn performs than baselines in metrics including distributional similarity. ProTDyn also shows reasonable performance in capturing state transitions consistent with molecular dynamics.", "questions": "1.Before ProTDyn, the paper “Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics” also discusses how the connections between score-based generative models and force fields can be leverage to train a diffusion model on MD simulations, and the trained score function can be used to simulate MD trajectories. It would be valuable to clarify the difference and advantage of ProTDyn when compared with these methods.\n\n2.It would be interesting to see how ProTDyn works on larger proteins. Particularly, it would both enhance the presentation of the paper and provide more evidence of the generalizability of ProTDyn if the predicted structure/distribution can be demonstrated with figures.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.The paper proposes an interesting view of enabling one model for both equilibrium distribution sampling and temporal molecular dynamic modeling.\n\n2.The experiment results prove the effectiveness of the method in distribution modeling compared with strong baselines.\n\n3.The paper is clearly written with the mothod being simple yet elegant.", "weaknesses": "1.The benefit of unifying thermodynamics and dynamics generation is not explicitly discussed in this paper. It would be interesting to see how the two tasks interfered with each other in terms of performance. This would also provide stronger support for the motivation of the unification.\n\n2.Some important details of the method are missing. For example, how the “two-layer rotary embedding scheme” is designed to combine temporal and residue positions. Given the model architecture leverages the ESM3 backbone, such details are important for the readers to understand the key adaptation to enabling the dynamics generation.\n\n3.The evaluation metric for the distributional similarity are along low dimensional collective variables. Specifically, distribution along the Rg and RMSD (which are single values not reflecting details of the protein structures) w.r.t. the native structure seems to be less significant in reflecting the true distribution in 3D Euclidean space with all the residues.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1R6MNIXOW8", "reviewer": "Reviewer_NTbS", "summary": "This paper introduces ProTDyn, a foundation PLM to unify equilibrium distribution sampling, time-dependent dynamics sampling as well as inpainting for fine-grained timesteps based on coarse grained timesteps. It has been trained on a large scale molecular dynamics dataset, and demonstrated promising performance in the generated sample distribution in comparison with baseline models. I find this methodology very interesting and novel, and has shown scaling up in the model and data.", "questions": "- Could you clarify the train/test splitting procedure? Was any filtering based on sequence identity or structural similarity\n- Have the authors considered other architectures such as those AF-based models? What was the specific rationale for choosing the ESM3 architecture?\n- BioEmu has been finetuned on experimental data after training on MD, so the comparison on MD distribution might not be the most direct comparison, which can be noted.\n- The method aims to generate coarse-grained dynamics. It'll be helpful to demonstrate the computational cost in comparison with MD simulation\n- Have the authors considered the asymptotic behavior of the time dependence? e.g. when the $\\delta t \\to \\inf$, it should approach an i.i.d. distribution", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**Novel methodology**\n- The method is novel to incorporate three tasks in one model:\n  1. i.i.d. equilibrium distribution sampling\n  2. time lagged predictions with multiple time scales\n  3. inpainting: given two coarse grained time steps, predict fine grained steps in between\n- This methodology overcomes the lack of capability in the description of dynamics/kinetics in the equilibrium sampling models\n\n**Transferability**\n- The model is trained on multiple time scales, showing transferability in the time dependence\n- The model is trained on a large scale of MD data, showing transferability in the chemical space\n\n**Performance**\n- The performance has been compared on MD sample distributions with proper baselines and benchmarks, and has demonstrated the advantage of this model", "weaknesses": "- **Generalization**: For the dynamics tasks, the test proteins were seen by the model during thermodynamics training, which may not represent true generalization. A stronger evaluation would involve holding out proteins based on sequence similarity\n- **Baseline**: This model is only compared against BioEmu as the only baseline. It can consider adding more baseline models, such as Alphaflow which is also trained on mdCATH, as well as a few other works that baked in the time dependence for dynamics generation of protein models.\n- **Inpainting task**: Dynamics inpainting is highlighted as a key capability, but its performance is only measured indirectly through the quality of the final, end-to-end generated trajectories. There are no specific benchmarks that isolate and quantify the accuracy of the inpainting process itself", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dex90fdW3x", "reviewer": "Reviewer_7aKu", "summary": "In this paper, the authors present ProTDyn which is a transformer operating on ESM3 structure tokens trained for conformation and dynamics sampling. The model is trained on a mixture of the alphafold synthetic database of structures and molecular dynamic datasets. It is trained for both equilibrium distribution sampling as well as jumping ahead in MD simulations or inpainting in between coarse time steps in MD simulations. The model improves over BioEmu, a recent deep learning model for conformer generation, in terms of distributional match to MD.", "questions": "Why did you decide to use a causal autoregressive distribution for the generation of structural tokens? In ESM3, an any-order autoregressive model was used to generate tokens in parallel which seems to make sense for protein data. Did you try any other parameterizations?\n\nHow were the proteins in Figure 2 selected? If these were cherry-picked as to best show the benefits of your method this should be stated clearly. Or if these are exhaustive of the CATH2 dataset, this should also be made clear because this is important for judging the significance of your findings.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The training of the model is simple in that all 3 tasks of thermodynamics, dynamics and dynamics inpainting are trained jointly and simultaneously in a single model. This avoids the need for any task specific fine-tuning which is a downside of prior approaches. All tasks are unified in terms of autoregressive modelling with different factorizations.\n\nThe experimental results look strong in terms of improvement over the BioEmu baseline model. Distibutional fit on MD test data looks quite convincingly better in Table 1 and in Figure 2.\n\nThe authors make some interesting observations in terms of the time resolution with which to sample their dynamics model. They find that sampling with a finer grid can lead to worse overall performance potentially due to error accumulation. This finding will be useful for further research into deep learning based dynamics models.", "weaknesses": "A main weakness of the paper is the lack of ablations with respect to the stated contribution of ProTDyn being a unified model for Thermodynamics and Dynamics generation. On L56-L58 in the introduction the authors describe how joint training can be mutually beneficial for both tasks. I would therefore expect experiments and ablations showing that this is indeed the case. You could train a version of your model on the tasks individually and compare performance with the multi-task trained model. I don't think comparing to BioEmu is enough to make this claim since BioEmu is pretrained on Thermodynamics style tasks and fine-tuned on dynamics tasks which can be seen as a form of multitask training.\n\nWith regards to Table 2, where these proteins are in your test set but BioEmu's training set, I am unsure how ProTDyn ended up with a different test set to BioEmu that made this comparison difficult. If you are using the same datasets as BioEmu, why not also use the same splits? In the end, it makes it very hard to draw any conclusions from Table 2 since the models are not comparable.\n\nThe paper is quite unclear with the usage of the term 'module'. The authors state that they have a 'thermodynamics module' and a 'dynamics module'. This would imply separate parts of the network that are being trained for these two tasks which would be in contradiction with the introduction describing a single unified model which is a key stated contribution of the work. I am unsure what the authors are referring to with regards to these modules and this should be made clearer. \n\nFurther, in terms of clarity, on L149 the authors describe their model as a 'multimodal protein language model'. In what way is this a multimodal model because as far as I am aware the model just generates structural tokens (and no other modalities).\n\n\nI am tending to accept this paper due to the strong experimental results with regards to BioEmu and simple training scheme. This model seems to make a significant improvement on the state of the art for conformational modelling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ggOklTNLme", "reviewer": "Reviewer_x8Ec", "summary": "The work proposes ProTDyn, a model that is able to do conformation ensemble generation and multi-timescale dynamics modeling within a single model.", "questions": "I'm slightly confused by Figure 1 as the authors mention it is a language model, but there does not seem to be language/text as input in the diagram. Could the authors please clarify? Are the authors using \"language model\" to highlight the fact that it is an autoregressive (causal/decoder) Transformer architecture and not that the input is language.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-written and easy to follow. The fact that the authors present a model that is able to perform simultaneously equilibrium conformation ensemble generation and multi-timescale dynamic trajectory generation is interesting and as far as I am aware novel.", "weaknesses": "As the paper is not near my main area of research, it is hard for me to make questions or comment on the weaknesses although I found the paper easy to follow and appreciated especially the explanation on page 4. \n\nThe paper is a rather standard application of ML modelling techniques but unfortunately I cannot comment on the strength of the results. For this reason I will leave a weaker confidence score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)