PROTDYN: A FOUNDATION PROTEIN LANGUAGE MODEL FOR THERMODYNAMICS AND DYNAMICS GENERATION

## Abstract

Abstract Molecular dynamics (MD) simulation has long been the principal computational tool for exploring protein conformational landscapes and dynamics, but its application is limited by high computational cost. We present ProTDyn, a foundation protein language model that unifies conformational ensemble generation and multi-timescale dynamics modeling within a single framework. Unlike prior approaches that treat these tasks separately, ProTDyn allows flexible independent and identically distributed (i.i.d.) ensemble sampling and dynamic trajectory simulation. Across diverse protein systems, ProTDyn yields thermodynamically consistent ensembles, faithfully reproduces dynamical properties over multiple timescales, and generalizes to proteins beyond its training data. It offers a scalable and efficient alternative to conventional MD simulations. Code is available at: https://github.com/Harrydirk41/ProTDyn .

## 1 Introduction

Proteins are the fundamental building blocks of life, carrying out essential functions such as catalysis, signaling, and transport. Understanding their conformation flexibility and dynamic nature is crucial for uncovering the molecular mechanism of protein functions Whisstock & Lesk (2003). Traditional computational methods, such as molecular dynamics (MD) simulations, have been optimized and widely used for decades to study biomolecular processes such as protein folding and unfolding Shaw et al. (2010); Robustelli et al. (2018). However, MD remains computationally expensive, since the simulation time step must be orders of magnitude smaller than the timescales of biologically relevant processes Izaguirre et al. (1999). As a result, simulating long-timescale protein dynamics is often infeasible.

Recent advances in machine learning, particularly deep generative models, have introduced powerful alternatives to model proteins. Generative approaches have been developed to sample equilibrium conformational ensembles Lu et al. (2023); Lewis et al. (2025); Jing et al. (2024a); Noé et al. (2019); Zheng et al. (2024); Wayment-Steele et al. (2024) and, separately, to learn protein dynamics Jing et al. (2024b); Raja et al. (2025); Lelièvre et al. (2023); Du et al. (2024); Fu et al. (2022). Recent models also incorporate the principles of statistical mechanics that correlates thermodynamics and dynamics Raja et al. (2025); Arts et al. (2023). However, scalable and transferable protein emulator models that can simultaneously describe both equilibrium conformational ensembles (thermodynamics) and conformational transitions across multiple timescales (dynamics) are still unavailable, which limits further progress such as accurately modeling protein biophysics, where both equilibrium ensembles and transition dynamics are essential for understanding protein–protein interaction, allosteric regulation, biocondensation, and conformational heterogeneity Brandsdal & Smalås (2000); Guo & Zhou (2016).

Based on recent progress and gaps, we introduce ProTDyn, a unified framework for generative modeling of Protein Thermodynamics and Dynamics. Unlike previous approaches that treat conformational ensemble generation and conformational dynamics propagation separately, ProTDyn unifies them within a single multimodal architecture. ProTDyn is trained on hundreds of thousands of protein sequences and over a million of conformations, leveraging both single-structure and equilibrium MD simulation data to train the model for comprehensive understanding of protein conformational space. Moreover, ProTDyn can perform multi-timescale training which enables modeling conformational transitions of diverse protein systems with timescale ranging from nanoseconds to microseconds. This flexible scheduling bridges short- and long-timescale dynamics. Together, these capabilities are mutually reinforcing: accurate thermodynamic ensembles provide stable baselines for dynamic propagation, while realistic dynamics improve the diversity and fidelity of generated equilibrium ensembles.

In sum, we demonstrate three key capabilities of ProTDyn:

- 1.
Thermodynamics generation: sampling independent and identically distributed (i.i.d.). equilibrium protein structures from the learned ensemble distributions that are consistent with Boltzmann statistics.
- 2.
Multiscale dynamics generation: generating temporally coherent trajectories at multiple time-resolutions, capturing both fast local fluctuations and slow global transitions.
- 3.
Dynamics inpainting: refining coarse time-resolution trajectories by recovering fine-grained time-resolution, physically plausible dynamic pathways.

By unifying thermodynamics and dynamics within a single generative framework, ProTDyn enables scalable, transferable, flexible, and computationally efficient protein modeling. In particular, compared to existing protein ensemble and dynamics generators, ProTDyn offers greater flexibility in generating conformational ensembles across diverse settings. We validate the accuracy and transferability of ProTDyn on multiple tasks, including generating conformational ensembles for proteins outside the training dataset and simulating protein dynamics beyond the training regime. Experimental results confirm that ProTDyn agrees well with reference MD simulations while generalizing effectively to unseen systems.

## 2 Background

Molecular dynamics.
Molecular dynamics (MD) simulates the time evolution of a molecular system by integrating Newton’s equations of motion.
For each particle $i$ in a molecular configuration $\mathbf{x}=(\mathbf{x}_{1},\ldots,\mathbf{x}_{N})\in\mathbb{R}^{3N}$, the equations are

$$ $M_{i}\ddot{\mathbf{x}}_{i}=-\nabla_{\mathbf{x}_{i}}U(\mathbf{x}_{1},\ldots,\mathbf{x}_{N}),$ (1) $$

where $M_{i}$ is the mass of particle $i$ and $U:\mathbb{R}^{3N}\to\mathbb{R}$ is the potential energy that is often modeled by a force field. By construction, an MD simulation at a fixed temperature $T$ will converge to the Boltzmann distribution of the system,

$$ $P(\mathbf{x})\propto e^{-U(\mathbf{x})/k_{B}T},$ (2) $$

where $k_{B}$ is the Boltzmann constant.

Deep generative modeling for proteins.
Recent advances in deep generative modeling have opened new opportunities to simulate protein conformational ensembles. These models can generate conformational ensembles and transitions within hours, offering an efficient alternative to conventional MD simulations.
One line of work targets *thermodynamics*, directly learning the stationary Boltzmann distribution $P(\mathbf{x}|s)$ over conformations from structural databases or equilibrium MD trajectories Lu et al. (2023); Lewis et al. (2025); Jing et al. (2024a); Noé et al. (2019); Zheng et al. (2024); Wayment-Steele et al. (2024).
Such models recover equilibrium ensembles but lack dynamical information.
In addition to modeling thermodynamics, complementary approaches focus on *dynamics*, learning the transition density $P(\mathbf{x}_{t+\Delta t}\mid\mathbf{x}_{t},s)$ to accelerate MD  Jing et al. (2024b); Raja et al. (2025); Lelièvre et al. (2023); Du et al. (2024); Fu et al. (2022).
Although these methods can predict short-time kinetics, they are often trained on limited MD data, restricting their ability to generalize to rare or long-timescale transitions.
Despite rapid progress, current approaches remain specialized in either thermodynamics or dynamics, overlooking their intrinsic connection based on statistical mechanics.
A unified foundation model capable of predicting both equilibrium ensembles and transition dynamics across scales has yet to be established.

Protein conformation representation.
Although proteins are inherently three-dimensional objects, recent advances have shown that conformations can also be represented as sequences of discrete tokens, enabling the use of powerful sequence modeling techniques.
In this work, we adopt the pretrained ESM3 Hayes et al. (2025) structure tokenizer to map protein conformations to tokenized sequences $\mathbf{c}\in\mathbb{Z}^{N}$, where each residue is assigned one of the $4{,}096$ structure tokens (plus $4$ special tokens).
These tokens provide a compact, learned representation of the local structural neighborhood around each residue.
The discretization is performed with a VQ-VAE encoder Van Den Oord et al. (2017), while a paired decoder reconstructs the generated token sequences back into three-dimensional coordinates.

## 3 Method

In this section, we will introduce the high-level framework of ProTDyn, as illustrated in Fig. [1](https://arxiv.org/html/2510.00013v1#S3.F1). ProTDyn is a multimodal protein language model designed to perform three complementary tasks within a single framework: (i) equilibrium conformation ensemble generation (thermodynamics), (ii) multi-timescale dynamic trajectory generation (dynamics), and (iii) fine-grained trajectories recovery from coarse time-resolution trajectories (dynamics inpainting). The model operates by autoregressively predicting the structure tokens of successive residues. Its architecture builds upon the state-of-the-art protein language model ESM3, and additionally incorporates a temporal positional embedding to encode dynamic transition information.

Figure: Figure 1: An illustrative framework of ProTDyn. ProTDyn is a protein language model that operates on discretized representations of protein sequence and structure. It leverages a powerful autoregressive transformer architecture to simultaneously perform three tasks: (i) equilibrium conformational ensemble generation (thermodynamics), (ii) forward trajectory generation across multiple timescales (dynamics), and (iii) recovery of fine-grained trajectories from coarse trajectories (dynamics inpainting).
Refer to caption: x1.png

### 3.1 Objective

We represent protein conformations in a discrete tokenized space and define three complementary learning objectives:

- 1.
Thermodynamics: the thermodynamics module of ProtDyn aims to learn the equilibrium distribution of conformations $P_{\theta}(\mathbf{c}\mid s)$. Specifically, it models the equilibrium conformation distributions autoregressively as
$P_{\theta}(\mathbf{c}\mid s)=\prod_{i=0}^{N-1}P_{\theta}(\mathbf{c}_{i}\mid\mathbf{c}_{<i},s),$
(3)
where $\mathbf{c}_{i}$ denotes the structure token of residue $i$, and $\mathbf{c}_{<i}$ represents all preceding residues. The thermodynamics module is learned by minimizing the cross entropy between $P_{\theta}$ and the observed protein equilibrium conformation ensemble distribution:
$\mathcal{L}_{\text{thermo}}(\theta)=-\mathbb{E}_{(s,\mathbf{c})\sim\mathcal{D}}\Bigg{[}\sum_{i=0}^{N-1}\log P_{\theta}(\mathbf{c}_{i}\mid\mathbf{c}_{<i},s)\Bigg{]},$
(4)
where $\mathcal{D}$ denotes the dataset of protein sequences and their equilibrium conformations.
- 2.
Dynamics: the dynamics module aims to learn the temporal correlations across multiple timescales $P_{\theta}\big{(}\mathbf{C}\mid s\big{)},$
where $\mathbf{C}=(\mathbf{c}^{0},\ldots,\mathbf{c}^{M\delta t})$ is a trajectory segment of length $M$ with time step $\delta t$. We factorize the trajectory distribution as
$P_{\theta}(\mathbf{C}\mid s)=\prod_{j=0}^{M}P_{\theta}(\mathbf{c}^{j\delta t}\mid\mathbf{C}^{<t},s),$
(5)
where $\mathbf{C}^{<t}$ represents all preceding conformations up to time step $t-1$. Each conformation $\mathbf{c}^{j\delta t}=(\mathbf{c}_{0}^{j\delta t},\ldots,\mathbf{c}_{N-1}^{j\delta t})$ is further decomposed residue-wise as
$P_{\theta}(\mathbf{c}^{j\delta t}\mid\mathbf{C}^{<t},s)=\prod_{i=0}^{N-1}P_{\theta}(\mathbf{c}_{i}^{j\delta t}\mid\mathbf{c}_{<i}^{j\delta t},\mathbf{C}^{<t},s).$
(6)
The model parameters are optimized by minimizing the negative log-likelihood of observed trajectory data:
$\mathcal{L}_{\text{dyn}}(\theta)=-\mathbb{E}_{(s,\mathbf{C})\sim\mathcal{D}}\Bigg{[}\sum_{j=0}^{M}\log P_{\theta}(\mathbf{c}^{j\delta t}\mid\mathbf{C}^{<t},s)\Bigg{]},$
(7)
where $\mathcal{D}$ denotes the dataset of protein sequences paired with dynamic segments of length $M$. In this project, we aim to capture protein dynamical behaviors at multiple timescales. Specifically, we let the model learn $\delta t=1\,\text{ns}$, $10\,\text{ns}$, and $100\,\text{ns}$ resolution. Additionally, we set a memory kernel of $M=10$, which corresponds to effective timescales of $10\,\text{ns}$, $100\,\text{ns}$, and $1000\,\text{ns}$, respectively.
- 3.
Dynamic inpainting:
The dynamics module enables trajectory generation with large timesteps, but this comes at the cost of losing fine-grained temporal resolution. To address this, we introduce a dynamic inpainting module that reconstructs physically plausible transition sequences between metastable conformational states. Formally, the inpainting task is to recover fine-resolution conformations trajectory $\mathbf{C}$ between two states $\mathbf{c}^{0}$ and $\mathbf{c}^{M\delta t}$, modeled from coarse trajectories $P_{\theta}(\mathbf{C}\mid\mathbf{c}^{0},\mathbf{c}^{M\delta t},s)$.
Similar to dynamics module, the dynamic inpainting between two states $\mathbf{c}^{0}$ and $\mathbf{c}^{M\delta t}$ is formulated as an autoregressive conditional generation problem:
$P_{\theta}(\mathbf{C}\mid\mathbf{c}^{0},\mathbf{c}^{M\delta t},s)=\prod_{j=1}^{M-1}P_{\theta}(\mathbf{c}^{j\delta t}\mid\mathbf{C}^{<t},\mathbf{c}^{0},\mathbf{c}^{M\delta t},s),$
(8)
and the training objective is:
$\mathcal{L}_{\text{dynI}}(\theta)=-\mathbb{E}_{(s,\mathbf{C})\sim\mathcal{D}}\Bigg{[}\sum_{j=1}^{M-1}\log P_{\theta}(\mathbf{c}^{j\delta t}\mid\mathbf{C}^{<t},\mathbf{c}^{0},\mathbf{c}^{M\delta t},s)\Bigg{]},$
(9)
The choice of time step is 1 ns and 10 ns and memory kernel follows the same as training of dynamics module.

Together, these objectives enable the model to simultaneously generate equilibrium ensembles and reproduce protein dynamics at multiple timescales. During training, ProTDyn is optimized to minimize losses from all three modules. The losses are combined using hyperparameter weights:

$$ $\mathcal{L}_{\text{ProTDyn}}=\omega_{1}\mathcal{L}_{\text{thermo}}+\omega_{2}\mathcal{L}_{\text{dyn}}+\omega_{3}\mathcal{L}_{\text{dynI}}.$ (10) $$

where $\omega_{1}$, $\omega_{2}$, and $\omega_{3}$ are hyperparameters.

### 3.2 Transformer architecture

We adopt a transformer backbone following ESM3  Hayes et al. (2025). In particular, we use Pre-LN instead of Post-LN, rotary embeddings Su et al. (2024) instead of absolute positional embeddings, and SwiGLU activations instead of ReLU. To encode both residue- and temporal-level positional information, we introduce a two-layer rotary embedding scheme. The first layer represents residue positions along the protein sequence, while the second layer encodes temporal positions. The temporal embedding is defined such that the smallest unit corresponds to 1 ns, which is the minimum time resolution accessible to our model.

## 4 Experiment

In this section, we describe the details of our training and evaluation setup. Our results demonstrate that ProTDyn achieves performance equivalent to, or even surpassing, state-of-the-art protein ensemble generators in equilibrium ensemble generation, while additionally capable of capturing long-timescale dynamics.

### 4.1 Training setup

Data Following BioEmu Lewis et al. (2025), we combine single-structure and equilibrium MD datasets for thermodynamics training. For the single-structure dataset, we use the Swiss-Prot subset from the AlphaFold database, which contains 542,378 sequence–structure pairs. For equilibrium MD simulations, we include two sources: mdCath and BioEmu. For dataset construction and simulation details, we encourage readers to read both sources for a comprehensive understanding.

- •
mdCath Mirarchi et al. (2024): 5,398 proteins with simulation lengths up to 500 ns.
- •
BioEmu Lewis et al. (2025): prolonged simulations of diverse protein systems. The BioEmu corpus includes several subsets:
–
Octapeptides Charron et al. (2025): 1,100 peptides of length 8, each simulated for 5 $\mu$s.
–
CATH1 Charron et al. (2025); Sillitoe et al. (2021): 50 CATH domains, each simulated for 100 $\mu$s.
–
CATH2 Sillitoe et al. (2021): 1,100 CATH domains, each simulated for 39 $\mu$s.
–
MEGAsim Tsuboyama et al. (2023): extended simulations of 271 wild-type proteins, together with 1 $\mu$s simulations for each of 22,118 single-point mutants.

For dynamics and dynamics inpainting training, we use simulations from mdCath at temporal resolutions of 1 ns, and from the Octapeptides CATH2, and MEGAsim subsets at temporal resolutions of 10 and 100 ns.

Model details For better sequence and structure representation, we use the pretrained sequence and structure embedding module from ESM3 and freeze it throughout the training. The backbone of ProTDyn consists of 24 transformer blocks, which together contain 1.4 billion parameters.

Optimization We trained the model using AdamW optimizer Loshchilov & Hutter (2017) with a learning rate of $4\times 10^{-4}$ and weight decay of $1\times 10^{-5}$. We used a learning rate scheduler that reduces the learning rate by a factor of 0.5 if the training loss does not improve for 5 consecutive epochs.

### 4.2 Inference setup

We conduct ablation studies with three types of sampling methods:

- 1.
Thermodynamics: independent and identically distributed (i.i.d.) sampling of protein conformational ensembles. In our experiments, we generated 2,000 ensembles for each protein.
- 2.
Dynamics (10 ns): forward generative simulations with a time step of $10~\text{ns}$. In our experiments, we generated 50 independent trajectories, each propagated for 100 steps, corresponding to a total of $50~\mu\text{s}$ of simulation time and 5,000 conformational ensembles.
- 3.
Dynamics (100 ns): Forward generative simulations with a coarse time interval of $100~\text{ns}$. To recover fine-grained details, each $100~\text{ns}$ interval is refined by dynamics inpainting into ten $10~\text{ns}$ sub-intervals. Similarly to the “Dynamics (10 ns)” sampling module, we generate 50 independent trajectories, each propagated for 10 coarse steps and refined by inpainting, also yielding $50~\mu\text{s}$ of simulation time and 5,000 ensembles.

For the experiment of the thermodynamics module, we generate ensembles for 50 proteins from the CATH1 dataset. These proteins are included in the thermodynamic training set and serve primarily as a benchmark against the baseline model. To further assess generalization, we also generate ensembles for 10 octapeptides that were not included in the training dataset.

For the experiment of the dynamics module, we apply both dynamic sampling modules to 50 proteins from the CATH1 dataset. Although these proteins are part of the thermodynamics training set, they are excluded from dynamic and inpainting training. This setting enables a direct evaluation of the generalization capacity of ProTDyn’s dynamic generation.

### 4.3 Evaluation

Distributional similarity Our main comparison of ensemble distribution quality is against the BioEmu model Lewis et al. (2025). We selected BioEmu as our benchmark baseline because it represents the most recent major advance in quantitative protein conformation ensemble modeling. In contrast, previous approaches were typically trained on very limited MD simulations of short duration. In fairness, we used a training dataset that closely matches the one used by BioEmu. We report the Jensen-Shannon divergence (JSD) between the reference MD simulations and predicted conformation ensembles along various collective variables (CVs). The
first set of CVs are low dimensional physical features: radius of gyration (Rg) and root
mean square distance (RMSD) w.r.t the native structure predicted by AlphaFold 3 Abramson et al. (2024). The second set of CVs are the top 2 independent
components from time-lagged independent components analysis (TICA) Schultze & Grubmuuller (2021), representing the slowest
dynamic modes of proteins. We evaluate distributional similarity on ensembles generated by all three sampling modules.

Dynamical content
We first evaluate the autocorrelation functions of the TICA components. To further assess the accuracy of predicted dynamics, we discretize the conformational space into states and construct Markov state models (MSMs) Husic & Pande (2018); Chodera & Noé (2014); Pande et al. (2010); Bowman et al. (2013) to estimate transition probabilities and stationary distributions. MSMs built from reference 100 $\mu$s MD trajectories serve as ground truth, while the MSMs constructed from the ProTDyn trajectories are used for evaluation. For direct comparison, we also construct MSMs from the first $25\%$, $50\%$, and $75\%$ of the reference MD trajectories, corresponding to $25$, $50$, and $75~\mu\text{s}$ of simulation time, respectively.
Following metrics from previous works Jing et al. (2024b); Raja et al. (2025), we report the Jensen–Shannon divergence (JSD) between the stationary distributions of Markov states obtained from ProTDyn and those from reference MD. In addition, we report the JSD between the corresponding transition probability matrices. We also sample dynamic trajectories from MSM, and compute the mean negative log-likelihood (NLL) of dynamic trajectories. (details of MSM construction and evaluation methods in Appendix LABEL:Appendix:_exp)

### 4.4 Result

Distributional similarity result
We visualize the free energy surface (FES) of the TICA projection in Fig. [2](https://arxiv.org/html/2510.00013v1#S4.F2). As shown, all three ProTDyn sampling modules recover the major conformational metastable states along with qualitatively correct energy barriers that separate them.

Figure: Figure 2: Free energy surface along the top two TICA components, parameterized from the backbone torsion angles of reference MD simulations. The TICA projection is then applied to conformational ensembles generated by the three sampling modules of ProTDyn: (1) ”Thermodynamics”, (2) ”Dynamics (100 ns)”, and (3) ”Dynamics (10 ns)”, as well as to ensembles generated by the baseline model BioEmu.
Refer to caption: x2.png

Figure: Figure 3: Representative conformational metastable states and dynamic transition pathways illustrated on the 2D TICA free energy surface (FES) for a CATH1 protein system: 1b43A02.
Refer to caption: x3.png

For quantitative assessment, we report the JSD of the conformation ensembles generated by ProTDyn using the three sampling modules and BioEmu, average over all proteins in the test data in Table [1](https://arxiv.org/html/2510.00013v1#S4.T1). Across all collective variables, ProTDyn shows good agreement with the reference $100~\mu\text{s}$ simulations MD distributions. Among the three sampling modules, “Thermodynamics” sampling yields the highest-quality ensembles. This is expected, as i.i.d. samples are directly generated from the equilibrium distribution and while the invariant probably from dynamics module may be affected by possible error accumulation in autoregressive dynamic simulations.

Interestingly, we observe that “Dynamics (100 ns)” produces higher quality ensembles than “Dynamics (10 ns)”. We hypothesize that the “Dynamics (10 ns)” module, which requires many successive steps to reach long timescales, is more susceptible to error accumulation and degradation of predictive accuracy. In contrast, the “Dynamics (100 ns)” module provides a more stable strategy: large time steps enable robust exploration of long-timescale conformational transitions, while dynamic inpainting subsequently refines each interval to recover fine-grained transition details.

Finally, we evaluate the generalization of the “Thermodynamics” module on 10 octapeptides unseen in the training dataset, with results reported in Table [2](https://arxiv.org/html/2510.00013v1#S4.T2). We observe a high level of agreement with the reference MD simulations and thermodynamic performance that is very close to the baseline BioEmu model. Note that these 10 proteins are included in the BioEmu training dataset but not for ProTDyn. This demonstrates the strong generalizability of ProTDyn’s thermodynamic module beyond its training dataset.

**Table 1: Distributional similarity evaluation on CATH1 test dataset. Metrics are reported as Jensen–Shannon divergence (JSD) over the radius of gyration (Rg), root-mean-square distance (RMSD) w.r.t the native structure, and the top two TICA components capturing slow protein dynamics. Evaluation is conducted for ProTDyn under three different sampling modules and for the baseline method BioEmu.**
| Model | Rg | RMSD | TICA |
| --- | --- | --- | --- |
| ProTDyn-Thermodynamics | 0.023 | 0.012 | 0.155 |
| ProTDyn-Dynamics (10 ns) | 0.052 | 0.032 | 0.278 |
| ProTDyn-Dynamics (100 ns) | 0.030 | 0.018 | 0.206 |
| BioEmu | 0.082 | 0.137 | 0.293 |

**Table 2: Distributional similarity evaluation on the Octapeptide test dataset, using the same evaluation metrics as in CATH1 test dataset. Note that the these proteins are used as training proteins for the baseline model Bioemu but not for our ProTDyn model.**
| Model | Rg | RMSD | TICA |
| --- | --- | --- | --- |
| ProTDyn-Thermodynamics | 0.034 | 0.020 | 0.207 |
| BioEmu | 0.031 | 0.020 | 0.134 |

Dynamical content result
We visualize the dynamic pathways of a test protein in CATH1 in Fig. [3](https://arxiv.org/html/2510.00013v1#S4.F3), where systems demonstrate multiple conformation states. Both ProTDyn dynamic sampling modules can recover the conformation ensembles and the rare transition events. For qualitative analysis, we visualize the autocorrelation of the first two TICA components over 800 ns across all test proteins in Fig. [4](https://arxiv.org/html/2510.00013v1#S4.F4). The autocorrelation functions from “Dynamics (100 ns)” sampling module closely reproduce the reference results, whereas the autocorrelation functions from “Dynamics (10 ns)” module decay much faster than the the reference results. This observation is consistent with our expectation that smaller time intervals accumulate errors more rapidly, leading to a loss of long-timescale correlations. Finally, we construct MSM and quantitatively evaluate the JSD of stationary distributions and transition probability from MSM and the negative log-likelihood of dynamic paths between full reference MD simulations, ProTDyn-generated trajectories, and different portions of the reference MD data (Table [3](https://arxiv.org/html/2510.00013v1#S4.T3)). The $50~\mu\text{s}$ trajectory generated with the “Dynamics (10 ns)” module recovers MSM stationary distributions and transition probabilities with accuracy comparable to $25~\mu\text{s}$ of MD, while the “Dynamics (100 ns)” module achieves MSM quality similar to $50~\mu\text{s}$ of MD. These results demonstrate the strong dynamical fidelity of ProTDyn and highlight the effectiveness of combining large-timestep generation with inpainting to recover fine-scale transitions.

Figure: Figure 4: Autocorrelation of the top two TICA components from 0 to 800 ns lag time, evaluated on four test CATH1 proteins using reference MD trajectories and dynamic trajectories generated by the two dynamic sampling modules of ProTDyn.
Refer to caption: x4.png

**Table 3: Dynamical contents evaluation of ProTDyn under two dynamic sampling modules. Evaluation metrics include Jensen-Shannon (JS) divergence of the stationary distribution of and transition probability between Markov states, and mean negative log-likelihood (NLL) of 200 ns transition paths.**
| Model | Stationary JSD | Transition JSD | NLL |
| --- | --- | --- | --- |
| ProTDyn-Dynamics (10 ns) | 0.042 | 0.105 | 0.806 |
| ProTDyn-Dynamics (100 ns) | 0.021 | 0.055 | 0.656 |
| 25 $\mu s$ MD | 0.040 | 0.117 | 0.815 |
| 50 $\mu s$ MD | 0.018 | 0.049 | 0.682 |
| 75 $\mu s$ MD | 0.007 | 0.019 | 0.639 |

## 5 Discussions

Limitations
Similar to other generative models, ProTDyn is constrained by the availability and scale of training data, especially equilibrium MD simulation data. Its performance could be improved with access to larger and more diverse datasets. In addition, the current training procedure manually specifies the memory kernel at each timescale, without a rigorous investigation of its optimal form. Recent studies have explored principled approaches to designing or learning memory kernels, and incorporating these insights could further strengthen the model. Wu et al. (2024); Ge et al. (2024)

Opportunities
Two important capabilities of ProTDyn that we have not explored in detail are transition path sampling and likelihood evaluation. Currently, ProTDyn employs transition path sampling purely as an inpainting technique to recover short-timescale details within long-timescale generative dynamics trajectories. However, rigorous transition path sampling requires specialized sampling strategies such as transition interface sampling, milestoning, and string methods Van Erp & Bolhuis (2005); Pan et al. (2008); Votapka & Amaro (2015), and general-purpose MD simulations are not the most suitable training data for this purpose.

A key distinction of ProTDyn is that, through autoregressive modeling of discrete protein tokens, it provides *exact* likelihood evaluation for both conformational ensembles and dynamic trajectories. This contrasts with popular diffusion- or flow-based protein ensemble generators, where likelihoods are only approximated and are often computationally prohibitive to evaluate Song et al. (2020). Exact likelihoods create opportunities for integration with physics-based energy functions, or even the development of a new top-down protein force field.

Finally, current generative models for proteins are not explicitly grounded in principles in statistical mechanics, such as detailed balance. Tolman (1925). With its likelihood evaluation capability, ProTDyn offers an opportunity to enforce such physical laws, thereby unifying thermodynamics and dynamics. This could help overcome the limitations of data scarcity and enable more physically consistent generative modeling.

## References

- Abramson et al. (2024)
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al.
Accurate structure prediction of biomolecular interactions with alphafold 3.
*Nature*, 630(8016):493–500, 2024.
- Arts et al. (2023)
Marloes Arts, Victor Garcia Satorras, Chin-Wei Huang, Daniel Zugner, Marco Federici, Cecilia Clementi, Frank Noé, Robert Pinsler, and Rianne van den Berg.
Two for one: Diffusion models and force fields for coarse-grained molecular dynamics.
*Journal of Chemical Theory and Computation*, 19(18):6151–6159, 2023.
- Bowman et al. (2013)
Gregory R Bowman, Vijay S Pande, and Frank Noé.
*An introduction to Markov state models and their application to long timescale molecular simulation*, volume 797.
Springer Science & Business Media, 2013.
- Brandsdal & Smalås (2000)
BO Brandsdal and AO Smalås.
Evaluation of protein–protein association energies by free energy perturbation calculations.
*Protein Engineering*, 13(4):239–245, 2000.
- Charron et al. (2025)
Nicholas E Charron, Klara Bonneau, Aldo S Pasos-Trejo, Andrea Guljas, Yaoyi Chen, Félix Musil, Jacopo Venturin, Daria Gusew, Iryna Zaporozhets, Andreas Krämer, et al.
Navigating protein landscapes with a machine-learned transferable coarse-grained model.
*Nature chemistry*, pp.  1–9, 2025.
- Chodera & Noé (2014)
John D Chodera and Frank Noé.
Markov state models of biomolecular conformational dynamics.
*Current opinion in structural biology*, 25:135–144, 2014.
- Du et al. (2024)
Yuanqi Du, Michael Plainer, Rob Brekelmans, Chenru Duan, Frank Noe, Carla P Gomes, Alan Aspuru-Guzik, and Kirill Neklyudov.
Doob’s lagrangian: A sample-efficient variational approach to transition path sampling.
*Advances in Neural Information Processing Systems*, 37:65791–65822, 2024.
- Fu et al. (2022)
Xiang Fu, Tian Xie, Nathan J Rebello, Bradley D Olsen, and Tommi Jaakkola.
Simulate time-integrated coarse-grained molecular dynamics with multi-scale graph networks.
*arXiv preprint arXiv:2204.10348*, 2022.
- Ge et al. (2024)
Pei Ge, Zhongqiang Zhang, and Huan Lei.
Data-driven learning of the generalized langevin equation with state-dependent memory.
*Physical Review Letters*, 133(7):077301, 2024.
- Guo & Zhou (2016)
Jingjing Guo and Huan-Xiang Zhou.
Protein allostery and conformational dynamics.
*Chemical reviews*, 116(11):6503–6515, 2016.
- Hayes et al. (2025)
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al.
Simulating 500 million years of evolution with a language model.
*Science*, 387(6736):850–858, 2025.
- Husic & Pande (2018)
Brooke E Husic and Vijay S Pande.
Markov state models: From an art to a science.
*Journal of the American Chemical Society*, 140(7):2386–2396, 2018.
- Izaguirre et al. (1999)
Jesús A Izaguirre, Sebastian Reich, and Robert D Skeel.
Longer time steps for molecular dynamics.
*The Journal of chemical physics*, 110(20):9853–9864, 1999.
- Jing et al. (2024a)
Bowen Jing, Bonnie Berger, and Tommi Jaakkola.
Alphafold meets flow matching for generating protein ensembles.
*arXiv preprint arXiv:2402.04845*, 2024a.
- Jing et al. (2024b)
Bowen Jing, Hannes Stärk, Tommi Jaakkola, and Bonnie Berger.
Generative modeling of molecular dynamics trajectories.
*Advances in Neural Information Processing Systems*, 37:40534–40564, 2024b.
- Lelièvre et al. (2023)
Tony Lelièvre, Geneviève Robin, Innas Sekkat, Gabriel Stoltz, and Gabriel Victorino Cardoso.
Generative methods for sampling transition paths in molecular dynamics.
*ESAIM: Proceedings and Surveys*, 73:238–256, 2023.
- Lewis et al. (2025)
Sarah Lewis, Tim Hempel, José Jiménez-Luna, Michael Gastegger, Yu Xie, Andrew YK Foong, Victor García Satorras, Osama Abdin, Bastiaan S Veeling, Iryna Zaporozhets, et al.
Scalable emulation of protein equilibrium ensembles with generative deep learning.
*Science*, pp.  eadv9817, 2025.
- Loshchilov & Hutter (2017)
Ilya Loshchilov and Frank Hutter.
Decoupled weight decay regularization.
*arXiv preprint arXiv:1711.05101*, 2017.
- Lu et al. (2023)
Jiarui Lu, Bozitao Zhong, Zuobai Zhang, and Jian Tang.
Str2str: A score-based framework for zero-shot protein conformation sampling.
*arXiv preprint arXiv:2306.03117*, 2023.
- Mirarchi et al. (2024)
Antonio Mirarchi, Toni Giorgino, and Gianni De Fabritiis.
mdcath: A large-scale md dataset for data-driven computational biophysics.
*Scientific Data*, 11(1):1299, 2024.
- Noé et al. (2019)
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu.
Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning.
*Science*, 365(6457):eaaw1147, 2019.
- Pan et al. (2008)
Albert C Pan, Deniz Sezer, and Benoît Roux.
Finding transition pathways using the string method with swarms of trajectories.
*The journal of physical chemistry B*, 112(11):3432–3440, 2008.
- Pande et al. (2010)
Vijay S Pande, Kyle Beauchamp, and Gregory R Bowman.
Everything you wanted to know about markov state models but were afraid to ask.
*Methods*, 52(1):99–105, 2010.
- Raja et al. (2025)
Sanjeev Raja, Martin Šípka, Michael Psenka, Tobias Kreiman, Michal Pavelka, and Aditi S Krishnapriyan.
Action-minimization meets generative modeling: Efficient transition path sampling with the onsager-machlup functional.
*arXiv preprint arXiv:2504.18506*, 2025.
- Robustelli et al. (2018)
Paul Robustelli, Stefano Piana, and David E Shaw.
Developing a molecular dynamics force field for both folded and disordered protein states.
*Proceedings of the National Academy of Sciences*, 115(21):E4758–E4766, 2018.
- Schultze & Grubmuuller (2021)
Steffen Schultze and Helmut Grubmuuller.
Time-lagged independent component analysis of random walks and protein dynamics.
*Journal of Chemical Theory and Computation*, 17(9):5766–5776, 2021.
- Shaw et al. (2010)
David E Shaw, Paul Maragakis, Kresten Lindorff-Larsen, Stefano Piana, Ron O Dror, Michael P Eastwood, Joseph A Bank, John M Jumper, John K Salmon, Yibing Shan, et al.
Atomic-level characterization of the structural dynamics of proteins.
*Science*, 330(6002):341–346, 2010.
- Sillitoe et al. (2021)
Ian Sillitoe, Nicola Bordin, Natalie Dawson, Vaishali P Waman, Paul Ashford, Harry M Scholes, Camilla SM Pang, Laurel Woodridge, Clemens Rauer, Neeladri Sen, et al.
Cath: increased structural coverage of functional space.
*Nucleic acids research*, 49(D1):D266–D273, 2021.
- Song et al. (2020)
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
Score-based generative modeling through stochastic differential equations.
*arXiv preprint arXiv:2011.13456*, 2020.
- Su et al. (2024)
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu.
Roformer: Enhanced transformer with rotary position embedding.
*Neurocomputing*, 568:127063, 2024.
- Tolman (1925)
Richard C Tolman.
The principle of microscopic reversibility.
*Proceedings of the National Academy of Sciences*, 11(7):436–439, 1925.
- Tsuboyama et al. (2023)
Kotaro Tsuboyama, Justas Dauparas, Jonathan Chen, Elodie Laine, Yasser Mohseni Behbahani, Jonathan J Weinstein, Niall M Mangan, Sergey Ovchinnikov, and Gabriel J Rocklin.
Mega-scale experimental analysis of protein folding stability in biology and design.
*Nature*, 620(7973):434–444, 2023.
- Van Den Oord et al. (2017)
Aaron Van Den Oord, Oriol Vinyals, et al.
Neural discrete representation learning.
*Advances in neural information processing systems*, 30, 2017.
- Van Erp & Bolhuis (2005)
Titus S Van Erp and Peter G Bolhuis.
Elaborating transition interface sampling methods.
*Journal of computational Physics*, 205(1):157–181, 2005.
- Votapka & Amaro (2015)
Lane W Votapka and Rommie E Amaro.
Multiscale estimation of binding kinetics using brownian dynamics, molecular dynamics and milestoning.
*PLoS computational biology*, 11(10):e1004381, 2015.
- Wayment-Steele et al. (2024)
Hannah K Wayment-Steele, Adedolapo Ojoawo, Renee Otten, Julia M Apitz, Warintra Pitsawong, Marc Hömberger, Sergey Ovchinnikov, Lucy Colwell, and Dorothee Kern.
Predicting multiple conformations via sequence clustering and alphafold2.
*Nature*, 625(7996):832–839, 2024.
- Whisstock & Lesk (2003)
James C Whisstock and Arthur M Lesk.
Prediction of protein function from protein sequence and structure.
*Quarterly reviews of biophysics*, 36(3):307–340, 2003.
- Wu et al. (2024)
Yue Wu, Siqin Cao, Yunrui Qiu, and Xuhui Huang.
Tutorial on how to build non-markovian dynamic models from molecular dynamics simulations for studying protein conformational changes.
*The Journal of Chemical Physics*, 160(12), 2024.
- Zheng et al. (2024)
Shuxin Zheng, Jiyan He, Chang Liu, Yu Shi, Ziheng Lu, Weitao Feng, Fusong Ju, Jiaxi Wang, Jianwei Zhu, Yaosen Min, et al.
Predicting equilibrium distributions for molecular systems with deep learning.
*Nature Machine Intelligence*, 6(5):558–567, 2024.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: 1.Before ProTDyn, the paper “Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics” also discusses how the connections between score-based generative models and force fields can be leverage to train a diffusion model on MD simulations, and the trained score function can be used to simulate MD trajectories. It would be valuable to clarify the difference and advantage of ProTDyn when compared with these methods.

2.It would be interesting to see how ProTDyn works on larger proteins. Particularly, it would both enhance the presentation of the paper and provide more evidence of the generalizability of ProTDyn if the predicted structure/distribution can be demonstrated with figures.
soundness: 3
strengths: 1.The paper proposes an interesting view of enabling one model for both equilibrium distribution sampling and temporal molecular dynamic modeling.

2.The experiment results prove the effectiveness of the method in distribution modeling compared with strong baselines.

3.The paper is clearly written with the mothod being simple yet elegant.
summary: The paper proposes a protein foundation model based on ESM3 that unifies equilibrium distribution modelling and dynamics generation for protein structures. Training of the model optimizes the loss for distribution modeling, dynamic trajectory prediction, and dynamic trajectory inpainting at the same time. To enable the modeling of dynamics, ProTDyn proposes two-layer rotary embedding scheme to combine time rotary embedding. Experiment results show that ProTDyn performs than baselines in metrics including distributional similarity. ProTDyn also shows reasonable performance in capturing state transitions consistent with molecular dynamics.
weaknesses: 1.The benefit of unifying thermodynamics and dynamics generation is not explicitly discussed in this paper. It would be interesting to see how the two tasks interfered with each other in terms of performance. This would also provide stronger support for the motivation of the unification.

2.Some important details of the method are missing. For example, how the “two-layer rotary embedding scheme” is designed to combine temporal and residue positions. Given the model architecture leverages the ESM3 backbone, such details are important for the readers to understand the key adaptation to enabling the dynamics generation.

3.The evaluation metric for the distributional similarity are along low dimensional collective variables. Specifically, distribution along the Rg and RMSD (which are single values not reflecting details of the protein structures) w.r.t. the native structure seems to be less significant in reflecting the true distribution in 3D Euclidean space with all the residues.

---

confidence: 5
contribution: 3
initial_rating: 8
presentation: 4
questions: - Could you clarify the train/test splitting procedure? Was any filtering based on sequence identity or structural similarity
- Have the authors considered other architectures such as those AF-based models? What was the specific rationale for choosing the ESM3 architecture?
- BioEmu has been finetuned on experimental data after training on MD, so the comparison on MD distribution might not be the most direct comparison, which can be noted.
- The method aims to generate coarse-grained dynamics. It'll be helpful to demonstrate the computational cost in comparison with MD simulation
- Have the authors considered the asymptotic behavior of the time dependence? e.g. when the $\delta t \to \inf$, it should approach an i.i.d. distribution
soundness: 4
strengths: **Novel methodology**
- The method is novel to incorporate three tasks in one model:
  1. i.i.d. equilibrium distribution sampling
  2. time lagged predictions with multiple time scales
  3. inpainting: given two coarse grained time steps, predict fine grained steps in between
- This methodology overcomes the lack of capability in the description of dynamics/kinetics in the equilibrium sampling models

**Transferability**
- The model is trained on multiple time scales, showing transferability in the time dependence
- The model is trained on a large scale of MD data, showing transferability in the chemical space

**Performance**
- The performance has been compared on MD sample distributions with proper baselines and benchmarks, and has demonstrated the advantage of this model
summary: This paper introduces ProTDyn, a foundation PLM to unify equilibrium distribution sampling, time-dependent dynamics sampling as well as inpainting for fine-grained timesteps based on coarse grained timesteps. It has been trained on a large scale molecular dynamics dataset, and demonstrated promising performance in the generated sample distribution in comparison with baseline models. I find this methodology very interesting and novel, and has shown scaling up in the model and data.
weaknesses: - **Generalization**: For the dynamics tasks, the test proteins were seen by the model during thermodynamics training, which may not represent true generalization. A stronger evaluation would involve holding out proteins based on sequence similarity
- **Baseline**: This model is only compared against BioEmu as the only baseline. It can consider adding more baseline models, such as Alphaflow which is also trained on mdCATH, as well as a few other works that baked in the time dependence for dynamics generation of protein models.
- **Inpainting task**: Dynamics inpainting is highlighted as a key capability, but its performance is only measured indirectly through the quality of the final, end-to-end generated trajectories. There are no specific benchmarks that isolate and quantify the accuracy of the inpainting process itself

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: Why did you decide to use a causal autoregressive distribution for the generation of structural tokens? In ESM3, an any-order autoregressive model was used to generate tokens in parallel which seems to make sense for protein data. Did you try any other parameterizations?

How were the proteins in Figure 2 selected? If these were cherry-picked as to best show the benefits of your method this should be stated clearly. Or if these are exhaustive of the CATH2 dataset, this should also be made clear because this is important for judging the significance of your findings.
soundness: 2
strengths: The training of the model is simple in that all 3 tasks of thermodynamics, dynamics and dynamics inpainting are trained jointly and simultaneously in a single model. This avoids the need for any task specific fine-tuning which is a downside of prior approaches. All tasks are unified in terms of autoregressive modelling with different factorizations.

The experimental results look strong in terms of improvement over the BioEmu baseline model. Distibutional fit on MD test data looks quite convincingly better in Table 1 and in Figure 2.

The authors make some interesting observations in terms of the time resolution with which to sample their dynamics model. They find that sampling with a finer grid can lead to worse overall performance potentially due to error accumulation. This finding will be useful for further research into deep learning based dynamics models.
summary: In this paper, the authors present ProTDyn which is a transformer operating on ESM3 structure tokens trained for conformation and dynamics sampling. The model is trained on a mixture of the alphafold synthetic database of structures and molecular dynamic datasets. It is trained for both equilibrium distribution sampling as well as jumping ahead in MD simulations or inpainting in between coarse time steps in MD simulations. The model improves over BioEmu, a recent deep learning model for conformer generation, in terms of distributional match to MD.
weaknesses: A main weakness of the paper is the lack of ablations with respect to the stated contribution of ProTDyn being a unified model for Thermodynamics and Dynamics generation. On L56-L58 in the introduction the authors describe how joint training can be mutually beneficial for both tasks. I would therefore expect experiments and ablations showing that this is indeed the case. You could train a version of your model on the tasks individually and compare performance with the multi-task trained model. I don't think comparing to BioEmu is enough to make this claim since BioEmu is pretrained on Thermodynamics style tasks and fine-tuned on dynamics tasks which can be seen as a form of multitask training.

With regards to Table 2, where these proteins are in your test set but BioEmu's training set, I am unsure how ProTDyn ended up with a different test set to BioEmu that made this comparison difficult. If you are using the same datasets as BioEmu, why not also use the same splits? In the end, it makes it very hard to draw any conclusions from Table 2 since the models are not comparable.

The paper is quite unclear with the usage of the term 'module'. The authors state that they have a 'thermodynamics module' and a 'dynamics module'. This would imply separate parts of the network that are being trained for these two tasks which would be in contradiction with the introduction describing a single unified model which is a key stated contribution of the work. I am unsure what the authors are referring to with regards to these modules and this should be made clearer. 

Further, in terms of clarity, on L149 the authors describe their model as a 'multimodal protein language model'. In what way is this a multimodal model because as far as I am aware the model just generates structural tokens (and no other modalities).


I am tending to accept this paper due to the strong experimental results with regards to BioEmu and simple training scheme. This model seems to make a significant improvement on the state of the art for conformational modelling.

---

confidence: 2
contribution: 2
initial_rating: 6
presentation: 2
questions: I'm slightly confused by Figure 1 as the authors mention it is a language model, but there does not seem to be language/text as input in the diagram. Could the authors please clarify? Are the authors using "language model" to highlight the fact that it is an autoregressive (causal/decoder) Transformer architecture and not that the input is language.
soundness: 2
strengths: The paper is well-written and easy to follow. The fact that the authors present a model that is able to perform simultaneously equilibrium conformation ensemble generation and multi-timescale dynamic trajectory generation is interesting and as far as I am aware novel.
summary: The work proposes ProTDyn, a model that is able to do conformation ensemble generation and multi-timescale dynamics modeling within a single model.
weaknesses: As the paper is not near my main area of research, it is hard for me to make questions or comment on the weaknesses although I found the paper easy to follow and appreciated especially the explanation on page 4. 

The paper is a rather standard application of ML modelling techniques but unfortunately I cannot comment on the strength of the results. For this reason I will leave a weaker confidence score.


Decision: Accept (Poster)