ICML 2025

Diffusion on Language Model Encodings for Protein Sequence Generation
Viacheslav Meshchaninov * 1 Pavel Strashnov * 2 Andrey Shevtsov * 2 Fedor Nikolaev 2 Nikita Ivanisenko 2
Olga Kardymon 2 Dmitry Vetrov 1
Abstract
Protein sequence design has seen significant ad-
vances through discrete diffusion and autore-
gressive approaches, yet the potential of con-
tinuous diffusion remains underexplored. Here,
we present DiMA, a latent diffusion framework
that operates on protein language model rep-
resentations.
Through systematic exploration
of architectural choices and diffusion compo-
nents, we develop a robust methodology that gen-
eralizes across multiple protein encoders rang-
ing from 8M to 3B parameters.
We demon-
strate that our framework achieves consistently
high performance across sequence-only (ESM-
2, ESMc), dual-decodable (CHEAP), and mul-
timodal (SaProt) representations using the same
architecture and training approach. We exten-
sively evaluate existing methods alongside DiMA
using multiple metrics across two protein modali-
ties, covering quality, diversity, novelty, and dis-
tribution matching of generated proteins. DiMA
consistently produces novel, high-quality and di-
verse protein sequences and achieves strong re-
sults compared to baselines such as autoregres-
sive, discrete diffusion and flow matching lan-
guage models. The model demonstrates versa-
tile functionality, supporting conditional gener-
ation tasks including protein family-generation,
motif scaffolding and infilling, and fold-specific
sequence design. This work provides a univer-
sal continuous diffusion framework for protein
sequence generation, offering both architectural
insights and practical applicability across vari-
ous protein design scenarios. Code is released at
GitHub.
*Equal contribution
1Constructor University, Bremen, Ger-
many 2AIRI, Moscow, Russia.
Correspondence to:
Viach-
eslav Meshchaninov <meshchaninov.viacheslav@gmail.com>,
Pavel
Strashnov
<strashnov@airi.net>,
Andre...