ICML 2025

The Diffusion Duality
Subham Sekhar Sahoo 1 Justin Deschenaux 2 Aaron Gokaslan 1 Guanghan Wang 1 Justin Chiu 3
Volodymyr Kuleshov 1
Abstract
Uniform-state discrete diffusion models hold
the promise of fast text generation due to their
inherent ability to self-correct. However, they are
typically outperformed by autoregressive models
and masked diffusion models. In this work, we
narrow this performance gap by leveraging a
key insight: Uniform-state diffusion processes
naturally emerge from an underlying Gaussian
diffusion. Our method, Duo, transfers powerful
techniques from Gaussian diffusion to improve
both training and sampling. First, we introduce
a curriculum learning strategy guided by the
Gaussian process, doubling training speed by
reducing variance. Models trained with curricu-
lum learning surpass autoregressive models in
zero-shot perplexity on 3 of 7 benchmarks. Sec-
ond, we present Discrete Consistency Distillation,
which adapts consistency distillation from the
continuous to the discrete setting. This algorithm
unlocks few-step generation in diffusion
language models by accelerating sampling by
two orders of magnitude. We provide the code
and model checkpoints on the project page:
https://s-sahoo.com/duo
1. Introduction
An eternal theme in mathematics is that discreteness
emerges from underlying continuity. From quantum me-
chanics, where the quantized energy states of electrons arise
as solutions to continuous wave equations, to the binary
logic of digital circuits, fundamentally driven by smooth
analog currents, discreteness has repeatedly and naturally
emerged from an underlying continuum. Our work contin-
1Computer and Information Science, Cornell Tech, NY, USA.
2School of Computer and Communication Sciences, EPFL Lau-
sanne, Switzerland 3Cohere, NY, USA. Correspondence to: Sub-
ham Sekhar Sahoo <ssahoo@cs.cornell.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the auth...