Title: Deterministic Discrete Denoising
Primary area: generative models
Keywords: discrete diffusion, deterministic denoising, herding algorithm, discrete generative modeling

Abstract: We propose a deterministic denoising algorithm for discrete-state diffusion models based on Markov chains.
The generative reverse process is derandomized by introducing a variant of the herding algorithm
with weakly chaotic dynamics, which induces deterministic discrete state transitions.
Our approach is a direct replacement for the stochastic denoising process,
requiring neither retraining nor continuous state embeddings.
We demonstrate consistent improvements in both efficiency and sample quality on text and image generation tasks.
Thus, this simple derandomization approach is expected to enhance the significance of discrete diffusion in generative modeling.
Furthermore, our results reveal that deterministic reverse processes, well established in continuous diffusion,
can also be effective in discrete state spaces.

TL;DR: Derandomization of the generative denoising process for discrete-state diffusion models