NeurIPS 2025

DINGO: Constrained Inference for Diffusion LLMs
Tarun Suresh∗, Debangshu Banerjee∗, Shubham Ugare, Sasa Misailovic, Gagandeep Singh
Department of Computer Science,
University of Illinois Urbana-Champaign
Abstract
Diffusion LLMs have emerged as a promising alternative to conventional autore-
gressive LLMs, offering substantial potential for improving runtime efficiency.
However, existing diffusion models fail to provably enforce user-specified formal
constraints, such as regular expressions, which makes them unreliable for tasks
that require structured outputs, such as fixed-schema JSON generation. Unlike
autoregressive models, which generate tokens sequentially, diffusion LLMs predict
a block of tokens in parallel. This parallelism makes traditional constrained decod-
ing algorithms, designed to enforce constraints with sequential token prediction,
ineffective at preserving the true output distribution. To address this limitation, we
propose DINGO, a dynamic programming-based constrained decoding strategy
that is both efficient and provably distribution-preserving. DINGO enables sam-
pling of output strings with the highest probability under the model’s predicted
distribution while strictly adhering to any user-specified regular expression. On
standard symbolic math and JSON generation benchmarks, DINGO achieves up to
a 68% points of improvement over unconstrained inference. The code is available
at DINGO.
1
Introduction
Autoregressive LLMs demonstrate impressive performance across a wide range of tasks, including
logical reasoning [Pan et al., 2023], theorem proving [Yang et al., 2023], and code generation [et.
al., 2021]. However, because they generate one token at a time, they can be slow when producing
long responses. Recent work has explored using diffusion models to accelerate token generation by
predicting blocks of tokens in parallel. For tasks such as logical reasoning, where the LLM output
is fed into symbolic solvers like Z3 [Fedoseev et al., 2024], synta...