Beyond Fixed: Variable-Length Denoising for Diffusion Large Language
  Models
Authors: name: Jinsong Li; name: Xiaoyi Dong; name: Yuhang Zang; name: Yuhang Cao; name: Jiaqi Wang; name: Dahua Lin
Published: 2025-08-01 17:56:07

Abstract: Diffusion Large Language Models (DLLMs) are emerging as a powerful
alternative to the dominant Autoregressive Large Language Models, offering
efficient parallel generation and capable global context modeling. However, the
practical application of DLLMs is hindered by a critical architectural
constraint: the need for a statically predefined generation length. This static
length allocation leads to a problematic trade-off: insufficient lengths
cripple performance on complex tasks, while excessive lengths incur significant
computational overhead and sometimes result in performance degradation. While
the inference framework is rigid, we observe that the model itself possesses
internal signals that correlate with the optimal response length for a given
task. To bridge this gap, we leverage these latent signals and introduce
DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive
Length Expansion for Diffusion Large Language Models. DAEDAL operates in two
phases: 1) Before the denoising process, DAEDAL starts from a short initial
length and iteratively expands it to a coarse task-appropriate length, guided
by a sequence completion metric. 2) During the denoising process, DAEDAL
dynamically intervenes by pinpointing and expanding insufficient generation
regions through mask token insertion, ensuring the final output is fully
developed. Extensive experiments on DLLMs demonstrate that DAEDAL achieves
performance comparable, and in some cases superior, to meticulously tuned
fixed-length baselines, while simultaneously enhancing computational efficiency
by achieving a higher effective token ratio. By resolving the static length
constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap
with their Autoregressive counterparts and paving the way for more efficient
and capable generation.