Title: Accelerating Discrete Diffusion Decoding with Parallel Scan
Primary area: foundation or frontier models, including LLMs
Keywords: Efficient AI System, Discrete Diffusion Model

Abstract: Diffusion-based language models provide strong controllability and parallel generation capabilities, but suffer from prohibitively high decoding cost. Block diffusion, a semi-autoregressive approach, alleviates this issue by reducing diffusion steps and enabling KVCache utilization, yet it restricts parallel decoding strictly within blocks, preventing inter-block parallelism. In this work, we identify a class of \emph{associative states} in diffusion models: \textbf{blocks that can be independently sampled without conditioning on the prefix, and later refined once the prefix becomes available, effectively performing a form of self-refinement}. Leveraging this property and inspired by classic parallel algorithms, we propose a novel \textbf{parallel-scan based decoding} framework. Our method incorporates two key techniques---\emph{local remasking} and \emph{global aggregation} ---to enhance stability and efficiency. We further introduce a systematic design space of prefix-network topologies, cache strategies, and parameter configurations, and conduct a preliminary exploration of this search space. Empirically, our training-free approach achieves up to \textbf{68 tokens/s} throughput and \textbf{60.7\% accuracy on GSM8K}, and we validate the effectiveness of the proposed techniques through ablations. Compared to mainstream semi-autoregressive methods, our results demonstrate that parallel decoding with structured parallel patterns remains a promising and underexplored direction for efficient inference in diffusion-based LLMs.