Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
Authors: Chengyue Wu
Hao Zhang
Shuchen Xue
Zhijian Liu
Shizhe Diao
Ligeng Zhu
Ping Luo
Song Han
Enze Xie
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion LLM
efficiency
Primary area: foundation or frontier models, including LLMs
TL;DR: Fast-dLLM boosts diffusion-based LLM inference speed by introducing block-wise KV caching and confidence-aware parallel decoding, achieving up to 27.6× throughput gains with minimal quality loss.

Abstract: Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation. However, the practical inference speed of open-sourced Diffusion LLMs often lags behind autoregressive models due to the lack of Key-Value (KV) Cache and quality degradation when decoding multiple tokens simultaneously. To bridge this gap, we introduce Fast-dLLM, a method that incorporates a novel block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models, enabling cache reuse with negligible performance drop. Additionally, we identify the root cause of generation quality degradation in parallel decoding as the disruption of token dependencies under the conditional independence assumption. To address this, Fast-dLLM also proposes a confidence-aware parallel decoding strategy that selectively decodes tokens exceeding a confidence threshold, mitigating dependency violations and maintaining generation quality. Experimental results on LLaDA and Dream models across multiple LLM benchmarks demonstrate up to 27.6× throughput improvement with minimal accuracy loss, closing the performance gap with autoregressive models and paving the way for practical deployment of Diffusion LLMs.

Published as a conference paper at ICLR 2026
FAST-DLLM: TRAINING-FREE ACCELERATION OF DIF- FUSION LLM BY ENABLING KV CACHE AND PARAL-
LEL DECODING
Chengyue Wu Hao Zhang"* Shuchen Xue²
Zhijian Liu²
Shizhe Diao²
Ligeng Zhu?
Ping Luo Song Han Enze Xie²
'The University of Hong Kong NVVDDI 'MIT
"Equal contribution.
ht https://nvlabs gi thub. 1o/Fast--Lii
ABSTRACT
Diffusion- -based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation. However, the practical inference speed open-sourced Diffusion LLMs often lags behind autoregressive models due the lack of Key-Value (KV) Cache and quality degradation when decoding multiple tokens simultaneously. To bridge this gap, we introduce Fast-dI.L.. LLM, a method that incorporates a novel block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models, enabling cache reuse with negligible performance drop. Additionally, we identify the root cause of generation quality degradation in parallel decoding as the disruption of token dependencies under the conditional independence assumption. To address this, Fast-dLLM also proposes confidence-awes parallel decoding strategy that selectively decodes tokens ex- ceeding confidence threshold, mitigating dependency violations and maintaining generation quality. Experimental results LLaDA and Dream models across multiple LLM benchmarks demonstrate up to 27.6x throughput improvement with minimal accuracy loss, closing the performance gap with autoregressive models and paving the way for practical deployment of Diffusion LLMs.
INTRODUCTION
Diffusion-baas based large language models (Diffusion LLMs) have recently attracted increasing attention due to their potential for parallel token generation and the advantages of bidirectional attention 1,000 tokens per second, and
mechanisms. Notably, Mercury (Inception Labs, 2025) runs Gemini Diffusion (Google DeepMind, 2025) by Google DeepMind has demonstrated the ability to generate over 1,400 tokens per second, highlighting the promise of significant inference acceleration.
However, current open-source Diffusion LLMs (Nie et al., 2025b; Ye et al., 2025) have yet to close such throughput gap in practice, and their actual speed often falls short of autoregressive (AR) models. This is primarily due to two issues. First, diffusion LLMS do not support key-value (KV) caching, critical component
AR models for speeding up inference. Second, the generation quality tends to degrade when decoding multiple tokens in parallel. For example, recent findings such those from LLaDA (Nie al., 2025b) indicate that Diffusion LLMs perform best when generating tokens one at a time and soon degrades when decoding multiple tokens simultaneously.
To bridge the performance gap with AR models that benefit from KV Cache, we present L.. fast aWnd practical diffusion-based language modeling framework. First, Fast-dL.LM introduces an approximate KV Cache tailored to Diffusion LLMs. While the bidirectional nature of attention in Diffusion LLMs precludes fully equivalent KV Cache, our approximation closely resembles an ideal cache in practice. To support KV Cache, we adopt a block-wise generation manner. Before generating block, we compute and store KV Cache of the other blocks to reuse. After generating the block, we recompute the KV Cache of all the blocks. Visualizations confirm the high similarity with adjacent inference steps within the block, and our experiments show that this approximation

---

Published as a conference paper at ICLR 2026
Qwen2.5-77
Rowns per step Througpou Beterabcccrdi)
2.01
GSHBK (5 (5--hh Accuracy (1)
54.40
42-
60-
8.1x Faster
Number of tolann
76-
LLDDAKKVCach
1.00
1,00
Dream
5.3X Faster
L-BNA-3-8
Paralial
1.-
21.20
16,50
20
6,70
10
Parallell
0.-
72-
LLaDA
LLaDA-
LLaDA iPoralie
UabA AVCcce Paralie!
70
10
23
30
Throughput (Tokensbec)
(x) Throughput ys. Accuracy across methouss
(b) Throughput and tokens per step acrows methods
Latanoy per stop
(8-shot) Gen. Length- 1024 Acc-77.3
LLaDA 1024 staps
0.28x
Labencc per samplec 268s, Thrmaghout toku/s
Parallel 100 staps
0.78
PrefixCache 140 staps
0.14s 20s. 13.0 toN./
1.4x
DualCache
19.3 105.22
2.1x
GSMBK (8-shot) Gen. Length-1024 760-76.0
(c) Eid to end speedup over vanilla LL.DD buseline.
Figure 1: Effectiveness of components of Fast-dLLM across different approaches. We use NVIDIA A GPU with a single batch size and inference speedup frameworks.. (a) Inference throughput (tokens/sec) and GSM8K (5-shot) accuracy across various designs and models under a maximum generation length of 256. Caching mechanism and paralicl decoding can significantly accelerate inference, while the combination provides up to an 8.1x increase in throughput with negligible accuracy reduction. (b) We break down the contributions of each method by showing both the number of tokens generated per step (line) and total throughput (bars). (c) With long prefilling (8-shot) and maximum generation length of 1024, our combined approach achieves up 27.6x end-to-end speedup compared the vanilla LLaDA baseline.
preserves model performance during inference. We further propose DualCache version that caches Keys and Values for both prefix and suffix tokens.
In parallel, Fast-dLLM investigates the degradation in output quality when generating multiple tokens simultaneously. Through theoretical analysis and empirical studies, we identify that simultaneous sampling of interdependent tokens under a conditional independence assumption disrupts critical token dependencies. To address this issue and fully exploit the parallelism potential of Diffusion LLMs, propose novel decoded simultancously. Instead of selecting the tokens with top K confidence to decode as in LLaDA, we select tokens with confidence larger than threshold. Our theoretical justification and experimental results demonstrate that this strategy maintains generation quality while achieving up to x3.3x inference speed-up.
strategy to select which tokens be safely
In summary, our contributions are threefold. First, Value Cache for Block-Wise Decoding, We introduce block-wise approximate KV Cache mechanism specifically designed for bidirectional attention. Our approach reuses cached activations from previously decoded blocks by exploiting the high similarity of KV activations between adjacent steps. By caching both prefix and suffix blocks, the DualCache strategy enables substantial computational reuse. Second, Confidence-Awarr Parallel Decoding. We propose a novel confidence-awaan aware parallel decoding method. Unlike prior approaches that select fixed number of tokens per step, our method dynamically selects tokens whose confidence exceeds a global threshold, enabling safe and effective parallel decoding. This approach significantly accelerates inference by 13.3x while preserving output quality. Third, State- of-the-Art Acceleration Results. We conduct comprehensive experiments on multiple open-source Diffusion LLMs (LLaDA, Dream) and four mainstream benchmarks (GSM8K, MATH, HumanEval, MBPP). Results demonstrate that our F consistently deliver snder-of-magniine with minimal degradation in accuracy, confirming the generality and practical value of
speedups
2

---

Published as a conference paper at ICLR 2026
approach for real-world deployment. Fast-dLLM achieves hgiher acceleration (up to 27.6x) when generation length is longer (1024).
PRELIMINARY
2.1 MASKED DIFFUSION MODEL
Diffusion models for discrete data were first explored in (Sohl-Dickstein et al., 2015; Hoogeboom al., 2021). D3PM (Austin al., 2021) generalized them with discrete-state Markov chain forward process parameterized by transition matrices Q, and learned the reverse process pe(xo) (x,) via ELBO maximization. CTMC (Campbell et al., 2022) extended this to continuous time, while SEDD (Lou et al., 2023) instead modeled the likelihood ratio p=(M) using Denoising Score Entropy. pr(m)
Among noise processes, Masked Diffusion Models (MDMs)-akee called absorbing state discrete diffusion-are prominent. MDMs replace tokens with special [MASK] token according to
(x)) = II Cat </ (1 t) + t
(1)
where E [0, interpolates between (f 0) and fully masked sequence (t = =).
Recent work (Shi et al., 2024; Sahoo et al., 2024: Zheng al., 2024; Ou al., 2024) shows MDM parameterizations are equivalent and that their training objective reduces to ELBO:
log < 1 Eyc
Σ
log po (z (cti22 20)) dl LMDM.
F:a [MAS]
2.2 GENERATION PROCESS MDMS
Directly reversing Equation is inefficient, altering only one token per step (Campbell et al., 2022; Lou al., 2023). faster strategy r-leaping (Gillespie, 2001), which lets multiple masked tokens be recovered in single step from to < t:
x * MASK] 2 = xi
Osje
II
Uoje (sje(arg (wt), (eje(ar"]aa)) t=
x; MASK x = [MASK]
(3)
4u (1)
x; [MAK
,. [MASK].
Here (a) 2x)) is model distribution over the vocabulary, extended to 40 ( (0) P) when condi- tioned on a prompt
Curse of Parallel Decoding Although r-leaping accelerates generation by sampling multiple tokens in parallel, the conditional independence assumption causes problems. For example, in "The list of poker hands that consist of two English words are: (Song & Zhou, 2025), valid pairs include "high card" or "full house," but independent sampling can yield incoherent pairs like "high house." Formally, MDMs approximate p(x), x xx)) by p( (x)) ((x/|x;;) ignoring dependencies such as p(a: [z,, ). This mismatch worsens when many tokens are unmasked simultaneously. degrading flueney and coherence.
3 METHODOLOGY
3.1 PIPELINE OVERVIEW
Our approach, FaMM- builds on the Masked Diffusion Model (MDM) architecture to enable effi- cient and high-quality sequence generation. To accelerate inference, the overall pipeline incorporates two key strategies: efficient attention computation through Key-Value (KV) Cache and a parallel decoding scheme guided by prediction confidence.
Specifically, we adopt Key- Key-Value Cache for Block- Wise Decoding, which allows reusing attention activations across steps and significantly reduces redundant computation. Within each block,
3

---

Published as a conference paper at ICLR 2026
Piampi
Black o
Block
Pronet
Block
Dlock 1
Prompe
MASK token
Decoded tokeo
Compute cache
Decadl token
(a) Pretix KV Cache for black-wise generition.
(b) DualCache: Bidirectional KV cache consains prefix suffis Cache.
Figure 2: Illustration of our Key-Value Cache for Elsck-Wise Decoding, (a) During prefix-only caching, the for the prompt and reused
KV cache is computed cache is updated after completing block to maintain consistency, with negligible overhead. (b) DualCache extends this approach by caching both pretix and masked suftix tokens, further accelerating decoding. The high similarity of KV activations across steps allows effective reuse with minimal approximation error.
multiple decoding sleps within each block. The
further propose Confidence-Aware Parallel Decoding, enabling selective updates of tokens based on confidence scores to improve efficiency while maintaining output quality.
By combining these strategies, Last-dLLM significantly speeds up inference for MDMs with minimal impact on generation performance. The overall procedure summarized in Algorithm 1.
3.2 VALUE CACHE FOR BLOCK-WIL WISE DECODING
As shown in Figure 2, adopt block wise decoding strategy to support the use of Key- Value (KV) Cache. Initially, we compute and store the KV Cache for the prompt, which is reused throughout Block 0. Within each block, the same cache is reused for multiple decoding steps. After completing the decoding of a block, we update the cache for all tokens (not just the newly generated ones). This cache update can be performed jointly with the decoding step, so compared there is no additional computational overhead. This approach results in an approximate decoding process, due to the use of full attention in masked diffusion models (Nie et al., 2025b; Ye al., 2025).
not using caching,
The effectiveness of our approximate KV Cache approach stems from the observation that KV activations exhibit high similarity across adjacent inference steps, as illustrated in Figure The red boxed region in Figure 3a highlights the similarity scores within a block, which are consistently close 1. This indicates that the differences in prefix keys and values during block decoding are negligible, allowing My eRfely rewseleesackee wi Diceviigat loss in accuracy.
Furthermore, we implement a bidirectional version of our KV caching mechanism, named DualCache. While E asproaches only onl (oe eo t Ba the &
fökens under bufl black wise NaPia Leneerease
BBBK REPFE (h aware desedipanteonn anterrilln dessgned to weteefattnnoooee red B b Set Tor Pi Piycl effec gltechyf wif.ititooonio wwcnnnssccssaaii.o ye muiigalee this condition minditionnlt Dloch wendeeedecccooo
f As sho T
sumple
eenendeeee die highigible. H
Concretely, at each iteration, rather than aggressively unmasking all masked tokens using their independent marginal probabilities, softmax probability). Only those with confidence exceeding a threshold are unmasked in the current step; the rest remain masked and are reconsidered in future steps. If token's 's confidence exceeds the threshold, we always unmask the token with the highest confidence to ensure progress and prevent an infinite loop. This strategy accelerates generation while reducing errors from uncertain or ambiguous predictions.
compute confidence score for each token (e.g., the maximum
critical question, however, is: When is theoretically justifiable to decode tokens in parallel using independent marginals, despite the true joint distribution potentially containing dependencies? We address this with the following formal result, which characterizes the conditions under which greedy parallel (product of marginal distribution) decoding is equivalent to greedy sequential (true joint distribution) decoding in the high-confidence regime, and quantifies the divergence between the two distributions.
4

---

Published as a conference paper at ICLR 2026
AM
AW
AM
Stop
(a) Prompt block
(b) Last block
Figure 3: Heatmaps of Key-Valus Activation Cosine Similarity Across Inference Steps in LLaDA-Instrus Cosine similarity heatmaps of Vey-Value activations for (a) the prompt and (b) the last (suffix) block. High similarity along the diagonal (i = j. red boxes) indicates that activations for adjacent inference steps are highly similar. This supports using an approximate block-wise KV Cache, allowing cached activations be reused for faster decoding with negligible impact on accuracy.
Prior to presenting the theorem, we will define the mathematical notation used in its statement. Let po(-|E)) denote the conditional probability mass function (PMF) given by an MDM condition on E (comprising prompt po and previously generated tokens). Suppose the model is to predict tokens for positions (1,.... .,,,..,,,, in not in E. Let
(Xi....., (x,,,.,;;,,))) X,,)) be the vector of n tokens, where each X1, takes values in vocabulary V. Let X E po (X,, ... ..,, (E) be the joint conditional PMF according to the model. Let Pj (X,, [) = Po (x, EE) be the marginal conditional PMF for position iz. Parallel decoding generates tokens using the product of marginals: q(X ()) = P5 (E).
The proof of Theorem and relevant discussions are in Appendix A.
Theorem (Parallel Decoding under High Confidence). Suppose there exists specific sequence of tokens xt = (Ii PX(,, - 31 E) 1 for some small > 0. Then, the following results hold:
Ii,,) such that for each j E {l....,nJ .T., the model has high confidence in Tiş²
Equivalence for Greedy Decoding: If (n 1)c S (i.e. c cs 1 then
argmax ((z]xxx((2EB argmax {(2|E)= xx
(4)
This means that greedy parallel decoding (selecting argmax g) yields the same result as greedy sequential decoding (selecting argmax p).
This bound is tight: if c ~4T, there exist distributions p( () satisfying the high-confidence marginal assumption for which argmax, F)= * argmax, g(2
2. Distance and Divergence Bounds: Let p(- EE) and q(-11)) be denoted as p and for brevity.
Lp Distance (p 2 1): For n > 1, Dp (p.g) < ((n 1)p + 2m) 2/pe Specifically, for Total Variation Distance (Drv (p.((p.) 1 Di (p. q)): (p g) < 3c-lc.
Forward KL Divergence: For 1, DKL (pllq) (n (n-1) ()(p(c) cln([V] - 1)), where Holc) -clne (1- e) In(1 - e) is the binary entropy function, and [V] is the size of the vocabulary.
Building this theorem, propose practical factor- parallel decoding strategy extension of the threshold strategy that adaptively selects how many tokens to decode in parallel based the confidence levels. Concretely, given the model's marginal confidence estimates for tokens in block, sort these confidences and select the largest such that (n + 1) c(")) < f. where f is fixed decoding factor hyperparameter and (( is the n-th highest confidence. At each step, the top-n tokens are decoded in parallel. This formulation mirrors the bound in Theorem and ensures that decoding only proceeds when the marginal confidence is sufficiently high to approximate the joint decoding reliably. contrast to the static threshold based strategy, factor-based decoding dynamically controls the degree of parallelism in theoretically grounded manner.
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Block-wise Confidence-awee aware
aware Parallel Decoding with (Dual) KV Cache Require: Pe. prompt answer length L. blocks K. block size B, steps block I' threshold т. use DualCache, strategy (threshold, factor). factor f
x Po: [MASK] .... [MXSX]]
Initialize Cache (single dual) for (fuse with decoding).
Cache Init
for k to K do
4:
(k 1yB, for to do
Pol kB
5:
6:
Use cache, run on s/ne) if use DualCache clse gloe) For masked v² compute confidence max, (-())))) strteggy == threshold then
I1 Cache Reuse # Confidence scoring
7:
8:
Unmask in 8,6) with e' always unmask else if strategy factor then
11:
Sort c in descending order (c cd ...,)....... e!!")
12:
Find largest such that (n 1)(1 dal)
13:
Unmask top-n tokens, always unmask the max e² end
14:
15:
if all gla,e) unmasked then
16:
break end
end for
19:
Update KV cache: if DualCahee prefix & suftix; elset prefix. // Cache Update
20:
end for
21:
return i
Table I: Comprchensive benehmark results the LLaDA-Instrus suite. Each cell presents the accuracy and the decoding throughput in tokens per second with relative speedup to the LLaDA baseline (bottom row, blue: tokens per second/orange: relative speedup). highest throughput speedup each configuration are highlighted.
Benchmurk
Gen Length
LLaDA
+Cache
Parallel
+Cache+Parallel (Fast-dLLLM)
256
79.3
79.5
79.2
78.5
GSMBK -hot)
6.7 (1x)
21.2 (3.2x)
16.5 (2.5%)
54.4 (8.11)
512
77.5
77.0
77.6
77.2
(1x
(3.3%)
(5.8
35.3 (11.0x)
256
33.5
33.3
33.4
33.2
MATH (4-shot)
(1x
23.7 (2.6x)
24.8 (2.7%)
51.7 (5.7%)
512
37.2
36.2
36.8
36.0
(Ix
19.7 (2.5x)
23.8 (3.0x)
47.1 (5.9x)
256
41.5
42.7
43,9
43,3
HumanEval (0-shot)
(1xx
40.7 (1.3x)
101.5 (33x)
114.1 (3.7x)
512
43.9
45.7
43.3
44.5
(1x4
29.3 (1.6x)
(3.1x
73.7 (4.0x)
256
29.4
29.6
28.4
28.2
MBPP (3-shot)
(1x.
(2.8
24.8 (4.1x)
44,8 (7.5x)
512
14.8
13.4
15.0
13.8
4.3 (lx)
10.1 (2.3x)
25.3
39.5 (9.2x)
256
57.1
53.6
57.1
54.0
IFEvll
19.7 (1.3x) ()x7
(2.0g)
(2.3x)
512
58.0
$7.5
$7.9
57.3
812(1))
(1.5())))
22.7
(1.7
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
All experiments are conducted on an NVIDIA A100 80GB GPU. The proposed approach, Fast- dLLM, comprises two components: Value Cache mechanism and a Confidence-Aware Parallel Decoding strategy. The KV Cache component introduces a hyperparameter, the cache block size, varied between 4 and 32. The parallel decoding strategy uses confidence threshold hyperparameter, explored the range of 0.5 to 1.0. Unless otherwise specified, we use PrefixCache with block size of 32 and the threshold to 0.9.

---

Published as a conference paper at ICLR 2026
Table 2: Comprehensive benchmark results Dream-Bart variants four tasks with different generation lengths (256 and 512). Each cell shows accuracy (top row) and decoding throughput in tokens per second with relative speedup to Dream-Baat baseline (bottom row, blue: tokens per second/orange: relative speedup). Numbers in yellow indicate the highest throughput and speedup per configuration.
Benchmark
Gen Length
Dream
+Cache
+Parallel
+Cache+Parallel (Fast- MI.LM)
75.0
74.3
74.2
74.8
GSM8K (5-sbot)
(1x)
32.5 (3.6%)
(1.66)
(5.3x)
76.0
74.3
73.4
74.0
(1x
(1.3%)
(1.9)
42,9 (5.6x)
256
384
36.8
37,9
37.6
MATH (4-shor)
(1x(
(3.0%)
(24x)
66.8 (5.9x)
512
39.8
38.0
39.5
39.3
(1x()))
26.8 (2.8%)
(3.2
63.3 (6.5x)
256
49.4
53.7
49.4
54.3
Hum (0-shot)
(1x)
(1.5%)
45.6 (2.0%)
62.0 (2.8x)
512
54,3
54,9
51.8
54.3
16.3 (1x)
27.8 (1.7%)
29.8 (1.8%)
52.8 (3.2x)
256
56,6
53.2
53.8
56.4
MBPP (3 shot)
(1x)
34.5 (3.1%)
318 (2.8%
76.0 (6.8x)
512
55.6
53.8
55.4
55.2
9.4((
26.7 (2.8%)
37.6 (4.0%)
73.6 (7.8x)
We evaluate Fast-dLLLM on two recent diffusion-based language models: LLaDA (Nie et al., 2025b), LLaDA- 1.5 (Zhu et al., 2025) and Dream (Ye et al., 2025). Bench- marks include four widely-used datasets GSMBKK MATH, HumanEval, MBPP and [FEval, to assess per- formance across diverse reasoning and code generation tasks. We also test under varying generation lengths to evaluate scalability and robustness.
No cache
GSMOK (S-thot) Acouracy
Throughpui houu (tolame
3.76 Speedap
In addition, we extend our evaluation to LLaDA-V (You al., 2025), a multimodal variant of LLaDA tailored for vision-language reasoning tasks. For this, we use two challenging multimodal benchmarks: Math Vista and Math MathVerse, which require solving math problems grounded in complex visual scenes.
55-
No cache
Block Slze
Figure 4: Impact of Cache Block Size on Ac- curacy and Throughput. The orange line il- lustrates the effect of varying cache block size on throughput, while the blue line depicts accu- racy.
Inference throughput is measured as the average number of output tokens generated per second, calculated over
the full sequence until the end-of-sequence (<eos>)
token is reached. This metric reflects true end-to-end decoding speed. All evaluations are conducted using the standardized 1nvevll library to ensure consistency and reproducibility.
4.2
MAIN RESULTS: PERFORMANCE AND SPERD
We evaluate Fa on LLaDA-Instruch and Dream-Base across four benchmarks (Tables and 2).
Introducing the KV Cache yields 2x-3.6x speedup over the vanilla backbone across tasks and lengths. Parallel decoding alone provides further acceleration, typically 4x-6x, especially at longer generation lengths.
Combining both gives the largest gains. On LLaDA, throughput improves up to 11x (GSM8K, len 512) and 9.2x (MBPP, len 512). On Dream-Bars Base, maximum gains are 7.8x (MBPP, len 512) and 5.6x (GSM8K, len 512). These results show the methods only effective individually but also highly complementary.
Importantly, these efficiency gains come with negligible accuracy cost: across all benchmarks accuracy remains within 1-2 points of the backbone, and in some cases even improves. Longer sequences, common in few-shot and code generation, benefit. disproportionately due greater cache
7

---

Published as a conference paper at ICLR 2026
OSNOK 1S 35-ahee Accumre
Das
OSMBIC 1S aae-aaeee Accurac
Das (140 Hip
Nintkoo by DAA takent
04
0.3
85
30
18
Threshold
Thresheld
Average TTokens per Step (G)
13)
(b)
shows the GSM8K (5-shot) accuracy across different confidence thresholds. Numbers along the red line indicate the average number of tokens decoded at cach step. The three dashed lines represent the accuracy of the baseline method when selecting the top 2, 4, or 8 tokens per step. (b) The number of inference steps required under varying confidence thresholds. (c) A comparison between method and the baseline on GSM8K (5-shot) accuracy, plotted against the average number of tokens per step. Our method consistently outperforms the baseline.
Figure 5: (a) The
Table 3: Performance and Speedup Comparison of LLaDA-V on Math Vista and Math Verse. Each benchmark includes results from Full Steps, Half Steps, and Fast-dLLM. Fast-dLLM significantly improves throughput (highlighted), with minimal accuracy loss.
Math
Math Verse
Metric
Full Steps
Half Steps
Fast-dLLM
Full Steps
Half Steps
Fast-dLLM
Accuracy (%)
59.7
56.6
28.5
28.3
28.6
Throughput (Speedup)
2.84 (1x)
5.56 (1.96x)
28.2 (9.9x)
2.75 (1x)
5.17 (1.88x)
23.3 (8.5x)
reuse and batch computation. We also evaluate L.L.DA-155, which achieves consistently higher accuracy and comparable or better throughput (Table 12).
Beyond text-only models, we test Fast-dLLM on multimodal LLaDA-V using Math Vista and Math- Verse. LLaDA-V is sensitive to block size, losing over 8% accuracy when reduced from 96 to on Math Vista. To mitigate this, we retain full block length and apply refresh-basee updates, yielding up to 9.9x speedup with minimal degradation (Table 3). On Math Verse, Fast-dLLM even slightly improves accuracy, showing robustness on vision- laioonlanguage reasoning.
Overall, improvements hold across architectures (LLaDA, Dream), task types (math reasoning, program synthesis), and modalities (text, vision), establishing Fast-dLLM as broadly applicable framework for accelerating masked diffusion LLMs.
4.3 ABLATIONS AND ANALYSIS
Table 4: Performance and Speedup Comparison on LLLDDA Between 5-Shot and 8-Shot Settings Generation Length 1024. This table compares the ac- curacy and throughput speedups of different decoding strategies under 5-shot and 8-shot configurations using a generation length of 1024. The results demonstrate how increased prefill length enhances effective- ness of caching strategies, particularly for DualCache.
Table 5: Impact of Generation Length Accuracy and Speedup Under 8-Shot for LLaDA. This ta- ble illustrates the effect of varying generation lengths (256, 512, and 1024) on decoding performance and efficiency for different caching stralegies under the 8- shot setting. Longer generation lengths lead to higher throughput gains, especially for DualCache, validat- ing the scalability our approach.
Parallel Decoding PretixCache
Parallel Decoding PretixCache
Sctting
LLaDA
LLaDA
Cacho
DualCache
No Cache
DualCache
77.0
77.4
75.2
74.7
77.6
77,9
77.3
76.9
shoht
(1x)))
(10.6x)
(13.
219.
256
(19()))
(3.3x)
(10.0))
(9.4x)
77.3
78.0
75.7
76.0
78.9
78.9
74,8
75.4
8-shot
(1xg
9.3 (13.3x)
13.0 (18.6x)
19.3 (27.6x)
512
(1x
(6.0
32.0 (13.9)))
36.4 (15.8%)
77.3
78.0
75.7
76.0
1024
0.7 (bx)
9.3 (13.3)))
13.0 (18.6x)
19.3 (27.6x
We perform extensive ablations to assess the contribution of different components in LLMM- focusing on prefill length, generation length, cache variants, block size, and confidence thresholds.
8

---

Published as a conference paper at ICLR 2026
Prefill and Generation Length Tables 4 and 5 show that longer prefill (n-shot) and generation lengths significantly boost speedup. For example, DualCache improves from 19.6x (5-shot, gen len 1024) to 27.6x (8-shot, gen len 1024). Speedup grows generation length increases (e.g., 9.4x at 256 tokens vs. 27.6x at 1024), consistent with amortizing computation over longer sequences.
Prefix KV Cache vs. DualCache DualCache generally surpasses prefix KV Cache, especially for long generations (e.g.. 27.6x vs. 18.6x at gen len 1024, Table 5). Accuracy remains competitive, confirming DualCache's ability to exploit parallelism and cache locality effectively.
Cache Block Size Figure shows that smaller block sizes maximize accuracy but add cache update overhead, while larger sizes risk mismatch. Block size 32 achieves the best trade-off, balancing throughput and accuracy.
Dynamic Threshold vs. Fixed Token-per-Step On GSM8K (Figure 5), our confidence-aware strategy consistently outperforms fixed baselines: it yields higher accuracy with comparable or fewer NFEs, generates more tokens per step, and approaches the accuracy of the 1-token baseline with much higher throughput.
Factor Decoding Fixed Strategies As shown in Figure and Table 11, factor-based decoding achieves competitive or better accuracy with fewer steps. Larger factors decode more tokens per step, reducing iterations while preserving performance. Compared threshold decoding, factor decoding maintains accuracy but achieves higher throughput via adaptive granularity (see Appendix C.4).
Decoding Efficiency and Limitations Section C.5 shows PrefixCache accelerates iiffusion-based or exceeding LLaMA throughput
LLMs like LLaDA by up to 5x in compute-bound settings, reaching small batch sizes. However, at larger batches falls behind LLaMA, since diffusion models incur higher overhead from full attention during decoding.
5 RELATED WORK
We put a short version here, the full Related Work is in Appendix D.
Diffusion LLM. Diffusion models, proven first in vision/audio, are re-shaping text generation. Discrete formulations re-cast noising/denoising as Markov chains (Austin et al., 2021), continuous- time processes (Campbell et al., 2022), score-matching (Lou et al., 2023) or masked-language (Shi al., 2024; Sahoo al., 2024; Zheng al., 2024); all maximize ELBO or entropy objectives and can be made equivalent. Plugging the masked objective into BART or LLaMA yields Diffusion-NAT (Zhou al., 2023), LLaDA LaMA (Nie al., 2025b; Gong et al., 2024) and Dream (Ye et al., 2025): 7 B- parameter diffusion LLMs that refine corrupted sequences in parallel and match AR quality while promising >10x speed-up.
tasks
LLM Acceleration. KV-cache (Vaswani, 2017) avoids recomputation in AR Transformers, but full-sequence diffusion invalidates it; Block flock-diffusion (Arriola et al., 2025) restores the cache by generating block-by block- bb-olock. Non-autoregressive (NAR) decoding outputs many tokens once (Xiao et al., 2023), yielding large latency gains yet lower quality. Diffusion LLMs are a new NAR family, but until now the speed benefit has been offset by accuracy loss (Nie al., 2025b).
6 CONCLUSION
We address key inefficiencies in Diffusion-barn bbase Large Language Models (Diffusion LLMs), which traditionally lack KV Cache support and suffer from degraded performance in parallel decoding. To bridge the gap with autoregressive models, we propose Fast-dLLM, a framework introducing an approximate KV Cache tailored to bidirectional attention via block-wise generation. We further mitigate token- laken-dependengee issues in parallel decoding with a Confidence-Awart strategy that enables safe, efficient multi-token generation. Experiments across benchmarks and baselines (LLaDA, Dream) demonstrate up to 27.6x speedup with minimal accuracy loss, establishing F as a practical path toward making Diffusion LLMss competitive for real-world deployment.

---

Published as a conference paper at ICLR 2026
REFERENCES
Anonymous. Oneflowseq:
Achieving one-step generation for diffusion language models via lightweight distillation. OpenReview, ICLR 2026 Conference Submission, 2025. Submission ID: P70ZWxOUHK.
Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregres- sive and diffusion language models, 2025. URL https: //ariiv../.iii////aa/eaaeeaaraaa org/ abs/////////// 2503309973.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34: 1981--1993, 2021.
Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via learnable parallel decoding. arXiv preprins 20XXi::250922/888 2025.
Andrew Camphell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 35:28266-28279, 2022.
Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai Helen Li, and Yiran Chen. Dpad: Efficient diffusion language models with suffix dropout. arXiv preprint arXiv:2508. 14148, 2025a.
Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. dparallel: Learnable parallel decoding for dllms. arXiv preprint 25Xi:2500.26488, 2025b.
Zixiang Chen, Huizhuo Yuan, Yongqian Li, Yiwen Kou, Junkai Zhang, and Quanquan Gu. Fast sampling via de-randomization for discrete diffusion models. arXiv preprint arXiv:2312.09/93, 2023.
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky TQ Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. arXiv preprint #rXiv:2407.15595 2024.
Daniel T Gillespie. Approximate accelerated stochastic simulation of chemically reacting systems. The Journal of chemical physics, 17164)177 2001.
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, et al. Scaling diffusion language models via adaptation from autoregressive models. arXiv preprint arXiv:2410.17891, 2024.
Google DeepMind. Gemini diffusion. https: /depmieepmmm/////////// nd goog e e/model s/ gemini-dis fiiniidifffssse fusiion 2025. Accessed: 2025-05-24.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. The llama 3 herd of models, 2024. URL https: https:: :/apps:://a/vv.../// http://awwwww.o///////////////// 21783.
Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusion- bert: Improving generative masked language models with diffusion models. arXiv preprint arXiv22211.1002. 2022.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 04:112444-2466 2021.
Rongjie Huang, Jiawei Huang. Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models, 2023. URL https: /aaxivivvi. www.ii.owgaaaa...... abs /2011 12661.
Inception Labs. Introducing mercury: The first commercial diffusion- language model. https:
wwwwww wwweennniiiit.mmccmoocco on labs. a a ng - rcury, 2025. Accessed: 2025-05-24.
10

---

Published as a conference paper at ICLR 2026
Ouail Kitouni, Niklas Nolte, James Hensman, and Bhaskar Mitra. Disk: A diffusion model for structured knowledge. arXiv preprint arXiv:23/2.05253, 2023.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequee pre-training for natural language generation, translation, and comprehension, 2019. URL https://arxix: org/abs/ 910 13461.
Anji Liu, Oliver Broadrick, Mathias Niepert, and Guy Van den Broeck. Discrete copula diffusion. arXiv preprint arXhv:2410.00190 2024.
Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.0623 2025.
Aaron Lou, Chenlin Meng. and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310. 1...6444,,, 2023.
Chenlin Meng, Kristy Choi, Jiaming Song, and Stefano Ermon. Concrete score matching: Generalized score matching for discrete data. Advances in Neural Information Processing Systems, 35:34532- 34545, 2022.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022. URL https: /ttp:///axxvvvvvv
abs /2122 10741.
Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text, 2025a. URL httpssttts////rxivvooo /rbbs/// 2410. 18514.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025b. URL https: //aaxix.org/aab....... / 2202/2502000 09992.
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406.037555 2024.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021. URL peattps://arivvee abss2100 12092.
Machel Reid, Vincent J. Hellendoom, and Graham Neubig. Diffuser: Discrete diffusion via edit-based reconstruction, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models, 2022. URL www.thtss//axiii.oott/ abs/2112 10752.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding, 2022. URL https: / http///axivv..roo/ abs/
Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. arXiv preprint arXivv2406.07522 2024.
Jiaxin Shi. Kehang Han, Zhe Wang. Amaud Doucet, and Michalis K Titsias. Simplified and generalized masked diffusion for discrete data. arXiv preprint arXiv:2406.0433 04329, 2024.
11

---

Published as a conference paper at ICLR 2026
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning. pp. PMLR, 2015.
Jiaming Song and Linqi Zhou. Ideas in inference-time scaling can benefit generative pre-training algorithms. arXiv preprini arXiv:2503.07154. 2025.
Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction. arXiv preprint NrXiv:2508.02558, 2025.
Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-time discrete diffusion models. arXiv preprint arXiv:2211.16750, 2022.
Ashish Vaswani. Attention is all you need. arXiv preprint arXiv:1706.033000 2017.
Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, and Tie yan Liu. A survey on non-autoregressivv generation for neural machine translation and beyond, 2023. URL https: //aaxv/aaavvv///a www.abs.///ooooooooooooooooo.o. 22224 09269.
Yi Xin, Qi Qin, Siqi Luo, et al. Lumina-dimoo: An omni diffusion large language model for multi-modal generation and understanding. arXiv preprint arXiv:2510.06:08, 2025.
Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, and Arash Vahdat. Energy-based diffusion language models for text generation. arXiv preprint arXiv:2410. 2024.
Donghaa Yang. Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation, 2023. URL https:/ //axxiv.. org/abs 22077. 09983.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. URL https:: /hkunlp gi thub . 1o /tll/////////oooooooooooooooool/ 20225/225///// dream.
Jiasheng Ye, Zaixiang Zheng. Yu Bao, Lihua Qian, and Quanquan Gu. Diffusion language models can perform many tasks with scaling and instruction -inettionniintuinng... arXiv preprint PXXv:23308.12219 2023.
Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933, 2025.
Runpeng Yu, Qi Li, and Xinchao Wang. Discrete diffusion in large language and multimodal models: A survey, 2025a. URL https:ttp///s. /tapp:://arxivv arxiv. http:////axxi..../a///as////s//s /2///.156.......
Runpeng Yu, Xinyin Ma, and Xinchao Wang. Dimple: Discrete diffusion multimodal large language model with parallel decoding, 2025b. URL htttt:://axxiiiii wttp:bs////bb./// 111.25155159990 16990.
Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yo Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. arXiv preprint arXiv:2409.022: 02908, 2024.
Lianmin Zheng, Wii-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46623, 2023.
Kun Zhou, Yifan Li, Wayne Xin Zhao, and Ji-Rong Wen. Diffusion-nat: Self-prompting discrete diffusion for non-autoregressive text generation, 2023. URL https: /ttp:::///arvvvo//o org/ abs/ 04044.
Fengqi Zhu, Rongzhen Wang. Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji- Ji-Rong Wen, and Chongxuan Li. Llada 1.5: Variance-reduced preference optimization for large language diffusion models, 2025. URL https: http//awvvvv..o...e//a www./bs////ooooooooooo.........a 2200. 9223.
12

---

Published as a conference paper at ICLR 2026
A PROOF
In this section, we will give the comprehensive proof and discussion of Theorem 1.
Proof. Step I: Show that x: the unique maximizer of g(x)).
Let pj = PP5(x; 3. E.) We are given pž > 1 Let cj 1 P2 = Pz(Xi, * 2i,1E)) (.). Thus,
of The product-of --agg-inals probability mass function (PMF) is
o( E)
II P3 (X,, 22|
To maximize ()) we must maximize each term p. (X,, = =2 =2)|E) independently. The condition (n 1)e I 1 implies < 1/(n 1). Since 2 1, it follows that 1/(n + 1) s 1/2. So, e S 1/2. Therefore, for the chosen Tzz²
-21 pj - p3 = [D) 1
1/2.
This means 24, is the unique maximizer for (-(-)))) So.
argmax ((2(E) = (Ii ................................ Fzm = x².
Step Show that x" is the unique maximizer of p(x).
We want to show p(x* > P(z|E) for all * x². Using the Bonferroni inequality:
= = (X,, = Ii, }1E) 2 Σ p(x,, # Iz, EB) = 1-
Σ o
Since ej <e for all j. we have Σラ= tjj ne. So,
p(x EE) > 1 nc.
Now consider any (21,..., 2n) such that F x=. This means there is at least one index A such that 2 # Dis. The event (X = 2) is sub-event of (Xix =x So,
E)z[E)= S (Xi, = =zx(E).
Since 2か # Tigs
(X E) S (Xin & ris E) c <C
Thus,
p( E) <E E. For p(x > p(=|E) hold, is sufficient that
ne 2,
which simplifies to 1 > (n + 1)c, or < this condition. The strict inequalities p(æ" [E) that p E) p(z Thus,
1
The theorem assumes (n + 1)e < 1, which is exactly
n-T
> 1- E
and p(=|E) S é ensure
argmax p(z E) = x
Combined with the argmax of q, this proves the main statement of Part
argmax p(z|E) argmax q(x x=
Step 3: Tightness of the bound a+I
The bound S 1 is tight. This means if .F.. one can construct scenario where the marginal T+1 conditions P,(X, = Iz, [E) > - hold, but argmax_ [( * x" (which is argmax, g(x E) long 1/2).
13

---

Published as a conference paper at ICLR 2026
Consider vocabulary V {0, 1} and let 21; = 0 for all j, so x=' (O....,0) 0). For each j E (I,...,,), let ey be the vector with 1 at position j and 0 elsewhere. Let x+1
1 (c n+1 > 0.
1
Set p(ey = カ+1 + n: A1 S < n and p( E) n+1 then x & argmax, p [z) E). The
marginal probabilities are:
Px 1|B) - B) = 1
1 - A1 S S n.
p) 0|E) = 1 1 (X;, = 1|3) = 1 - Ec
-1 > 1 - E,
1
because
1 1
1
n + 1) n + 1 <<-
(c-
n + 1
So, the marginal condition (X 2i, [) > 1 - c (with Iaj = 0) holds. As shown,
argmax, p(= can be made different from x". Thus, if n+ the argmax of p and q may not be the same.
Step 4: Bound the Lp distance. Let A, be the event (Xi, = Niy }-
Dp (p. g)8 = [p(x= F.) q( [P)P +
Σ Ip(=|F)- [=
-
The term A)())=A)1BB) - [Ti= p( (Ay|B)) (using p ((A,1 E) for py (X1, Fıy |B) can be bounded. Since
1 1-
Σ
& S p(n -
min p(A, E) = =1 - max ',
1si5n
15/5n
j=1
n
n
n
Σ ,
II (1-6)) ())
II (() S
max oj
15/5n
j=1
j=1
j=1
Thus,
1p(0* (E) g(x= [E) (n 1)e.
For + w": p(z(B) < and (2 e. So,
|x(z|E) q( x E|] <e.
The sum Exper [p(z[E) {z(|E)) can be bounded:
Σ (<=1E)-g(z(E))) g((|E>| < Σ
=
+ +(xx fx"l))
p(x + -1 -1==\)
<1 11- Σ ( = Σ 4 < ne.
j=1
j=l
<(X +
<1 - II (1-
2 nc.
j=1
1=1
So.
Σ p(z|b)- q(>|->) #
2ne.
Then,
Σ |e(2(E) {= ( sup |p(2|F)) (((|E)))011
Σ |(((EE) ((([E)
< cc-1((nn) = 2nc
Therefore,
Dr (p.9)* < <((n 1)c}p + 2ne ((n - 1)p + 2n)eP.
14

---

Published as a conference paper at ICLR 2026
So,
Dp (p. < ((n - 1)p + 22)
For p =
D1 g) < (n 1 2n)e = (3n 1)E.
And for Total Variation Distance,
Drv(p, Di (p.0) < 3n
2 6.
Step 4: Bound the forward KL divergence.
DKL. (pllg) E(z)E) (3 (x, ()).
(x([E)
The conditional total correlation can be expanded using the chain rule:
[X i........ X2, E)
I(X X N 1
Each term is bounded by the conditional entropy:
T(Xi;: Xiy...., Xi, (E) S H(xi, (E).
The conditional entropy HX;;, lE) is bounded. Since Px Fig EE) > 1 c, it implies
Pa (X;, * Fig IE) = c < e. The entropy is maximized when the remaining probability eg is spread uniformly, leading
H (X,; [b)= & Ho(()) + é In([V) < Ho(e) + enn(
Summing (n- 1) such terms (for k = I1):
DKL (plla) < (n (() + cn([ 1)).
Remark 1. Assumption of a Well-Defined Joint pe .X........ X., E):; The theorem and proof rely on po (Xi....... X,, [E) being a well-defined joint probability mass function from which the marginals (X, [E) are consistently derived. This implies that the joint PMF is coherent and its definition does not depend on a specific factorization order beyond what captured by the conditioning on E. In practice, while MDM may not strictly satisfy this property. its behavior typicully offers close approximation. The theorem holds for an idealized po that possesses these properties. As MDMs become larger and more powerful, their learned distributions might better approximate such consistency.
The conditions and bounds provided in the theorem (c.g., (n + 1)e < 1) are derived from a worst-case analysis. This means the bounds are guaranteed to hold if the conditions are met, regardless of the specific structure of pe (X [E) beyond the high-confidence marginal property. In practice, the actual case might be "better behaved" than the worst-case scenario. For instance, the dependencies between X,, and Xi, (given E) might be weaker than what the worst-case construction assumes. Consequently, the argmax equivalence! (Result 1) might still hold frequently even if (n + 1)e slightly greater than I (but not much larger). The condition identifies a threshold beyond which guarantees break down in the worst case, but practical performance can be more robust. Similarly, the actual Lnp distances or KL divergence might be smaller than the upper bounds suggest if the true joint po X [E) is closer to the product of marginals ((X(E) EE) than the worst-car case configurations.
Worst-Case Analysis:
B
CASE STUDY
B. EFFECT OF CACHING STRATEGIES ON RESPONSE QUALITY
Table 6 presents qualitative comparisons of responses from the Original, PrefixCache, and DualCache methods for both an arithmetic prompt and an coopl-cooplttion prompt. For the arithmetic question, all three methods correctly compute 3 bolts through similar step-by-step reasoning, differing only in minor wording. For the dtfom-completinn give accurate explanations of its meaning. These results indicate that different caching strategies preserve semantic fidelity and reasoning quality across both factual and conceptual queries, while primarily serving to improve efficiency without degrading interpretability correctness.
task, all methods produce the standard completion and
15

---

Published as a conference paper at ICLR 2026
Table 6: Qualitative comparison of responses across methods.
Prompt: A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?
Original
PrefixCache
DualCache
The robe takes bolts blue fiber. It also takes half that much white fiber, so it takes 2/2 = 1 bolt of white fiber.
The robe takes 2 bolts of blue fiber. It also takes half that much white fiber, so it takes 2/2 = bolt of white fiber.
The robe takes bolts blue fiber. It also takes half that much white fiber, so it takes 2/2 = 1 bolt of white fiber.
In total, the robe takes 2 3 bolts of fiher. So, the value is 3
In total, the robe takes 2 1 bolts fiber. So, the value is 3
In total, takes bolts 1 bolt 3 bolts of fiber. The final result is
Prompt:
Finish the idiom and explain its meaning: bird in the hand worth...
Original
PrefixCache
DualCache
bird in the hand worth two in the bush.
A bird in the hand is worth two the bush.
bird in the hand worth two in the bush.
This idiom means that having something certain and guaranteed is more valuable than the possibil- ity of something that is uncertain or risky. It suggests that s's better to have a sure thing than to risk losing something valuable. The phrase comes from the idea that having bird in one's hand is more secure than having two birds in the bush, which could be lost or harmed.
This idiom means that having something certain and guaranteed is more valuable than the possibil- ity of gaining something more but is uncertain. It suggests that it's better to have a smaller amount of something that you know you have, rather than risking larger amount of something that you don't have. The phrase is often used to encour- age people to focus on what they already have and avoid chasing after something that may not mate- rialize.
This idiom means that having something certain and guaranteed is more valuable than the possibil- ity something greater but uncer- tain. It suggests that it's better to have a small, certain gain or reward rather than to risk larger, poten- tially uncertain reward. In other words, it's better to be safe than sorry.
Table 7: Qualitative comparison of responses with varying block size for DualCache.
Prompt: A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?
Block Size 8
Block Size 16
Block Size 32
The robe takes bolts of blue fiber. It also takes half that much white fiber, so it takes 2/2 1 bolt of white fiber.
The robe takes 2 bolts of blue fiber. It also takes half that much white fiber, it takes 2/2 bolt of white fiber.
The robe takes bolts of blue fiber. It also takes half that much white fiber, so it takes 2/2 1 bolt of white fiber.
total, the robe takes 2 + = 3 bolts of fiher. So, the value is 3
In total, the robe takes 2 + 3 bolts of fiher. So, the value is 3
total, the robe takes 2 + = bolts of fiber. So, the value is 3
Prompt: Finish the idiom and explain its meaning: A bird in the hand is worth...
Block Size
Block Size 16
Block Size 32
bird in hand worth two in the bush.
A bird the hand is worth two the bush.
bird in the hand worth two in the bush.
This idiom means that having something certain and secure is more valuable than the possibility of gaining something more but is uncertain or risky. It emphasizes the importance of taking precau- tions and holding onto what you already have, rather than risking everything for a potential gain.
This idiom means that having something certain and secure more valuable than the possibility of something that is uncertain or risky. It suggests that it's better to have sure thing in your pos- session rather than taking chance on something that may not work out. The phrase is often used to advise against taking unnecessary risks and to prioritize safety and se- curity over potential gains.
This idiom means that having something certain and guaranteed more valuable than the possibil- ity of something greater but uncer- tain. It suggests that it's better to have a small, certain gain reward rather than to risk larger, poten- tially uncertain reward. In other words, it's better to be safe than sorry.
16

---

Published as a conference paper at ICLR 2026
Table 8: Qualitative comparison of responses under different threshold settings.
Prompt: A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?
Threshold 0.7
Threshold 0.8
Threshold 0.9
The robe takes bolts of blue fiber. It also takes half that much white fiber, so it takes 2/2 1 bolt of white fiher.
The robe takes 2 bolts of blue fiber. It also takes half that much white fiber, it takes 2/2 bolt of white fiber.
The robe takes bolts of blue fiber. It also takes half that much white fiber, so it takes 2/2 I bolt of white fiber.
total, it takes takes 2 1 = bolts of fiber. So, the value is 3 (NFE; 9)
In total, the robe takes 1 = 3 bolts of fiber. So, the value is 3 (NFE: 12)
total, the robe takes + = bolts of fiber. So, the value is 3 (NFE: 20)
Prompt: Finish the idiom and explain its meaning: A hird in the hand is worth...
Threshold 0.7
Threshold 0.8
Threshold 0.9
A bird in the hand is worth two in the bush.
A bird in the hand is worth two in the bush.
A bird in the hand is worth two in the bush.
This idiom means that having something certain and guaranteed more valuable than the possibil- ity of something that is uncertain or risky. It suggests that it's 'ss better to have thing than risk losing something valuable. The phrase comes from the idea that having a bird in one's 's hand is more secure than having two birds in the bush, which could be lost or harmed. (NFE: 74)
This idiom means that having something certain and guaranteed is more valuable than possibil- ity of something that is uncertain or risky. It suggests that it's better to have sure thing than to risk losing something valuable. The phrase comes from the idea that having a bird in one's 's hand is more secure than having two birds in the bush, which could he lost harmed. (NFE: 79)
This idiom mcans that having something certain and guaranteed more valuable than the possibil- ity of something that is uncertain or risky. It suggests that it's better to have
thing than risk losing something valuable. The phrase comes from the idea that having a bird in one's 's hand is more secure than having two birds in the bush, which could he lost or harmed. (NFE: 86)
B.2 EFFECT OF BLOCK SIZE IN DUALCACHE
Table 7 compares outputs from DualCache with varying block sizes (8, 16, and 32). Across both the arithmetic and idiom prompts, all configurations return identical answers, with consistent logical steps and explanatory clarity. No qualitative differences are observed in output content, suggesting that DualCache is robust to block size settings for these types of queries. This allows practitioners to adjust block size for computational efficiency without worrying about a loss of output quality.
B.3 IMPACT OF DYNAMIC THRESHOLD SETTINGS
Table 8 explores the effect of varying dynamic threshold values (0.7, 0.8, 0.9). For the arithmetic and idiom tasks, all thresholds yield correct answers with coherent explanations, confirming that threshold tuning does not affect accuracy for straightforward prompts. However, higher thresholds noticeably increase the number of function evaluations (NFE), e.g., from 9 to 20 for the arithmetic task and from 74 to 86 for the idiom task. This suggests that threshold choice primarily influences computational cost, with quality remaining stable-enabli trade-offs between speed and resource usage for different application scenarios.
B.4 MULTIMODAL GENERATION WITH LLADA-V
To qualitatively analyze the effecttivnness of our Fast-dLLM framework in multimodal scenarios, we conduct a visual case study where the model is tasked with generating a highly detailed image deseription. As illustrated in Figure 10, both the baseline model and our Fast-dLLM are given the same visual input and user prompt: "Please describe the image in detail."
The baseline model requires 63.0 seconds to complete the generation, producing detailed and poetic description of the rural landscape. It highlights elements such as the weathered wooden barn, the soft pink sky, and the tranquil atmosphere.
17

---

Published as a conference paper at ICLR 2026
User: Please describe the image in delail.
Baseline (63.0 sees); The image captures serene rural landscupe. Dominating the seene a wooden barn, its weathered exterior a testament to the passage of time. The burn is nestled in field of green grass, which stretches out to meet the horizon. In the distance, range of mountains rises, their peaks shrouded in blanket of clouds. The sky above is a soft pink, suggesting either dawn or dusk. The perspective of the image is from low angle, making the barn appear grand und imposing. The overall composition of the image creates sense of depth and tranquility, inviting the viewer to imagine themselves in this peaceful rural sctting.
Fast-dLLM (6.8 secs): The image captures serene rural landscape. Dominating the seene is a wooden barn, its rustic charm enhanced by the gray shingles ils roof. The burn nestled grass, which stretches towards the horizon. The field is framed runge of mountains, their peaks dusted with snow, udding touch tranquility the scene. The sky above is soft pink, suggesting either carly morning late afternoon. The perspective of the image from imposing. overall composition harmonious blend man-made structures natural world.
field of tall
angle, making barn appear grand and
image creates
Figure Comparison between baseline and Fast-dLLM on a visual description task. Fast-dLLM produces comparable and faithful image caption in fraction of the decoding time.
In contrast, our Fast-dLLM completes the task in just 6.8 seconds nearly 10x speedup while maintaining rich visual detail. It further enhances the description with additional grounding (e.g.. "gray shingles its roof", "touch of tranquility"), reflecting a strong alignment with both appearance and mood cues from the image. Notably, the generated caption retains compositional depth and stylistic fluency, illustrating the model's ability balance fluency and factuality even under diffusion- based parallel decoding.
This highlights how LLAda-V with Fast-dLLM decoding enables high-quality vision-language generation at significantly improved efficiency, paving the way for faster and more interactive multimodal applications.
C EXPERIMENT DETAILS
C.1 FURTHER EXPERIMENTS WITH LLADA-V
Table Effect block length on performance (MathVista, 48 Steps)
Block Length
8
16
32
96
Accuracy (%)
51,2
50.7
51.8
52.3
59.7
Throughput (tok./s)
6.1
6.2
5.5
5.5
5.6
Table 10: MathVista Performance with Fast-dLLM at different reffresh intervals (block length 96)
Refresh Interval
4
16
Accuracy (%)
59.2
59.2
58.2
57.1
56.6
Throughput (tok./s)
15.9
19.5
21.1
25.2
28.2
In Table 9, we investigate how the choice block length affects the performance of LLaDA-V V on Math Vista under fixed decoding length of 48 steps. The results show that the model achieves the highest accuracy with a block length of 96. However, when reducing the block size to or 4, the accuracy drops significantly by over 8%.
18

---

Published as a conference paper at ICLR 2026
Given this sensitivity to block length, we choose not to break the output into small blocks for updating caches individually. Instead, we keep the block length fixed at 96 and adopt refresh-based strategy: the cache is updated only every decoding steps using the most recent full block. As shown in Table 10, increasing the refresh interval leads to consistent gains in throughput-ffrom 13.9 tokens/s interval 2 to 28. 2..2 tokens/s at interval 32. While accuracy drops slightly with larger intervals, it remains above 56.6%, suggesting that aggressive refresh scheduling can yield substantial speedups with only minor performance degradation.
C.2 PERFORMANCE COMPARISON BETWEEN THRESHOLD AND FACTOR STRATEGY
Table 11: Performance comparison between Threshold and Factor confidence-aware decoding on GSM8K and MATH benchmarks with generation lengths of 256 and 512. Each block shows accuracy (top row) and throughput with speedup (bottom row), Factor decoding provides favorable trade-offs most settings.
Benchmark
Gen. Len
Threshold
Factor
256
78.5
77.5
GSM8K (5-shot)
54.4 (8.1x)
78.5 (11.7x)
512
77.2
74.8
35.3 (11.0x)
47.1 (14.7x)
256
33.2
32.0
MATH (4-shot)
51.7 (5.7x)
78.3 (8.6x)
512
36.0
35.2
47.1 (5.9x)
64.6 (8.1x)
We compare the performance of our frreshold-based and factor- based confidence-awee awaree parallel decoding strategies on GSM8K and MATH benchmarks (Table 11). While the threshold strategy achieves marginally better accuracy in most settings (e.g., 78.5% 77.5% GSM8K with 256 tokens), the factor strategy demonstrates substantially superior throughput performance.
Specifically, factor decoding achieves 1.4-1.5x higher throughput than threshold decoding across all settings. On GSM8K with 256 tokens, factor decoding reaches 78.5 tokens/sec (11.7x speedup) compared 54.4 tokens/sec (8. 1x.1x speedup) for threshold decoding. This throughput advantage becomes even more pronounced on longer generation tasks-for GSM8K with 512 tokens, factor decoding attains 47.1 tokens/sec while threshold only achieves 35.3 tokens/sec.
The results demonstrate that factor decoding offers a compelling trade-off: it sacrifices minimal accuracy (typically 1-3%) in exchange for significant throughput improvements (40-50% higher). This makes factor decoding particularly attractive for aiteccy-seniiive applications where the slight accuracy reduction acceptable. The consistent pattern across both benchmarks and generation lengths validates the robustness of the factor strategy's 's theoretical foundation, which adaptively controls parallelism based on the confidence bound (n + 1)e K.
C.3 COMPARISON BETWEEN LLADA AND LLADA-1.5
We compare the performance of LLaDA and its enhanced version LLaDA-115 across both GSM8K (5-shot) and MATH (4-shot) benchmarks under two generation length settings (256 and 512 tokens), as shown in Table 12. Each cell reports accuracy and decoding throughput (in tokens per second), along with the relative speedup over the greedy bascline.
Across GSM8K settings, LLaDA-1.5 consistently improves accuracy over the original LLaDA, achieving notable +2.2% absolute gain at 256-token generation and +3.2% at 512-token generation. Furthermore, it maintains strong decoding efficiency, with throughput reaching 59.4 tokens/sec at 256 tokens, improving upon LLaDA's 54.1 tokens/sec under the same setting.
On the MATH benchmark, accuracy between the two versions remains comparable. However, L.. slightly improves throughput at 256 tokens (53.7 vs. 51.7) while incurring mild
19

---

Published as a conference paper at ICLR 2026
Decoding Steps vs Parallel Token Count
95% Confidence Avg Taken Count
Average Parallei Token Count
20
60
80 100
Step Index
Figure 7: Average number tokens generated at each decoding step. Blue line shows the mean token count, and the shaded area denotes the 95% confidence interval.
Table 12: Performance comparison between LLaDA and L.a.D--1.5. Each cell presents the accuracy and the decoding throughput tokens per second with relative speedup the L.LaDA baseline (hottom row, blue: tokens per second/orange: relative speedup).
Benchmark
Gen Length
LLaDA (Fast-dLLM)
LLaDA 1.5 (Fast-dLLM)
256
78.5
80.7
GSM8K (5-shot)
54.1 (8.1x)
59.4 (8.9x)
512
77.2
80.4
35.3 (11.0x)
33.0 (10.3x)
256
33.2
32.6
MATH (4-shot)
51.7 (5.7x)
53.7 (5.9x)
512
36.0
35.1
47.1 (5.9x)
41.1 (5.1x)
efficiency regression the 512-token setting (41.1 vs. 47.1). This suggests that while LLaDA-1.5 introduces enhancements beneficial for shorter or moderate decoding contexts, longer sequences may require further optimization.
Overall, LLaDA-1.5 consistently provides either superior accuracy or better decoding speed across settings, demonstrating better performance-efficienss trade-off's and highlighting the benefit of incorporating adaptive improvements on top of the base LLaDA architecture.
C.4 ANALYSIS PARALLEL TOKEN COUNTS ACROSS DECODING STEPS
To better understand the behavior of factor-based parallel generation, analyze the average number of tokens generated at each decoding step. Specifically, we collect statisties from all intermediate steps of the sampling process and compute the average number of tokens generated in parallel per step. The results are visualized in Figure along with 95% confidence interval indicating cross-sample variability.
As shown in Figure 7, the average number tokens generated in parallel gradually increases during the early to middle stages of decoding, peaking roughly between step 30 to step 60. After this peak, the parallelism tends to slightly decline toward the end of generation. This suggests that the model becomes confident in generating outputs during the mid-decoding phase, allowing it to produce more tokens simultancously. Toward the final steps, the decoding process tends to become more conservative, reducing the number of tokens produced at each step.
20

---

Published as a conference paper at ICLR 2026
GSMOK 15 30--hhe Accurar
GSHOK -ahoo Accunacy
Selerid
3,50
Inforrncce Stege
-20
3:79
67
16
1.5 Factor
1.5
25 30
35
46
45
500
Factor
Average #Tokens per Step
13)
(b)
1G)
Figure 8: (a) GSM8K (5-shol) accuracy across different factor values using our factor-based decoding strategy. Numbers above each point indicate the average number tokens decoded per step. The dashed lines show the accuracy of the baseline method with 2 or tokens per step, and the non parallel (1 token/step) baseline. (b) The corresponding number of inference steps needed under each factor setting. Our method generally requires significantly fewer steps than fixed-step haselines. (c) Accuracy versus average number of tokens decoded per step on GSM8K (5-shot). Our factor-based decoding achieves better accuracy-e -euuacc-efficiecc trade-offs compared to baselines. The red "Selected" point represents the setting chosen in our main results.
Throughput Size
Generation
Method
PrefixCache
600
32
LLaDA
LLaMA
Throughpu: (Token/S)
400
300
100
Batch Size
Figure Throughput comparison between PrefixCache. LLaDA, and LLaMA under different generation lengths and batch sizes. All models are evaluated on aa NVIDIA A100 GPU with the prefill length fixed at 256.
The shaded confidence interval reveals greater variance in later decoding steps, indicating instability and inconsistent generation behavior across samples. This is expected since tail-end decoding steps tend to handle only a few remaining tokens required to complete the output, and the number of remaining tokens could differ widely among different samples (c.g., due to early completion padding).
These observations are important for understanding how decoding efficiency be optimized: increasing parallelism during high-confidence phases (middle steps) offers computational savings, while conservative behavior near boundaries maintains quality.
C.5 THROUGHPUT COMPARISON UNDER VARYING BATCH SIZES
All experiments are conducted on an NVIDIA A100 GPU, with the prefill length fixed to 256 tokens. The generation length is varied among 16, 32, and 64 tokens, and batch sizes range from 1 to 32. This setup reflects realistic deployment scenarios, allowing the evaluation decoding efficiency under diverse conditions.
21

---

Published as a conference paper at ICLR 2026
It should be noted that parallel decoding allows multiple tokens to be generated simultaneously affected by dummy input tokens. To ensure fairness, we focus solely on the acceleration provided by caching techniques.
PrefixCache is designed as an acceleration mechanism for LLaDA, a diffusion- LLM, and suc- cessfully boosts the throughput significantly. Figure 9 shows that PrefixCache achieves consistent improvements across all batch sizes and generation lengths, making it particularly suited for scenarios with smaller generation lengths and larger batch sizes. For instance, with a generation length of 16 and batch size of 32, PrefixCache achieves a throughput of over 211 tokens/s, signifi- cantly outperforming the native LLaDA which reaches only 43 tokens/s, demonstrating nearly 5 5x improvement.
While LLaDA exhibits limited scalability with increasing batch sizes-its throughput plateaus after batch size this limitation is inherent to iuffusion-based LLMs, which are compute- bound by nature. In contrast, LLaMA, an autoregressive (AR) model, benefits greatly from large batch sizes. As the batch size increases, LLaMA shifts from being memory-bound to compute-bound, allowing it to achieve high absolute throughput at larger batch settings.
These results highlight the practical advantages of PrefixCache in accelerating compute-bound diffu- sion models like LLLDA, especially for latency-critical and high-throughput applications. Further- more, the scalability and efficiency provided by PrefixCache bridge the gap between diffusion-based LLMs and AR models like LLaMA, showcasing its importance for large-scale deployment settings.
D RELATED WORK
D.1 DIFFUSION LLM
Diffusion models have emerged as transformative paradigm in generative modeling, initially achieving remarkable success in continuous domains such as image (Rombach et al., 2022; Nichol al., 2022; Ramesh al., 2021; Saharia et al., 2022) and audio synthesis (Yang et al., 2023; Huang et al., 2023) before expanding into natural language processing. Recent advancements in discrete diffusion models (Austin et al., 2021: Nie al., 2025a;b; Hoogeboom et al., 2021; Campbell et al., 2022; He et al., 2022; Meng et al., 2022; Reid et al., 2022; Sun et al., 2022; Kitouni et al., 2023; Zheng al., 2023; Chen al., 2023; Ye al., 2023; Sahoo et al., 2024: Shi al., 2024; Zheng et al.. 2024; Gat et al., 2024; Yu et al., 2025b;a) have reshaped the landscape of text generation, offering viable alternative to autoregressive (AR) paradigms in large language models (I.LMs). These models address the inherent challenges of discrete data by redefining noise injection and denoising processes through innovative mathematical formulations.
Theoretical Foundations of Discrete Diffusion Diffusion models for discrete data were first explored in (Sohl-Dickstein et al., 2015; Hoogeboom et al., 2021). Subsequently, D3PM (Austin et al., 2021) provided a more general framework. This framework models the forward noising process discrete state Markov chain using specific transition matrices. For the reverse process, D3PM learns parameterized model of the conditional probability of the original data given a noised version by maximizing the Evidence Lower Bound (ELBO). CTMC (Campbell et al., 2022) further extended Markov Chain (CTMC). In
D3PM to a continuous-time setting, formalizing it as a continuous- distinct approach, SEDD (Lou et al., 2023) learns the reverse process by parameterizing the ratio of marginal likelihoods for different data instances a given noising timestep. This ratio model is then trained using a Denoising Score Entropy objective. More recently, research on Masked Diffusion Models (MDMs) by MDLM (Shi al., 2024; Sahoo et al., 2024: Zheng et al., 2024) and RADD (Ou et al., 2024) has introduced significant clarifications. These studies have demonstrated that different parameterizations of MDMs can be equivalent.
Integration with Pre-trained Language Models A critical breakthrough involves combining discrete diffusion with existing LLM architectures. Diffusion-NAT (Zhou et al., 2023) unifies the denoising process of discrete diffusion with BART's (Lewis et al., 2019) non-autoregressive decoding, enabling iterative refinement of masked tokens. By aligning BART's inference with diffusion steps, this approach leverages pre-trained knowledge while maintaining generation speed 20x faster than comparable AR transformers. Similarly, the LLaDA (Nie et al., 2025b) and DiffuLLaMA (Gong et al., 2024) framework scales diffusion to 7B parameters using masked denoising, while LLaDA
22

---

Published as a conference paper at ICLR 2026
and Dream (Ye et al., 2025) demonstrating competitive performance with autoregressive baselines like LLaMA3 (Grattafiori et al., 2024) through recursive token prediction across diffusion timesteps. Ditiia (Xin et al., 2025) extends discrete diffusion modeling to multimodal tasks, achieving state-of-the-arr performance on text-to-image generation and image understanding by utilizing fully discrete diffusion across modalities with higher sampling efficiency than hybrid AR-Diffusion paradigms.
Beyond text-only generation,
One-Step and Few-Step Diffusion Models Recent work has focused on reducing the number of denoising steps required for high-quality generation. OneFlowSeq (Anonymous, 2025) distills multi-step diffusion teachers into one-step generators through MeanFlow-basee supervision and Jacobian- product signals, achieving state-of-the-ar Seq2Seq performance with significantly reduced inference time while requiring fewer trainable parameters. dParallel (Chen et al., 2025b) introduces certainty-foree forcing distillation that trains diffusion models to achieve high certainty on masked tokens more rapidly and parallel, reducing decoding steps from 256 30 on GSM8K with 8.5x speedup. Learning to Parallel (Learn2PD) (Bao et al., 2025) proposes a lightweight adaptive filter model that predicts whether each token position matches the final output, achieving up to 22.58x speedup on LLaDA without performance degradation, and up to 57.51x when combined with KV-Cache.
D.2 LLM ACCELERATION
Key-Value Cache Key-Value (KV) Cache is a fundamental optimization technique in modern large language model (LLM) inference with Transformer architecture ( (Vaswani, 2017). It enables efficient autoregressive text generation by storing and reusing previously computed attention states. However, it is non-trivial to apply KV Cache in diffusion language models such as LLaDA due to full attention. Block diffusion (Arriola et al., 2025) overcomes key limitation of previous diffusion language models by generating block-by-bloce so that key and values of previously decoded blocks can be stored and reused. Several recent works have proposed specialized caching mechanisms for diffusion LLMs. dLLM-Cache (Liu et al., 2025) introduces training-free adaptive caching framework that combines long-interval prompt caching with partial response updates guided by feature similarity, achieving up to 9.1x speedup. Sparse Sparse-dLLM (Song et al., 2025) proposes the first training-free free dynamic cache eviction method featuring delayed bidirectional sparse caching, leveraging the observation that pivotal tokens remain salient across decoding steps while low-relevance tokens stay unimportant, achieving up to 10x higher throughput. DPad (Chen et al., 2025a) restricts attention to a structured subset of suffix tokens through sliding window and distance-decay dropout strategy, delivering up to 61.4x speedup while maintaining comparable accuracy.
non-Auutoeerressiee Generation Non-autorrerresivv (NAR) generation marks a fundamental shift from sequential token generation by enabling the simultaneous generation of multiple tokens, signifi- cantly accelerating inference (Xiao al., 2023). Initially introduced for neural machine translation, NAR methods have since been extended to variety of tasks, including grammatical error correction, text summarization, dialogue systems, and automatic speech recognition. Although NAR generation offers substantial speed advantages over autoregressive approaches, it often sacrifices generation quality. Diffusion LLMs represent a recent paradigm for non-autoregressive text generation; however, prior work (Nie et al., 2025b) has struggled to realize the expected acceleration due to a notable drop in output quality.
E LLM USAGE
During manuscript preparation, we used large language models -strictly for language polishing of paragraphs and sentences (grammar, flow, and tone). These tools were not used to generate ideas, design experiments, or determine conclusions. All technical content, methodology, and interpretations were written, verified, and approved by the authors. To reduce risks of factual drift or citation errors, we required human review of every model-edited sentence and cross-checker all references against primary sources. The authors take full responsibility for the accuracy and integrity of the manuscript.
23

---

Published as a conference paper at ICLR 2026
1.00
Cosine Similarity
0.90
0.85
0.80
Step
Step 16 vs 64
Layer Index
Figure KV cache similarity across layers between adjacent diffusion steps.
F KV ACTIVATION SIMILARITY
As shown in the figure, adjacent diffusion steps (e.g.. step 16 vs. 17) consistently exhibit high cosine similarity across all layers, indicating that our caching mechanism is well-justified for short-term reuse. However, the similarity decreases in deeper layers step intervals widen (e.g., step 16 vs. 64), suggesting that KV activations evolve more rapidly toward the top of the network. This non-uniformity across layers and steps points to opportunities for improvement. In future work, plan to explore layer-aware or adaptive caching strategies that more precisely account for per-layer similarity patterns.
G DPARALLEL COMBINED WITH FAST-DLLM
Our confidence-ane aware decoding strategy remains effective when applied to distilled few-step diffusion models like dParallel, which naturally exhibit higher confidence due to distillation. Using threshold of 0.5, dParallel combined with our decoding method already achieves large speedups. Moreover, by further integrating to 92.0 tokens/s) with a slight accuracy improvement (from 76.5% to 77.1%) on GSM8K (see table above). The reduced number of decoding steps per block in dParallel improves cache reuse fidelity, making our caching design even more precise. As for One-Step Diffusion LLM, while it explores aggressive timestep reduction, its evaluation focuses primarily on perplexity rather than downstream tasks, so we could not include it in our setting. Exploring how our techniques generalize to extreme distillation regimes remains a valuable direction for future work.
KV cache, observed an additional 1 1.84x throughput gain (from 49.9
Table 13: Performance comparison LLaDA baseline, dParallel (threshold 0.5), and dParallel with prefix caching (threshold 0.5) on GSM8K in terms of throughput (tokens/s) and accuracy (%). The dParallel model benefits from intillation-inducee higher confidence, enabling effective application of confidence-aware decoding. Integrating KV cache further boosts throughput 1.84x with slight accuracy gain.
Model
Throughput (tokens/s)
Accuracy (%)
Baseline (LLaDA)
6.7
79.3
dParallel (Threshold 0.5)
49.9
76.5
dParallel + Prefix Cache (Threshold 0.5)
92.0
77.1
24

---

Published as a conference paper at ICLR 2026
H MEMORY USAGE
While our biock-wise KV caching mechanism introduces changes in memory access patterns, does not significantly increase memory usage. As shown in the table below, both PrefixCache and DualCache reuse KV entries from a single full-sequence cache, split across blocks rather than duplicated. For instance, in the 256+256 case, DualCache reduces computation by over 16x (7.68T + 0.48T FLOPs), while memory only increases slightly (15.07G - 15.36G). Even at 1024+1024, VRAM remains well within 16.6GB, showing practicality for large-scale input. PrefixCache reuses only the prefix, while DualCache retains all but the current decoding block, providing more effective reuse at minimal extra memory cost. Overall, our method achieves substantial speedup with marginal VRAM overhead, and scales well to long sequences and large models.
Table 14: FLOP count (T) and peak VRAM usage (G) under different input and generation sequence lengths (256+256, 512+512, 1024+1024) for LLaDA baseline, PrefixCache, and DualCache. DualCache achieves over reduction computation (7.68T + 0.48T FLOPs) with only marginal memory increase (15.07G - 15.36G) the 256+256 case, demonstrating high computational efficiency and memory scalability.
256+256
512+512
1024+1024
Method
FLOPs (T)
VRAM (G)
FLOPs (T)
VRAM (G)
FLOPs (T)
VRAM (G)
LLaDA (Baseline)
7.68
15.07
15.36
15,20
30.71
15.44
PrefixCache
3.84
15.63
7.68
16.31
15.36
17.69
DualCache 0.48
15.36
0.48
15.76
0.48
16.57
I COMPARE WITH RELATED WORKS
While dLLM-Cache focuses on content-based adaptive KV caching using value similarity, our block- wise KV cache relies on temporal similarity across adjacent steps, offering complementary approach. DPad targets a different axis by pruning redundant suffix tokens to reduce attention cost, whereas our method optimizes the decoding process itself via caching and confidence-awee Notably, both methods are compatible with our approach, and can be combined for multiplicative speedups. acknowledged the DPad paper. We these works as complementary, addressing different bottlenecks in dLLM inference, and appreciate the opportunity to position Fast-dLL. within this evolving landscape.
parallel decoding.
Table Accuracy (%) and throughput (tokens/s) on GSM8K 5-shot for 256 and 512-token sequences. Fast- dLLMM denotes our full system (comfidence-aware parallel decoding + block-wise KV cache). dLL.-Cachhe adopts value-similarity-harr orthogonal and can be combined with others for multiplicative gains (e.g.. Paaudd-Paaalll-rrefiiico reaches 54.0 tokens/s 256 length with 77.6% accuracy).
adaptive caching, whereas DPad prumes redundant suffix tokens. Our approach
256 tokens
512 tokens
Method
Ace ( (%)
Thru (tok/s)
Ace (%)
Thru (tok/s)
dLLM-Cache
79.2
18.0
78.0
9.2
DPad
79.1
7.3
80.1
7.2
+ Parallel
80.5
20.8
81.0
21.2
+ Parallel + PrefixCache
77.6
54.0
79.2
53.8
Fast-dLLM (ours)
78.5
54.4
77.3
35.3
Cache
79.5
21.2
77.0
10.4
Parallel
79.2
16.5
77.6
18.6
25


Reviews:
[{"review_id": "k1UjdTqEOb", "reviewer": "Reviewer_SE2t", "summary": "This paper proposes Fast-dLLM, a training-free inference acceleration framework for diffusion-based large language models (dLLMs). Unlike autoregressive LLMs, diffusion LLMs cannot directly leverage KV caching or efficient parallel decoding due to their bidirectional denoising process. The authors introduce two key techniques to address this:\n\n* Block-wise Approximate KV Cache: exploits temporal similarity between diffusion steps to reuse attention states across steps, enabling cache reuse in bidirectional attention without retraining.\n\n* Confidence-Aware Parallel Decoding: decodes multiple tokens simultaneously when their predicted confidence exceeds a threshold, with theoretical justification showing approximation to sequential decoding under high-confidence conditions.\n\nExperiments on several diffusion LLMs (e.g., LLaDA, LLaDA-V, Dream) across text, code, and multimodal reasoning tasks demonstrate up to 27.6x speedup with negligible accuracy loss.", "questions": "How sensitive are the results to the cache window size and confidence threshold? Could a learning-based mechanism adaptively set these parameters?\n\nWhat is the additional memory footprint introduced by storing bidirectional caches compared to AR caching?\n\nWill the proposed caching mechanism be compatible with TensorRT deployment?\n\nCould the proposed methods extend to multimodal diffusion transformers (e.g., text-to-image or video diffusion models)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Timely and practically relevant: addresses the major efficiency bottleneck of diffusion-based LLMs, a direction gaining interest.\n\n2. Training-free approach: requires no retraining or model modification, making it directly applicable to existing dLLMs.\n\n3. Comprehensive evaluation: covers both text and multimodal reasoning tasks, with consistent gains across benchmarks.\n\n4. Strong empirical results: large acceleration factors (up to 27.6x) with small degradation make the method attractive for deployment.", "weaknesses": "1. Applicability to distilled or few-step diffusion LLMs. It remains unclear whether the proposed caching and confidence-aware decoding strategies would remain effective for distilled diffusion LLMs that operate with only a few or even a single denoising step (e.g., dParallel, arXiv:2509.26488; One-Step Diffusion LLM, OpenReview:P7OzWxOUHK). The reviewer acknowledge that these are concurrent works, while such aggressive timestep reduction is becoming a key trend, similar to continuous diffusion distillation in image/video models. Caching-based acceleration mainly benefits multi-step teacher models, but may offer limited or no gain for student variants without hurting the accuracy, restricting practical adoption.\n\n2. Memory overhead. The block-wise KV caching mechanism likely introduces memory costs, especially for long sequences or large models. The paper does not quantify the memory–speed trade-off or report actual VRAM usage, which is important for understanding deployment feasibility.\n\n3. Baseline comparisons. Recent concurrent and closely related works, such as dLLM-Cache (arXiv:2506.06295) and dPad (arXiv:2508.14148), are not compared or discussed. Even if concurrent, a conceptual comparison highlighting methodological similarities and distinctions would help position Fast-dLLM within this rapidly evolving research landscape.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y1GOiB4vfP", "reviewer": "Reviewer_eGsn", "summary": "This paper introduces Fast-dLLM, a training-free acceleration framework for diffusion-based LLMs that addresses their slow inference speed compared to AR models. The method incorporates two key innovations: (1) a block-wise approximate KV Cache mechanism tailored for bidirectional attention that enables cache reuse with negligible performance drop, and (2) a confidence-aware parallel decoding strategy that selectively generates multiple tokens simultaneously based on confidence thresholds to preserve generation quality. Experimental results on LLaDA and Dream models across multiple benchmarks demonstrate remarkable throughput improvement with little accuracy loss, effectively closing the performance gap with autoregressive models.", "questions": "1. Why does the throughput plateau when the block size continues to increase in Fig4 ?\n2. Is the KV activation similarity pattern largely consistent, or can it be different among layers?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel adaptation of KV Cache to bidirectional diffusion models via block-wise approximation, with insightful analysis showing high cosine similarity between adjacent steps.\n2. Theoretical foundation with Theorem 1 proving the equivalence between greedy parallel and sequential decoding under certain conditions.\n3. Comprehensive ablation studies covering key hyperparameters (block sizes, thresholds, generation lengths) and evaluation across models and benchmarks.", "weaknesses": "The evaluation of models relies on only four benchmarks (GSM8K, MATH, HumanEval, MBPP for LLaDA), which primarily focus on math reasoning and code generation, missing important capability dimensions like commonsense reasoning (HellaSwag), factual knowledge retrieval (TriviaQA), and real-world code generation (LiveCodeBench, BigCodeBench) that would provide a more comprehensive understanding of the method's generalization and potential failure modes across diverse task types.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jJbC8THbQE", "reviewer": "Reviewer_hYMS", "summary": "This paper proposes a novel training-free acceleration framework Fast-dLLM, which aims to address two key bottlenecks in Diffusion LLMs: the lack of KV caching and the quality degradation associated with parallel decoding. First, the paper introduces a block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models. This approach is justified by demonstrating the high similarity of KV activations across adjacent inference steps, which enables cache reuse with a negligible performance drop. Second, the paper posits that the quality degradation in parallel decoding stems from the conditional independence assumption, which disrupts critical token dependencies. To mitigate this, it proposes a confidence-aware parallel decoding strategy that selectively decodes only those tokens exceeding a confidence threshold. This approach is shown to reduce dependency violations while maintaining generation quality. The authors validate their method through comprehensive experiments on the LLaDA and Dream models across multiple LLM benchmarks, and results demonstrate that Fast-dLLM achieves significant throughput improvements (up to 27.6x) while incurring negligible degradation in generation quality.", "questions": "1.The paper's appendix C.4 demonstrates that the 'Factor' decoding strategy significantly outperforms the 'Threshold' strategy in terms of throughput, with only a minor accuracy trade-off. However, the main experimental results (e.g., Table 1) are reported using the slower 'Threshold' strategy. Could the authors clarify why the seemingly superior 'Factor' strategy was not used for the main results? Does the 'Factor' strategy have other un-discussed drawbacks that led to this decision?\n\n2.Could the authors comment on the robustness of the approximate KV cache strategy for very long sequence generation? Is there a risk that approximation errors will accumulate over many blocks, leading to a loss of attentional fidelity to the initial prompt?\n\n3.The qualitative comparisons in appendix B are based on a simple arithmetic prompt, could the authors provide more diverse qualitative examples (e.g., generating idioms or logical pairs) where the risk of dependency failure is more prominent?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.The paper introduces a novel, training-free framework that tackles two foundational challenges in dLLM inference.\n\n2.The proposed methods are well-justified and supported by solid theory and experiment phenomenon. The approximate KV cache is empirically validated by the high similarity of KV activations in adjacent steps. The parallel decoding strategy is theoretically supported by Theorem 1, which proves the equivalence of greedy parallel and sequential decoding under high-confidence conditions, motivating the threshold-based approach. Furthermore, the identification of these two phenomena is a valuable contribution, offering clear insights and new avenues for solving these challenges in dLLM inference.\n\n3.The paper provides extensive and sufficient experiments. Main results validate the method's effectiveness. A wide range of ablation studies on variables (e.g., cache block size, confidence thresholds, generation lengths) assess scalability and robustness, with clear analysis and interpretation of the results.\n\n4.This paper is writen well and clear. The structure is logical and easy to follow, making the core concepts and contributions readily understandable.", "weaknesses": "The experimental evaluation is primarily limited to comparisons against the baseline dLLM pipelines. The paper lacks an comparison to other existing acceleration techniques for Diffusion LLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "agHE0I5YL9", "reviewer": "Reviewer_MBqz", "summary": "This paper investigates the slow inference speed of diffusion-based large language models, stemming from the lack of Key-Value (KV) cache support and quality degradation in parallel decoding. The proposed Fast-dLLM method introduces a block-wise approximate KV cache tailored for bidirectional attention and a confidence-aware parallel decoding strategy. This strategy dynamically selects tokens above a confidence threshold, enabling efficient cache reuse and safe, training-free parallel token generation.", "questions": "1. The paper does not clarify how to automatically tune the commonly used 0.9 confidence threshold across different model scales, temperatures, or tasks, nor whether a simple calibration method (e.g., based on confidence–accuracy curves) can be applied. It also leaves open how to select the factor-based hyperparameter \\(f\\), and whether a universal default exists or task-specific tuning is required.\n\n2. The paper does not quantify the computational cost (FLOPs or wall-clock time) of recomputing all blocks after completing one, nor does it explore whether incremental updates—such as refreshing only neighboring blocks or subsets of Keys/Values—could achieve comparable accuracy more efficiently. Profiling or throughput measurements would help clarify these trade-offs.\n\n3. In the MathVista experiments, Fast-dLLM exhibited a noticeable performance degradation. Could you give a detailed analysis?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method is training-free, making it an easily applicable plug-in for compatible diffusion language models.\n2. It includes a comprehensive set of ablation studies analyzing the impact of key components and hyperparameters, such as cache block size, confidence threshold, cache variants, and generation/prefill length. \n3. The empirical results are significant, showing marked improvements in throughput across multiple benchmarks with minimal accuracy loss.", "weaknesses": "1. The mechanism of 'DualCache' is not clearly explained; it is unclear how caching suffix tokens saves computation and accelerates inference. Additionally, more details on the extra memory overhead introduced by the KV cache need to be disclosed.  In addition, the paper claims that the cache update introduces “negligible overhead” in Figure 2, but it provides no concrete explanation or timing comparison. \n\n2. The novelty of the block-wise KV cache contribution is unclear. The manuscript fails to sufficiently differentiate its caching method from existing work like Block-diffusion, and the experimental section lacks relevant diffusion model baselines. \n\n3. The reported speedup ratios raise concerns about potential metric inflation. Furthermore, the datasets used are primarily focused on math and code problems, lacking experimental data on benchmarks specifically designed for evaluating inference acceleration.\n\n4. The conceptual relationship and practical distinction between threshold- and factor-based strategies are insufficiently clarified. It is unclear whether the factor-based rule is a relaxed, adaptive, or independent variant, or under which conditions one should be preferred. A clearer rationale linking these strategies would improve comprehension and applicability.\n\n5. While the paper qualitatively illustrates KV similarity, it does not provide quantitative measures showing how approximation errors change with block size, decoding length, or modality, making it difficult to assess cache mismatch risks. It would be helpful to report KV similarity decay curves across decoding steps for different block sizes and tasks, correlate these similarities with downstream task metrics (e.g., accuracy, EM, BLEU) to establish a “similarity threshold → refresh” rule.\n\n6. The assumption of a “well-defined joint PMF with self-consistent marginals” may not hold for real diffusion LLMs trained approximately. The implications of this idealization are not discussed in detail, limiting interpretability of theoretical guarantees.\n\n7. The paper does not include a comparison with autoregressive (AR) models. As a result, it remains unclear whether the diffusion + Fast-dLLM approach is competitive with state-of-the-art AR systems in realistic serving scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)