d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Authors: Yuchu Jiang
Yue Cai
Xiangzhong Luo
Jiale Fu
Jiarui Wang
Chonghan Liu
Xu Yang
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion-based large language models
Key-value caching
Inference acceleration
Primary area: foundation or frontier models, including LLMs

Abstract: Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The anonymous evaluation codes are available at \url{https://anonymous.4open.science/r/d2Cache-5538}.

Published as a conference paper at ICLR 2026
D'CACHE: ACCELERATING DIFFUSION-BASER LLMs VIA DUAL ADAPTIVE CACHING
Yuchu Jiang
Yue Caia.
Xiangzhong Luo 2.2x Jiale Fu
Jiarui Wang
Chonghan Liu
Xu Yangl.3
KKy Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary
Applications (Southeast University), Ministry of Education "Qiyuan Tech
Southeast University
kamichan seu edu.en, xiangzhong 1wo@seu. edu.e en
ABSTRACT
Diffusion-basee large language models (dLLMs), despite their promising perfor- mance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we in- troduce Dual aDaptive Cache (daCache). which training-free approximate KV cache framework for accelerating dLLM inference. d'Cache features two- stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, Cache naturally offers a more reliable decod- ing alternative, which enable quasi left-to-right generation and mitigate pre- mature overconfidence in tokens at the end of the sequence. Extensive experi- mental results on representative dLLMMs (i.e., LLaDA and Dream) demon- strate that d'Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code available https://qit wom/Koaniii www.aaaaiihhiaaaaa chanw chanw/d2Cache d2Cache
INTRODUCTION
Diffusion models have recently achieved remarkable success in generating continuous data like im- ages (Yang et al., 2023), but text generation-a fundamentally discrete task-has long been dom- inated by autoregressive models (ARMs) (Touvron et al.) 2023 Achiam et al.) 2023 Guo et al.]
2025). Building on the foundations of ARMs, recent studies have successfully extended diffusion processes to discrete language modeling and further scaled up these models (Nic et al.
2025 Ye
al., 2025 Li al. 2025c). These diffusion-
large language models (aLLMs) offer sev-
eral key advantages over ARMs, such as mitigating the "reversal curse" (Berglund et al.
2023 and
capturing high-level global semantic patterns (Nagarajan al., 2025).
Despite their potential, recent dLLMS still face substantial efficiency challenges (Wu et al. 2025
Due to bidirectional attention, dLLMMs cannot benefit from the standard key-value (KV) cache as ARMs do. As shown in Figure [ (a), ARMs leverage causal attention to sequentially generate new tokens and append each new token to the end the sequence. This autoregressive process naturally enables the reuse of earlier KV states when generating the next token (Li et al. 2024). In contrast, as shown in F dLLMs feature an iterative decoding process over a fixed-length sequence, where masked tokens progressively replaced with decoded tokens. However, under bidirectional attention, updating even a single masked token changes the context seen by all other tokens (Ye al. 2025 Nie et al.) 2025). As result, the KV states of the entire sequence must be recomputed at each decoding step, making dLLMMs inherently incompatible with the standard KV cache.
To address the above efficiency challenges, recent studies (Ma al.) 2025 Wu et al. 2025 Liu
et al. 22025: Hu et al.) 2025) have explored approximate KV cache to accelerate dLLM inference. These studies build on the following key observation: for a subset of tokens, their KV states often
"Corresponding Author

---

Published as a conference paper at ICLR 2026
Masked loken
New atlention interaction
Step t
Decoded loken
Reuseble attention interacion
Cactred of 10 takens
(Skipping S/10 computations)
Stage 1: Certainty prior
prior-guided guided selection from masked tokens (colored by cartainty magnituda)
Cached token
Selected taken updale
Stage 2: Attention-awe -awellen--wweeeeeee selection from remaining tokens (colored by attontion magnitude)
Step t
Step t +
Step
(a) Causal attention
(b) Bidirectional attention
(c) deCchee: Two-stage fine-grained token selection
Figure 1: (a) In ARMs, causal attention requires each token to interact only with its preceding tokens. (b) In dLLMs, bidirectional attention requires each token to attend to both its preceding and subsequent tokens, such that any modification in the subsequent tokens necessitates recomputation of the entire sequence. (c) The proposed d adaptively selects small subset of tokens in dLLMs and updates their KV states through a two-stage fine-grained process. The KV states of the remaining tokens can be approximately cached for reuse in subsequent decoding step.
exhibit high similarity across consecutive decoding steps. This enables to approximately reuse these KV states, which can avoid redundant computations and reduce the overall inference cost. In prac- tice, these studies typically divide the sequence into a static segment, where their KV states can be approximately reused, and a dynamic segment, where their KV states need to be frequently updated within a fixed window of decoding steps. However, these studies are coarse- coarse-grained and apply the same strategy to all tokens within both static and dynamic segments. As result, they either suffer from limited flexibility (Wu et al. (025) or require complicated tuning (Liu et al.] (025). Moreover, since coarse- coarse-grained designs cannot capture the fine-grained token-level dynamics of KV states, they inevitably reuse KV states that should be updated, or update KV states that can be safely reused, thus limiting the achievable acceleration gains.
To address these limitations, we seek to develop an effective fine ane-ggrined approximate KV cache strategy, which can adaptively select tokens and update their KV states at each decoding step rather than within a fixed decoding window. To this end, we first perform fine-grained analysis to in- vestigate the KV state dynamics d...Ms. Our results show that, for masked tokens, their KV states evolve through three phases: (1) a gradual-change phase during the early decoding steps, (2) a rapid-change phase in the few steps immediately preceding their decoding, and (3) a stable phase after being decoded. Notably, we find that it is sufficient to update the KV states of masked tokens only during the rapid-change phase.
Nonetheless, unlike masked tokens, prompt and decoded tokens exhibit substantially smaller KV state dynamics across consecutive decoding steps. This makes the above phase-based caching strat- egy less effective and necessitates another caching alternative for prompt and decoded tokens. In- spired by prior KV cache research in ARMs (Feng et al., 2024) Cai et al.) (0024), which reveals that attention unevenly distributed and concentrated on a small subset of bokens-thus allowing to prune the KV states of less important ones-we investigate whether dLLMs exhibit the same attention behavior. Our results confirm that attention in dLLMs is likewise concentrated on a small subset of tokens, especially prompt and decoded tokens. Therefore, similar to KV cache pruning, we can adaptively update the KV states of tokens that receive consistently higher attention, whereas the KV states of the remaining tokens can be safely cached for reuse in subsequent decoding step.
Motivated by the above observations, we propose Dual aDaptive Cache (daCache), a training-free approximate KV cache framework for accelerating dLLMM inference, as shown in Figure | (c). Specifically, C'cch features a two-stage fine-grained selection strategy that identifies tokens and adaptively updates their KV states at each decoding step, while the KV states of the remaining tokens can be cached and reused. In the meantime, d' Cache also naturally delivers a more reliable decod- ing option, which seamlessly enables quasi left-to-right generation and thus mitigates premature overconfidence in the tokens at the end of the sequence. Extensive experiments on representative dLLMS (Le., LLaDA (Nee et al. 2025) and Dream Yee al. 2025)) demonstrate that d not
2

---

Published as a conference paper at ICLR 2026
only achieves substantial inference speedups, but also yields consistent improvements in generation quality. Finally, we summarize our main contributions as follows:
We present a fine-grained analysis on the KV state dynamics in dLLMs, which explicitly reveals a three-phase decoding pattern and uneven attention distribution.
Building on the above findings, we propose a training-free approximate KV cache framework, namely Cachhe to accelerate dLLM inference. d'Cache features a two-stage fine-grained selec- tion strategy to identify tokens and adaptively update their KV states at each decoding step, while the KV states of the remaining tokens can be cached for reuse in subsequent decoding step.
Extensive experiments demonstrate that d² Cache can achieve substantial inference speedups while consistently improving generation quality across various dLLMs and datasets.
RELATED WORK
Diffusion-based large language models (dLLMs). Building on the success of diffusion models
in continuous domains, such image and video generation (Yang et al.] 2023 Ho al. 2022)
recent studies have extended diffusion models to discrete language tasks (Sahoo et al. 2024; Shi
al. 2024, Nie et al.) 2024; Arriola et al. (025). Unlike autoregressive models (ARMs) that
generate tokens sequentially (Touvron et al. |023] Achiam et al. 2023 Guo et al. 2025). dLLMs
feature an iterative denoising process over masked sequences, which can enable bidirectional context modeling and inherently support parallel decoding (Li et al., 2025c b, Xu et al. 2025). More
recently, large-scale dI.LMs, such as LLaDA (Nie et al. (025) and Dream (Ye et al.) (025)) have
demonstrated competitive performance on reasoning and folllwiion tasks, establishing themselves as a promising alternative to ARMs. Despite their promising performance, their reliance on bidirectional attention necessitates substantial inference overheads, which significantly hinder their practical deployments.
Approximate KV cache for dLLMs. Due to bidirectional attention, dLLMs cannot directly ben- efit from the standard KV cache (Li et al. (025c) as ARMs do. To address this limitation, re- cent studies have observed that the KV states in dLLMs remain highly similar across consecutive decoding steps. Building on this observation, several approximate KV caching techniques have recently emerged (Liu et al. 2025 Ma et al., 2025 Wu et al.) 2025 Hu al., 2025). Among them, dLLM-Cache (Liu et al. 2025) partitions the input sequence into two segments prompt and response- and updates their KV states different frequencies. dKV-Cache (Ma et al.) 2025) intro-
duces a one-step delayed KV caching scheme, in which decoded tokens are stored not at the current decoding step but at the subsequent decoding step. Fast-dI.LM (Wu et al., (025) features block-witg semi-aute semi-autoregressin
decoding and caches all KV states except those in the current decoding block. However, due to the coarse-grained nature, these methods inevitably reuse KV states that should be actively updated or update KV states that can be safely reused, which thus suffer from inferior acceleration gains. A comprehensive comparison between our dCCache and two concurrent similar works (i.e., dLLM-Cache and Fast-dLLM) is provided in Section B of the Appendix.
3 PRELIMINARIES
3.1 GENERATION PROCESS OF DLLMS
As shown in Nie et al. 2025) dLLMs feature an iterative denoising paradigm to generate text over T' discrete decoding steps, where fully masked initial sequence is progressively transformed into fully unmasked final output. Formally, let V denote the token vocabulary, which includes a special masked token [MASK]. The inference process of dLLMs begins with an initial sequence yo of length L, which is simply constructed by concatenating a prompt segment with a response segment ro that consists of a masked tokens. We denote the set of indices corresponding to these masked tokens as Mo {pl pl L..... [pl + 1}.
At each decoding step t E ,O...., T - 1]. the corresponding sequence ye is first fed into the given dLLM as input, which produces probability distribution p(æ: yr) over the vocabulary for each masked position of. Based this distribution, the most confident token predictions X, and their
3

---

Published as a conference paper at ICLR 2026
180
10k
Second Principal Component
100
90% of takens
50
Step
Decoded
Step 98
Frequency
Cumulative Porcentage
Step 255
-50
50
-100
Step
Gradual Change Change
10
20
30
40
50
60
90
100
0.2
Stable
-180
0
0.0
250
-000
-150
-100
-50
50
100
150
0.10
100
150
200
250
First Principal Component
Sequence Distance
(b)
Figure 2: (a) PCA visualization of 77th masked token's trajectory on LLaDA-8B-Inss Instructt with GSM8K (I=328, n=256, and T=256). (b) Sequential distances between token pairs decoded in adjacent steps.
associated confidence scores S₁ can be derived as follows:
X, = { if arg max p(a a vr): i M.).
=EV S, = { s/
(1)
v() i M,).
where F-) is function that measures the token-level prediction confidence score.
Furthermore, the decoding process employs scheduling function to generate set of indices 4, which specifies the masked positions in y/ to be replaced with their predicted tokens:
= G(t, St, Br), where vi+1 { a if E Tt.
otherwise.
In practice, the scheduling is typically performed either by randomly sampling subset of M, or by choosing those masked positions with the highest confidence scores (Nie et al. 2025). Subsequently, the masked index set for the next decoding step is updated Mc+1 M, Le. After T iterations, when the condition Mr- 0 holds, the whole generation process is stopped and we get the final sequence yr with no remaining masked tokens.
3.2 KV STATE DYNAMICS AND DECODING ORDER IN DLLMS
Recent studies approximate KV cache in dLLMs have shown that the KV states of certain tokens exhibit high similarity across adjacent decoding steps (Wu et al. 2025 Liu et al. 2025). Leveraging this redundancy, they first partition the entire sequence into static segment and dynamic segment, after which they cache the KV states tokens in the static segment for reuse. Despite its efficacy, this segment-level partitioning scheme is coarse-grained and totally ignores the fine-grained token- level dynamics. To bridge this gap, we begin with masked tokens and perform experiments on LLaDA-8B-Inste with GSM8K explore how their KV states evolve during generation.
KV state dynamics in dLLMs. To analyze the dynamics of KV states for masked tokens, we employ principal component analysis (PCA) project their layer-averaged key states into two di- mensions and visualize their trajectories across decoding steps. As shown in Figure 2 (a), the KV states of masked tokens evolve through three phases: (1) gradual-change phase during the early decoding steps (i.e., steps 0-64), (2) rapid-change phase in the few steps immediately preceding their decoding (i.e., steps 64-986 64-98), and (3) stable phase after being decoded (i.e., steps 98-255). We find that is sufficient to update the KV states of masked tokens only during the rapid-change phase, whereas the KV states of masked tokens from the other two phases can be safely cached for reuse. More importantly, this does not degrade the final generation quality, as shown in Figure
Decoding order in dLLMs. Building on the above findings, a natural question arises: how can we determine whether masked token is about be decoded before its actual decoding essentillly "chicken-an and-agg aedd problem? To shed light on this, we randomly sample 64 examples from GSM8K, in which analyze the sequential distance between token pairs decoded in adjacent steps. As
4

---

Similarity of attention allocations in adjacent steps. Building on the above findings, we further calculate the sum of absolute differences rollout values across all pairs of decoding steps. As shown in Figure (b), the attention allocations across adjacent decoding steps highly similar. This suggests that the attention allocation of the current decoding step can be used to approximate Published as a conference paper at ICLR 2026
Summed Rollout
Nigh
Promot Diecaded
40
Decoding Step
80
low sitention
Adjecent sleps
Query Position
similar
rollout values
100
60
80
240
300
256
150 200
40
80
120
160 200
240
Key Position
Decoding Step
(b)
Figure 3: Attention rollout analysis over sequence, where the example and setting are the same as Figure 2 (a) Attention rollout visualization at step 126, showing the sum of rollout values over all key positions (top) and the pairwise rollout values across different positions (bottom). (b) The total absolute differences rollout values between each two adjacent decoding steps.
tends to decode the next masked token from positions close the recently decoded token, with 90% of tokens falling within distance of 10. This reveals an interesting decoding pattern: dLLMs tend to decode masked tokens located near previ- ously decoded tokens. Therefore, we can estimate whether masked token is about be decoded according to the density of decoded tokens in its local context.
shown in Figure 2 (b),
3.3 ATTENTION DISTRIBUTIONS IN DLLMS
Prior research on ARMs has observed that attention is not uniformly distributed but instead con- centrated on small subset of salient tokens (Xiao et al.) 2023). This observation has served as the foundation for various optimization techniques, which apply differentiated strategies to tokens based their importance (Feng et al. 22224] Cai et al. 2024 Zhu al. 2024). This naturally raises the following question: can the above observation from ARMs generalize to dLLMs? answer this question, conduct experiments I.LDDA-8B. -B--t-trt with GSM8K to analyze the attention distribution.
Attention salience among tokens. Inspired by prior attention studies ARMs, employ tention rollout (Abnar & Zuidemal 2020) to visualize how attention propagates across tokens. The attention rollout algorithm aggregates cumulative attention by recursively multiplying the attention matrices across layers, yielding a global attribution map that highlights how information propagates from input tokens the final output. More details about the attention rollout algorithm are provided in Section [.22 As shown in Figure 3 (a, bottom), queries consistently attend to a small subset of key positions prompt and decoded tokens, revealing that these tokens dominate the attention dis- tribution compared to other tokens. As shown in Figure 3 top), masked tokens receive negligible attention, which is substantially lower than that allocated to both prompt and decoded tokens.
that of the next decoding step. In light of this, analogous to KV cache optimization techniques in ARMs, KV state updates can thus be restricted to tokens that receive higher attention.
DºCACHE: DUAL ADAPTIVE CACHE
Motivated by the observations in Section 3, we present Dual aDaptive Cache (dPCache), a training- free approximate KV cache framework for accelerating dI..M inference. To exploit approximate
5

---

Published as a conference paper at ICLR 2026
KV cache in dLLMs, Cache seeks to adaptively identify tokens whose KV states should be ac- tively updated at each decoding step. while caching the remaining tokens for reuse in a subsequent decoding step.
Overview of Cache. We first group tokens in dLLMs into three categories: prompt tokens, introduce two-stage fine- grained token selection strategy. 1 Certainty prior-guided selection from masked tokens. After each forward pass, d=Cache assigns each masked token a score combining prediction confidence and a certainty prior, which represents the density of known tokens (i.e., prompt or decoded ones) in the local context. d'Cache then adaptively selects a subset of masked tokens with higher scores. In light of this, d'Cache naturally enables an alternative decoding scheme guided by the certainty prior rather than prediction confidence alone, yielding higher reliability (see Table 2 @ Attention- aware selection from remaining tokens. Furthermore, for the remaining tokens (especially prompt and decoded tokens), Cacac adaptively selects a subset of tokens with higher attention activations. Finally, for the tokens selected in these two stages, d' updates their KV states at each decod- ing step, while caching the KV states of the remaining tokens for reuse subsequent decoding step. An intuitive example of this two-stage token selection is provided in Figure (c).
masked tokens, and decoded tokens. Based on this categorization,
4.1 STAGE 1: CERTAINTY PRIOR-GUIDED SELECTION
As shown in Figure 2 (b), the decoding order in dLLMs is highly localized: 90% of subsequent tokens are decoded within a distance of 10 from the most recently decoded token. Building on this finding, we introduce a certainty prior, defined as the density of known tokens (i.e., prompt or decoded tokens) in the local context. In practice, the certainty prior can capture structural certainty, where higher value indicates that the masked token is more likely to be decoded sooner.
Formally, at each decoding step t E [0, [o...., - 1), the sequence Y/ fed into the given dILMM to generate predictions X, for the masked tokens 24, together with their corresponding confidence scores S With the above in mind, natural definition of certainty density is the proportion of known tokens (i.e., prompt or decoded tokens) within a fixed local window. However, this definition ignores the effect of relative distance among tokens: intuitively, a known token that is closer masked token x" should impose stronger constraints on x' than another known token that is farther away. To capture this intuition, we introduce the following position-aware certainty density:
-(1== =0 & - ( s.d. & (li - ji) = =xp (- 1 -
D(i) =
(3)
20
where denotes the position of the masked token x² and j denotes the position of each known token in the sequence. In practice, the Gaussian function e(-) assigns larger weights to known tokens that are closer to x' and smoothly diminishes the impact of distant ones, making (--) distance-aware aggregation of certainty from all known tokens. The effect of weighting is further controlled by the hyperparameter a. which denotes the standard deviation of the Gaussian function o(.). A larger broadens the positional scope considered by D1),, thereby causing the certainty density of different of to converge. Finally, we incorporate D(-) into S to measure the certainty prior and select the masked tokens with the top-k calibrated scores, with their indices forming the candidate set M*.
M arg topk D(i)- st.
PEM
This formulation ensures that token selection considers both prediction performance and certainty density, which thus can provide principled foundation for more reliable token selection.
& Certainty prior-guided decoding. The above certainty prior delivers a novel decoding altemna- tive: masked tokens can be decoded according to their certainty prior rather than their prediction confidence. We demonstrate that the certainty prior-guided decoding can achieve more reliable decoding performance than the default confidence-basse tuition here that the certainty prior-guided decoding can preserve a quasi left-to-right decoding order, since masked tokens located closer to known tokens exhibit higher structural and predictive certainty. This quasi left-to-right decoding order effectively mitigates the issue of premature over- confidence in sequence termination during the early decoding steps (Huang et al. 2025).
decoding, as shown in Table 2 The in-
For the simplicity of notation, we omit the subscript for the current step in the remainder of this paper.

---

Published as a conference paper at ICLR 2026
4.2 STAGE 2: ATTENTION- AWARE SELECTION
In Section 4.1 we present certainty prior-guided selection, which explores masked tokens whose KV states should be updated each decoding step. In this section, extend the selection process the remaining tokens. Notably, we observe that attention rollout (Abnar & Zuidema, 2020 widely used attention analysis technique in ARMs-can effectively generalize to dLLMs, particularly for
analyzing prompt and decoded tokens, making it well suited for our subsequent token selection.
As described in Abnar& & Zuidemal 2020), the attention rollout algorithm aggregates cumulative at- tention by recursively multiplying the attention matrices across layers, yielding global distribution map that reveals how information propagates from input tokens to the final output. Formally, let U denote the indices of the remaining tokens. At the decoding step + 1, the input of the given dLLM no longer the full sequence +,, but instead a subset of it:
Vi+1 = (vt+1 E M UU}.
(5)
To further derive U, at each decoding step t, we first collect the attention scores A "x from each layer 7 E {I....,A N... where H/ and N denote the number of attention heads and layers. We then average the resulting attention scores across all heads to obtain A and expand A into full-sized attention matrix F() E RLXL as follows:
( 2,
A e
if MTEM otherwise,
(6)
where e, is the one-hot vector with value of at position i. Following Abnar Zuidema (2020). further define the per-layer transition matrix ( by combining the expanded attention matrix E with the residual connection (i.e., an identity matrix I) and applying row-wise normalization:
( normalize sum 1o-1 sam--to (F + T).
The cumulative attention rollout matrix C is then iteratively computed, starting with ((O)
I:
C (( ((I-1),
(8)
The final rollout matrix ( captures the end-to-end influence between all token pairs. To quantify the overall contribution of each token, we further derive influence score cy for each token by summing the columns of ((N) follows:
9- Σ J
Finally, we sort tokens according to their influence scores and directly select the indices of the smallest set whose cumulative probability exceeds the predefined threshold p, thus forming U.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Models, datasets, metrics and hardware. Following recent conventions (Wu et al. 2025). we eval-
uate deCache on the Base and Instruct variants of two representative dLLMs (i.e., LLaDA-SB (Nie et al. 2025) and Dream-vol vB (Ye et al. (025)). which are denoted as -.aDD--Base/lng and Dream- Base/Inst. Following BLLM-Cachi (Liu al. 2025). we evaluate d'Cache on six benchmarks,
cluding GSM8K (Cobbe et al. 2021). MBPP (Austin et al. 2021), HumanEval (Chen et al. 202 Math-500 (Lightman et al. 2023), GPQA (Rein et al. 2024), and MMI.U-Pro (Wang et al. 2024 to assess performance across diverse reasoning, code generation and general tasks. The performance is reported in terms of task accuracy, which is evaluated using the 1m-eval- -haalaaaaaessse framework (Gao report both inference throughput and latency, where through-
al. 2024). For fair comparisons, put denotes the average number of tokens generated per second and latency denotes the average inference time per sample. All experiments are performed NVIDIA 3090 24GB GPUs.
Baselines. We consider three baselines, including Vanilla and two representative approximate KV cache methods (i.e., dl M-Cchhe and Fast-dI.LMM. For Vanilla, each decoding step, the masked
7

---

Published as a conference paper at ICLR 2026
Table I: Comprehensive evaluation results on LLaDA-Inst (Nie al. 2025) and Dream-Inst Ye
et al. 2025). Bold numbers indicate the best results and green texts denote the speedup ratios.
LLaDA-Inst
Dream-Inst
Dataset
Method
Throughput 1
Latency(s) 1
Score
Throughput 1
Latency(s) 1 Score
Vanilla
2.77 (1.0x)
110.26
77.6
2.62 (1.0k)
85.94
76.7
GSM8K
ddLLM-Cache
8.29 (3.0x)
30.34
76.8
7.50 (2,9x)
33.75
74.6
4-shor
Fast-dLLLM
9.64 (3.5x)
26.15
77.0
10.12 (3.9x)
24.88
77.0
Gen. Len. 256
Caach
(4.1x)
22.41
79.2
12.25 (4.7x)
21.36
78.2
Vanilla
2.48 (1.0x)
199.99
38.0
2,73 (1.0x)
182,78
52.0
MBPP
dLLM-Cache
6.97 (2.8x)
71.79
7.07 (2.6x)
71.13
3-shor
Fast-dLLM
(2.7x)
73.27
7.29 (2.7x)
69.47
$2.0
Gen. Len. = 512
d'Cache
11.42 (4.6x)
43.86
39.4
12.47 (4.6x)
40.32
58.0
Eval
Vanilla
4,99 (1.0x)
105.76
45.1
4.39 (1.0x)
114.86
56.7
dLLM-Cache
8.67 (1.7x)
57.48
44.5
5.35 (1.2x)
94.33
56.5
O-shor
Fast-dLLM
7.90 (1.6x)
63.12
43.9
7.89 (1.8x)
63.84
56.1
Gen. Len. 512
d
14.00 (2.8x)
35.44
48.2
14.06 (3.2x)
36.61
61.6
Vanilla
3.08 (1.0x)
82.51
38.4
3.51 (1.0x)
71.05
45.2
Math-500
dLLM-Cache
6.71 (2.2x)
37.84
38.2
7.19 (2.0k)
35.36
44.2
4-shor
Fast-dLLM
10.61 (3.4x)
23.79
38.0
10.72 (3,1x)
23.52
44,4
Cache
12.02 (3.9x)
20.19
38.0
13,80 (3,9x)
18,80
44,6
Vanilla
6.14 (1.0x)
43.34
25.2
6.43 (1.0))
41.14
30.1
GPQA
dLLM-Cache
11.51 (1.9x)
22.33
27.2
10.91 (1.7x)
23.62
31.0
Fast-dLLM
12.41 (2.0x)
20.66
25.7
11.75 (1.8x)
21.79
34.6
256
d'Cache
15.04 (2.4x)
17.08
28.4
14,65 (2.3x)
17.52
31.5
Vanilla
1.76 (1 (1.0x)
152.62
37.5
2,15 (1.0k)
126.31
47.9
MMLU-Pro
dLLM-Cache
6.79 (3.9x)
38.29
38.1
7,82 (3.6x)
34.09
46.5
5-shor
Fast-dLLM
8.91 (5.
29.00
37.1
9.74 (4.5x)
27.69
45.9
Gen. Len, 256
d'Cache
(5.4x)
27.60
33.1
10.12 (4.7x)
25.77
46.8
Vanilla
3.54 (1.0x)
115.73
43.6
3.64 (1.0x)
103.68
51.4
dLLLM-Ccche
8.16 (2.3x)
43.01
43.8
7.64 (2.1x)
48.71
50.9
AVG
Fast-dLLM
(2.7x)
39.33
43.4
9,59 (2.6x)
38.53
51.7
Caache
12.24 (3.5x)
27.76
12.89 (3.5x)
26.73
53.4
Table 2: Comparisons of different decoding schemes under the default NAR setting, where Con denotes the confidence-based decoding and CP denotes our certainty prior-guided decoding.
LLaDA-Inst
Dream-Inst
Method
GSMSK
MBPP
HumanEval
Math-500
AVG
GSMSK
MBPP HumanEval Math-500
AVG
(Vanilla)
76.7
56.7
45.2
NAR w/ Conf
$7.5
18.2
42.1
26.4
36.0
51.6
34.2
26.8
3.2
29.0
Only 22
44.5
43.6
Scmi-AR w/ C'cch
76.9
40.2
46.6
38.6
50.6
76.0
53.8
56.7
42.0
57.1
NAR w/ &ache
79.2
39.4
48.2
38.0
51.2
78.2
58.0
61.6
44.6
60.6
position with the highest confidence is replaced with its predicted token. For dLL.-Caach and Fast- dLLM, we employ their default configurations as reported in Liu et al. (2025): ; Wu et al. (2025). For Instruct variants, all baselines adopt block- wise cemi-autoregressivve decoding (semi-AR) with block size of 32, whereas the Base variants are evaluated in fully non-autogressive (NAR) manner. More details are provided Section of the Appendix.
Implementation details. Unless otherwise specified, the standard deviation of the Gaussian func- tion set to 10.0. the number masked tokens selected per step fixed 32, the cumulative probability threshold p is set to 0. and the decoding is performed under the certainty prior.
5.2 MAIN RESULTS
The evaluation results on LLaDA-Inst and Dream-Inst are summarized in Table [ Notably, we ob- serve that Cacch achieves the best overall performance on average across all benchmarks, which delivers the highest throughput, the lowest latency, and the best score, consistently outperforming all baselines. Across all models and datasets, our d'Cache obtains an average 3.5x speedup over Vanilla. Taking Dream-Inst GSM8K example, our Cache improves the inference through-
8

---

Published as a conference paper at ICLR 2026
0
0= 10
0 40
18
40
120
Phases
80
Phase Only
Phase 2 Only
16
120
100
14
160
Decoding Step
200
12
Throughput
240
Score
10
80
NAR
60
40
120
20
2
240
80
120
160
200
240
GSMBK
Math 500
HumanEval
MBPP
Token Position
Figure 5: Comparisons of different update strate-
Figure 4: Visualization of the decoding order us- ing certainty prior with different and NAR de- rapid-change phase (Phase 2 Only) and updating coding. Each dot at (i, t) indicates that the token tokens during both the gradual-change and rapid- at position is decoded at step t.
gies, including updating tokens only during the
change phases (Phases & 2).
put from 2.62 to 12.25 tokens per second, leading to 4.7x inference speedup. More importantly, these substantial inference speedups are achieved without sacrificing accuracy. as the attainable score on average across six datasets remains comparable to better than Vanilla. Furthermore, compared to dl.hhM and Fast-dLLLM, our d'Cache can also deliver hetter performance in terms both inference efficiency and accuracy. For example, compared to Fast-dLLM, our d'Cache yields 1.3x inference speedup on Dream-Inst, while maintaining +1.7% accuracy on average across six datasets. These results clearly demonstrate the efficacy of d'Cache, which benefits from its two-stage fine- grained selection strategy.
5.3 ABLATIONS AND ANALYSIS
Certainty prior-guided decoding vs. confidence-based decoding. As discussed Section 4.1 C'Cache naturally delivers an alternative decoding scheme: masked tokens can be decoded accord- ing their certainty prior rather than their prediction confidence. evaluate efficacy, we further compare our certainty prior-guided decoding with the standard confidence-basee decoding under the default NAR setting. As shown in Table 2 our certainty prior-guided decoding delivers more reliable performance than the confidence-basee decoding under the default NAR setting. We also ob- serve that certainty prior-guite guidedd decoding and semi-AR decoding achieve comparable performance, because both approaches constrain the model to decode in quasi left-to-right manner. Although they share a similar intuition, only the combination of certainty prior-guided decoding and d"Cache delivers the best performance among all evaluated configurations.
Effect of a on decoding order. We visualize the decoding step for each masked position using LLaDA-Inst on 64 randomly sampled examples from GSM8K. As shown in Figure 4 we compare NAR decoding with our certainty prior-guided decoding, where the hyperparameter a (see Equa- tion (3) set to 10, 40, and 80. We find that NAR decoding exhibits a distinctive "U-shaped" trajectory: tokens at both sequence boundaries are first generated, which then converge towards the center (Huang et al. 2025). At the first glance, this behavior seems inconsistent with our ear- lier observation that dL. tend to prioritize decoding masked tokens adjacent to known tokens (i.e., prompt or decoding tokens). This discrepancy, however, stems from the supervised fine-tuning (SFT) of LLaDA-Inst where the excessive number of [ECS] tokens in the training data biases the model towards producing an unnatural number of [EOS] tokens during inference (Nie et al. 2025). In contrast, certainty prior-guided decoding yields generation order, where smaller a makes the generation closer to autoregressive decoding.
natural and controllable left-to-right
Computational redundancy during the gradual-change phase. As discussed in Section 3.2 the KV states of masked tokens evolve through three phases: gradual-change. rapid-change and stable. It is thus natural to update the KV states of masked tokens during both the gradual-change and rapid-

---

Published as a conference paper at ICLR 2026
256
Gen. Len.
66-
pass@1
60-
-9-
58
2-
54-
2
pw24
Throughput
Throughput
(a)
(b)
Gen, Len. 1024
34
60-
pass@ 1
54-
51.
Les 512
0-64
Gen Len 3324
10
Throughput (c)
[d)
Figure 6: Hyperparameter sensitivity analysis of p. k, and a on Dream-Inst and HumanEval.
change phases, while caching them for reuse during the stable phase. However, our analysis shows that it is sufficient update the KV states of masked tokens only during the rapid-change phase. To shed light on this, we conduct an ablation on Dream-Inst, in which we compare the full-update strategy (updating tokens during both the gradual-change and rapid-change phases) with our default selective-update strategy (updating tokens only during the rapid-change phase). As shown in Fig- 33 our default selective-uddate strategy (i.e., Phase Only) delivers higher inference throughput than the full-update strategy (i.e., Phases & 2), while maintaining a comparable or even better score. This finding reveals a counterintuitive property of dLLMs: increased computation does not necessarily translute into improved performance. Instead, selectively updating only the most critical tokens can reduce computational redundancy and, in some cases, even yield better performance.
Hyperparameter sensitivity analysis. To determine the optimal hyperparameters, we conduct sys- tematic experiments on Dream-Inst and HumanEval with generation lengths of 256, 512 and 1024. As shown in Figure 6 (a-c), the number of masked tokens updated per step is the dominant factor: performance improves as & increases but saturates- and may slightly indicating that & 32 offers the most stable gains across settings of p and sequence lengths. The cumulative probability threshold p. which regulates the retained probability mass and thus affects throughput, does not monotonically improve performance with larger values. We additionally exam- ine the Gaussian standard deviation a in Equation (3), which governs the locality of certainty-prior selection.
k 32,
CONCLUSION
In this paper, propose Dual aDaptive Cache (dPCache), training-free approximate KV cache framework for accelerating dLLM inference. Through a fine-grained analysis of KV state dynamics, we uncover two key insights behind dLLMs: (1) the KV states of masked tokens exhibit substantial changes only in the few steps immediately preceding their decoding, indicating that their KV states can be reused beyond this phase; and (2) attention distributions are highly skewed towards a small subset of prompt and decoded tokens, indicating that the KV states of low-attention tokens can be reused. Building on these insights, d'Cache introduces a two-stage fine-grained selection strategy that adaptively identifies tokens and updates their KV states at each decoding step, whereas the KV states of the remaining tokens can be safely cached for reuse in subsequent decoding step, thus substantially reducing redundant computations and improving inference efficiency. Extensive experiments on representative dLLMs (i.e., LLaDA and Dream) demonstrate that d' Cache achieves substantial inference speedups, while also yielding consistent improvements in generation quality.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work is supported by Jiangsu Province Carbon Peak Carbon Neutrality Science and Technology Innovation Special Fund Project (Grant No. BT2025029); National Natural Science Foundation of China(625760911 the Southeast University Big Data Computing Center; Southeast University Kunpeng& & AscendCenter of Cultivation.
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00922, 2020.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303, 08774, 2023.
Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autore- gressive and diffusion language models. arXiv preprint arXiv:2503. 09573, 2025.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry. Quoe Le, al. Program synthesis with large language models. arXiv preprint arXiv:2108.0772: 2021.
Lukas Berglund, Meg Tong. Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Kor- bak, and Owain Evans. The reversal curse: Lims trained on" is b" fail learn" is a". arXiv preprint arXiv:2309. 12288, 2023.
Zefan Cai, Yichi Zhang. Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong. Yue Dong, Junjie Hu, et al. Pyramidkv: Dynamic kv cache compression based on pyra- midal information funneling, arXiv preprint arXiv:2406.02000 022.000..00 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, llya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10. 14168, 2021.
Yuan Feng, Junlin Lv, Yukun Cao, Xike Xie, and S Kevin Zhou. Ada-kv: Optimizing kv cache evic- tion by adaptive budget allocation for efficient Imm inference. arXiv preprint arXiv:2407. 1550, 2024.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. http:::::
/ttEtpp://zenddo. www.do.oooooooo .aceeddoorgg. org/ rec. rocrrda rds / 600
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning, arXiv preprint arXiv:2501. 12948 2025.

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022.
Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprini urXiv:2505.21467, 2025.
Pengcheng Huang, Shuhao Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, and Tong Xiao. Pe-sampler: Position-aware calibration of decoding bias in masked diffusion models. arXiv preprint arXiv:2508. 13021, 2025.
Haoyang Li, Yiming Li, Anxin Tian, Tianhao Tang, Zhanchao Xu, Xuejia Chen, Nicole Hu, Wei Dong, Qing Li, and Lei Chen. A survey on large language model acceleration based on kv cache management. arXiv preprint arXiv:2412.1944) 2024.
Pengxiang Li, Shilin Yan, Joey Tsai, Renrui Zhang, Ruichuan An, Ziyu Guo, and Xiaowei Gao. Adaptive classifier-free guidance via dynamic low-confidence masking. arXiv preprint arXiv:2505.20199 2025a.
Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508. 19982, 2025b.
Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A survey on diffusion language models. arXiv preprini arXiv:2508./003: 2025c.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 2023.
Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dilm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint (XXi:2506600699 2025.
Xinyin Ma, Rumpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models. arXiv preprint w1Xiv:2505.1578,, 2025.
Vaishnavh Nagarajan, Chen Henry Wu, Charles Ding, and Aditi Raghunathan. Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction. arXiv preprint arXiv:2504.13: 15266, 2025.
Shen Nie, Fengqi Zhu, Chao Du Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410. 2tti:224001181144 2024.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint MXXi:25002.0992. 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. In First Conference on Language Modeling, 2024.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 111330013110144 2024.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103166 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023.
12

---

Published as a conference paper at ICLR 2026
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pu pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266-9528 95290, 2024.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training free acceleration of diffusion Ilm by enabling cache and parallel decoding. arXiv preprint arXiv:2505.22618, 2025.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprini arXiv:2309. 17453, 2023.
Chenkai Xu, Yijie Jin, Jiajun Li, Yi Tu, Guoping Long, Dandan Tu, Mingcong Song, Hongjie Si, Tianqi Hou, Junchi Yan, al. Lopa: Scaling dllm inference via lookahead parallel decoding. arXiv preprint arXiv:2512.16229, 2025.
Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: comprehensive survey of methods and applications. ACM computing surveys, 56(4):1-39, 2023.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487, 2025.
Xingyu Zhu, Shuo Wang, Jinda Lu, Yanbin Hao, Haifeng Liu, and Xiangnan He. Boosting few-shot learning via attentive feature regularization. In Proceedings the AAAI conference artificial intelligence, volume 38, pp. 7793-7801, 2024.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In this work, we employ large language models (LLMs) as general- purpose auxiliary tools, which are mainly used in the following two scenarios:
Writing and editing: LLMs assist in revising the manuscript by enhancing its clarity, grammar, and stylistic consistency.
Code generation: LLMs assist in programming tasks, including debugging and generating illustrative code snippets.
The authors are fully responsible for the entire content of this paper, including sections in which LLMs provide writing assistance. We note that LLMs are not involved in research ideation, experi- mental design, data analysis, and therefore do not meet the criteria for authorship.
B RELATIONSHIPS WITH CONCURRENT WORKS
Prompt
Block
Block 2
Prompt
Response
Prompt takan
(AAKK token
Cachad token
steps
steps
Compute
Compute only
(a) Prefix eache Fast-dLLM
(b) dLLM Cache
Figure 7: Illustration of existing approximate KV cache works. (a) Fast-dLLM, the tokens of the current block and all subsequent blocks are recomputed. Once block has been fully decoded, the KV cache at all positions refreshed. (b) In dLLM-Cache the prompt and response update their corresponding segment cache at intervals of Kp and K,, steps, respectively. During steps when the response not updated, a subset of response tokens is still updated in each layer.
We note two concurrent works on approximate KV cache for dLLMs, including dLLM-Cache (Liu et 2025) and Fast-dILLM (Wu al. (2025). While both share the same motivations of accelerating dLLM inference through approximate KV cache, our Cachh is fundamentally different.
First and foremost, as shown Figure 7 dLLM-Cache and Fast-dLLM both operate the coarse- grained segment level, which partition the input sequence into multiple segments and apply dif- ferent KV state updates to each segment. For instance, dLLM-Cache divides the input sequence into two segments promppt and response- updates their KV states at different frequencies. Similarly, Fast-dLLM relies on bioek-wise eutmeeuttrrerrsssvve decoding, which divides the input sequence into multiple blocks (or segments) and sequentially generates these blocks from left to right with tailored KV state updates each block. Nonetheless, due the coarse-graing gaaseee nature, dLLM-Cache and Fast-dLLM inevitably reuse KV states that should be updated update KV states that can be reused, thus limiting the achievable inference gains.
In contrast, our CaCcche operates at the fine-grained token level, which adaptively identifies tokens whose KV states should be updated at each decoding step. while caching the KV states of the remaining tokens for reuse in subsequent decoding step. Thanks to the fine- grained token selection, our d' Cache achieves significant inference speedups while maintaining strong generation quality across different tasks, compared to both dL.LM-Cach and L..
14

---

Published as a conference paper at ICLR 2026
C BASELINE HYPERPARAMETERS
In this section, we provide more details about the hyperparameter configurations for the baseline methods (ie.,, Fast-dLLM (Wu et al. 2025) and dLLMMCachh (Liu al. (025)) across different models and datasets. For FLM,-dLLM, we closely follow common practices in prior work and set the block size 32 for all models (Wu et al. 2025). For dLLM-Cache, we consider its key hyperpa- rameters Kp and Kr, where Kp denotes the prompt refresh interval and K, denotes the response refresh interval. To ensure fair comparisons, we employ the default configurations as reported in Liu et al. (2025). which are also summarized in Table
Table 3: Configurations of dl LL-Chec Kp and K, are the refresh interval of prompt and response.
Dataset
Model
Kp
K
LLaDA-8B-Bass
25
5
-LaDD--B-Instruu
50
7
GSM8K
Dream-v0-7B-BBan
100
8
Pream-v0-7B-Instien
25
2
LLaDA-8B-Bass
50
5
LLaDA-
MAAAA-B--Instrue
25
5
HumanEval
Dream-vO-7B-DD
5
1
Dream-v0-7BBInsttuu
50
LLaDA-8B-Ban
50
8
LLaDA-SB-Instrus
50
1
Math-500
100
4
Pream-v0-7B-Insteee
50
1
LLaDA-8B- -B--ase
25
4
LLaDA-SB- Instrucc
100
5
MBPP
Dream-v0-701 Saann-v0-BB-ass
25
8
Dream-v0-77 Seaam-00-7BIIntrrus
10
8
ALaDA-SB-Instruue
50
6
Dream-v0-7D1
100
8
GPQA
Pream-v0-7B-Insee
10
8
LLaDA-8B- MADDA-SB-Base
100
6
LLaDA-8B -ns-tuutt
50
3
MMLU-Pro
Dream- (0-7B-Base
25
2
Pream-v0-7B--e
5
1
D DISCUSSIONS
D.1 MEMORY OVERHEAD OF CACHING
We conduct a thorough analysis and profiling of the memory overhead of caching. Note that the KV cache used by dLLMMs consumes the amount of memory that required by autore- gressive LLM (ARM) of the same scale. Specifically, for sequence length L, number of layers N, and hidden dimension an ARM dLLM stores 2 L N d floating-point values for the KV cache. d'Cache additionally stores an attention- roolouut matrix of size L x L, which is typically negligible. We report the peak memory usage on Dream Inst for generation length of 1024 across four datasets. As shown in the Table 7 for example GSM8K, where the average prompt length approximately 800-resulling in a sequence length of roughly 1.8k-the additional memory con- sumption of d'Cache is nearly identical to that of Fast-dLLM (Wu et al. 2025).
D.2 DECODING ORDER OF DLLMS
In Section 4.1
decoding, which forces the model to generate in quasi-left-to-righh order. A natural question arises: if dI.LMs behave more like autoregressive
we proposed certainty-prion
15

---

Published as a conference paper at ICLR 2026
Comparisons of using only the first layers to compute attention rollout (Rollout-5) and using all layers to compute attention rollout (Full-rollout) on Dream-Inst. Bold numbers indicate the best scores, and green texts denote the speedup ratios relative to the Vanilla method.
Table 4:
Dataset
Method
Throughput 1
Score
GSM8K
Vanilla
2.62
76.7
4-shot
Rollout-5
(2.61(4.8x)
71.8
Gen. Len. 256
Full-rollout (Ours)
12.25(4.7x)
78.2
MBPP
Vanilla
2.73(1.0x)
52.0
3-shot
Rollout-5
13.10(4.8x)
57.2
Gen, Len. = 512
Full-rollout (Ours)
12.47(4.6x)
58.0
HumanEval
Vanilla
4.39(1.0x)
56.7
O-shot
Rollout-5
14.20(3.2x)
62.2
Gen. Len. = 512
Full-rollout (Ours)
14.06(3.2x)
61.6
Moth-500
Vanilla
3.51(1.0x)
45.2
4-shot
Rollout-5
13.99(4.0x)
40.2
Gen. Len. = 256
Full-rollout (Ours)
13.80(3.9x)
44.6
models (ARMs), does this violate the original intention of enabling parallel, any-order generation? Here, we argue that the answer is not simply "no""
Unlike ARMs, which can generate tokens only at the immediately adjacent next position, dLLMs produce predictions over the entire sequence, which is the source of their any-order generation ca- pability. However, high-quality predictions are not available all positions. Thus, selecting which tokens to decode that is, determining the decoding order-is is crucial for generation quality. As shown in Nie et al. (025), LLaDA-Instruct tends to become prematurely overconfident EOS to- kens near the end of the sequence, and therefore proposes block-wise semi-autoregrresive decoding (semi-AR), which constrains the model to decode from left to right at the block level while gener- ating in parallel within each block. Compared with fully non-autoreeressive decoding. block-wise semi-AR preserves the model's 's sequential reasoning ability to a large extent, as shown in Table [
In paper, experiments in Section 3.2 show that dLLM consistently prefers decode tokens close to known positions. This observation explains why block-wist wise semi-AR effective: enforcing quasi-left-to-righe generation ensures that each token is decoded only when the contextual informa- tion is sufficiently rich. Our certainty-prior decoding shares the same intuition, but provides a more conceptual formulation.
Although dLLMs need to decode in quasi-left-to-rightt order to maintain sequential reasoning ability, they still retain substantially greater flexibility during generation. For example, when the model encounters a position where all next-token candidates have low confidence, an AR model must commit to one choice. In contrast, dLLM can decode further positions and delay the decision until the extended context provides adequate evidence, thereby exploiting its non-AR modeling capacity. A concrete example pronoun resolution in ambiguous contexts. Suppose the prompt is: "Alice thanked Mary because -.. had helped with the project".
At the blank position, an AR model must immediately choose between "she" and "Alice", even though the correct antecedent remains unclear without additional context. A dLLM, however, can tentatively consider both possibilities, continue decoding subsequent positions, and use the extended context to determine whether the sentence is likely continue "... she had provided key data", or " Alice needed assistance". before committing to the final token.
Moreover, even under quasi-AR decoding, each masked token still attends to the entire context (unlike ARMs, where tokens can only attend to previous positions), so the original advantages of dLLMs, such bidirectional modeling and parallel decoding, remain preserved.
16

---

Published as a conference paper at ICLR 2026
Table 5: Comprehensive evaluation results on LLaDA-Base Nie al. 2025 and Dream- et al. 2025). Bold numbers indicate the best results and green texts denote the speedup ratios.
Ye
LLaDA Base
Dream- Pream-Base
Dataset
Method
Throughput 1
Latency(s) 1
Score
Throughput 1
Latency(s) 1 Score
Vanilla
2.31 (1.0x)
112.39
70.4
2.67 (1.0x)
96.29
71.7
GSM8K
+ ddLLM-Cache
7.72 (3,3x)
33,30
69.3
9.28 (3,5x)
27.88
64.7
4-shor
Fast-dLLLM
9.94 (4.3x)
25.82
70.3
10.07 (3.8x)
25.44
71.4
Gen. Len. 256
Caach
(4.9x)
22.57
12.37 (4.6x)
21.74
73.5
Vanilla
2.52 (1.0x)
195.59
39,2
2,81 (1.0k)
177.14
51.4
MBPP
dLLM-Cache
6.52 (2.6x)
77.20
38.6
7.73 (2.8x)
64.75
49.8
3-shor
Fast-dLLM
(2.8x)
73.16
7.07 (2.5x)
72.40
$2.6
Gen. Len. = 512
d'Cache
8.62 (3.4x)
43.41
38.0
12.67 (4.5x)
40.10
53.6
Eval
Vanilla
(1.0x)
100.54
32.3
5,45 (1.0x)
92.11
dLLM-Cache
9.04 (1.8x)
55.60
31.7
5.47 (1.0x)
91.72
51.8
O-shor
Fast-dLLM
8.13 (1.6x)
62.96
31.7
7.81 (1.4x)
88.02
53.7
Gen. Len. 512
d
14.36 (2.9x)
35.60
33.5
14.36 (2.6x)
37.18
61.0
Vanilla
3.14 (1.0x)
80.44
32.2
3.55 (1.0x)
71.54
39.0
Math-500
dLLM-Cache
9.83 (3.1x)
25.94
29.6
9.70 (2.7x)
26.08
35.2
4-shor
Fast-dLLM
10.93 (3.5x)
23.41
27.0
10.69 (3.0x)
23.94
39.4
Cache
10.80 (3.4x)
20.13
30.4
13,86 (3,9x)
18.63
39.6
Vanilla
6.27 (1.0x)
42.35
30.4
6.54 (1.0)
40.55
32.8
GPQA
dLLM-Cache
11.32 (1.8x)
22.69
31.0
11.12 (1.7x)
23.10
34.6
Fast-dLLM
12.67 (2.0x)
20.24
31.0
11.92 (1.8)
21.45
31.5
256
d'Cache
15.32 (2.4x)
16.77
30.8
13.02 (2.0x)
18.64
32.6
Vanilla
1.53 (1.0x)
143.45
38.1
2,13 (1.0x)
127.08
46.1
MMLU-Pro
dLLM-Cache
6,86 (4.5%)
37.96
37.4
7.45 (3.5x)
34,81
44.6
5-shor
Fast-dLLMM
8.96 (5.9x)
28.83
9.42 (4.4x)
27.31
45.9
Gen. Len, 256
d'Cache
(6.3x)
27.60
39.1
9.71 (4.6x)
26.73
44.4
Vanilla
3.47 (1.0x)
112.46
40.4
3,86 (1.0x)
100.79
48.7
dLLM-Cache
8.55 (2.5x)
42.12
39.6
8.46 (2.2x)
44.72
46.8
AVG
Fast-dLLM
9.61 (2.8x)
39.07
9,50 (2.5x)
43.09
49.1
Cacch
(3.4x)
27.68
12.67 (3.3x)
27.17
50.8
D.3 LIMITATIONS AND FUTURE WORK
Although Cacch delivers substantial inference speedups across multiple models and datasets while maintaining comparable performance, several limitations have also emerged. Below we summarize these limitations and further outline potential directions for future work.
Larger-scale dLLMs. In this work, we closely follow recent representative practices (Wu et al. 2025 Liu et al. 2025) to evaluate Cacch LLaDA-8B (Nie et al. 2025) and Dream-78 7B (Ye et al. 2025). ). We note that, at this moment, LLaDA-8B and Dream-7B are the only publicly available dense dLL.s. As future dLL.Ms continue scale up in depth, width, and context length, their bidirectional attention patterns will become even more costly to maintain during decoding. This trend further highlights the importance of more effective caching schemes. We view extending d"Cache to larger-scale dLLMs togtser with more effective caching schemes-as promising direction for future work, especially as model sizes and application demands continue to explode.
Lightweight variants of attention rollout. Although attention rollout is not a performance bot- tleneck in our d'Cache, its cost can become significant when it is applied to larger- seaeer-scale models. More efficient approximations are therefore desirable. We evaluate a lightweight variant that com- putes rollout using only the first five layers on Dream-Inst and four datasets. shown in Table reducing the rollout depth from 28 to yields slight improvement in decoding speed while notice- ably degrading performance on math reasoning tasks (GSM8K and Math-500); in contrast, code- generation tasks (HumanEval and MBPP) exhibit minimal performance loss. Designing lightweight rollout variants that can identify key tokens still remains an important direction for future work.
Alternative scoring functions for contextual contribution. We currently employ Gaussian func- tion to characterize how a masked token influences its surrounding context. While this approach performs well empirically, context-adaptive formulations may further enhance performance.
17

---

Published as a conference paper at ICLR 2026
Table 6: Comprehensive evaluation results on LLaDA-Inst (Nie al. 2025) and Dream-Inst YYe et al. 2025) with semi-AR parallel decoding. Bold numbers indicate the best results and green texts denote the speedup ratios.
LLsDA-Inst
Dream-Inst
Dataset
Method
Throughput
Latency(s)
Score
Throughput f
Latency(s)
Score
Vanilla
2.77 (1.0x)
110.26
77.6
2.62 (1.0x)
85.94
76.7
GSM8K
Parallel
8.53 (3.1x)
33.95
77.6
13.93 (5.3x)
21.68
74.2
4-shor
dI..M-Cache
25.62 (9.2x)
10.26
77.0
36,00 (13.7%)
8.21
74.3
Gen. Len, 256
Fast-dLLM
25.15 (9.
10.65
77.6
32.75 (12.5%)
8.35
74.1
PCCche
38.16 (13.8x)
7.26
76.9
46.69 (17.8%)
6.13
75.7
Vanilla
2.48 (1.0x)
199.90
14.4
2.73 (1.00)
182.78
52.0
MBPP
Parallel
29.92 (12.1x)
19.61
38.8
38.06 (13.9x)
14,95
51.6
3-shor
+ LLCcM-cchhe
72.52 (29.2x)
7.94
38.0
89.31 (32.7x)
6.11
51.8
Gen. Len. 512
+ Fast-dLLM
43.29 (17.5x)
12.44
51.04 (18.7%)
10.40
d'Cache
119.81 (48.3x)
4.64
108.39 (39,7x)
5.11
Vanilla
4.99 (1.0x)
105.76
45.1
4.39 (1.0x)
114.86
56.7
HumanEval
Parallel
15.74 (3.2x)
37.63
45.1
39.78 (9.1x)
20.53
dLL.M-achee
27.88 (5.6x)
20.54
48.50 (11.0x)
14.75
53.7
Gen. 512
Fast-dLLM
25.14 (5.0x)
21.76
43.3
48.94 (11.1x)
13.67
$7.3
dPCache
48.39 (9.7x)
11.89
46.6
104.40 (23.8x)
6.30
57.3
Vanilla
3.08 (1.0x)
82.51
3.51 (1.0x)
71.05
45.2
Math-500
Parallel
8.80 (2.9x)
31,99
38.4
12.75 (3.6x)
23.99
44,4
4-shor
d.L.M-Cache
17.90 (5.8x)
15.53
37.8
25.16 (7.2x)
12.06
43.0
Gen. Len. 256
Fast-dLLLM
24.49 (8.0x)
11.01
37.4
28.68 (8.2x)
9.73
43.4
+ Cachh
34.69 (11.3x)
8,03
38.6
37.83 (10.8x)
7.74
42.6
Vanilla
6.14 (1.0x)
43,34
25.2
6,43 (1.0k)
41.14
30.1
GPQA
Parallel
44.61 (7.3x)
15.98
25.7
128.85 (20.0x)
2.87
31.3
0-shor
+ dLLM-Cache
50.17 (8.2x)
10.78
28.4
174.83 (27 2x)
2.06
33.3
Gen. Len. 256
Fast-dLLM
(6.9x)
10.13
136.28 (21.2x)
2.31
d'Cache
89.03 (14.5x)
6,89
162.95 (25.3x)
2.08
Vanilla
1.76 (1.0x)
152.62
37.5
2,15 (1.0x)
126.31
47.9
MMLU-Pro
Parallel
(5.0x)
58.86
17.87 (8.3x)
27.12
-...MM--ache
19.76 (11,2x)
18.95
35.2
41.83 (19.5x)
8.88
48.9
Gen. 256
Fast-dLLM
19,46 (11.1x)
16.07
37.1
35.46 (16.5%)
8.94
47.1
d'Cache
28.85 (16.4x)
13.24
35.1
54.56 (25.4x)
6.42
46.1
Vanilla
3.54 (1.0x)
115.73
39.7
3.64 (1.0k)
103.68
51.4
Parallel
19.40 (5.5x)
33,00
43.8
41.87 (11.5x)
18.52
50.2
AVG
+ dLLM-Cache
35.64 (10.1x)
14,00
44.1
69,27 (19.0x)
8.68
50.8
+ Fast-dLLLM
30.03 (8.5x)
13.68
43.3
$5.53 (15.3%)
8.90
51.5
+ d d'Cache
59.82 (16.9x)
8.66
44.4
85,80 (23.6x)
5.63
51.2
E ADDITIONAL EXPERIMENTAL RESULTS
E. EXPERIMENTAL RESULTS ON THE BASE VARIANTS
In addition to the Instruct variants of 8B Nie al. 2025 and Dream-v0-7B (Ye et al. 2025), we also conduct experiments on their Base variants, which are denoted as LLaDA- Base and Dream-Bass respectively. As shown in Table 5 our Cache consistently outperforms other approximate KV cache methods terms of both average inference efficiency and accuracy across six datasets.
E.2 EXPERIMENTAL RESULTS UNDER PARALLEL DECODING SETTINGS
To enable fair comparison across all methods and to verify the generalizability of d"Cahe under al- ternative decoding strategies, we evaluate all approaches using the parallel decoding strategy. where the threshold is set to 0.9 following Wu et al. (2025). As shown in Table our method achieves up to 48.3x acceleration over the segeletooee--er-teeee bascline while maintaining performance com- parable to all other baselines, which clearly demonstrates the broad applicability of dCCach..
18

---

Published as a conference paper at ICLR 2026
Table 7: Performance comparison on Dream-Ins with generation length of 1024.
Dataset
Method
Throughput (tokens/s)
Latency(s)
Score 1
Memory (GB)
Vanilla
1.54 (1.0x)
671.35
68.46
19.26
GSM8K
Fast dLLM
4.18 (2.7x)
245.29
67.85
19.39
4-shot
dLL.MCachee
3.33 (2.2x)
308.62
68.76
20.28
Gen. Len. 1024
d'Cac
8.58 (5.6x)
119.69
66.29
19.35
Vanilla
1.89 (1.0x)
541.04
43.6
19.15
Math-500
Fast dLLM
4.43 (2.3x)
231.45
42.6
19.27
4-shor
dLLM-Cache
2.75 (1.5x)
373.1
40.4
20.2
Gen. Ten. 1024
d'Cache
9.55 (5.1x)
107.29
41.2
19.28
Vanilla
2.62 (1.0k)
393.47
56.71
19.06
HumanEval
Fast dLLM
4.77 (1.8x)
214.5
58.53
19.16
0-shor
dLLM-Cache
3.03 (1.2x)
338.21
60.97
19.79
Gen. Len. 1024
Cachh
11.74 (4.5x)
87.64
64.02
19.14
Vanilla
1.95 (1.0x)
526.86
52.8
19.12
MBPP
Fast dLLM
4.45 (2.3x)
229.91
52.4
19.23
3-shor
d.L.M-Cache
4.59 (2.4x)
223.05
54.2
19.94
Gen. Len. 1024
dache
9.76 (5.0x)
105.05
56.4
19.24
Vanilla
2.00 (1.0x)
533.18
55.39
19.15
Fast dLLM
4.46 (2.2x)
230.29
55.35
19.26
AVG
dLLM-Cache
3.43 (1.7x)
310.75
56.08
20.05
CCCch
9.91 (5.0x)
104.92
56.98
19.25
E.3 EXPERIMENTAL RESULTS UNDER LONG-CONTEXI SETTINGS
To further assess our method's performance under long-context settings, we further evaluate our method Dream-Inst under with a longer generation length 1024. As shown in Table we observe that other methods- the context length increases. In contrast, d'Cache maintains substantial speedups without per- formance loss even in long-context scenarios, owing to its fine-grained two-stage token selection. These results demonstrate that Cache also performs well in long contexts.
to their coarse- coarse-grained nature-cxpenne exxerrence severely degraded accelcration
E.4 MORE VISUALIZATION RESULTS ON ATTENTION ROLLOUT
In this section, we present additional examples of attention rollout corresponding to the sample used in Figure As shown in Figure X
the attention pattern also aligns with our findings in Section 3.3
E.5 MORE VISUALIZZATION RESULTS ON KV STATE DYNAMICS
To substantiate our findings in Section 3.22 visualize additional KV state dynamics. In Figure which visualizes the trajectories of the key and value states of the same masked token during de- coding, both closely aligned in both trajectory shape and magnitude, and both exhibit the same gradual-rapid-stable dynamic pattern. This result suggests that, for both key and value states, it is sufficient update them only during the rapid-change phase, where these KV states be safely cached for reuse during the other two phases. We hypothesize that this rapid change arises because tokens are particularly sensitive changes in their local context. Specifically, at step token xf located near another masked token z!! that is decoded, then step & + the embedding of af changes from [MASK] to the embedding of a concrete token. This provides ai with additional contextual information; the smaller the distance i -1, the more tightly constrained the context be- comes, thereby substantially altering the model's representation of x These observations motivate the introduction of distance-aware decay into the certainty density, defined in Equation (3)
a masked
19

---

Published as a conference paper at ICLR 2026
Attention Analysis Step 54
Attention Analysis Step 127
Query Position Summed Rollout
Query Position Summed Rollout
5-
Promot Desoded
Prompt Decoded
4-
2
2
1-
0.02
0.01
100
100
200
300
300
100
150
200
250
300
0.00
50
100
150
200
250
300
0.00
Key Position
Key Position
(b)
Attention Analysis at Step 128
Attention Analysis at Step 219
Query Position Summed Rollout
5-
Prompt Masked
Query Position Summed Rollout
4-
Prompl
Masked
2
2
0.01
0,01
100
100
200
200
300
Key Position
Key Position
(c)
(d)
Figure 8: Visualization of attention rollout on LLaDA-Inst (Nie al.
2025) with GSM8K, which is
generated using the same sample and configuration as in Figure
20

---

Published as a conference paper at ICLR 2026
Second Principal Component
100
Step 87
Second Principal Component
100
Step 87
80
50
Step 111 Decoded
Step Decoded
-50
step
Step 255
Step
Step 255
Gradual Change Change
Gradual Change Rapid Change Stable
-60
150
Stable
-200
-200
-000
-150
-100
-50
50
100
200
250
300
-200
-150
-100
-50
0
50
100
150
200
250
300
Principal Component
Principal Component
(a) Key trajectory for the 91st masked token.
(h) Value trajectory for the 91 masked token.
Second Principal Component
Second Princlpal Component
Step 154
Step 154
100
Step Decoded
100
Step 173 Decaded
50
50
Stop 255
Step 255
Gradual Change Change
Gradual Change
-100
Step
-100
Step
Rapid Change
Slable
Stable
-150
-150
200
-150
.100
-50
50
100
200
250 300
-300
-250
-200
-150
-100
-50
100
160 200
First Principal Component
First Principal Component
(c) Key trajectory for the 186th masked token.
(d) Value state trajectory for the 186th masked token.
Figure 9: Visualization of PCA-projected trajectories of LLaDA-Inss -nnt on GSM8K, which are gener- ated using the same sample and configuration as Figure (a).
21


Reviews:
[{"review_id": "6MWwtBBF7n", "reviewer": "Reviewer_4Cqp", "summary": "Diffusion-based large language models (dLLMs) outperform autoregressive models (ARMs) in mitigating the \"reversal curse\" and capturing global semantics but suffer from low inference efficiency—they cannot use standard KV caches due to bidirectional attention, and existing coarse-grained approximate KV solutions lack flexibility or cause redundancy. To address this, the authors propose D2Cache, a training-free framework built on two insights: masked tokens’ KV states only need updates in the \"rapid-change phase\" before decoding, and dLLMs’ attention concentrates on a small subset of prompt/decoded tokens. D2Cache uses a two-stage strategy: 1) selecting masked tokens via a \"certainty prior\" (combining prediction confidence and nearby known token density) for KV updates, enabling quasi left-to-right generation; 2) choosing high-impact prompt/decoded tokens via attention rollout. Experiments on LLaDA-8B and Dream-v0-7B show D2Cache achieves 3.2×–4.0× average speedups (e.g., 4.7× on Dream-Inst/GSM8K) while maintaining/improving generation quality, outperforming baselines like dLLM-Cache and Fast-dLLM. It advances dLLMs’ practical deployment by bridging performance and inference efficiency.", "questions": "+ **Discussion 1**: How do you think your method is compatible with token sparsification techniques [1], which have been proven useful for LLMs [2]? Could you add related discussion?\n\n+ **Discussion 2**: While the paper presents the method as a training-free framework, it raises the question of its potential trainability. It would be interesting to investigate whether the framework can be made trainable and how fine-tuning might affect its performance.\n\n**I would be happy to increase my rating if my views are given a thorough discussion.**\n\n[1]: Wu X, Zeng F, Wang X, et al. PPT: Token pruning and pooling for efficient vision transformers[J]. arXiv preprint arXiv:2310.01812, 2023.\n\n[2]: Wan Z, Wu X, Zhang Y, et al. D2o: Dynamic discriminative operations for efficient generative inference of large language models[J]. arXiv preprint arXiv:2406.13035, 2024.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Originality**: Breaks from prior coarse-grained (segment/block-level) KV caching for dLLMs, proposing a token-level adaptive strategy. It’s driven by two dLLM-specific insights (masked tokens’ KV states only need updates in the \"rapid-change phase\"; dLLMs’ attention concentrates on prompt/decoded tokens) and introduces \"certainty prior-guided decoding\" to fix NAR’s premature overconfidence bias.  \n\n- **Quality**: Rigorous validation covers 2 dLLMs (LLaDA-8B, Dream-v0-7B), 4 tasks (reasoning, code generation), and key metrics (throughput, latency, accuracy). Comprehensive ablations (decoding schemes, update phases, hyperparameters) isolate design impacts, with mathematical formalization (Gaussian-weighted density, attention rollout) ensuring theoretical soundness.  \n\n- **Clarity**: Follows a logical \"problem→insight→solution→validation\" flow. Visualizations and plain-language formula explanations enhance accessibility, while full parameter specs, public datasets, and planned code release support reproducibility.  \n\n- **Significance**: Achieves 3.2×–4.0× average inference speedups (up to 4.7×) while maintaining accuracy, resolving dLLMs’ deployment bottleneck. The token-level caching paradigm inspires optimization for other bidirectional models, and its benchmarks set a reference for future work.", "weaknesses": "+ **Narrow validation scope**: Only tests mid-sized dLLMs (8B/7B) and 4 reasoning/code tasks—lacks large-scale dLLMs (17B+/70B) and non-technical tasks (dialogue/creative writing). If it is possible, add experiments on large dLLMs and conversational datasets (e.g., DailyDialog).  \n\n+ **Fixed hyperparameters**: Uses static k=32/p=0.1/σ=10 across all scenarios, no adaptive tuning. If it is possible, add hyperparameter sensitivity analysis (e.g., long vs. short sequences) and lightweight dynamic adjustment (as a supplement to ablation study, beyond section D.4).  \n\n+ **Unreported memory overhead**: Ignores peak GPU memory usage—critical for edge deployment. If it is possible, add memory metrics and analyze speed-accuracy-memory trade-offs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kHe0XNQkgA", "reviewer": "Reviewer_KmGv", "summary": "Authors proposes d2Cache, a training-free approximate cache that updates only a small subset of tokens’ KV states and reuses cached KV for the rest. Token selection is two-stage: Certainty-prior for masked tokens and attention-aware selection. The accuracy and efficieny are improved by this method.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. KV cache for dLLM inference is new, where standard KV cache doesn’t apply.\n\n2. Simple design with limited moving parts and clear hyperparameters.\n\n3. consistent speedups and several accuracy improvements.", "weaknesses": "1. Please explore hyperparameters of k  and p. these likely trade speed vs. quality and are important for deployment.\n\n2. Results use gen lengths 256–512; authors may do experiement on L=1-4k. The rollout cost and cache efficacy under long prompts should be evaluated\n\n3. How are layer norms, residuals, and MLP activations treated when some tokens are stale, as freezing only K V may mismatch with new hidden states.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4zlhR3myt9", "reviewer": "Reviewer_1oge", "summary": "To solve diffusion-based LLMs’ (dLLMs) inference inefficiency (cannot use standard KV caches), the paper proposes d2Cache, a training-free KV cache framework. It leverages two insights: (1) masked tokens’ KV states only need updates in the \"rapid-change\" phase; (2) dLLM attention concentrates on a small token subset. d2Cache uses two-stage selection (certainty prior for masked tokens, attention rollout for prompt/decoded tokens) and guided decoding to avoid premature termination. Experiments on 2 dLLMs (LLaDA-8B/Dream-7B) and 4 benchmarks show 3.2×–4.0× speedup over vanilla dLLMs, outperforming baselines while improving accuracy.", "questions": "1. Scalability to large models: How does d2Cache’s performance (speedup, accuracy) change when applied to 13B/70B dLLMs?  Can lightweight optimizations (e.g., sparse attention rollout, only using top-5 layers) reduce this overhead without losing accuracy?\n2. Adaptation to long sequences: For sequences longer than 512 tokens (e.g., 1k PubMed abstracts, 2k legal documents), how should the Gaussian function’s σ be adjusted? Does a dynamic σ (e.g., σ = sequence_length × 0.02) maintain accuracy while avoiding excessive updates?\n3. Failure case root cause: On LLaDA-Inst/MBPP, why does d2Cache’s accuracy (12.4) lag behind Fast-dLLM (13.8)? Are there specific code patterns (e.g., nested loops, import statements, function definitions) where the two-stage selection fails?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Empirically grounded design with measurable evidence: The framework  relies on fine-grained analyses of dLLM behavior. For example, PCA visualizations confirm masked tokens’ three-phase KV evolution (e.g., LLaDA-8B-Inst/GSM8K shows rapid changes in steps 64–98), and attention rollout proves 80% of attention concentrates on 20% of prompt/decoded tokens. These insights show the \"phase-based updates\" and \"high-attention selection\" core strategies.\n2. Most dLLM efficiency methods trade accuracy for speed, but d2Cache improves both. Its certainty prior-guided decoding mitigates vanilla dLLMs’ \"U-shaped\" bias (tokens at sequence boundaries decoded first, leading to premature, boosting accuracy while accelerating inference. For instance, on Dream-Inst/HumanEval, it raises throughput from 4.39 to 14.06 and accuracy from 56.7 to 61.6.", "weaknesses": "1. The paper only tests 7B/8B parameter models and sequences ≤512 tokens. For larger models (e.g., 13B/70B dLLMs), the computational overhead of attention rollout may scale sharply with model size, offsetting speed gains. It is interesting to discuss the proposed method on the experiments at scale.\n2. The paper ignores scenarios where d2Cache underperforms. For example, on LLaDA-Inst/MBPP, its accuracy (12.4) is lower than Fast-dLLM (13.8) and dLLM-Cache (12.8)， although the proposed method is better at throughput and latency. It is interesting to explain whether code generation requires more frequent masked token updates (k=32 is too small), attention rollout misses code-specific critical tokens (e.g., function call syntax), or the certainty prior fails to capture code structure dependencies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gwr72QII10", "reviewer": "Reviewer_E2JM", "summary": "This paper proposes D2Cache, a training-free approximate key-value (KV) cache framework designed to accelerate inference in diffusion-based large language models (dLLMs). Unlike autoregressive models (ARMs), dLLMs rely on bidirectional attention and cannot directly reuse KV states efficiently.\n\nD2Cache introduces a two-stage fine-grained token selection process:\n1. Certainty Prior-Guided Selection: Selects masked tokens to update based on both their prediction confidence and local token density, yielding a more reliable quasi-left-to-right decoding order.\n2. Attention-Aware Selection: Updates KV states only for tokens that receive higher attention weights, identified using attention rollout.\n\nThrough this adaptive caching mechanism, D2Cache reduces redundant computations and achieves significant speedups (3–4×) over baselines such as dLLM-Cache and Fast-dLLM, while maintaining or slightly improving task performance across math datasets.", "questions": "1. Since the proposed decoding strategy (certainty prior-guided) enforces a quasi-left-to-right order, how does it differ from simply using an autoregressive model? If the pipeline behaves like a “quasi-AR” system, does the original advantage of dLLMs (e.g., bidirectional modeling, parallel decoding) still hold?\n2. Were the baseline methods (dLLM-Cache, Fast-dLLM) tuned optimally for the same tasks? Given the performance gap, could hyperparameter choices or block sizes explain some of the observed differences?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a dual adaptive caching scheme that fine-tunes token-level updates for dLLMs. This is a clear step forward compared to previous coarse-grained cache methods like Fast-dLLM and dLLM-Cache.\n2. D2Cache demonstrates up to 4× acceleration without significant performance loss. The method achieves competitive or better results than baselines across multiple reasoning and programming datasets.", "weaknesses": "1. The baseline (confidence-based decoding) performs much worse than the proposed certainty prior-guided method, even when acceleration isn’t applied. This large performance gap raises doubts about whether the comparison is fair or due to confounding factors. It would be useful to include random sampling or alternative decoding methods (e.g., semi-autoregressive decoding) for stronger baselines. \n\n2. There’s ambiguity between speedup runs (Table 1) and default non-autoregressive runs (Table 2). Where Table 1 evaluates speedup under different methods, and Table 2 evaluates decoding schemes (confidence-based vs certainty-prior). However, both report exacly same performance scores, which makes it unclear whether the performance metrics are decoupled from the acceleration setting or derived from identical runs.\n3. The experiments focus only on math and code-related tasks. It remains unclear how the proposed method generalizes to general tasks. For example, perplexity should be evaluated as a basic metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)