Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
Authors: Xu Wang
Chenkai Xu
Yijie Jin
Jiachun Jin
Hao Zhang
Kai Yu
Zhijie Deng
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion Large Language Models
Discrete Diffusion Models
Inference Acceleration
KV Cache
AR-Diffusion Hybrid
Primary area: foundation or frontier models, including LLMs

Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs for text generation, with the potential to decode multiple tokens in a single iteration. However, none of the existing open-source dLLMs have achieved superior inference speed over AR LLMs of similar size.  This paper breaks this barrier based on a simple and effective strategy named discrete diffusion forcing (D2F). D2F equips dLLMs with two key capabilities: (1) block-wise autoregressive generation to enable KV cache utilization; (2) prediction of following tokens without requiring completion of prior blocks for inter-block parallel decoding. In this way, the vanilla dLLMs are refurbished into an AR-diffusion hybrid paradigm for efficient inference. D2F can be implemented with an asymmetric distillation process based on pre-trained dLLMs to achieve rapid convergence.We further propose a pipelined parallel decoding algorithm, which enables a trade-off between efficiency and efficacy. Empirically, D2F dLLMs achieve more than $\mathbf{2.5\times}$ inference speed than LLaMA3 and Qwen2.5 on GSM8K. Compared to the vanilla dLLMs like LLaDA and Dream, the acceleration can be more than $\mathbf{50\times}$ while maintaining comparable output quality.

Published as a conference paper at ICLR 2026
DIFFUSION LLMS CAN Do FASTER-THAN-AH INFERENCE VIA DISCRETE DIFFUSION FORCING
Xu Wang's Chenkai Xu' Yijie Jin 133 Jiachun Jin', Hao Zhang Zhijie Deng"
Shanghai Jiao Tong University University of California San Diego "Shanghai University (wangxu60, 32435xck, Jochun-1tmm 1 31 gted)esttu.edi
1y1243156 reshu.ddu.co adu.on
140
GSMBK-6-ehot
HumanErnl-O-that
122
Tokens/Seconn
502
82-
P.5a TPG
1A4 TPS
35.4
48.0
62-
3,5
7.5
16.7
2.8
12.6
LLane braceed
LEaDE instrrcct
Broath dame-
La Mattrrt
par Brean Sape
Llatk tnytracttttto
Part LLaDe iostrrcc
Fast Bream Praam-dace-1 dace-
La dov
Owen daae--7
D2p LLa.t Laadt-instrrrt-hh
Droai
Figure 1: D2F dLLMs surpass AR LLMs in inference speed for up to 2.3x. Comparison of inference throughput among D2F dLLMs, vanilla dLLMs like Dream- Beaam-Bacc-7B (Ye ct al.
2025) and
(Nie et al. (2025), previous SOTA acceleration method Fast-
dLLM (Wu et al. 2025), and similarly-sized AR baselines (Yang et al. 2024a Grattationi
el al. 2024). The max generation length set to 512.
ABSTRACT
Diffusion Large Language Models (dLLMs) have emerged as a promising alter- native to autoregressive (AR) LLMs for text generation, with the potential to de- code multiple tokens a single iteration. However, none of the existing open- source dLLMs have achieved superior inference speed over AR LLMs of simi- lar size. This paper breaks this barrier based on a simple and effective strategy named discrete diffusion forcing (D2F). D2F equips dLLMs with two key capa- bilities: (1) block blice-wise autoregressive generation to enable KV cache utilization; (2) prediction of following tokens without requiring completion of prior blocks for inter-block parallel decoding. this way, the vanilla di dL.MM are refurbished into an AR-diffusion hybrid paradigm for efficient inference. D2F can be im- plemented with an asymmetric distillation process based on pre-trained dLLMs achieve rapid convergence. We further propose pipelined parallel decoding algorithm, which enables trade-off between efficiency and efficacy. Empiri- cally, D2F dLLMs achieve more than 2.5x inference speed than LLaMA3 and Qwen2.5 on GSM8K. Compared to vanilla dLLMs such as LLaDA and Dream, D2F delivers over 50x acceleration while preserving comparable output quality. Moreover, it boosts DiffuCoder by up to 11.8x speedup without sacrificing per- formance. Code is available at https: //gthub com/ BJTU-DENG- Lab/ Discrete-Dlffust Forcinnn
INTRODUCTION
Large Language Models (LLMs) have maintained dominant position text generation for a long
time (Achiam et al. 2023 Touvron et al. 2023a Yang et al. 2025 Grattatiori et al. 2024). Recently,
Equal contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
GSMBk-4-shot
MBPP-3-shot
80
Scoe
50
60
50-
Throughput (tokens/sec)
100
150
181.3
200
50
Throughput (tokens/sec)
160
150
200
250
300
Methods
Parameter Details
02F Dream Base 7B
dLL Cach
Loct= connf-005
Lade Lconf-0.8
stops- 512
Druam-ase-75
LLMNAS-Isste Irsts-isstracc-e
acter conf-quy
ader conteg.7
stapu=256
Paet-EL LM
Qwen2 5-
L conf-885
r-
stepen 128
Figure 2: Throughput vs. performance trade-off. As shown, D2F achieves more favorable tradc-off' compared to vanilla dLLMs. Refer to Section 4.3 for the details of the hyperparameters Thks and Teonf-
Diffusion Large Language Models (dL.LMs) have emerged as a promising alternative to L.MSs (Ye al. 2025 Nie et al. 2025; Zhu et al. 2025), acknowledged by their potential to generate multi- ple text tokens in paralel.. For example, closed-source dLLMs such as Gemini Diffusion (Google DeepMind 2025 Mercury (Inception et al. 2025). and Seed Diffusion (Song et al. 2025 can yield thousands of tokens per second, 5-10 times faster than autoregressive (AR) LLMs of similar size.
However, the speed merits of dLLMs have not been demonstrated within the open-source commu- nity. Approaches to bridge the gap include designing KV cache strategies (Arrioaa et al., 2025
Liu
eet al. 2025 Ma et al. 2025) and improving parallel sampling algorithms (Wu al.] 2025 Wei
et al. 2025 Hu et al. 2025) For instance, Block Diffusion (Arriola et al. 2025) turns dLLMM
into a block-wise sequential generation paradigm to leverage KV cache. Yet, precludes the inter- block parallelism, a crucial factor for efficient inference. Fast-dLLM (Wu et al. (2225) also adopts a block-wise generation order facilitate the reuse of states of generated tokens and incorporates confidence- baasdd remasking for parallel decoding. Nonetheless, the cached states can be biased after subsequent tokens are decoded due the bidirectional nature of the involved attention.
This paper achieves the first breakthrough in accelerating dLLMss to faster-than-AR regime. Con- ceptually, we aim to embrace block-wise sequential generation facilitate KV cache utilization, yet reject the dilemma that the decoding of subsequent blocks must wait for preceding blocks to be fully denoised. This implies a novel AR-diffusion hybrid paradigm, which, however, cannot be realized through naive teacher-forcing ing information to predict subsequent content. Interestingly, this insight shares the spirit with the diffusion foreing (DF) (Chen et al., 2024a) technique developed specifically for continuous-spat space diffusion models. In this sense, this paper forms an extension of DF to discrete data, giving rise to the discrete diffusion forcing (D2F) method for dLLMM acceleration.
training. This is because teacher-forcing requires complete preced-
Concretely, D2F dLLMs learn to denoise a sequence of token blocks with monotonically increasing mask ratios in parallel. Naturally, preceding blocks can finish before subsequent ones, allowing their KV states to be cached for subsequent computations. Note that we constrain the attention to be block-wise causal to ensure the KV cache remains accurate. For training efficiency, we distill D2F dLLMs from existing bidirectional attention dLLMMs using an asymmetric distillation loss. In inference, we design a pipelined parallel decoding algorithm which enables inter-block parallelism and offers a decent trade-off between inference efficiency and performance (see Figure 2).
Distilled on the Respoke-Stratorn 17k SXM4-40GB GPUs, D2F can accelerate LLaaAA-Intrrut-SB (Nie et al.) 2025) and Dream-Base- 7B (Ye et al. 2025) for over 50x without degradation on mathematical and programming bench- marks, including GSM8K (Cobbe et al. 20021), MATH (Hendrycks et al. 2021). HumanEval (Chen) et al. 2021) and MBPP (Austin et al. 2021b). More importantly, D2F-Dream Dream-BBas-7BB is up to (Grattafiori et al.) 2024) GSM8K and 1.6x faster
17k (Bespoke Labs, 2025) for 12 hours with NVIDIA A
Base-7B
2.5x faster than LLaMA3- HumanEval, establishing the first open-source dLLMs that outrun AR ones. Moreover, D2F can accelerate DiffuCoder (Gong et al. 2025) by up to 11.8x without sacrificing performance.
2

---

Published as a conference paper at ICLR 2026
DLLMS
ditTision
large
language
Monosnniclly Incressing Masks
DLLMS
<tmacke
mnnovative
fusion
masett
difftsiion
4morki
large
<maskk>
<maski
DZF dLLM
Pre-Trained dLLM
Block-wisc
Budinctional Full Attmiion
Figure 3: overview discrete difflusion forcing (D2F). During training. answer divided into blocks with progressively increasing masking ratios. D2F dLLM is specified with a block-wise causal attention mask, and trained to mimic the prediction of pre-trained bidirectional dLLM conditioned on partially denoised preceding tokens. This enables biter-block parallel decoding with KV cache compatibility during inference.
In summary, this work makes the following key contributions:
Discrete Diffusion Forcing (D2F), novel framework adapting diffusion forcing to the diserete domain. Unlike prior work, D2F is specifically designed to unlock inter-block parallelism for inference acceleration.
An asymmetric distillation strategy that efficiently refurbishes pre-trained bidirectional dLLMs into block-wise causal models, avoiding the high cost of training from scratch.
A pipelined parallel decoding algorithm which leverages accurate KV caching to decode future blocks before prior ones fully generated, ensuring high throughput.
The first faster-than-AR dLLMs. Our models achieve up to 2.5x speedup over LLaMA3 (Grattafiori et al. 2024) and Qwen2.5 (Yang al. 2024a) on GSM8K, estab- lishing a new speed benchmark for open-source dLLMs.
RELATED WORK
Diffusion Large Language Models (dLLMs). The landscape of language generation has long been defined by autoregressive models (Achiam et al. 2023 Guo et al. 2025; Liu et al. 2024). These models, renowned for their high-quality output, are inherently limited by a sequential, token- by-token decoding process. To overcome this latency bottleneck, dLLMs have emerged Ye al.
2025 Nie et al.] 2025 Google DeepMind 20255] Inception et al., 2025 Zhu et al. 2025 Gong al. 2025). Instead of generating text sequentially, dI.LMs operate by iteratively denoising fully masked sequence, a process that enables the parallel prediction of all tokens at once. This approach, which draws from non-auutreerrssive principles, utilizes a bidirectional attention mechanism to achieve a more holistic understanding of context. Recent large-scale dLLMs, such as LLaDA (Nic) al. 2025), trained from scratch, and Dream (Ye et al., 2025), initialized from pre-trained AR weights, have demonstrated outstanding performance competitive with leading ARMs, establishing dLLMs as a powerful alternative paradigm for high-quality parallelizable generation.
Acceleration of dLLMs. dLLMs suffer from slower inference than autoregressive models due to incompatibility with standard KV cache and limited parallelization. Existing acceleration methods
fall into two categories. First, paching-based approaches (Liu et al. 2025 Wu et al.) 2025 Ma jet al. 2025) develop approximate schemes to reuse computations for static sequence parts, as stan- dard KV cache is incompatible with bidirectional attention. Second, sampling optimization methods reduce decoding steps through confidence- aware strategies (Wu et al. (2025). auxiliary model guid- ance (Israel al. 2025), adaptive decoding speeds (Wei et al.) 2025). However, these methods achieve limited speedups due to inherent approximations and auxiliary model overhead, often fail-
ing to match the efficiency of comparable AR (Achiam et al. 2023 Touvron al.] 2023a Yang

---

Published as a conference paper at ICLR 2026
et al. 2025 Touvron al. 2023b Grattafiori et al., 2024) models. Our approach fundamentally restructures generation into a block aoockk-uttrrressvv... framework that is compatible with standard KV cache, while enabling prediction of subsequent tokens without completing previous blocks.
AR-diffusion hybrid models. Recent studies have explored incorporating the speed advantages of autoregressive models into ilffussionbbsse frameworks, particularly in tasks such video genera- tion (Yin et al. 2025 Po et a 2025; Sun et al. 2025 Huang et al., (025). For the video generation task, is common to model the temporal dependencies between frames using an autoregressive ap- proach, while applying denoising within each frame independently. This hybrid architecture lever- ages the acceleration benefits of KV cache enabled by the autoregressive modeling paradigm, while preserving the high generation quality of diffusion-basse methods.
3 PRELIMINARY: DIFFUSION LARGE LANGUAGE MODELS (DLLMS)
Diffusion models, originally developed for continuous data, have achieved state-of theotf--t--rttt-- results in
image synthesis (Esser al. 2024 Chen al., 2024b Podell al. 2023 Labs et al.
2025) and
video generation (Peng et al. 2025 Zheng et al., 2024 Yang et al., 2024b Bao et al.
2024). In
recent years, advances the theory of discrete diffusion Austin al. 202 Shi al.
2024 Lou
et al.
2023 Campbell et al. 2022) have facilitated the emergence of large-scale dLLMs (Nie et al.,
2025
Ye et al., 2025) for text generation tasks, which have demonstrated performance competitive with their AR counterparts, while offering the potential for parallel generation.
The majority of successful dLLMs operate under masked diffusion paradigm (Nie et al. This process begins with a forward process, where an original text sequence of T. tokens, yo (vf....,W) v..).. is progressively corrupted into a noisy sequence Yf over a continuous time schedule E (0, 1J. This corruption is routinely achieved by replacing original tokens independently with special [MASK] token. Typically, we can define the conditional distribution as:
2025
(1 1
t, if y vi
"(Y')1") II (ox/l)?) where (vvi1!
if y [MASK]
Consequently, Y represents fully masked sequence. dLLMs are designed to learn a parameterized model Po Po to reverse the forward process, hence enabling denoising from the fully masked sequence y1 to language samples y This formulation allows the model po predict all mask to- kens in the sequence simultaneously at each inference step, forming the basis of dL.LMs' theoretical potential to surpass the AR paradigm for the high-speed, parallel generation.
However, practical implementations of dLLMs face severe inference efficiency bottlenecks. First, the of bidirectional attention mechanisms fundamentally conflicts with KV cache, leading to significant redundant computation across denoising steps. Second, the reliance on a conditional in- dependence assumption for parallel decoding makes it hard to generate interdependent tokens (Song & Zhou 2025), so more iterative steps are required for high-quality outputs. While prior works attempt mitigate these issues, for instance, by introducing block-wise sequential generation to enable KV cache (Arriola et al.) (025) or implementing approximate KV cache (Wu et al.) 2025) Liu et al.] 2025), they fail to simultaneously achieve both precise KV cache and efficient parallel de- coding. Consequently, no open-source dLLMs has yet matched the inference speed of AR models.
4 METHOD
The paper derives discrete diffusion forcing (D2F) to accelerate the inference of dLLMs to faster- than-AR AR regime. Refer to Figures and for an overview of the training and inference of D2F.
4.1 DISCRETE DIFFUSION FORCING
D2F endows dLLMs with two critical capabilities: (1) block-level AR generation, which enables efficient standard KV cache and significantly reduces computational redundancy; and (2) inter- parallel decoding. where the model is trained to predict future blocks from incomplete, partially reconstructed predecessors to maximize the number of decoded tokens per inference step. This naturally gives rise AR-diffusion hybrid modeling paradigm, but naive teacher-forcing training.
4

---

Published as a conference paper at ICLR 2026
MLLMSs new
that
Semi-activated Block Fully-activated Block
GLLMs are new type of that
Cached Block
MLL are new type
dLLMs are new type of
mix ideas from
They
dLLMs
type of Al that mix key ideas from moderm tech
They
LLM
dLL are new type of Al that mix key ideas from modern tech.
They blend
LLM
dLLMs are now type of Al that mix key ideas from modem tech. They blend diffiasion
LLM to
Figure 4: Visualization of the pipelined parallel decoding of D2F dLLMs. As shown, a pipeline of blocks are decoded in parallel. new block is dynamically added when the completion ratio of the block exceeds a threshold Tudi (= 3 here). The new block is initially semi-activated and will transition to a fully-activated state when its predecessor reaches the completion threshold Taet (= here). The fully-activated blocks differ from setivactivated ones in that they would decode multiple tokens in each step more aggressively.
Algorithm Asymmetric Distillation for D2F
Require: Pre-trained dLLM P D2F dLLM pe; block size k; training dataset D.
1: while training do
2: Sample a sequence Y from D.
3: Divide y into N blocks (YB,...., YBy each of size k.
4: Sample a monotonic noise schedule {f1..... .. where t < ... tN.
5: For each E (1,.... N.,, corrupt YR, to Yt using Eq.
6: Predict distributions for each block YD, with:
7: Teacher (global view): Pe- (Y |Y
Y
8: Student (causal view): PY B, Y B.: ..... Y:
9: Update po based the loss LD2F defined in Eq.
as employed in block diffusion (Arriola al. 2025), cannot lead the second capability. To address this, D2F trains the model to perform conditional denoising of the current block based on a partially denoised prefix, enabling more coherent and efficient sequential generation.
Concretely, D2F partitions clean sequence Y into N blocks of size k. Let B; := ( 11 denote the token indices in the i-th block and YB, denote the corresponding subsequence. In the forward process, D2F applies a monotonically increasing noise schedule (41 < l2 < ... < IN) to the N blocks, i..., Y {Y ................................ Y J. Namely, the earlier blocks Y are progressively less masked (i.e., more complete), while later blocks remain increasingly masked (i.e., more uncertain). For the reverse process, D2F trains a f- p-parameterized model to characterize:
po Po
II po PY . Y
Intuitively, the learned model can first complete the decoding of preceding blocks while simultane- ously advancing the denoising of subsequent ones, which effectively enables inter-block parallel de- intra-block attention re-
coding. By preserving a causal attention structure across mains bidirectional-we can cache the KV states of already decoded blocks for exact reuse, thereby reducing redundant computations and improving inference efficiency.
Connection diffusion forcing (Chen et al. 2024a). From high-level perspective, our approach bears a strong conceptual resemblance to diffusion forcing (DF) (Chen et al. 2024a), originally de- veloped for continuous-space diffusion models and notably applied in video generation (Yin al. 2025). Both methods involve predicting the tokens of the next block conditioned on noisy, incom- plete premise, and our framework can viewed as an extension of DF to discrete sequences. Such principled extension motivates our naming of discrete diffusion
4.2
ASYMMETRIC DISTILLATION
Noting that training a dLLM with billions of parameters from scratch can be costly (Ne et al. 2025 Ye et al. 2025), propose to distill D2F dI..M from pre-trained vanilla di L. available in
5

---

Published as a conference paper at ICLR 2026
Algorithm 2 Pipelined Parallel Decoding for D2F Inference
Require: D2F model thresholds Tadd: Tact, Tconf-
1: Initialize Y= (Yn,) with block mask tokens.
2: while generation is not complete do
3: if' the ratio of decoded tokens in YB_, exceeds Tadı and < EOS] > not Y then fully masked block with semi-activated state to Y.
4: Append
5: for the active block YB, in Y do
6: if the ratio of decoded tokens in YB.., exceeds Tact then
7:
YB, to be fully-activated
8: Forward pass of Y with D2F dLLM ps using cached KV
9:
for the active block YB, in Y do
10:
Let Jj record the set of token positions in B; with > Toont prediction confidence if YB, is fully-activated and Jj then Add the token position with the highest confidence to J;
12:
13:
Sample tokens with positions in Jj and remask other positions
Update KV cache for completed blocks
the open-source community. Let pø- denote the standard bidirectional teacher dLLM and pe the student D2F dLLM (0 is initialized as ) The distillation minimizes the following loss
DEL (Yo N ........ 1B) (Y IY ........ YBN
CD2F B CIN
(3)
where DxL. represents the KL divergence aggregated over the mask tokens. As shown, the distillation asymmetric- the teacher Pa- predicts for each block Yg, with a global view of all noisy blocks while the student po learns to approximate using only a causally restricted view. In this way, the mask prediction capabilities of existing dLLMS can be embedded into new D2F dLLM. This objective also connects to Caus (Yin et al. 22025), which distills existing holistic diffusion video generators into streaming ones. As for model architecture, the student differs from the teacher solely in attention masks it uses the block- wise causal attention instead of bidirectional one.
We summarize the whole algorithmic process in Algorithm
4.3 PIPELINED PARALLEL DECODING
introduce pipelined parallel decoding algorithm for the inference of D2F dLLMs. Specifically, we maintain a sliding window of active blocks and dynamically append new fully-masked block when the decoding progress of the last block exceeds threshold Tadd- The dynamic strategy significantly reduces the per-step computational cost compared to maintaining full sequence of massive blocks throughout inference.
As illustrated in Figure 4
Observing that aggressive decoding of newly added block can degrade performance, we incorpo- rate a dual-state decoding mechanism. Concretely, the newly added block initialized semi- activated state to enable conservative parallel decoding, and will become fully activated when its predecessor has finished Tac: of the decoding i.c., sufficient contextual information has been accu- mulated to support aggressive decoding of the latter block. Following Fast-dLLM (Wu et al.) (2255), semi-activated blocks admit tokens with confidence above a threshold Teonts while fully activated blocks additionally enforce the selection of the most confident token when such token exists.
The synergy between the dynamic block management and dual-state mechanism conjoins per-step efficiency and inter-block parallelism. is interesting note that approach also shares concep- tual similarities with prior work in video generation (Teng et al. 2025). More algorithmic details provided in Algorithm and hyperparameter analysis is in Table
EXPERIMENTS
This section details the experimental setup and presents the results of D2F dl Ms..M.

---

Published as a conference paper at ICLR 2026
Test Set
Method
TPS 1
Latency (s) & Gen. Length Score
LLaDA-Instrute
7.2 (1.0x)
32.3 (1.0x)
231
77.4
dLLM-Cache
20.1 (2.8x)
11.5 (2.8x)
231
77.5
GSM8K
Fast-dLLM (Prefix-Cache)
33.3 (4.6x)
7.0 (4.6x)
232
77.8
4-shot
Fast-dLLM (Dual-Cache)
35.2 (4.9x)
6.6 (4.9x)
232
78.9
D2F-LLaDA
52.5 (7.3x)
2.8 (11.5x
144
77.3
LLaDA-Instruct
0.9 (1.0x)
71.4 (1.0x)
65
39.0
ULLM-Cache
2.3 (2.6x)
28.3 (2.5x)
37.0
MBPP 3-shot
Fast-dLLM (Prefix-Cache)
13.0 (14.4x)
4.9 (14.6x)
64
37.6
Fast-dLLM (Dual-Cache)
15.3 (17.0x)
3.8 (18.8x)
58
36.4
D2F-LLaDA
47.6 (52.9x)
1.4 (51.0x)
68
38.0
LLaDA-Instruct
2.8 (1.0x)
38.8 (1.0x)
107
36.0
dLLM- MaCahhh
(1.6x)
23.3 (1.7x)
104
39.0
HumanEval
Fast-dLLM (Prefix-Cache)
13.7 (4.9x)
7.4 (5.2x)
102
38.4
0-shot
Fast-dLLM (Dual-Cache)
19.2 (6.9x)
5.2 (7.5x)
100
35.4
D2F-LLaDA
81.6 (29.1x)
1.6 (24.3x)
133
40.2
LLaDA-Instrut
21.1 (1.0x)
11.5 (1.0x)
243
23.7
ULLM-Cache
26.9 (1.3x)
9.1 (1.3x)
246
23.2
Math
Fast-dL (Prefix-Cache)
47.7 (2.3x)
5.2 (2.2x)
246
22.4
4-shot
Fast-dLLM (Dual-Cache)
42.5 (2.0x)
5.8 (2.0x)
246
22.4
D2F-LLaDA
90.2 (4.3x)
4.3 (2.7x)
384
29.1
Table 1: Performance comparison of various acceleration methods on LLaDA-Instruct-SB. Speedup ratios are shown in (green). All baseline methods use the default sampling configuration from the original L.LaA implementation. See Appendix D for detailed hyperparameters.
5.1 EXPERIMENT SETTINGS
We perform evaluation on two representative dLLMs: LLaDA-Instruct-SB (Nie et al. 2025) and
Dream-Base-78 (Ye et al. 2025 backbones shared by prior acceleration work for fair compar- ison. For our distillation- basiattiooo---dddddd training, we utilize a dataset derived from the Bespoke-Strane Saroooke-Stttts-111
use two publicly available collections sourced from the Hugging Face Hub. where a third party had previously generated responses to the problems from Bespoke- -trrktos--ttt----t 17k using the Qwen2.5-7B model (Yang et al. 2024a). These collections are pre-filtered to maximum length of 600 tokens. The q_proj, v v_pro k_proje and o_pro mod- ules of D2F dLLMs are tuned with LoRA (Hu et al., 2022) technique, using the rank 32, scaling factor of 32, and the dropout rate of 1. During training, We truncate or pad all sequences to a final length 1024 tokens, and employ block size of 16. We applied block-wise monotonically increasing mask ratios, with maximum threshold of 0.7 and minimum threshold of 0.2, inspired by Block Diffusion (Ariila et al. 2025). During inference, unless otherwise specified, Tevel is set to 0.9, Tada is optimizer with a constant learning rate of 10-5, All training and inference are conducted on a setup comprising 8 NVIDIA A 1000SXM4440GB GPUs. Detailed hyperparameter configurations for in- ference on each benchmark are provided in the Appendix D In addition, we conduct experiments on DiffuCoder (Gong al. 2025). and report the results in Appendix
benchmark (Bespoke Labs 2025). Specifically,
0.1, and Taxi set to 0.95. The models are trained for 12 hours using Adam W
5.2
MAIN RESULTS
Benchmarks. Following established conventions, performance evaluation of D2F is conducted across mathematical reasoning and code generation benchmarks, including GSM8K (Cobbe et al, CSSMK-Coff oo-ain--f-Thuughh reasoning variant of GSM8K), Math (Hendrycks et al. HumanEval (Chen et al. 2021), and MBPP (Austin et al. 2021b).
2021
2021
Baselines. Comprehensive comparisons
established between D2F and state-of-the-an accelera- tion strategies, including Fast-dLLM (Wu et al. (025) and dLLM-Cache (Liu et al. (225)., imple- mented on A.L.DA-nnstuccttt (Nie al. 2025) and Dream-Base-7Bt (Ye al. 2025 architectures.

---

Published as a conference paper at ICLR 2026
Test Set
Method
TPS 1
Latency (s) 4 Gen. Length Score
Dream-Bast
9.5 (1.0x)
26.8 (1.0x)
255
75.0
dlL.M-achhe
26.0 (2.7x)
9.8 (2.7x)
255
72.0
GSM8K-CoT
Fast-dLLM (Prefix-Cache)
50.3 (5.3x)
5.1 (5.3x)
255
76.6
8-shot
Fast-dLLM (Dual-Cache)
49.8 (5.2x)
5.1 (5.3x)
255
75.0
D2F-Dream
91.2 (9.6x)
2.8 (9.6x)
256
77.6
Dream-Base
10.4 (1.0x)
24.6 (1.0x)
256
56.2
dLLM-Cache
25.5 (2.5x)
10.0 (2.5x)
256
52.6
MBPP 3-shot
Fast-dLLM (Prefix-Cache)
71.6 (6.9x)
3.6 (6.8x)
256
56.4
Fast-dLLM (Dual-Cache)
73.2 (7.0x)
3.5 (7.0x)
256
51.0
D2F-Dream
105 (10.1x)
2.3 (10.7x)
240
55.2
Dream-Base
20.2 (1.0x)
12.6 (1.0x)
255
54.3
dLLM-Cache
23.2 (1.1x)
11.0 (1.1x)
255
55,5
HumanEval
Fast-dLLM (Prefix-Cache)
62.4 (3.1x)
4.1 (3.1x)
255
54.3
0-shot
Fast-dLLM (Dual-Cache)
60.0 (3.0x)
4.3 (2.9x)
255
53.0
D2F-Dream
73.2 (3.6x)
3.1 (4.1x)
227
54.3
Dream-Bast
9.9 (1.0x)
25.8 (1.0x)
256
35.8
dLLM-Cache
12.7 (1.3x)
20.2 (1.3x)
256
34.5
Math
Fast-dLLLM (Prefix-Cache)
65.6 (6.6x)
3.9 (6.6x)
256
37.6
4-shot
Fast-dLLMM (Dual-Cache)
67.0 (6.8x)
3.8 (6.8x)
256
37.1
D2F-Dream
98.8 (10.0x) 2.6 (9.9x)
256
35.4
Table Performance comparison of various acceleration methods on Dream-Base-7B. Speedup ratios relative to the baseline are shown in (green). The max generation length is set to 256. See Appendix D for detailed hyperparameters.
Additional benchmarking against leading auto-regressive (AR) LLMs of comparable scale, specif- ically I-MtA3--stttut-8E (Grattafioni et al. 2024) and Qwen2.5 --ase--BB (Yang et al. (204a) demonstrates the efficacy of D2F in achieving faster-than-AR inference speeds.
Quantitative Results. As shown in Figure D D2F represents the first open-source dLLMs to sur-
pass faate-of-the-an AR LLMs in inference speed. The maximum generation length to 512 for all methods to ensure fairness. Concretely, kens/s on GSM8K. This constitutes a 2.5x speedup over ILLaMAA--Inttrur--BBB (48.0 tokens/s) and 2 speedup over Qwen2.5- (52.7 tokens/s).
Trr-Dream-Base--1 achieves a throughput of 1 19.9 to-
D2F also significantly accelerates baseline dLLMs while maintaining equivalent average perfor- mance. As shown in Table D2F-LLaDA-Instrut-te achieves 52.9x speedup (47.6 tokens/s vs. baseline 0.9 tokens/s) with minimal performance difference (38.0 vs. baseline 39.0), and con- sistently outperforms prior dLLM acceleration techniques. This owes to D2F's early termination via <BOS> detection on Instruct models, avoiding unnecessary generation. On MATH, it reaches 90.2 tokens/s, 2.1 x speedup over Last-dI.LM's Dual-Cache (42.5 tokens/s). For fairness, the baseline hyperparameters follow the settings in (Nie et al., 2025). Similarly, as shown in Table [] D2F-Dream-Base-1 attains 91.2 tokens/s on GSMSK-CoT, corresponding
9.6x speedup over Dream-Base-7B (9.5 tokens/s) with slight performance improvement, and a 1.8 x speedup over Fast- dLLM (49.8 tokens/s). Since Dream- Base struggles to generate the stop token correctly, we set unified maximum length of 256 for all methods in this comparison, following the setting in dLLM- cache (Liu et al. 2025). We vary max lengths (512 vs. 256) to strictly align with respective baseline protocols for fair comparison. These results demonstrate that D2F not only surpasses existing accel- eration methods but also enables dLLMSs to exceed AR LLMs throughpur, substantially enhancing their practical applicability. More detailed hyperparameter settings are provided in Appendix
5.3 ABLATIONS AND ANALYSIS
This section presents ablation studies to dissect the contributions of key components within D2F. All ablation experiments are conducted the Dret---ra-BBeee model unless otherwise specified.
8

---

Published as a conference paper at ICLR 2026
120
76.0
120
50
Score
77.5
170.
135-
53
130
52
250
55
49
10
48
32
64 50
96
32
48
64
60
Figure 5: Ablation study the block size during inference. The block size for inference tested with integer multiples of the training block size (16). All experiments were conducted with maximum length of 512, Teonf = 0.9, Tase = 0.1, and Tast 0.95.
Benchmark
Tact
Model
TPS f Score
Tact
Tconf
Tadd
TPS 1
Score f
random
147.2
49.6
0.95
0.95
105.2
76.9
D2F
171.2
54.6
0.7
107.2
77.2
MBPP 3-shot
0.95
random D2F
148.5
48.6
0.5
106.3
77.3
0.90
175.5
53.2
0.1
104.0
77.7
random D2F
150.6
47.6
0.90
124.5
74.7
0.85
177.5
52.6
0.7
126.2
75.7
0.90
0.5
124.7
76.2
0.95
random D2F
114.5
77.1
0.1
122.1
76.4
119.9
77.2
GSM8k
0.85
136.8
72.6
4-shot
0.90
random D2F
116.7
76.5
123.5
76.4
0.7
139.0
74.2
0.85
0.5
138.5
74.0
random 118.8
75.9
0.85
0.1
135.4
75.0
D2F
124.8
75.4
Table 3: Ablation of inference hyperparameters on GSM8K-4-shot.
Table 4: Ablation of the noise scheduling on MBPP-3-shot and GSM8k-4-shot. Comparison of D2F and random noise schedules.
performance trade-off for parruuhpuut-peffrmmance trade-off. Figure illustrates the employs fixed Tadd 0.1 and different methods on GSM8K and MBPP.
block size of 32, with unified thresholds Tact = Toonf being varied. Results demonstrate that D2F- Dream-Base-7B achieves significantly higher efficiency than baselines. For instance, on GSM8K, Ser-Dream-Base--i attains 150.9 tokens/sec with score of 7 71.2, achieving 3.1x x the throughput of LLaMA3-Instruct- (48.0 tokens/sec) while exceeding its score (70.1). In contrast, Dream-Base 7B exhibits substantial performance degradation at higher throughput: reducing sampling steps from 512 128 causes its GSM8K score drop from 71.4 42.8. This demonstrates the superior capability of D2F-Dream-Basse Ba in maintaining performance during accelerated inference.
Effect of block size for inference. Figure 5 demonstrates the influence of block size on infer- ence performance and speed. Overall, increasing block size consistently reduces throughput while yielding an initial performance improvement followed by deterioration. Optimal block size selec- tion moderately reduces throughput but maximizes performance-fee for example, block size of 48 achieves the peak GSM8K score 77.5, surpassing smaller block sizes such as 16.
Ablation on inference hyperparameters. Table studies Tconf, Tacts and Tada- When Tad Tacts new blocks are immediately activated (single-state). Our dual-state design (Fass < Tax)) introduces a conservative initial state and consistently performs better. For example, with Tat 0.85, lowering Tadd from 0.85 to 0.7 increases the score from 72.6 to 74.2 and throughput from 136.8 to 139.0 TPS; further reduction yields additional gains with only slight throughput loss.

---

Published as a conference paper at ICLR 2026
CONCLUSION
In this work, we introduce Discrete Diffusion Forcing (D2F), a novel training paradigm for ds.Ms. D2F employs a generation scheme that conditions on partially predicted tokens from previous blocks to predict the next block, thereby supporting KV cache and enabling parallel generation across mul- tiple blocks, resulting in significantly faster inference. Empirically, extensive experiments demon- strate that D2F achieves the milestone of being the first dLLM to support faster-than- inference.
ETHICS STATEMENT
This work does not involve human subjects, animal experiments, or sensitive data. Therefore, it does raise any ethical concerns.
REPRODUCIBILITY STATEMENT
To promote reproducibility, plan release the source code for both training and inference the proposed method in the future. The released code and accompanying instructions will enable other researchers to reproduce our results and build upon this work.
LEGEONTSS
This work was supported by Shanghai Key Technology R&D Program "New Generation of Informa- tion Technology" (No. 25311103700,, NSF of China (Nos. 62306176, 92470118), Natural Science Foundation of Shanghai (No. 232R 1428700), CCF-ALIMAMA TECH Kangaroo Fund (NO. CCF- ALIMAMA OF 2025010), and Ant Group.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autore- gressive and diffusion language models. arXiv preprint arXiv:2503.09573 2025.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 17981-17993 2021a.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry. Quoc Le, al. Program synthesis with large language models. arXiv preprint arXiv:2108.0775 077332, 2021b.
Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to- video generator with diffusion models. arXiv preprint arXiv:2405.04233, 2024.
Bespoke Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning dis- tillation. https://www.bet http::/wwwwbspoollbbbb..ii.a wwwps///////gbo/oooeeeoooo streeesttttttttt---eeeeee whw.eeumraassnaae--eetttieeeeeee 2025. Accessed: 2025-01-22.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 55:28266-282799 2022.
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-24125, 2024a.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2024b.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.03377 0333744 2021.
Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, et al. Sdar: A synergistic for scalable sequence generation. arXiv preprint 1exxii:2510.06.:00 2025.
paradigm
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10, 14168, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu. Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration. arXiv preprint arXiv:2506.2066 20639,, 2025.
Google DeepMind.
Gemini Diffusson. 2025. Accessed: 2025-05-24.
gemi ni di fusion

---

Published as a conference paper at ICLR 2026
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXi::240.221783, 2024.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprini arXiv:2501. 12948, 2025.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.035: 2021.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)3, 2022.
Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprint arXiv:2505.21400 2025.
Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J Yang. Jiaheng Liu, Chenchen Zhang, Linzheng Chai, et al. Opencoder: The open cookbook for top-tier code large language models. arXiv preprint arXiv:2411.04900 2024.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. arXiv preprint arXiv:2506. 08009, 2025.
Labs Inception, Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang. Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, et al. Mercury: Ultra-fast language models based on diffusion. arXiv preprint arXiv:2506.17200 2025.
Daniel Isracl, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion Ilms via adaptive parallel decoding. arXiv preprint arXiv:2506 00413, 2025.
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL htttt:: /ttp///aavvv//aa///a//eee/aaaa// org abs /20../506...0 15742
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang. Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseck-v3 technical report. arXiv preprint arXiv:2412.19437 2024.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chat- gpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36:21558-214 21572, 2023.
Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.06260 2025.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310. 16834, 2023.
Xinyin Ma, Rumpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models. arXiv preprint 2577: 2025.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09999. 2025.
Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang. Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, et al. Open-sora 2.0: Training a commercial-level video generation model in 200 k. arXiv preprint arXiv:2503.096-2 2025.
12

---

Published as a conference paper at ICLR 2026
Ryan Po, Yotam Nitzan, Richard Zhang, Berlin Chen, Tri Dao, Eli Shechtman, Gordon Wetzstein, and Xun Huang. Long-context state-space video world models. arXiv preprint arXiv:2555.20171, 2025.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167, 2024.
Jiaming Song and Linqi Zhou. Ideas in inference-time scaling can benefit generative pre-training algorithms. arXiv preprint arXiv:2503.07154 2025.
Yuxuan Song, Zheng Zhang. Cheng Luo. Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuchang Yang, Hongli Yu, Xingwei Qu, Yuwei Fu, Jing Su, Ge Zhang, Wenhao Huang, Mingxuan Wang, Lin Yan, Xiaoying Jia, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang. Yonghui Wu, and Hao Zhou. Seed diffusion: A large-scale diffusion language model with high-speed inference, 2025.
Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, SiYu Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with auto-regressive diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7364- 7373, 2025.
Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, al. Magi-1: Autoregressive video generation at scale. arXiv preprint 2rXiv:2505.132111 2025.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 1397,,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykow, Soumya Batra, Prajjwal Bhargava. Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint (xXiv:2307.09288 2023b.
Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Dongrui Liu, and Linfeng Zhang. Accelerating dif- fusion large language models with slowfast: The three golden principles. arXiv preprint arXiv:2506. 10848, 2025.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao. Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training- free acceleration of diffusion Ilm by enabling kv cache and parallel decoding. arXiv preprint arXiv:2505.2240 22618, 2025.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng. Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. 2024a.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 0083028005009388 2025.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06011 2024b.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. RRI.ttps:
13

---

Published as a conference paper at ICLR 2026
Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang, From slow bidirectional to fast autoregressive video diffusion models. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 22963- 22974, 2025.
Zangwei Zheng, Xiangyu Peng. Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20400 2024.
Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. Llada 1.5: Variance-reduced preference optimization for large language diffusion models, arXiv preprint arXiv:2505 19223, 2025.
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In this work, large language models (LLMs) were employed solely as writing assistants. Their role was limited to polishing the language, improving clarity, and refining the overall readability of the manuscript. No part of the conceptual development, experimental design, data analysis, or interpretation of results relied on LLMs.
B ABLATION STUDIES ON D2F COMPONENTS
To isolate the performance gains from different components of our D2F framework, we conduct ablation studies on both the D2F-LLaDA 3 and D2F-Dream (Table models. We compare two configurations:
This configuration utilizes the block-wise causal attention mechanism to en- able standard KV cache but does not employ the parallel, asynchronous block generation pipeline. Generation proceeds serially, one block at a time.
Cache-only:
Cache + Para: This the full D2F method, combining KV cache with our pipelined parallel decoding algorithm.
The results clearly demonstrate that while enabling KV cache (Cache-only) provides a substantial speedup, the addition of parallel decoding pipeline (Cache Para) further accelerates inference by a significant margin (c.g., from 2.4x to 7.3x on GSM8K for LLaDA), highlighting the efficacy of the asynchronous generation strategy.
Advantages of D2F training over fully random masks. As shown in Table 4 the D2F training strategy significantly outperforms baseline utilizing random noise schedules per block. On MBPP, D2F demonstrates comprehensive superiority, achieving a 5.0-point score improvement (54.6 versus 49.6) and 24.0 TPS gain Taa 0.95. These results substantiate that the structured progressive noising of D2F delivers more effective training objective than fully random masking.
C CONTROL EXPERIMENT FOR DATA CONTRIBUTION
To prove that our performance gains stem from the D2F framework and not the training data, we conducted control experiment. We created a control model, Dream-Base" by directly fine-tuning the original Dream-Base on the same distillation dataset. For a fair comparison, this fine-tuning used LoRA with the exact same parameters our D2F training.
The results are in Table While Dream- * shows minor score improvements on some tasks, its inference speed drops significantlly This speed degradation due the computational overhead of the added LoRA layers. This control experiment confirms that our D2F methodology, rather than the data, is the key driver of the exceptional inference acceleration.
D HYPERPARAMETER DETAILS
Table details the specific hyperparameter configurations used for both the baseline models and our D2F models across all evaluated benchmarks. Bascline settings are adopted from their respective original works to ensure fair comparison. For D2F models, specify the maximum generation length, the inference block size, and the key thresholds for our pipelined parallel decoding algorithm: the token addition threshold (Foda), the block activation threshold (Tacz), and the token confirmation confidence (Teonf).
E ADDITIONAL EXPERIMENTAL RESULTS
We further train a D2F model based on DiffuCoder-Instrrue (Gong et al., 2025). Specifically, we selected 30K samples from the SFT-stage training data of OpenCoder (Huang al.) 2024) and follow the same training configuration as D2F-Dream, training for 28,000 iterations. The result- ing model, D2F-DiffuCoder, evaluated HumanEval, HumanEval+, and MBPP+ (Liu et al.
15

---

Published as a conference paper at ICLR 2026
Test Set
Configuration
TPS f
Latency (8)
Gen. Length Score 1
GSM8K
Cache-only
17.5 (2.4x)
8.3 (3.9x)
145
78.1
4-shot
Cache Para
52.5 (7.3x)
2.8 (11.5x)
144
77.3
MBPP
Cache-only
18.1 (20.1x)
3.8 (18,8x)
69
37.6
3-shot
Cache Para
47.6 (52.9x)
1,4 (51.0x)
68
38.0
HlumanEval
Cache-only
28.0 (10.0x)
4.8 (8.1x)
135
40.2
0-shot
Cache Para
81.6 (29.1x)
1.6 (24.3x)
133
40.2
Math
Cache-only
30.6 (1.5x)
12.6 (0.9x)
385
29.6
4-shot
Cache Para
90.2 (4.3x)
4.3 (2.7x)
384
29.1
Table 5: Ablation study of our proposed method (DZF-LLaDA) on the LLaDA-Instruct model. Performance ratios relative the -nstr--nsttuct baseline (from Table [ are shown in (bright
green).
Test Set
Method
TPS f
Latency (8)
Gen. Length
Score f
GSM8K-CoT
Dream-Base
9.5 (1.0x)
26.8 (1.0x)
255
75.0
8-shot
Dream-Basee
6.7 (0.7x)
29.9 (0.9x)
199
77.8
MBPP
Dream-Basn
10.4 (1.0x)
24.6 (1.0x)
256
56.2
3-shot
Dream-Base B
4.2 (0.4x)
27.4 (0.9x)
114
57.4
HumanEval
Dream-Bast
20,2 (1.0x)
12.6 (1.0x)
255
54.3
0-shot
8.8 (0.4x)
14.2 (0.9x)
125
51.8
Math
Dream -Base
9.9 (1.0x)
25.8 (1.0x)
256
35.8
4-shot
Dream- Dream-Base*
5.4 (0.5x)
28.6 (0.9x)
154
33.4
Table Comparison between the baseline Dream-Base model and fine-tuned version, Dream- Base*. The fine-tuned version, highlighted with a gray background, is obtained by directly fine- tuning on the distillation data. Performance ratios relative to the bascline are shown in (bright green), and the highest score in each test set is marked in bold.
Test Set
Configuration TPS
Latency (s)
Gen. Length Score t
GSMSK-CoT
Cache-only
40.7 (4.3x)
6.3 (4.3x)
256
77.8
8-shot
Cache Para
91.2 (9.6x)
2.8 (9.6x)
256
77.6
MBPP
Cache-only
40.5 (3.9x)
5.9 (4.2x)
240
54.2
3-shot
Cache Para
105 (10.1x)
2.3 (10.7x)
240
55.2
HumanEval
Cache-only
43,6 (2.2x)
5.1 (2.5x)
222
53.7
0-shot
Cache Para
73.2 (3.6x)
3.1 (4.1x)
227
54.3
Math
Cache-only
39.6 (4.0x)
6,5 (4.0x)
256
35.9
4-shot
Cache Para
98.8 (10.0x)
2.6 (9.9x)
256
35.4
Table Ablation study of our proposed method (D2F-Dream) on the Dream-Base model. Perfor- mance ratios relative to the Dream-Bas Base baseline (from Table 2 shown in (bright green).
16

---

Published as a conference paper at ICLR 2026
Benchmark
Configuration
length
block_size
Tadd
Tact
Teon.f
Method: D2F-LLaDA
Baseline
256
8
GSM8K
D2F
512
64
0.7
0.95
0.9
Baseline
512
32
MBPP
D2F
512
32
0.9
0.95
0.9
Baseline
512
32
HumanEval
D2F
512
32
0.1
0.95
0.9
Baseline
256
256
MATH
D2F
512
32
0.1
0.95
0.9
Method: D2F-Dream
Baseline
256
GSM8K-COT
D2F
256
32
0.1
0.95
0.9
Baseline
256
MBPP
D2F
256
48
0.1
0.95
0.9
Baseline
256
HvmanEval
D2F
256
32
0.9
0.95
0.95
Baseline
256
MATH
D2F
256
64
0.1
0.95
0.9
Table Sampling hyperparameters for the experiments Table and Table Baseline settings are adopted from prior work. For D2F, the length parameter only sets a maximum generation limit without affecting the sampling distribution due its block-wise nature. Notably, for the Dream- based model, we match D2F's maximum length to the baseline's, as the base model often fails to generate proper termination tokens.
2023), achieving up to a 10x speedup while maintaining performance comparable to the original RiffuCoder-Instruct The results are shown in Table
Test Set
Method
TPS 1
Score 1
HumanEval+
DiffuuCoderInntrrut
11.05 (1.0x)
65.2
0-shot
D2F-DiffuCoder
82.06 (7.4x)
65.9
MBPP+
DiftuCooderInstrruct
3.94 (1.0x)
61.9
0-shot
D2F-DiffuCoder
46.53 (11.8x)
61.4
HumanEval
DiffuuCoder-nstrruct
11.05 (1.0x)
72.0
0-shot
D2F- DifffffuCoddr
82.06 (7.4x)
71.3
Table 9: Comparison between the baseline DiffuCCoder-Istruuct model and D2F-DiffuCoded (Since the HumanEval+ benchmark includes HumanEval, for convenience report the average speed HumanEval+ for both the HumanEval and HumanEval+ tasks.)
F PERFORMANCE VARIANCE AND ERROR BARS
Unlike autoregressive models or standard diffusion sampling which involve stochasticity, our D2F inference uses a greedy, confidence-bart based sampling algorithm (with temperature set to 0) that is deterministic. Consequently. D2F results do not exhibit variance across runs.
However, to address concerns regarding the statistical significance of our baselines, quantify the variance of the stochastic Daselines-Vant Vanilla dLLM (Dream-Bant Base) and dLLM-Cache (Liu et al. 2025). We conducted 3 runs GSM8K (4-shot, 256 length) using different random seeds. As
17

---

Published as a conference paper at ICLR 2026
shown in Table 10 the performance variation is minimal (standard deviation se 0.3), confirming that the improvements achieved by D2F are significant and not due to random fluctuations.
Seed
Vanilla dLLM (Score)
dLLM-Cache (Score)
72.5
72.3
72.6
72.7
72.0
72.1
Mean + Std
72.37 + 0.32
72.37 + 0.31
Table 10: Performance variance of stochastic baselines on GSM8K (4-shot). The minimal standard deviation confirms the stability of the baseline measurements.
G COMPARISON WITH BLOCK DIFFUSION (SDAR)
To further justify the necessity and advantages our D2F framework, we compare it against SDAR (Cheng al. 2025), state-of-the-ard block diffusion model trained from scratch. We eval- uate SDAR-4B-blocd
on GSM8K (4-shot, 256 length) using the same block size as D2F.
As shown in Table II while SDAR achieves competitive scores, its inference speed is significantly limited. Even with reduced confidence thresholds, SDAR maxes out at 46.7 TPS. contrast, D2F (as shown in Figure and Table 7 achieves significantly higher throughput (up 90+ TPS) with comparable accuracy.
Model
Confidence
TPS
Score
0,9
29.7
79.2
0.8
35.9
78.6
SDAR-4B-block32
0.7
39.6
76.8
0.6
46.7
72.3
Table 11:
Performance trade-off of SDAR-4B on GSM8K. Despite being specialized block diffu- sion model, SDAR lacks the inter-block parallelism of D2F, resulting in lower throughput.
Advantages of D2F over Block Diffusion models like SDAR:
1. Higher Inference Speed via Parallelism:
D2F's pipelined parallel decoding removes the sequential bottleneck between blocks. In SDAR, block (++ must wait for block / to finish; in D2F, future blocks start decoding early, leading significantly higher TPS.
2. Training Efficiency via Distillation: SDAR requires expensive pre-training from scratch on massive datasets (trillions of tokens). In stark contrast, D2F adapts existing are-trained dLLMs (e.g., Dream-7B) using only ~12 hours of fine-tuning on a small dataset (17k samples). This makes D2F far more accessible and cost-effective solution for refurbishing existing models into efficient block-wiss generators.
18


Reviews:
[{"review_id": "P8P0f78oVF", "reviewer": "Reviewer_au9N", "summary": "The paper proposes Discrete Diffusion Forcing (D2F). At inference time, the framework combines the strengths of diffusion LLMs (dLLMs) and autoregressive LLMs (AR LLMs). It adopts an AR‑like blockwise sequential generation scheme to exploit the KV cache while preserving parallel decoding across blocks in dLLMs, so later tokens can be predicted without waiting for the previous block to finish. This hybrid of AR and diffusion substantially accelerates dLLM inference. During training, the method performs asymmetric distillation from a pretrained dLLM teacher with standard bidirectional attention to a student that has only a causally restricted view. Experiments on multiple benchmarks show that D2F significantly speeds up dLLM inference beyond AR LLMs while maintaining accuracy.", "questions": "1) Can the main experiments include error bars to establish statistical significance?\n2) Tables 1 and 2 show large variation in speedup across different benchmarks and different dLLM base models for D2F, for example 52.9× versus 4.3×. What explains this variance? Does it indicate sensitivity to model architecture or to the data type used at inference time?\n3) How sensitive is D2F to generation length?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The work introduces an original hybrid paradigm that enables KV‑cache friendly, AR‑style generation while retaining dLLMs’ cross‑block parallelism, and it provides a tailored asymmetric distillation procedure to train the model.\n2) According to the paper, D2F yields the first open‑source dLLMs that surpass state‑of‑the‑art AR LLMs in inference speed, and achieves more than 10× speedup on some benchmarks over dLLM baselines without D2F.\n3) The experimental study is comprehensive, with comparisons against strong AR LLM and dLLM baselines on common benchmarks, along with ablations on key hyperparameters.", "weaknesses": "1) Training relies on a pretrained dLLM as the teacher, which may limit scaling to stronger future D2F variants. In addition, D2F does not accelerate training, so the compute cost remains substantial.\n2) The main figure (Figure 3) is information‑sparse; a clearer depiction of the asymmetric distillation would improve readability. Table 3 could be half‑width, since the current layout leaves excessive white space.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QbTznkFmqv", "reviewer": "Reviewer_QtKg", "summary": "This paper proposes discrete diffusion forcing that critically enables decoding of the new block to start even before the previous blocks are fully decoded. This unlocks the parallelization not only within a single block but also across blocks. By further inheriting KV-cache brought by block decoding, D2F achieves promising speedup without sacrificing much sample quality. Experiments on math and coding benchmarks over two models demonstrate the effectiveness of the proposed approach in terms of achieving more speedup while maintaining the accuracy.", "questions": "1. It would be more helpful to elucidate the speedup by showing the accuracy-speedup curves for the proposed approach and the baselines, instead of only showing one single point on the curve (the numbers in the table). i.e, plot the curve of Fast-dLLM and dLLM-Cache, if applicable.\n\n2. How is the accuracy-efficiency tradeoff of Block Diffusion [1]? This can be investigated by doing block diffusion distillation and sweep over different numbers of decoding steps at inference time. This is important for ablating the necessary of diffusion forcing distillation over naive block diffusion distillation.\n\n\n[1] Arriola et al. Block diffusion: Interpolating between autoregressive and diffusion language models.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper tackles a critical problem in discrete diffusion model: inference acceleration. D2F unlocks parallelization even across different blocks, which is a very important feature to significantly enhance the speed.\n\n2. The method includes a distillation training for D2F and a customized inference procedure. The distillation helps mitigate the bias in block diffusion that requires the previous blocks to be fully decoded.\n\n3. The presentation is clear and the proposed approach is clean and very easy to follow.", "weaknesses": "1. The technical novelty is somewhat bounded since the work is an adaptation of previous diffusion forcing literature (especially video diffusion) to discrete diffusion models. The concern is not significant though, given the promising empirical performance.\n\n2. Several experiment results that are key to compare the accuracy-efficiency frontier and understand the design of the proposed approach are missing. See Q1 and Q2 for more details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4Byzn0K4P9", "reviewer": "Reviewer_WETF", "summary": "This paper presents \"Discrete Diffusion Forcing\" (D2F), a framework to make diffusion LLMs (dLLMs) faster than autoregressive (AR) LLMs. D2F reformulates the dLLM as an AR-diffusion hybrid, which solves the core problem of dLLMs not being able to use a standard KV cache. The method uses asymmetric distillation to train a block-wise causal student dLLM to mimic a bidirectional teacher, using a monotonically increasing noise schedule. This enables a \"pipelined parallel decoding\" algorithm during inference, where future blocks can be processed before previous blocks are fully finished, all while using an exact KV cache. The authors report significant speedups (up to 2.5x vs. LLaMA3, 50x vs. vanilla dLLMs) with comparable performance.", "questions": "1. Could the authors please clarify the performance of the LLaMA3-Instruct-8B baseline on GSM8K? Figure 2 implies a score of ~77, but the text in Section 5.3 states 70.1. Which number should be used for comparison, and why is there a discrepancy?\n\n2. Similarly, could the authors provide a single, clear {TPS, Score} pair for the D2F-Dream-Base-7B model on GSM8K that represents the main claim? The numbers in Figure 1, Figure 2, and Section 5.3 all seem to refer to different operating points, making a direct comparison difficult.\n\n3. The inference pipeline (Alg. 2) seems complex. How sensitive is the model's performance (both speed and quality) to the choice of $\\tau_{add}$, $\\tau_{act}$, and $\\tau_{conf}$? Table 3 suggests they interact, but is there a \"default\" set of parameters that works well across most tasks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Significant Milestone: The paper achieves \"faster-than-AR\" inference with an open-source dLLM, a significant milestone. The reported speedups (2.5x vs. LLaMA3, 50x vs. LLaDA) are extremely impressive.\n\n2. Effective Training Strategy: The \"asymmetric distillation\" with a \"monotonically increasing mask schedule\" is a clever adaptation of Diffusion Forcing to the discrete domain. It effectively trains the model to predict from an incomplete prefix, enabling the parallel pipeline.\n\n3. Solves the KV Cache Problem: The method reframes the dLLM as a block-wise causal model, allowing the use of a standard, exact KV cache. This is a fundamental improvement over prior work that relied on approximate caching.\n\n4. Strong Ablation Studies: The ablations clearly validate the design, showing that the parallel pipeline adds significant speed over caching alone, the structured noise schedule is critical, and the gains are not just an artifact of the data or tuning.", "weaknesses": "1. Inconsistent/Confusing Performance Claims: The paper's headline claim of \"faster-than-AR\" performance is made confusing by seemingly inconsistent numbers across the text and figures. This makes the exact performance trade-off difficult to assess.\n\n- LLaMA3 Baseline: In Figure 2, the LLaMA3-Instruct-8B baseline (star) is plotted with a GSM8K score of ~77. However, the text in Section 5.3 states its score is 70.1. This is a significant discrepancy.\n\n- D2F Performance: The paper reports multiple different performance points for its own model. Figure 1 claims 119.9 TPS (which corresponds to a score of ~75 in Figure 2). But Section 5.3 reports a different operating point of 150.9 TPS and a 71.2 score.\n\n- This inconsistency makes it difficult to definitively conclude if D2F is \"faster and comparable,\" \"faster and slightly worse,\" or \"much faster and slightly better\" than its key AR competitor.\n\n2. Inference Complexity: The proposed \"pipelined parallel decoding\" (Algorithm 2) introduces significant new complexity to the inference process. It requires careful tuning of three new, interacting hyperparameters ($\\tau_{add}$, $\\tau_{act}$, $\\tau_{conf}$) in addition to the block size, which could be a barrier to practical adoption and tuning compared to the simplicity of standard AR decoding.\n\n3. Incremental Novelty: The paper is transparent that its core idea is an \"extension of DF (Diffusion Forcing)\" (Chen et al., 2024a) and \"connects to CausVid\" (Yin et al., 2025). While the adaptation to the discrete domain is novel and the engineering is strong, the fundamental concept of forcing a model to predict from a noisy prefix via distillation is not entirely new. This makes the contribution more of a highly successful and non-trivial adaptation than a completely new paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5R04nyCaTI", "reviewer": "Reviewer_gxgg", "summary": "This paper proposes Discrete Diffusion Forcing (D2F), a framework for faster text generation with diffusion-based large language models. D2F combines asymmetric distillation from a bidirectional teacher with block-wise causal decoding to enable partial KV caching. The authors report substantial inference speedups over both prior diffusion LLMs and standard autoregressive models while maintaining competitive accuracy on reasoning and coding benchmarks.", "questions": "KV caching explanation. One source of speedups is the use of block-wise KV caching. Without the strictly causal generation over blocks some blocks are getting unmasked with previous blocks now fully unmasked. How is the KV caching implemented here? Or is KV caching only used for fully unmasked blocks? This is related to the ablations in the Appendix (table 5 and 7) which show that the pipelined decoding leads to 2-3x speedups.\n\nThe paper introduces a form of asymmetric self-distillation to train the model. This is one of the methodological contributions of the paper. It is unclear to me how much this is needed. Does this have any benefits over training the model from scratch (apart from training cost)? Is the asymmetric component needed? You could imagine training with the teacher using exactly the same setup as the student (standard distillation); that could perform similarly? (I understand the teacher is fully bidirectional, but should be able to handle such task as well.)", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Clear motivation: Tackles the critical limitation of diffusion LLMs, slow inference, and provides a coherent framework aimed at bridging the efficiency gap with autoregressive models.\n\nTechnical integration: The combination of block-wise causal decoding, diffusion distillation, and pipelined inference is well-engineered and practically significant.\n\nStrong empirical results: Shows large speedups with minimal accuracy loss.\n\nAccessible open-source implementation: The authors promise to open source the code.", "weaknesses": "Limited novelty. Most structural components (block-wise causal diffusion, caching, block parallelism) are drawn from prior work. I understand the novelty comes from the asymmetric distillation and the generation of later blocks without fully generated earlier blocks?\n\nInsufficient Justification for Asymmetric Distillation. The paper justifies distillation primarily as a way to reduce training costs. However, it doesn't provide an ablation comparing its asymmetric method to a \"from-scratch\" D2F model (even on a smaller scale) or a simpler \"standard\" distillation. This makes it hard to isolate how relevant this distillation mechanism is.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)