dParallel: Learnable Parallel Decoding for dLLMs
Authors: Zigeng Chen
Gongfan Fang
Xinyin Ma
Ruonan Yu
Xinchao Wang
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion language model
parallel decoding
efficiency
Primary area: generative models
TL;DR: We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.

Abstract: Diffusion large language models (dLLMs) have recently drawn considerable attention within the research community as a promising alternative to autoregressive generation, offering parallel token prediction and lower inference latency. Yet, their parallel decoding potential remains largely underexplored, as existing open-source models still require nearly token-length decoding steps to ensure performance. To address this, we introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling. We identify that the key bottleneck to parallel decoding arises from the sequential certainty convergence for masked tokens. Building on this insight, we introduce the core of our approach: certainty-forcing distillation, a novel training strategy that distills the model to follow its original sampling trajectories while enforcing it to achieve high certainty on masked tokens more rapidly and in parallel. Extensive experiments demonstrate that our method can dramatically reduce the number of decoding steps while maintaining performance. When applied to the LLaDA-8B-Instruct model, dParallel reduces decoding steps from 256 to 30 on GSM8K, achieving an 8.5× speedup without performance degradation. On the MBPP benchmark, it cuts decoding steps from 256 to 24, resulting in a 10.5× speedup while maintaining accuracy.

Published as a conference paper at ICLR 2026
DPARALLEL: LEARNABLE PARALLEL DECODING FOR DLLMS
Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, Xinchao Wang"
National University of Singapore zigeng 9&u. nus. eduu, xinchao@nus.edu..
ABSTRACT
Diffusion large language models (dLLMs) have recently drawn considerable at- tention within the research community promising alternative to autoregres- sive generation, offering parallel token prediction and lower inference latency. existing
Yet, their parallel decoding potential remains largely underexplored, open-source models still require nearly token-length decoding steps to ensure per- formance. To address this, we introduce dParallel, simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling. We identify that the key bottleneck parallel decoding arises from the sequential certainty convergence for masked tokens. Building on this insight, we introduce the core of our approach: vertainty-foreing distillation, a novel training strategy that dis- tills the model to follow its original sampling trajectories while enforcing it to achieve high certainty on masked tokens more rapidly and in parallel. Extensive experiments across various benchmarks demonstrate that our method can dramati- cally reduce the number of decoding steps while maintaining performance. When applied to the LLaDA-8B-Instrucc model, dParallel reduces decoding steps from 256 to 30 on GSM8K, achieving an 8.5x speedup without performance degra- dation. On the MBPP benehmark, it cuts decoding steps from 256 to 24, re- sulting in a 10.5x speedup while maintaining accuracy. Our code is available at https: / / thub. cwm/czgl / 2255/Paaaall.
INTRODUCTION
Diffusion large language models (dLLMs) (Yu et al., 2025a; Zhang et al., 2025; Yi et al., 2024) have emerged as a promising alternative autoregressive LLMs (Achiam et al., 2023; Bai al., 2023). By leveraging bidirectional attention, they overcome the sequential generation bottleneck and enable parallel, random-order text generation, offering the potential for substantial improvements in inference efficiency. This potential has already been demonstrated in proprictary models such as Mereury (Labs al., 2025), Gemini-Diffusse
and
(Song al., 2025).
However, realizing this parallelism in existing open-source dLLMs remains challenging. Open im- plementations such as LLaDA (Nie et al., 2025; Zhu al., 2025) and Dream (Ye al., 2025), still require a number of decoding steps proportional to the sequence length to maintain generation quality, resulting limited inference efficiency. Many recent efforts have attempted to accelerate ds. Some approaches (Ma et al., 2025; Liu et al., 2025; Wu et al., 2025; Hu et al., 2025) re- duce the time cost per decoding step by enabling KV caching. Other works (Israel et al., 2025; Wei et al., 2025; Li et al., 2025a;b; Gwak et al., 2025; Ben-Hami
al., 2025) focus on optimizing par- allel sampling algorithms to accelerate inference by reducing the necessary decoding steps. Despite these advancements, existing methods have yet to fully unlock the parallel potential of dLLMs, as highly parallel decoding consistently leads to degraded performance.
This paper focuses training dLLMs unleash their potential for parallel decoding. We identify the core bottleneck as their sequential certainty convergence. Although dLLMMs predict all masked tokens in parallel at each step, the certainty of these predictions still converges in a left-to-right quential order. This sequential propagation of certainty prevents the model from reliably determining
"oorrespoding Author

---

Published as a conference paper at ICLR 2026
Original L.a.DA-S--Intrrrct
GSM8K Ace: 75.7%
Avg Step: 256
Step 0: [Prompt] To determine how many
Step 1: [Promp[] To determine how many kilometers
Step 2: [Prompt] To determine how many kilometers
can
Step 3: [Prompt] To determine how many kilometers
can run
Step 4: [Prompt] To determine how many kilometers
can run
Step 5: [Prompt] To determine how many kilometers Lily
Speedup 599
Step 6: [Prompt]
To determine how many kilometers Lily can run
in
hours,
Step 7: [Prompt]
To determine how many kilometers Lily can run
eight hours,
Step 8: [Prompt]
To determine how many kilometers Lily run
eight hours, we
Step 9: [Prompt]
To determine how many kilometers Lily can run
in
eight hours, we need
dParallel (Ours):
GSM8K Ace: 76.1% %
Avg Step: 30
Step 0: [Prompt] To determine how many
Step I: [Prompt] To
determine how many
kilometers
Lily
can
run
in
eight
hours,
we
need
Figure I: Our method achieves highly parallel decoding. Compared to the original LLaDA Model, dParallel decodes over 8 tokens per step on GSM8K while preserving the accuracy.
multiple tokens simultaneously. forming the key bottleneck to highly parallel decoding. Employing naive teacher forcing or diffusion forcing (Chen et al., 2024) training is insufficient to resolve this issue, as they solely focus on trajectory alignment. Consequently, a new training paradigm centered on predictive certainty itself is needed for dLLMs to further unlock parallelism.
Building on this insight, we present certainty-forcing distillation, a simple and effective training strategy that directly leverages token certainty a training signal. The core idea is to convert dLLM's inherently sequential certainty propagation into a more parallel convergence process. Con- cretely, guide pretrained dLLM to self-distill along its original autreeevoreeeesivvve decoding trajectory to maintain trajectory consistency, while simultaneously minimizing its predictive entropy correctly predicted masked tokens enforce high certainty. Certainty-forciin enables more to- kens to reach high certainty in parallel at each step, thereby significantly extending the boundary of parallel decoding in dLLMs.
We evaluate the effectiveness of our method on two representative open-source dLLMs: LLaDA, native dLLM trained from scratch, and Dream, dLLM initialized from an autoregressive LLM. Comprehensive experiments across multiple benchmarks demonstrate that our approach signifi- cantly reduces the number of decoding steps in dLLMs, while maintaining comparable performance. For instance, when applied to the tion in decoding steps on GSM8K (Cobbe et al., 2021), yielding an 8.5x speedup without sacrificing accuracy (Fig.1), 1). On MBPP (Austin et al., 2021b), it further reduces decoding steps by 91%, deliv- cring 10.5x accelcration while maintaining performance. Furthermore, the training process of our method is highly efficient and low-cost. Leveraging Low-Rank Adaptation (LoRA) (Hu et al., 2022), the training can be completed in just 10 hours on only eight A5000 GPUs with 24 GB memory each.
model, our approach achieves an 88% reduc-
In conclusion, present dParallel, learnable approach that unleashes the potential of parallel decoding in dLLMs, drastically reducing the number of decoding steps. Our analysis identifies the core bottleneck as the sequential convergence of certainty across masked tokens. To address this, we introduce a certainty-forcin distillation strategy that ensures consistency with the original generation trajectory while encouraging masked tokens to attain high certainty faster and more in parallel. Extensive experiments demonstrate the effectiveness of our method, This work establishes a new baseline and provides a foundation for future research on few-step and parallel dLLMs.
2 RELATED WORKS
Diffusion Language Models. recent years, diffusion models (Ho et al., 2020; Song et al., 2020) have established dominance in the field of visual generation (Rombach et al., 2022; Podell et al., 2023; Ruiz et al., 2023; Zhang et al., 2023). However, their application to text generation remains highly challenging. Masked diffusion models (Shi et al., 2024; Austin et al., 202 Sahoo al., 2024: Zheng et al., 2024; Lou et al., 2023)have emerged as a promising approach, modeling language in the discrete space by predicting masked tokens, thereby offering the potential for fast and parallel
2

---

Published as a conference paper at ICLR 2026
decoding. Building upon this idea, two representative dLLMs, LLaDA (Nie al., 2025) and Dream (Ye et al., 2025), have recently attracted significant attention from the community, demonstrating that dLLMs can achieve performance comparable to autoregressive LLMs at the billion-parre parameerr scale. Beyond these developments, there is also growing interest in reasoning dL..Ms (Zhao et al., 2025; Wang et al., 2025b; Zhu et al., 2025), multimodal dLLMMs (You et al., 2025; Yu al., 2025b; Yang et al., 2025; Li et al., 2025c), and code generation (Gong et al., 2025; Xie et al., 2025) dLLMs.
Accelerating Diffusion Language Models. The potential of dLLMs in inference efficiency remains largely underexplored. Recent studies have increasingly focused on accelerating the decoding pro- cess of dLLMs. Some approaches (Ma et al., 2025; Liu al., 2025; Wu et al., 2025; Hu et al., 2025: Chen et al., 2025) aim to reduce the time cost for each decoding step by enabling caching mechanisms and employing token dropping during inference. Other works (Israel al., 2025; Wei et al., 2025; Li et al., 2025a;b; Gwak et al., 2025; Ben-Hamu et al., 2025) focus on reducing the total number of decoding steps by designing improved sampling strategies. (Wang et al., 2025a; Arriola et al., 2025)have been proposed that combine the generative paradigms of dLLMs and autoregressive LLMs, training models to realize more efficient inference pipelines. SDTT Deschenaux & Gulcehre (2024) employs progressive distillation to reduce the inference steps. Further effort Xu & Yang (2025) leverages quantization techniques to construct lightweight dLLMss
addition, hybrid methods
3 PRELIMINARIES
Masked Diffusion Language Models (MDLMs). Unlike AR-LLMs that predict tokens in a strict left-to-right fashion, MDLMs (Shi al., 2024; Austin et al., 2021a; Zheng et al., 20r generation as probabilistic process consisting of a forward masking corruption and reverse de- noising recovery. The forward process corrupts a clean sequence zo into 2, level [0, 1]:
dx | mo) = II (1 (1-())o 0(t =b) + L = [MASKI)].
(1)
The reverse process is parameterized by mask predictor po. which attempts to recover To from 2t- At each step, the model predicts all masked tokens jointly:
Po | xx))
II
Pa(z) xx),
f:an== [MANK]
The training objective, defined the negative log-likelihood restricted to masked positions, has been shown to upper bound the model's negative log-likelihood (Ou et al., 2024):
L
[(0)= = - Eo,
Σ 1=1
1 [MASK]] log mx)))
Sampling Process. Inference proceeds through a discretized reverse process: at each step the model predicts distributions for all masked tokens in parallel, samples provisional tokens, and then applies a dynamic remasking strategy to determine which positions remain masked for further refinement. Unlike autoregressive decoding, this procedure allows multiple tokens to be determined in parallel, thereby enabling more flexible and potentially faster generation.
4
METHOD
4.1 THE BARRIERS TO PARALLEL DECODING
Diffusion language models are designed, in principle, for highly parallel token prediction. Yet in practice, this theoretical promise breaks down. To understand this discrepancy, we analyze the cer- tainty dynamics of token predictions in dLLMs, revealing why their potential for parallel decoding remains unrealized.
Certainty Correlates with Prediction Accuracy. We first establish that token-level certainty is reliable indicator of prediction correctness. Using I.LaDA-8B-Insrruct the GSMSK test set
3

---

Published as a conference paper at ICLR 2026
1,00
1
1,00
0.90
Confidencee
Confidence
0.60
0.20
0.20
0.29
0/00
0.7
0.5
09
000
16
22
Sap
0400
16
Average Determined Confidence
Token Index
Decoding Step Index
(a) Acc vs. Average Determined Conf
(b) Token Confidence in various steps
(c) Confidence Converges in Sequential
Figure 2: Empirical Studies: (a) The average confidence score exhibits a positive correlation with generation accuracy. (b) Token confidence propagates sequentially during the decoding process. (c)
Convergence trajectories confidence for different tokens.
(Cobbe et al., 2021), we adopt remasking strategy with varying confidence thresholds and record the average determined confidence of tokens. Fig 2 (a) shows strong positive correlation between token confidence and the generation correctness: tokens resolved at higher confidence consistently achieve higher accuracy, whereas low-confidence commitments lead to frequent errors. This result confirms that high certainty necessary condition for accurate generation.
Certainty Converges to Peak Sequentially. The high certainty is not achieved in parallel. Instead, propagates sequentially through the sequence. At any given decoding step, the model predicts all masked tokens, but only small subset, typically those adjacent to already known context, attain high confidence. The vast majority tokens remain a low-confidence regime until new context becomes available. Once a confident token is committed, it provides a new conditioning context that allows another subset to rise in certainty at the next step.
This dynamic is illustrated in Fig 2 (b), which shows the average confidence of tokens progressing as a left-to-right propagation over decoding steps. Fig 2 (c) further confirms this at the individual token level, showing confidence trajectories that converge to high certainty in a staggered, sequential order. Together, these findings reveal that high certainty does not emerge in parallel but unfolds sequentially through iterative context enrichment.
The Fundamental Bottleneck. The key bottleneck is the sequential convergence of certainty. While true parallelism requires committing many tokens in single step, dL.LM gains high certainty only for a few neighboring tokens per iteration. Forcing multiple commitments too early introduces low- confidence predictions, causing cascading errors and performance degradation.
Key to Unlocking Parallelism Potential. The above insight illuminates a clear path forward: could guide the model to achieve peak confidence in parallel across multiple token positions, we could break the sequential bottleneck. However, traditional training strategies, such as teacher forcing and diffusion forcing (Chen et al., 2024), are inadequate for this purpose, as their focus on trajectory alignment overlooks the dynamics of predictive certainty. Consequently, unlocking greater parallelism dLL.Ms requires a new training paradigm that directly optimizes for certainty. We therefore propose certainty-forcing disillation, a novel strategy that reshapes the model's cer- tainty dynamics by using token certainty itself as a direct training signal.
4.2 CERTAINTY- DISTILLATION
We propose certainty-foriing distillation, straightforward approach that enforces parallel certainty along the original trajectory without altering it. An overview is shown in Fig 3.
Teacher Trajectory Generation. Let Ma, be the teacher model pre-trained vanilla dLLM), and let Mng be the student model, initialized as an identical copy. We train on a dataset D ( } where each ( is an instruction prompt. For each prompt, the teacher Me, generates target response trajectory using a cemi-autoregressive remasking strategy with total length L and block size La. producing a sequence Y (v1, y2. Wt)) This sequence is partitioned into N contiguous blocks (B1, Bz...., By} such that N x Ls. where the n-th block is defined as V.....,V
B, (U(n-1)La+1> -- Vat.)) for (1,..., N).
4

---

Published as a conference paper at ICLR 2026
Token Confidence
Generation Trajectory
[MASK]
[MASK]
[MASK]
[MASK]
[MASK]
Certainty-Forcing
DifTusicn
[MASK]
Diffusion
Langwage
[MASK]
[MAKK]
[MAKK]
Loss
Diffusion
Diffusion
Language
Medels
arv
[MASK]
Goneration
Diffosion
Language
Madels
Ffficient
Training Ohjjctivv
Figure 3: Overview of proposed certainty-forcing distillation. The dLLM is self-distilled along its original generation trajectory, ensuring consistency with the trajectory throughout training while encouraging token certainty to converge faster in parallel rather than sequentially.
Semi-Autooregressive Forward Masking. To simulate the trajectory generation process for train- ing, perturb the clean trajectory Y and create noisy input sequence Y by applying semi- autoregressive structural masking scheme. We first first uniformly sample block index n (0,... {O,..., N- 1). The sequence is then divided into three distinet parts based this index: (1) Context Blocks (i S nl,)): Tokens within the first n blocks remain unmasked, serving as the model's context. (2) Active Block (nLs < S (n + L)): This the block currently being generated, where its kens are randomly replaced by the token [MASK] with masking probability pm q. (3) Future Blocks (i (n Ly): All tokens subsequent blocks are fully masked, as they have not yet been generated. This procedure yields noisy token at each position i, defined as:
V1:
if S nT. (Context)
ys [MASK]
with probability 1
D:
if nLa (n + I)Ls (Active Block)
with probability q [MASK],
if (n + L) (Future)
The resulting sequence Y simulates intermediate state in the semi-autoregressive generative pro- cess, where the model is predicting the (n + 1)-th block given the context of the first blocks.
Training Objective. Our objective differs from standard dLLM pre-training, which typically aims to predict all masked tokens across the sequence. Instead, we restrict the learning signal to the masked tokens within the active block B,+1- Our training objective is for the student model not only to replicate the target sampling trajectory within the active block but also to parallel achieve maximal certainty in its predictions.
To enforce consistency between the student model's generated trajectory and that of the teacher, we apply standard Cross-Entropy (CE) loss on the masked tokens of the active block, denoted as Ma:
LConsistency
log log Y),
(5)
[Ma IMM
iEML
where (0( Y) denotes the probability assigned by the student model to correct token y: at position conditioned on the noisy sequence Y. However, conventional CE loss is insufficient target. It focuses solely correctness, and once the correct token is predicted, the gradient quickly vanishes, offering no incentive to further increase confidence.
for certainty-maximiin
To explicitly encourage highly confident predictions, introduce term that directly minimizes the entropy of the model's output distribution, incorporating a temperature parameter T. This loss is applied only the masked tokens in the active block that the student model already predicts
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Certainty-Forcingg Distillation (CFD)
Require: Teacher Mør, student Mes: target trajectory set D = {Y")} Ji= temperature T > 0;
weight 0; optimizer O; token length L; block length Lu; mask ratio q E (0.
Notation: (p) Evev p(t) log p(v)
1: for jurj = /teration do
2: Sample Y ~ D. n L 1}:
3:
(Y,M,) + HOW.HARRROOARRMMMIINNNCCCC Ln);
4:
(Y);
5:
pi(v) + softmax(zi).. Pi (v) + softmax(z//T).. Vo EV
6:
CCousistency t MMl M-MalM'Eee log log):(y:):
7:
M. EiE Ma arg maX
CCeraainty + 1llMel > MM -0 H(p; (1));
8:
9: LCED t CConsistency + Ceeraiinty:
10: Os +- O(Os, VogLCrD).
end for
correctly. Formally, we define the set of correctly predicted tokens as
Me { i fie
Ma
arg max (o (Y) y: vEV
(6)
where V denotes the vocabulary. The certainty- loss is then defined the average entropy of the predictive distributions for these tokens:
Ceetannty
Σ iEM.
(-Σ DEV
polu (;T) log 17:7)
[Md
where (o T) denotes the temperature-scaled softmax distribution. Minimizing this term encourages the student model to generate sharper, higher-certainty distributions over the correct tokens, where T controls the strength of the certainty enforcement.
The overall training objective is combination of consistency loss and the cortainty-forcing loss:
LCFD Connsisteccy + 3 G.Cettiiytyy
(8)
where B a hyperparameter balancing the objective of matching the teacher's trajectory with the objective of enforcing high certainty. We find that this simple distillation strategy significantly celerates the parallel convergence of certainty in dLLMs, thereby unlocking their inherent potential for parallel decoding. The overall training pipeline is summarized in Algorithm
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We evaluate the effectiveness of our method on two representative open- source dLLMs: LLaDA-SB-Instruuc (Nie al., 2025) and Dream-7B-Instruet (Ye al., 2025). The training is conducted using the LoRA technique (Hu et al., 2022). For semi- autt-rrrooreeeeivvee masking, we set the block length Ls 32 for LLaDA and L, 256 for Dream, with a fixed masking ratio of 50%. The certainty loss is applied with temperature of T 0.5. Full training config- urations are provided in the appendix. During inference, our models adopt an entropy-threshold semi-autoregressive remasking strategy which is inherently consistent with training objective.
Implementation Details.
Training Data. As self-distillation approach, we use prompts from publicly available training datasets and let the pretrained model generate its own responses training data. For LLaDA-8B- Instruct, we sample prompts from the GSM8K (Cobbe et al., 2021), PRM12K (Lightman et al., 2023) training set, and part of the Numina-Math dataset (Li et al., 2024). We generate target tra- jectories using a semi- autrerrutrrer strategy with a sequence length of 256 and block length of 32. We further filter out responses containing incorrect answers and finally get about 92k samples. For

---

Published as a conference paper at ICLR 2026
Table I: Evaluation results LL For all methods, adopt semi- semi-autoregressive remasking strategy with a total sequence length of 256 and a block length of 32. For our approach, the entropy threshold
either 0.45 or 0.5 for different tasks.
Benchmark Method
#Steps
Latency.
Speedup
Accuracy
LLaDA-&B-Instrul
256
18.6s
1.0x
75.7%
Dual-Cache
256
9.7s
1.9x
72.9%
GSM8K -CoT
Few-step Decoding
4.7g
4.0x
68.6%
(0-shot)
Conf-threshold Decoding
5.2s
3.6x
75.5%
Consistency Distillation
4.7s
4.0x
69.9%
dParallel (Ours)
2.2s
8.5x
76.1%
-LaDA--B-Instteee
256
50,9s
1.0x
33.5%
Dual-Cuche
256
11.3s
4.5x
32.6%
MATH
Few-step Decoding
64
12.7s
4.0%
26.3%
(4-shot)
Conf-threshold Decoding
97
17.6s
2.9
33.2%
Consistency Distillation
64
12.7s
4.0%
28.0%
dParallel (Ours)
46
8,9s
5.7%
31.5%
LLaDA-8B-Instre
256
23.5s
1.XX
38.4%
Dual-Cache
256
9,8s
24XX
34.1%
HumanEval
Few-step Decoding
5,9s
4.0%
19.5%
(0-shot)
Conf-threshold Decoding
77
6.7s
3,5X
37.2%
Consistency Distillation
5.98
4.0X
19.5%
dParallel (Ours)
33
2.9s
8.2x
40.2%
LLaDA-8B -B----sutttt
256
50.1s
1.xx
42.4%
Dual- Dual-Cache
256
10.7s
4.7%
39.8%
MBPP
Few-step Decoding
64
12.5s
4.0%
19.6%
(3-shot)
Conf-threshold Decoding
68
12.8s
3,9X
41.6%
Consistency Distillation
12.5s
4.0%
25.0%
dParallel (Ours)
4.8s
10.5x
40.8%
Pream-7B-Instrenn adopt the same trajectory generation strategy, and additionally generate code data using prompts from subset of the AceCode dataset (about 10k) (Zeng et al., 2025). Impor- tantly, all training tokens are generated by the model itself, without introducing any external data as targets.
Evaluation Details. We evaluate our models across multiple benchmarks, including two mathe- matics datasets (GSM8K and MATH (Lewkowycz et al., 2022)) and two code generation datasets (HumanEval (Chen et al., 2021) and MBPP (Austin al., 2021b)). For GSM8K, we append chain-of-thought (CoT) prompt (Wei et al., 2022) after each question. We report accuracy, the av- erage number of decoding steps, latency, and speedup ratio to provide comprehensive evaluation. All efficiency evaluations are conducted NVIDIA RTX 6000 Ada GPUs.
Baselines. We evaluate the original dLLM under its official default inference setting, and further compare our approach with four baselines that seek to accelerate the generation: (1) Dual-Cache: enable KV-cache on both prefix tokens and suffix tokens (Wu et al., 2025). (2) Few-step Decoding: Decoding:
reducing the number of decoding steps used by the original dLLM. (3)
apply adaptive remasking based on the model's confidence in predicting masked tokens (Wu et al., 2025; Yu et al., 2025b), with the confidence threshold set 0.90 0.95 depending the task. (4) Consistency Distillation: training the dI.MM to predict all remaining masked tokens from inter- mediate state along its own generation trajectory (Luo et al., 2023). The training data and LoRA configuration are the same as our method.
5.2 MAIN RESULTS
Results on the Native LLaDA Model. As shown in Table 1, directly reducing the decoding steps of the original model leads to substantial drop in performance. Consistency distillation has only a marginal effect on LLaDA, offering slight improvement over the original model under the same number of steps. The confidence-threhhe lelism is limited, averaging only 3-4 tokens decoded per step. In contrast, our method significantly pushes the boundaries of parallel inference in dL.LMs, achieving than 8 tokens decoded per
remasking strategy preserves accuracy, but its paral-

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation results Dream-8B-Instrus The original model the official inference setting with a sequence length of 256. Other methods adopt semi cemi-autoregressvve remasking with the same length and block size of 32. The entropy threshold for our method is set either 0.45 0.5.
Benchmark
Method
#Steps
Latency.
Speedup
Accuracyl
Treat-7B-Instruet
256
17.2s
1.0x
82.9%
Dual-Cache
256
8.2s
2.1x
79.5%
GSM8K -CoT (0-shot)
Few-step Decoding
4.38
4.0x
59.0%
Conf-threshold Decoding
61
4.0s
4.3x
81.9%
Consistency Distillation
4.3s
4.0x
75.6%
dParallel (Ours)
2.5s
6.9%
82.1%
Dream-7B-Insee
256
17.58
1.0x
39.5%
Dual-Cuche
256
8.2s
2.1x
38.8%
MATH
Few-step Decoding
64
4,4s
4.0%
16.7%
(0-shot)
Conf-threshold Decoding
93
6.18
2.9X
38.9%
Consistency Distillation
64
4.4s
4.0%
29.6%
dParallel (Ours)
63
4.1s
4.2%
38.3%
Dream-7B-Instrun
256
25.9s
1.0X
52,4%
Dual-Cache
256
8.48
3.1X
47.0%
HumanEval IInstruct
Few-step Decoding
6,5s
4.0%
16.5%
(0-shot)
Conf-threshor Decoding
71
7.38
3,5X
53.1%
Consistency Distillation
6.4s
4.0X
34.2%
dParallel (Ours)
37
3.8s
6.9X
54.3%
Dream- 1aann--B-Instruu Instrccttt
256
19.8s
1.0x
58.8%
Cnal-Cache
256
8,9s
2.2x
52.8%
MBPP
Few-step Decoding
64
5.0s
4.0%
25.0%
-Instruct
(0-shot)
Conf-threshold Decoding
43
3,38
5.9X
56.4%
Consistency Distillation
5.0s
4.0%
37.4%
dParallel (Ours)
29
2.2s
8.8x
56.2%
020
0.30
23.6
0.00
8.2
033
Cor Juckold
Crelsbodsit
Mode
Modd
Bame Miski
Stade
Speddop
Spredag
Speoko
Trade GSMEK
LLADA Trade ofTos BananEval
Trods GSMIK
Trade HunazEval
Figure 4: Comparison of speed-accuracy trade-off curves between confidence-threshold decoding and our method, (a) and (b) show results on the LLaDA model for GSM8K and HumanEval, respec- tively. (c) and (d) present results on the Dream model for GSM8K and HumanEval benchmarks.
step average while still maintaining performance. Notably, for LLaDA, trained using only prompts from mathematical tasks, yet the model still exhibited a remarkable improvement in paral- lel decoding ability on code tasks.
Results on the AR-initialized Dream Model. As shown in Table 2, our method also demonstrates superior performance on the Dream model, which is initialized from an AR-LLM. Compared to other approaches designed to reduce the number of decoding steps, dParallel achieves a substan- tially higher speedup while maintaining accuracy, thereby greatly enhancing decoding parallelism. It is worth noting that we observed risk of degeneration toward the original AR L.LM when train- ing Dream with autoerevoreee masking. To avoid this issue, we employed standard random masking over the entire sequence instead. Consequently, the acceleration gains of our method on Dream are slightly lower than those observed on LLaDA.
Superior MeffieecyyPefformancc Trade-off. In Fig 4, compare our method against the original model with confidence- thereshhld decoding in terms of the efficiency-perfooeeeee performance trade-off curve. Our approach achieves substantially better trade-off. On LLaDA with GSM8K, the same 9.4x speedup, our method attains 16.5% higher accuracy than aofidennce-treshhen decoding. On Hu- manEval, at the same 9 speedup, method improves accuracy by 21 1..%. Results on Dream
8

---

Published as a conference paper at ICLR 2026
1.00
Bes Modd
1.00
Scze Modd
Cref- Oxff-Bleskkk)
0.39
Casfidewee
00.00
0.30
0.33
Thiles Teclex
Thikes Trdox
Token Confidence at Decoding Step
Average Token Confidencs Decoding Step 16
Figure 5: Average token confidence at the 8th and 16th decoding steps for LL Model on GSM8K. The proposed certainty-forein strategy reshapes the original sequential cer- lainty convergence into a faster and more parallel convergence process.
Table 3: Ablation study on different training strategies of our method using the LLaDA model.
Consistency Certainty Loss
Semi-AR Masking
GSMSK-Col (0-shot)
Human val (0-shot)
Loss
#Steps
Speed
Acc f
#Steps
Speed
Acc
53
4.5x
73.5%
71
3.6x
36.0%
23
10.4x
57.8%
28
9.8x
30.5%
44
5.5x
73.3%
61
4.3x
32.9%
30
8.5x
76.1%
33
8.2x
40.2%
exhibit similar curve. These findings strongly demonstrate that method effectively broadens the boundary of parallel decoding in diffusion language models.
Faster and Parallel Certainty Convergence. As illustrated in Fig the original dLLM exhibits a sequential convergence token certainty, where each step produces high confidence only for a small set of neighboring tokens, while the majority remain low-confidence range. Confidence- based decoding can extend the boundary of token certainty but still follows sequential propagation pattern. contrast, our dParallel, trained with certainty-foreinc distillation, transforms this process into significantly faster and more parallel convergence of certainty. Such parallel convergence further unlocks the potential of dLLMs for highly efficient parallel decoding.
5.3 ABLATION STUDY
Ablation Study on Training Strategy. We conducted an ablation study to validate the effectiveness of our proposed certainty-forcing distillation, with the results shown in Table 3. When the certainty- forcing loss is removed, the remaining consistency loss insufficient to alter sequential con- vergence pattern of the dLLM, resulting in speed and performance similar to the baseline model. Conversely, applying only the certainty loss without enforcing trajectory consistency achieves high decoding speed but leads forward masking effectively aligns the trajectory generation process with self-distillation, yielding superior efficiency and performance. These results collectively demonstrate that each component in the training process is essential.
sharp performance drop. Finally, our of semi-autoregressive
A Ablation Study Masking Ratio. We conduct an ablation study to determine the optimal masking ra- tio, training LLaDA for one epoch with various set- tings as shown in Table 4. We find that fixed mask- ing ratio of 50% yields the best performance, offer- ing significant acceleration while preserving accu- racy, In contrast, both higher and lower fixed ratios, as well as random ratios, lead to noticeable accu- racy degradation. This suggests that a 50% ratio cre- ates optimal trade-off between the training signals
Table 4: Performance of our method applied to ----D-----nnttuctt on GSMSK with dif- ferent masking ratios used in the forward process during training.
Masking Ratio
#Steps
Speed
Acc t
Random
36
7.1x
72.4%
25%
35
7.3x
69.9%
75%
38
6.7x
73.7%
100%
31
8.3x
71.1%
50%
36
7.1x
76.3%
for consistency and certainty by balancing masked and unmasked tokens. Importantly, training with this fixed ratio does not impair the model's 's ability handle variable ratios during inference.

---

Published as a conference paper at ICLR 2026
Prompt: Lily can run kilometers per hour for hours. After that, she runs kilometers per hour. How many kilometers can she run in hours? Please reason step by step, and put your final answer within (boxed]]..
LLaDA-SB-Instruct
Decoding Steps
dParallel (Ours)
Decoding Steps
To determine how many kilooeeeess Lily can run in hours, we need parts: distance she
determine how many kilometees Läly can run in 8 hours, we need calculate the distance she cuvers in each segment of her and then sum these distances.
break down problem first 4 hours and the distance sho cuns in the next bours.
First, let's culculate the distance she runs in the first 4 hours:
1. *Calculate the distance run the first hours:**
She speed 12 kilomndirrs per hour hours. Distance Speed x Time
Speed: killmeteess hour
Time: hours
km/h
km
Distance:: Speed Time 12 km/h x hours 48 kilometers
Next, calculate the distance
the hours:
2. *
*Calculate the distance the hours
Shee runs at speed of kilometers per hour for hours.
Speed: 6 killometers per hour Time: hours
Distance Speed Time
Distance 6 kmnh x hours 24 km
Distance Speed x Time kmmh x hours 24 kilometers
Now, we add the distances from both parts to find the total distance
3. **Sum the distances from both segments::*
run
Total distance Distance first hours Distance in the
Total distance - Distance in the first hours Distance in the next hours
4 hours
Total distance 48 km 24 72 kilometers
Total distance - 48 km 24 km 72 km
Therefore, the total distance Lily hours (bex
Therefore, the total distance Lily can run in 8 hours is (72x kilometers.
Figure 6: Case study on LLaDA-8B-Instrucc Model with chain-of-thought reasoning problem.
CASE STUDY
In Figure 6, we present case study of our dParallel model. When tackling the same Chain-of- Thought (CoT) math reasoning problem, dParallel achieves answer quality using only 28 decod- ing steps that is on par with the original -LaDA-8B-Instruch model requiring 256 steps. Furthermore, because dParallel selects the reasoning path with the lowest information entropy, the total number of tokens required to answer the question is slightly reduced. This case study clearly demonstrates the effectiveness of our approach in enhancing the decoding parallelism of dLLMs.
CONCLUSION
In this paper, present dParallel, simple yet effective method that unleashes the parallel de- coding potential of dLLMs. At the core of our approach is certainty-forcing distillation, a novel training strategy that maintains trajectory consistency while compelling high-certainty predictions, thus overcoming the sequential certainty propagation issue. Extensive experiments across various benchmarks validate the effectiveness of our method. Our work establishes new baseline for par- allel decoding in dLLMs and explores a new avenue for dLLM training paradigms.
ACKNOWLEDGEMENT
This project is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier (Award Number; MO0-T2EP20122-0000
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint wwx.v:2303.08777 2023.
Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang. Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autore- gressive and diffusion language models. arXiv preprint arXiv:2503.09573 2025.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 7987799111 1-17993, 2021a.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang. Carrie Cai, Michael Terry. Quoe Le, et al. Program synthesis with large language models. arXiv preprint Stti:2008.07755 2021b.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, and Brian Karrer. Accelerated sampling from masked diffusion models via entropy bounded unmasking. arXiv preprint arXiv:2505.248: 24857, 2025.
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neurul Information Processing Systems, 37:24081-24122 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained code. 2021.
Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai Li, Yiran Chen, et al. Dpad: Efficient diffusion language models with suffix dropout. arXiv preprint arXiv:2508. 14148, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 14168, 2021.
Justin Deschenaux and Caglar Gulcehre. Beyond autoregression: Fast Ilms via self-distillation through time. arXiv preprint urXiv:2410.21035, 2024.
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration. arXiv preprint arXiv:2506.26633 2025.
Dachoon Gwak, Minseo Jung, Junwoo Park, Minho Park, ChaeHun Park, Junha Hyung, and Jaegul Choo. Reward--eighted sampling: Enhancing non-autoregressive characteristics in masked dif- fusion llms. arXiv preprint vrXiv:2509.00707, 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.

---

Published as a conference paper at ICLR 2026
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):, 2022.
Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprint arXiv:2505.22 24467 2025.
Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion Ilms via adaptive parallel decoding. arXiv preprint arXiv:2506.0044 00413, 2025.
Inception Labs, Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang. Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, et al. Mercury: Ultra-fast language models based on diffusion. arXiv preprint arXiv:2506. 17298, 2025.
Aitor Lewkowyez, Anders Andreassen, David Dohan. Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo. et al. Solving quantitative reasoning problems with language models. Advances neural information processing systems, 35:3843-3857, 2022.
Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang. Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):9, 2024.
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, and Dahua Lin. Beyond fixed: Variable-length denoising for diffusion large language models. arXiv e-prints, pp. arXiv-2508, 2025a.
Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508. 19982, 2025b.
Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, and Aditya Grover. Lavida: large diffusion language model for multimodal understanding. arXiv preprint arXiv:2505.16839 2025c.
Hunter Lightman. Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe, Let's verify step by step. In The Twelfth International Conference Learning Representations, 2023.
Zhiyuan Liu, Yieun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dilm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint (xXXx:2506606955,, 2025.
Aaron Lou, Chenlin Meng. and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. 2023.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023.
Xinyin Ma. Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models. arXiv preprint arXiv:2505. /5781, 2025.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint TXXi::5502.09992.. 2025.
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406. 20t.::236000333366.. 2024.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint 207.122307.01992 2023.
12

---

Published as a conference paper at ICLR 2026
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 22500- 22510, 2023.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37: 1113301101888 2024.
Jiaxin Shi, Kehang Han, Zhe Wang, Araad Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint XXX::2000002502, 2020.
Yuxuan Song, Zheng Zhang, Cheng Luo. Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuchang Yang, Hongli Yu, Xingwei Qu, et al. Seed diffusion: large-scale diffusion language model with high-speed inference. arXiv preprint arXiv:2508.02 2025.
Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, and Zhijie Deng. Diffusion Ilms can do faster-than-arr inference via discrete diffusion forcing, arXiv preprint arXiv:2508.091 2025a.
Yinjie Wang, Ling Yang. Bowen Li, Ye Tian, Ke Shen, and Mengdi Wang. Revolutioniz- ing reinforcement learning framework for diffusion large language models. arXiv preprint arXiv:2509. 06949, 2025b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, al. Chain-of- of-thoughtt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Dongrui Liu, and Linfeng Zhang. Accelerating dif- fusion large language models with slowfast: The three golden principles. arXiv preprint arXiv:2506. 10848, 2025.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training- free acceleration of diffusion 11m by enabling cache and parallel decoding. arXiv preprint arXiv:2505.22611: 2025.
Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, et al. Dream-coder 7b: An open diffusion language model for code. arXiv preprint SSt.:2509.01142 2025.
Chen Xu and Dawei Yang. Dilmquant: Quantizing diffusion- based large language models. arXiv preprint arXiv:2508. 14090, 2025.
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505. 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models. arXiy preprint arXiv:2508.15448 2025.
Qiuhua Yi. Xiangfan Chen, Chenwei Zhang, Zehai Zhou, Linan Zhu, and Xiangjie Kong. Diffusion models in text generation: a survey. PeerJ Computer Science, 10:e1905, 2024.
Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu. Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.1693 16933. 2025.
13

---

Published as a conference paper at ICLR 2026
Rumpeng Yu, Li, and Xinchao Wang. Discrete diffusion in large language and multimodal models: A survey. arXiv preprint 1rXV5 2025a.
Runpeng Yu, Xinyin Ma, and Xinchao Wang. Dimple: Discrete diffusion multimodal large language model with parallel decoding. arXiv preprint arXiv:2505.16990 16990, 2025b.
Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. Acecoder: Acing coder rl via automated test-case synthesis. ArXiv, abs/2207.01188 2025.
Lingzhe Zhang, Liancheng Fang, Chiming Duan, Minghua He, Leyi Pan, Pei Xiao, Shiyu Huang, Yunpeng Zhai, Xuming Hu, Philip Yu, et al. A survey on parallel text generation: From parallel decoding diffusion language models. arXiv preprint arXiv:2508.08712, 2025.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023.
Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning. arXiv preprint arXiv:2504. 122 2025.
Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming- Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. arXiv preprint arXir:2409.02998, 2024.
Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505. 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A MORE IMPLEMENTATION DETAILS
In Table 5, we present the training configuration used for the certainty-foreing distillation process. For data generated by LLLDA-SB-Insteee standardized sequence lengths by padding truncating with the end-of-sequence token to fixed length of 384 tokens. In contrast, for Dream-7B-Instee response length of 256 tokens per sample without modification. Additionally, we set the balance weight 3 = for all training. We also used a complementary mask training strategy to improve token utilization.
(Nie et al., 2025) and LLaDA-115 (Zhu et al., 2025), we
(Ye et al., 2025), we preserved the original
Our training was conducted on two NVIDIA H100 GPUs, with per-GPU mini-batch size of 1 and gradient accumulation step of 32, resulting in effective global batch size of 64. Notably, despite the relatively large model sizes, the adoption of parameter-efficine 2022) and the use of shorter sequence lengths kept the memory footprint remarkably low. The entire training process required only 23 GB of GPU memory, meaning that it can be efficiently reproduced even on multiple consumer- -gade GPUs with 24 GB of memory each. This efficiency highlights the practicality of our approach, as it enables large-scale distillation training to be carried out on widely accessible hardware rather than being restricted to specialized high-memory accelerators.
fine-tuning (PEFT) (Hu et al.,
Table 5: The training configuration for certainty-forcing distillation across three base models.
Base Model
LoRA Rank
LoRA Alpha
Learning Rate
Lr-Schedule
Batchsize
Epoch
-LLaDA-8B-nsttrutt
32
32
2e-5
constant
64
6
a.I---
128
128
2e-5
constant
64
4
Dr-steut-
16
16
2e-5
cosine
64
3
Table 6: Evaluation results LLaDA-1.5 Model across four benchmarks.
Benchmark
Method
#Steps
Speedup Latency
Accuracyt
GSM8K
LLaDA-8B-Instre
256
19.1s
1.0x
76.0%
(0-shot)
dParallel (Ours)
30
2.3s
8.5x
76.3%
MATH
LLaDA-88 8BB IB-Instruct
256
50.0s
1.0x
34.0%
(4-shot)
dParallel (Ours)
45
8.7s
5.7x
32.1%
HumanEval
256
22.0s
1.0x
41.5%
(0-shot)
dParallel (Ours)
46
4.0s
5.6x
40.2%
MBPP
-LDDA-8B-Instrre
256
49.0s
1.0x
43.2%
(3-shot)
dParallel (Ours)
26
5.1s
9.8x
41.6%
B MORE EXPERIMENTAL RESULTS
In Table 6, we report the performance of applying our method to the LLaDA-1.5 model. Extensive evaluations across four standard benchmarks demonstrate the strong effectiveness of our approach on this reinforcement learning based model. Specifically, we reduce the original 256 decoding steps required by the baseline model to only 26-46 steps. This dramatic compression of the decoding steps delivers substantial acceleration in generation speed, while the same time preserving accuracy and reliability across tasks.
In Figure 7, we present the average token confidence the LLaDA-SB-Instruun model on GSM8K, measured across the first 160 positions over the initial 16 decoding steps. The results reveal that the original dLLM exhibits a clear sequential convergence of token certainty: each step yields high con- fidence for only a narrow band of neighboring tokens, while the majority remain in a cowcconfidence range. Although confidence-based decoding can extend the certainty frontier, still follows this se- quential propagation pattern. By contrast, our proposed dParallel, trained with forcing
15

---

Published as a conference paper at ICLR 2026
distillation, reshapes this process into substantially faster and parallel convergence of cer- tainty. This parallel convergence further unlocks the efficiency potential of dLLMs, enabling highly parallel decoding.
C
CASE STUDY
We also present additional case studies in Figure 8, Figure and Figure 10. Our dParallel achieves significantly reduced decoding steps while maintaining the generation quality.
D LIMITATIONS AND FUTURE WORK
The primary limitation of our method is its reliance on the performance of the pretrained dLLM. While approach achieves substantial gains in inference efficiency by unleashing the potential of parallel decoding and maintains strong accuracy, it cannot significantly improve the performance the base model itself is weak.
As a next step, we plan to extend our certainty-forcing strategy to the pretraining stage dLLMs and substantially scale up the training data explore the performance boundary of our approach. Currently, we have only used relatively small dataset of around 10k math problems. We believe that by dramatically increasing both the size and diversity of the training data, our method can yield further improvements: not only activating highly parallel decoding, but also enhancing overall model performance and demonstrating stronger generalization.
E ETHICS STATEMENT
This work adheres the ICLR Code of Ethics. Our study does not involve human subjects or sensitive personal data. All datasets used are publicly available and properly licensed. While our method focuses on improving the efficiency of diffusion language models, we recognize potential risks of misuse in harmful applications and encourage responsible use aligned with ethical and legal standards.
F REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure the reproducibility of our work. Detailed descriptions of model architectures, training objectives, and experimental settings provided in the main text and Appendix. All datasets used are publicly available, and their preprocessing steps are documented in the main paper and appendix. Additionally, include pseudocode and implementation details to facilitate replication, and source code is provided in the supplementary materials.
16

---

Published as a conference paper at ICLR 2026
Hecading
Decading
Bue Model
Daee Moeil
0/0
Token tnsks
Tiken Iaake
Beeadiing
Decading Step
Bue Model
Dae Hoeil
Token Iraket
000
16
Tikee fodes
132
150
340
100
Decadiing
Uecaading Step
Buo Model
Diee Moeil
Token trake
0.0
16
Tikee
135
340
100
Decading
Decaadnng Step
Boo Model
Daee Moeil
Gord (eella)
Token trcket
124
100
0/0
Tiken laake
Decalinng
Decating
Boo Model
Bate Moeil
Gard: (reoia
Tskea Innks
0.0
16
Tikea
112
135
340
100
Decudling Step
Boo Model
1.0
Bae Hoeil
Oko
Takes Tisks
144
190
16
130
Decnding Step
Deceding Step 14
Boo Model
0.1
Takea tinks
144
160
000
Tikee Eides
Deceding Step
Decoding
100
0.3
0.2
15
Figure 7: Average token confidence over the first 160 tokens across the first 16 decoding steps of the LLaDA-8B-Instre model on GSM8K. Our certainty-forcing strategy transforms the sequential certainty convergence of the baseline into faster and more parallel convergence process.
17

---

Published as a conference paper at ICLR 2026
Prompt: Lily can run 12 kilometers per hour for 4 hours. After that, she runs kilometers per hour. How many kilometers can she run 8 hours? Please reason step by step, and put your final answer within [hoxed
LLaDA--BBInstruue
# 256 Decoding Steps
d dParallel (Ours)
28 Decoding Steps
To dotermine how many kilometers Lily can run in 8 hours, we noed break down the problem into parts: the distance first 4 hours and the distance she runs in the next 4 hours.
To determine how many kilomeeeess Lily can run in hours, we meed calculate the distunce she covers in each segment of her and then sum these distances.
the
First, let's calculate the distance she runs the first 4 hours:
*Calculate distance run the first hours:**
She runs a spead of kilometers per hour for hours.
Speed: 12 kilometers hour
Distance Speed x Time
Time: hours
Distance km/h hours 48
Distancee: Speed x Time 12 km// x hours 48 kikoneters
Next, let's calculate the distance she the next hours:
*Calculatc the distance zun the noxt hours:*
She runs at a speed of 6 killmmters per hour for 4 hours.
-Spoed: 6 kilometers per hour
Distance Speed Time
-Time: hours
Distance km/h x hours 24 km
Distance Spoed x Time krrh x hours 24 kilometers
Now, we add the distances from both parts to find the total distunce
*SSum distanccs from both sogments:**
she
8 hours:
Total distance Distance in the first hours Distance in the next
Total distance Distance in the first 4 hours Distance the next
hours
hours
Total distance 48 km 24 km 72 kilometers
Total distance 48 km + 24 km 72 km
Therefore, the total distance Lily can run in 8 hours (boxed(72)..
Therefore, the total distance Lily can run in hours (boxed(72) kilumeters.
Prompt: Natalia sold clips to her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Please reason step by step, and put your final answer within bboxed (3.
# 256 Decoding Steps
dParallll (Ours)
# 30 Decoding Steps
To determine the number of clips Notalia sold in April and May, we noed to follow these steps:
To deternine the number of clips Natalia sold April May. we need to follow these stops:
1. Identifyy the number of clips Notalia sold April.
1. ** CCllcllate the number of clips sold in May:
2. Calculate the number clips Natalia sold May.
Natalli sold clips friends in May.
Add the number of clips sold in April and May to find the total.
Shee sold half as many clips in May as she did in April.
Since she sold clips in April:
Identify the number of clips Natalia sold April. Natalia sold clips in April.
Step I:
Therefore, the number of clips sold in May is (( Virac (48) (2) 24
Calculate the number of clips Natalia sold in May. Natalia sold half as many clips in May she did April. Therefore, we calculate the number of clips sold May as follows:
Step 2:
**Calculate total number clips sold in months
Add the number of clips sold in April the number of clips sold May.
wext (Number of clips sold May) Vfrcc [48) (2) - 24
Total clips sold Clips sold April Clips sold in May
Total clips sold +
Step 3: Add the mamber clips sold in April and May to find total.
Total clips sold 72
We add the number of clips sold April and May:
Wext Total mumber of clips sold)
24 721
Thas, the total number of clips Natalia sold April and May ((boxee (72))).
Thus, total number clips Natalia sold April and May (22))
Figure 8: Case study on LLaDA-8B- ---------Intttttt Model with shain-of-thought reasoning problem.
18

---

Published as a conference paper at ICLR 2026
Prompt: typing import List
has elements(numbers: List(flont); float) bool:
-" Chock in given list of numbers, any numbers closer to each other than given threshold.
500 has close clements([10, 2.0, 3.0), 0.5)
False
5o0 has close element((1.0, 2.8, A,, 5.0, 2.3).
Thme
LLaDA-8B-Iastruee
256 Decoding Steps
dParallel (Ours)
15 Decoding Steps
implementation has, elements' function:
"python
from lyping import List
ppython
from typing import List
def close eloments(number:: isstffloat) threshold: float)
def bas close elementssmummees: List floar,, threshold: flout) i bool: Cheok if in given list numbers, are any two numberr closer to each other thon
-en Chock if given list of numbers, are two numbers closer given threshold.
given thresbold.
has close elemmets([1 (1.0, 2.0, 3.0). 0.5)
close elements([1 3.0), 0.5)
False
False
sas has close, bacer__eeemnts[[111 I.O, 2.8, 3.0, 4.00 5.0, 2.0), 0.3).
close catee__eeeeems[_l.0
2.0. 0.3)
Trie
True
range((en/sumbees)
for in (ange(lentmunbers)):
for in rangeti I, Jon(numberss):: abu(numberssi] numbers(j) threshold:
for in rangeti 1, len(numbers)))
if (rabs(tumberss] numbers(j]) < threshold:
return True
retur True
retum
False
Figure Case study on LLaDA-SB-Instruc Model with naive code generation task.
Prompt: Write function to check the given number woodhall or not..
ILLaDA-SBB-Intrucc
256 Decoding Steps
dParallel (Ours)
49 Decoding Steps
determine i' given number "woodball." what a "woedball" is. For the sake of this example, let's assume "woodball" number
Sure, help you with that. woodhall number multiple both and Here's a Python function to check if given number woodball:
multiple finction in Python to check il's given number is a "woodball".
simple
" "python
"python
def is woodballkmubber..
is woodbulllnumber):
Check if number is multiple of both and number % and number %
Check the given number 'woedhalf (i.e,, multiple of
return True
clsc: return False
Parameters:
number (int): The mumber to check.
Example usage:
Returns:
printris woodhall(15)) Ourpot:
bool: True if number 'woodball', False otherwise.
printiis woodballi 30)) Ourut True
printris (0)) Ourjut:
return number % =
printtis woodball (7) Ouput: False
Example usage: print(is woodball(9)) printlis woodball(5) False
divisible by using
the modulus operalor '%. If both conditions are met, it returs "Truc" indicating
woodball.
finction, ii woodball', check if the number is divisible by 3. If the number is divisible by with remainder. "woodball," and function returns 'Trac'. Otherwise, it returns 'allee.
modulus operator '%6" to
Figure 10: Case study on LLaDA-80 8B ----8B--Innnt -n-srrutttt Model with iustruction-basee code generation task.
19


Reviews:
[{"review_id": "FRzWmYgLCl", "reviewer": "Reviewer_Aj7x", "summary": "This paper proposes dParallel, a learnable method to accelerate dLLMs by reducing the number of required decoding steps. The authors first identify a key bottleneck to parallel generation: the \"sequential certainty convergence,\" where the model's confidence in its predictions propagates from left-to-right rather than emerging in parallel. To address this, they introduce \"certainty-forcing distillation,\" a training strategy that fine-tunes a pre-trained dLLM using a new objective. This objective combines a standard consistency loss (to follow the original model's generation trajectories) with a certainty-forcing loss (an entropy minimization term) that encourages the model to become highly confident in its correct predictions more rapidly. The method is evaluated on top of two SOTA academia-setting models LLaDA and Dream models, shwoing significant reductions in decoding steps (e.g., 8.5x on GSM8K) while maintaining the original model's accuracy.\n\nHowever, I am recommending a borderline score at this time due to two major reservations that temper the paper's claims and require clarification:\n1.  **Conflation of Training and Inference Strategies:** The experimental setup makes it difficult to isolate the true benefit of the proposed *training* method. The dParallel-trained model is evaluated with a specific entropy-based sampler, while the baselines (which are untrained) use different samplers. The gains may be attributable to the superiority of the entropy sampler itself, rather than the certainty-forcing distillation. (I could be wrong but feel free to correct me on this authors)\n\n2.  **Concerns About Specialized Fine-Tuning:** The distillation process uses a relatively small (100k samples) dataset generated from specific domains (e.g., math and code). This raises the concern that dParallel is effectively a form of specialized fine-tuning, and its impressive performance might not generalize to broader, out-of-domain tasks.\n\nI am open to increasing my score if the authors can provide clean explanations or cleanly ablate the effects of their training method and demonstrate its generalizability beyond the distillation domains or have discussions on addressing this weakness too.", "questions": "1.  To isolate the effect of your proposed training method, could you provide results for the original, untrained LLaDA model using the *same entropy-threshold sampler* you use for dParallel? This is a crucial experiment to demonstrate that the gains are primarily from certainty-forcing distillation and not just the choice of sampler.\n\n2.  Have you evaluated dParallel on broader, general-purpose benchmarks (e.g., MMLU, ARC, HellaSwag) to confirm that the specialized distillation on math/code data does not cause performance regression on out-of-domain tasks?\n\n3.  Regarding the certainty-forcing loss (Eq. 7), what is the typical size of the set `Mc` (correctly predicted tokens) relative to `Ma` (all masked tokens) during training, which kinda of distribution does it follow w.r.t. `t` ? I might miss some details feel free to correct me? Does this ratio change significantly, and how does it affect the stability and effectiveness of the learning signal?\n\n4.  The paper argues that consistency distillation alone is insufficient. However, in Table 1, the performance drop for \"Consistency Distillation\" is quite severe (e.g., 75.7% -> 69.9% on GSM8K). This seems worse than simple few-step decoding. Could you clarify why this baseline performs so poorly? Does this suggest a potential issue with the self-distillation setup itself?\n\nMinor questions:\n- some discussion of the semi-autoregressive masking strategy during training would be beneficial. Why was a block size of 32 chosen for LLaDA but 256 for Dream? How does this choice interact with the goal of promoting parallel certainty?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The identification and visualization of \"sequential certainty convergence\" is a clear and valuable contribution that helps formalize why parallel decoding in dLLMs is challenging.\n\n2. The certainty-forcing loss, which minimizes entropy only on correctly predicted tokens, is an elegant way(IMO) to encourage higher confidence without penalizing the model during its exploration phase.\n\n3. The reported speedups (8.5x-10.5x) with little to no performance drop on challenging reasoning benchmarks are substantial and demonstrate the high potential of the approach.", "weaknesses": "- **Experimental Design Conflates Variables:** This is the most critical weakness. The paper compares `(dParallel-trained model + entropy sampler)` against `(original model + other samplers)`. To truly validate the contribution of the *certainty-forcing distillation*, the essential baseline is missing: `(original model + entropy sampler)`. Without this comparison, it is impossible to disentangle the gains from the training method versus the gains from the inference-time sampler. It is plausible that the entropy sampler is simply a better fit for dLLMs, and the training method provides only a marginal extra benefit.\n\n- **Overfitting danger**: The distillation data is sourced from math and code problems. While the main evaluation is on math and code benchmarks, this raises a serious question about generalization. Does this specialized training harm the model's performance on general-purpose language tasks like commonsense reasoning (e.g., MMLU, HellaSwag) or instruction following? The lack of evaluation on these standard benchmarks is a major omission and leaves the true \"cost\" of this training unclear.\n\n-   **Insufficient Ablation of the Method:** The paper ablates the training objective components but provides limited analysis on key design choices. For example, the certainty loss is applied only to tokens the model already predicts correctly. What percentage of tokens does this typically affect during training? How does this dynamic change over the course of training? Furthermore, the impact of the temperature `T` and the loss weight `β` are not explored, making it hard to assess the method's sensitivity. I understand that one paper might not cover everything but it is worth including some sort discussions and inform the future researchers who might be using this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D8b1vOWYVR", "reviewer": "Reviewer_Shkg", "summary": "The paper introduces dParallel to address sequential certainty convergence in Diffusion LLMs. dParallel minimizes the entropy of top tokens to boost confidence in the model’s predictions, and makes inference semi-autoregressive by breaking down parallel decoding into blocks wherein tokens within a block are predicted in parallel, and blocks themselves follow autoregressive decoding. dParallel significantly reduces the latency in comparison to several recent works without taking a hit on accuracy.", "questions": "* How would this fit with speculative decoding with a large autoregressive model (sharing the same vocabulary) since dParallel seems to satisfy all the requirements for a drafter.\n* What would the final diffusion model look like if it was trained from scratch with dParallel’s objective (without the distillation)?\n* How does LoRA compare to full finetuning for dParallel?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The latency reductions obtained by dParallel are substantial.\n* The paper provides clear ablations on various components of the training algorithm.", "weaknesses": "* dParallel uses self distillation to train the LoRA parameters which requires running a large number of forward passes through the base model making training computationally expensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3VYPn2zDIx", "reviewer": "Reviewer_GE2Q", "summary": "The paper introduces dParallel, a method to unlock the parallel decoding potential of diffusion large language models (dLLMs).\nWhile dLLMs conceptually support parallel token prediction, existing models still require nearly token-length decoding steps to maintain quality.\n\nTo overcome the bottleneck that token confidence builds up left-to-right, they propose certainty-forcing distillation (CFD), a self-distillation technique that encourages the model to reach high certainty across masked tokens in parallel, without disrupting the original decoding trajectory.\n\nWhen applied to LLaDA-8B-Instruct and Dream-7B-Instruct, dParallel dramatically reduces decoding steps (e.g., from 256 → 30 on GSM8K, ≈ 8.5× speedup) while maintaining or slightly improving accuracy.", "questions": "1. How does certainty-forcing behave on non-mathematical text (e.g., story generation or instruction following)?\n\n2. Does parallel certainty hold for open-ended or ambiguous tokens?\n\n3. The paper shows that confidence correlates with correctness, but could the model become over-confident and harm diversity?\n\n4. How robust are results to the choice of β (balance weight) or temperature T in the entropy loss?\n\n5. Is there a theoretical or empirical limit to how few decoding steps dLLMs can reach before degradation (e.g., 5–10 steps)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper well identifies sequential certainty convergence as the fundamental bottleneck in parallel decoding is a sharp and well-motivated observation.\n\n2. Certainty-forcing distillation is conceptually clear and easy to implement. It reuses a pre-trained dLLM as both teacher and student, requiring only modest computational cost (LoRA fine-tuning on consumer GPUs).\n\n3. Across math and code benchmarks (GSM8K, MATH, MBPP, HumanEval), dParallel achieves 5–10× decoding acceleration with minimal or no accuracy drop.\n\n4. The algorithm and implementation details (LoRA config, pseudocode, appendix) are precise and easy to follow and visualization of token confidence propagation (Figures 2 & 5) nicely supports the paper’s core claim about parallel convergence.", "weaknesses": "1. The “certainty-forcing” idea extends existing self-distillation and entropy-minimization concepts rather than introducing a wholly new framework. Its originality lies mainly in the application to dLLMs.\n\n2. The method doesn’t fundamentally enhance reasoning or knowledge capability, and experiments rely on ~ 100 k self-generated math/code prompts; scaling behavior on larger, more diverse datasets or multilingual tasks remains unexplored.\n\n3. The experiments  lack of natural-language, long-form, or open-ended generation tasks makes it unclear how general the method is beyond structured tasks.\n\n4. While empirically convincing, there’s no formal discussion of how certainty dynamics relate to convergence or stability of diffusion decoding, which could also benefit to reveal why easoning or knowledge capability is not enhanced. \n\n5. Recent works like Block Diffusion (Arriola et al. 2025) or Dimple (Yu et al. 2025b), which also target parallel decoding, are cited but not empirically compared.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "muA02D6Cbk", "reviewer": "Reviewer_FJ9p", "summary": "This paper aims to further enhance the parallel decoding capability of Diffusion Large Language Models (DLLMs) through training. The authors argue that the suboptimal performance of current DLLMs in parallel decoding stems from their inadequate confidence distribution, which fails to support effective parallel generation. To address this, they propose a training strategy that explicitly encourages high-confidence predictions across multiple positions. Extensive experiments demonstrate that the trained model achieves significantly improved reasoning efficiency compared to the original model.", "questions": "See Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The acceleration of parallel decoding in DLLMs is a promising and worthwhile direction for further exploration.\n2. Studying parallel decoding from the perspective of confidence distribution offers a novel and interesting angle.\n3. The presented experimental results demonstrate a clear and substantial acceleration effect.", "weaknesses": "1. I have significant concerns regarding the motivation of the proposed method. For example, in Figure 3, the authors’ approach is based on the assumption that the confidence distribution shown in the lower figure is superior to that in the upper one. However, since a given prefix can often be completed in multiple reasonable ways, it is natural and intuitive for the confidence to be higher near the prefix. The authors should provide further clarification and evidence to justify this assumption.\n\n2. I also have concerns regarding the role of Equation (7). Directly optimizing Equation (5) would inherently contribute to optimizing Equation (7) as well. Therefore, the authors should provide a more convincing explanation of the specific role and necessity of Equation (7), for instance, from an optimization perspective or through an analysis of the gradient behavior.\n\n3. Since both the training and test datasets consist of similar mathematical and coding problems, I am concerned about whether the proposed method can truly serve as a generalizable paradigm for accelerating inference through fine-tuning, or if the observed acceleration merely results from overfitting to specific training data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)