Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
Authors: Wenrui Bao
Zhiben Chen
Dan Xu
Yuzhang Shang
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion LLM
Primary area: foundation or frontier models, including LLMs

Abstract: Autoregressive decoding in large language models (LLMs) requires $\mathcal{O}(n)$ sequential steps for $n$ tokens, fundamentally limiting inference throughput. Recent diffusion-based LLMs (dLLMs) enable parallel token generation through iterative denoising. However, current parallel decoding strategies rely on fixed, input-agnostic heuristics (e.g., confidence thresholds), which fail to adapt to input-specific characteristics, resulting in suboptimal speed-quality trade-offs across diverse NLP tasks. In this work, we explore a more flexible and dynamic approach to parallel decoding. We propose **Learning to Parallel Decode (Learn2PD)**, a framework that trains a lightweight and adaptive filter model to predict, for each token position, whether the current prediction matches the final output. This learned filter approximates an oracle parallel decoding strategy that unmasks tokens only when correctly predicted. Importantly, the filter model is learned in a post-training manner, requiring only a small amount of computation to optimize it (minute-level GPU time). Additionally, we introduce **End-of-Text Prediction (EoTP)** to detect decoding completion at the end of sequence, avoiding redundant decoding of padding tokens. Experiments on the LLaDA benchmark demonstrate that our method achieves up to **22.58×** speedup without any performance drop, and up to **57.51×** when combined with KV-Cache.

Published as a conference paper at ICLR 2026
LEARNING TO PARALLEL: ACCELERATING DIFFUSION LARGE LANGUAGE MODELS VIA LEARNABLE PARAL- LEL DECODING
Wenrui Bao**, Zhiben Chen Dan Xu², Yuzhang Shang University of Central Florida, MobbiAA, *KKUST we we769358@ucf. edu; zhiben@mobi.ai
ABSTRACT
Autoregressive decoding in large language models (LLMs) requires O(n) se- quential steps for n tokens, fundamentally limiting inference throughput. Re- cent diffusion-basee LLMs (dLLMs) enable parallel token generation through it- crative denoising. However, current parallel decoding strategics rely on fixed, input-agnostic heuristies (e.g., confidence thresholds), which fail adapt input- specific characteristics, resulting in suboptimal speed-quality trade-offs across di- verse NLP tasks. In this work, we explore a more flexible and dynamic approach to parallel decoding. We propose Learning to Parallel Decode (Learn2PD), framework that trains lightweight and adaptive filter model predict, for each token position, whether the current prediction matches the final output. This learned filter approximates an oracle parallel decoding strategy that unmasks to- kens only when correctly predicted. Importantly, the filter model is learned in a post-training manner, requiring only a small amount of computation to optimize it (minute-level GPU time). Additionally, we introduce End-of- Prediction (EoTP) to detect decoding completion at the end of sequence, avoiding redun- dant decoding of padding tokens. Experiments on the LLaDA [Nie al., 2025] benchmark demonstrate that our method achieves up to 22.58x speedup without any performance drop, and up 57.51x when combined with KV-Cache. To fa- cilitate reproducibility and future research, the project page and code are publicly available Project Page and GitHub Repository.
INTRODUCTION
Large Language Models (LLMs) [Zhao et al., 2023, Ziyu et al., 2023, Minaee al., 2024] have demonstrated remarkable capabilities across a wide spectrum of natural language processing (NLP) tasks. However, most state-of-the-an art LLMs rely autoregressive (AR) decoding [Brown et al., 2020, Radford et al., 2019, Vaswani et al., 2017], which generates output tokens sequentially. Al- though this approach delivers strong generation quality, it inherently suffers from limited inference efficiency due to its strictly sequential nature [Leviathan et al., 2023, Stern et al., 2018]. To overcome this bottleneck, diffusion-based LLMs (dLLMs) [Nie et al., 2025, Ye al., 2025] have been pro- posed as a compelling alternative by enabling parallel token generation through iterative denoising, potentially achieving sublinear complexity (Sohl-Dickstein al., 2015, Li et al., 2022].
Diffusion-basee LLMs (dLLMs) produce or iteratively refine the entire token sequence via denois- ing steps rather than predicting tokens one by one, so token-wise predictions at each step can be computed in parallel. Especially, most dLLMs adopt semi-auloregressie decoding [Arriola et al., 2025], which divides the target sequence into contiguous blocks and decodes the blocks from left to right. It facilitates token- paken-parallelism by trading small amount of autoregressive constraint for substantially higher parallel throughput, while still preserving essential left-to-right dependencies. To fully unlock these benefits, further development of parallel decoding strategy that leverage this approach is needed. Current methods employ static heuristics, for example, confidence-based
"qqual contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
GSMSK (5-shot) Generation Length-1024 Acc-7 7cc-77.6%
LLaDA
0.54
Learn2PD
TPS: 6.63 tokensis
Acc 77.3%
[See. 3.3) &
12,21x
EaTP
TPS: 12.26 tolknns's
Ace 79.8%
(Sec. 3.3)
22.58x
&
TPS: 16.37 tokens's
Acc 74.0%
KV Cache
57,51x
Figure 1: Effectiveness of our proposed approaches. We report the throughput and accuracy on GSM8K (5-shot, Generation Length=1024) with LLaDA and our proposed methods under four set- tings: (1) vanilla decoding, (2) Learn2PD policy, (3) Leam2PD and EoTP mechanism, (4) Learn2PD and EoTP integrated by KV Cache. Our proposed methods, Lear22DD and EoTP, yield a 22.58x speedup over the vanilla baseline while simultaneously preserving the original accuracy. Integra- tion with KV Cache achieves a further improvement in throughput to 16.37 tokens/sec (a 57.51x speedup), with only minimal loss in accuracy.
sampling [Chang al., 2022] prioritizes the most confident tokens for parallel decoding. Although these methods speed up inference, their static decoding strategies lead to poor generation quality.
Targeting this static limitation, we pose an intuitive question: Instead of relying on a one-rule-ftt--ee decoding strategy, can we adopt a flexible, case-by-case one for parallel decoding? To answer this, analyzed the model's token-level decoding behavior and found that current models often remask tokens that have already been correctly predicted, leading to unnecessary computational redundancy. Taking advantage of this finding, we propose that an effective parallel decoding strategy should be capable of eliminating such redundancy. To realize this goal, we first establish an oracle baseline: Extremely Greedy Parallel (EGP), which unmasks each token immediately upon correct predic- tion. In the oracle, we use the reference answers unmask token when its prediction matches the ground truth. Our analysis reveals that this oracle can achieve a 15-20x speedup without quality loss, demonstrating substantial potential to improve parallel decoding. However, its dependence on unavailable ground truth makes it infeasible in practice.
To approximate this oracle, we propose Learning to Parallel Decode (Learn2PD), the first learned parallel decoding policy for dLLMs. The framework learns to predict when to finalize a token that is, when we have sufficient confidence to accept its current prediction. The key insight is that diffu- sion models exhibit predictable confidence patterns [Song et al., 2020, Nichol & Dhariwal, 2021]: the confidence score for each token can be treated as an informative feature. Fluctuations in these scores capture the model's internal state of acceptance or doubt regarding its predictions. Specif- ically, we train a lightweight filter model fo that predicts whether each token has been correctly generated. The filter model is optimized in the post-training phase, requiring minute-level GPU time for convergence. Once trained, this filter model remains fixed and requires no gradient updates during inference. The filter takes the model's confidence scores as input and outputs binary deci- sion for each token to indicate whether should be remasked. Surprisingly, a simple two-layer MLP [Tolstikhin al., 2021] performs exceptionally well at this task, as the block-level confidence pat- terns provide sufficient information for accurate convergence prediction, thus eliminating the need for complex architectures task-specific feature engineering.
Another finding from the EGP oracle is that even when the (End-of-Texee token is unmasked, the model continues the decoding process for subsequent tokens. When the generation length is 1024, this inefficiency is responsible for 90% of the computational waste. To reduce the excessive decod- ing steps after the [End-of-Text] token, we introduce End-of-Text Prediction (EoTP) mecha- nism. EoTP can terminate decoding as soon as the [End-of-Texi] token is confidently generated, which avoids redundant computation and further boosts decoding efficiency.
Our method accelerates dLLMs by eliminating redundant decoding operations, thereby preserving generation quality. Experimental results demonstrate remarkable 22.58x speed-up LLaDA while fully maintaining its performance. Importantly, our method is orthogonal to existing opti- mizations: when combined with KV caching, the speedup compounds to 57.51x accompanied by only slight degradation in accuracy (See Figure 1). In summary, our contributions are threefold:
We propose a novel and adaptive framework, Learn2PD that predicts which tokens have been correctly decoded, approximating the oracle Extremely Greedy Parallel Decoding strategy.
2

---

Published as a conference paper at ICLR 2026
2. We also propose Enx-of-Text Prediction (EoTP) mechanism to reduce the unnecessary decod- ing steps, which significantly boosts inference efficiency.
3. We extensively evaluate method on various dLLMs across four representative benchmarks: GSM8K, MATH, HumanEval, and MBPP. Our method consistently achieves order-of-man -fagentuu- inference acceleration with negligible aceuracy loss. Specifically, method attains significant 22.58x acceleration without any degradation in accuracy.
RELATED WORK
2.1
DIFFUSION-BANE BASED LARGE LANGUAGE MODELS
The integration of diffusion models with large language models (LLMs) is emerging and promis- ing direction in generative AI. Early work adapted continuous diffusion to discrete data domains [Sohl-Dickstein al., 2015, Hoogeboom al., 2021 J, leading to D3PM [Austin et al., 2021a], which introduced a Markov chain-based based framework for discrete noise injection and denoising trained via ELBO maximization. This was extended to continuous time by CTMC [Campbell al., 2022]. In parallel, SEDD [Lou et al., 2023] learned the reverse process by modeling the ratio of marginal prob- abilities using a denoising score entropy objective, while Masked Diffusion Models such as MDLM [Shi et al., 2024, Sahoo et al., 2024, Zheng et al., 2024] and RADD Ou al. [2025] provided further theoretical simplifications and formalized connections between parameterizations. A key break- through has been the incorporation of diffusion into existing LLM architectures: Diffusion-NAY [Zhou et al., 2023] aligned the denoising process with non-autoregressive decoding, enabling high- speed generation, while models like LLaDA [Nie et al., 2025]. DiffuLLaMA [Gong al., 2025], and Dream (Ye et al., 2025] successfully scaled diffusion-based decoding to billion-parane parameter models, significantly improving inference efficiency without compromising output quality.
2.2 ACCELERATE DIFFUSION-BASED LARGE LANGUAGE MODELS
Followed by mature diffusion large language models, their acceleration methods are also under de- velopment. Concretely, dllm-Cache [Liu et al., 2025] proposes a training-free, adaptive caching framework that performs long-interval prompt caching and short-interval, value-similarit-guidee partial response updates. Fast-dLLM [Wu et al., 2025] introduces block-wise approximate KV caching and confidence-aware parallel decoding rule that only decodes tokens whose marginal confidence exceeds a threshold. Hu et al. [2025] propose FreeCache to approximate KV states by reusing stable prompt/block activations across steps. They also introduce Guided Diffusion to de- cide which tokens to unmask each step without retraining. SlowFast Fast-Sampling [Wei et al., 2025] proposes dynamic two-stage sampler that alternates cautious exploratory phase with a fast phase that aggressively decodes high-confidence tokens within that span. Prophet [Li et al., 2025] mon- itors the top-2 logit gap and commits all remaining tokens in one shot via early-commit decoding once it is sufficiently confident. However, these accelerate methods are often static and lack flexibil- ity. To address this, we propose Learn2PD, novel dynamic remasking method that achieves more efficient inference acceleration by reducing the unnecessary and repetitive decoding steps. More- over, we also introduce EoTP to avoid redundant decoding when the answer does not span the full generation length.
3 METHODOLOGY
In this section, we present Learn2PD, a learned approach to accelerate diffusion language model inference through adaptive parallel decoding. We begin by reviewing the fundamentals of diffusion language models and their current parallel decoding strategies (Section 3.1.1). Through empiri- cal analysis, we reveal critical inefficiency: existing methods unnecessarily remask significant proportion of correctly predicted tokens, leading to redundant computation (Section 3.1.2). This observation motivates stability and approximate an oracle parallel decoding strategy (Section 3.2). Finally, we introduce an early-stopping mechanism further eliminate padding token overhead (Section 3.3).
core contribution- training lightweight filter model to predict token
3

---

Published as a conference paper at ICLR 2026
Prompt
Response
Prompt
Response
podkie
Mask predictor
Blaoc
Mask predictor
Motched
Unmatcced
Praticion
Filter fo
Bonaak
Masked token
Remask
Unmasked token
(e) Extromely Greedy Parallel Decoding
(b) Learning to Parallel Decoding
Figure 2: A Conceptual Overview of pipeline and method. (a) Extremely Greedy Parallel (EGP). This strategy compares the predicted tokens with the reference answer and only remasks the tokens that do not match in these comparisons. the inference process, after the model generates predictions and confidences for each token, the confidence of each token is fed into a filter model fo to determine which tokens need to be remasked. This determination then guides the subsequent remasking procedure.
(b) Learning to Parallel Decoding (Learn2PD). During
3.1 PRELIMINARY
3.1.1 DIFFUSION LARGE LANGUAGE MODELS
Forward Process. Given an input sentence No 10, 1- and noise level E [0, 1]. where V and I, represent the vocabulary size and sentence length. The forward process randomly and independently masks out tokens through the following Markov chain:
4 (x mo)
II (1 1)-
m/} +
1fxr = m)
where zf denotes the i-th element of ær, denotes the mask token [Devlin al., 2019], 24 denotes the noisy data at time t, and (-)-) is the data distribution (.)
Reverse process. The reverse process iteratively recovers masked tokens by predicting data dis- tribution from a masked sequence. Transitioning from corruption level t to an earlier level s, where 0 < t can be approximated as
xt + m, x². = wi
L-1 (a) xxt) = II Geje we) Gote (xe an))
xt m, x²
- 4ota (x) xx))
xt m, x
(2)
where m represent the [MASK] and (oje(-) the data prediction distribution by the model [Ho al., 2020]. Given prompt (c1,-.., the response generated in K discrete steps. In each step k., mask predictor pa takes (2) input and predicts the distribution of sequence. The estimate of the sequence (()) is generated via greedy decoding:
(0) arg max Po (y c, (() yET
arg max po (o. ((),
To improve the sample quality, the unmasking tokens with low confidence would be remasked. This approach follows a common practice in non-autoregreesivve generation for improving output fidelity [Ghazvininejad et al., 2019]. For each position i, the model predicts so (k) and computes its confidence 4, which is given by:
Low-Confidence Remasking.
C; Pe (yō. (l) (A))
The tokens corresponding to the n lowest confidence would be set to [MASK] again, where n is calculated by the noise level

---

Published as a conference paper at ICLR 2026
Frrlt carpot prodction Ato
Trtt cormec prodecco dea
stap
32
Figure 3: Samples of gaps in different datasets: GSM8K and HumanEval. The red line means the first correct prediction step, and the blue line means the actual decoding step.
3.1.2 UNNECESSARY REPETITIVE DECODING
Building on the iterative inference process dis- played in Section 3.1.1, we investigate the unnecessary and repetitive decoding condi- tions in diffusion-based large language mod- els. We conducted experimental analyses with L.aDA-8B-Instruse
GSMSK
et al., 2025] on two widely used datasets: GSM8K [Cobbe et al., 2021] and HumanEval [Chen et al., 2021]. We choose LLaDA as our base model due to its state-of-the-art performance and availabil- ity of pre-trained checkpoints across multi- ple scales. During the inference process, the model generates predictions for each masked token and re-masks those with low confidence. To maximize accuracy, existing dLLMs of- adopt conservative re-masking strategy, which results in many correctly predicted to- kens being unnecessarily re-masked and re- predicted. Specifically, we measured the ex- tent of redundant and repetitive decoding, de- fined as the number of times the model con- tinues to decode a token after it has already matched the reference answer. In this paper, we refer to the answer produced by LLaDA under the standard generation process the reference answer."
199
100
HumnsnFval
133
100
Figure 4: Distributions of gaps in different datasets: GSMSK and HumanEval. These two histograms show the distribution of step gaps for each token between the actual decoding step and the step with the first correct prediction.
Analysis of unnecessary repetition. As illustrated in Figure 4, we tally up the distribution of the step gaps between the decoding step and the step with the first correct prediction. For each dataset,
For all analyses in this section, L.LaDA's Generation Length at 128 and Block Size 32.
5

---

Published as a conference paper at ICLR 2026
Block
Block 2
Block
Block 6
60
Freuueccy
20
0
7
2
5
5
Number of steps
Number steps
Number of
Number steps
Figure 5: Distribution of decoding steps per block with Extremely Greedy Parallel (EGP) strat- egy. Histograms illustrate the number decoding steps performed in each block when using our -nDts----Isttruct strategy with LLaDA-8B Instruct on GSM8K based on 100 samples.
we randomly sample 10 questions to conduct the experiment. We find that most of the tokens still need to be decoded more than 10 times, even though they are already correct. And Figure 3 shows one sample from each dataset. The red line means the first correct prediction step, and the blue line means the actual decoding step. It is clear that the model performs many unnecessary decoding steps before unmasking the tokens.
3.1.3 EXPECTED INFERENCE PROCESS: EXTREMELY GREEDY PARALLEL (EGP)
Based on the above findings, we observe that a large portion of tokens are remasked as [MASK] and decoded multiple times Motivated by this, we define the Extremely Greedy Parallel (EGP) oracle (Figure 2a) as: at each step k, unmask token i and only if M(*);, ys, where ye is the reference answer for token This oracle achieves optimal speedup by never remasking correct predictions.
after they have already been decoded to the reference answer.
Acceleration Potential. To evaluate the efficiency of our strategy, we compared the number of the GSM8K dataset under the Ex-
decoding steps required per block for MAADA-SB-Instrus tremely Greedy Parallel policy versus the standard decoding regime. Similarly, we fix the Gener- ation Length to 256 and the Block Size to 32.
As shown Figure 5, the results are striking. Our strategy achieves a median of decodings per block while maintaining the same accuracy. In contrast, LLaDA with the vanilla setting requires 32 decodings per block. This demonstrates a substantial opportunity for efficiency gains, without compromising output quality.
3.2 LEARNING TO PARALLEL DECODING
Although our Extremely Greedy Parallel strategy performs well, this oracle requires ground truth tokens that are unavailable during inference. To address this, we propose a novel approach: Learn- ing to Parallel Decoding (Learn2PD) (Figure 2b). Our goal is simulate the EGP strategy after each decoding step to select tokens and decide whether to remask. We can reformulate this as an optimization problem by using Binary Cross-Entropy Loss (BCELoss) [De Boer et al., 2005]:
arg min
1 m
y: log (1 log(1 -pa)
(5)
where yi indicates whether token 4 should be remasked under the EGP strategy: means it should be remasked and means it can be unmasked. During the inference process, a threshold 7 is applied discretize Pt into either 0 or 1. The p. is generated from which called the filter model fa- In this algorithm, the only trained parameters are 6. Therefore, the parameters of the diffusion large language model remain unchanged. Then, the training loss should be:
LBCE
yi log (z1) + (1 y.) log(1 - (2)))
(6)
m
where is the output of the filter model (logit). We show the algorithms for training and inference in Algorithm and Algorithm 2. The filter fo takes the confidence of the prediction as input and
returns the logits indicate the probability of remask. And in order ensure remains in

---

Published as a conference paper at ICLR 2026
the range [0, 11, apply a sigmoid function before it is passed to the dLLMs. Critically, the filter model fo adds negligible overhead during inference. Our experimental results in Section quantitatively demonstrate that the achieved speedup vastly outweighs this minimal overhead.
Algorithm Training
Algorithm 2 Inference
Require:
Diffusion large language model M, filter model fe: prompt set Spreepe, reference answer sel Enferees, generation length Lezz. leurinng 1,, block size
Require: Diffusion large language model M. lilter model fa, prompt set Sprcnpe generation length Lyra. block size s. filler threshold for each i; Zpoonpt do
1: repeat
4. length(z;;);
concat(ar;; (24)
2: 2t Ziprerge- Tv Rntermee length(w:))
for 6 U, Aig do 3:
3:
concat( 2,, (m)
4:
for = O, Aym
4:
while .M * 0 do
M (1,2.....
5:
.M- - (1,2.....5)
6:
conf., pee, (x)) logit, (sont))) logit then
6:
while
do M(X)
7:
7:
8:
fo:
p = ra then
8:
(i), Xutbi++s + pro
9:
1/ 1,
pres. (1)
9:
end end while
10:
10:
X4 16
11:
end for
ebs
12:
response; Xegitt Lea-1
12:
13:
end if
BCELOSS(i, fo(conf())
14:
end for
response
15:
0 0- y Vol.
16:
end while
17:
18:
until converged
3.3 END-OF-TEXT PREDICTION
Input tokens
Besides the methods mentioned earlier, we observed that when the generation length of a diffusion large language model is in- creased to 1024, the generation time rises significantly for the same question com- pared to a length of 256, even though the final answer length remains unchanged. According to the analysis of the gener- ated output, we find that the extra length is filled with the [EoT] token, and the additional decoding time spent repeat- edly decoding the [EoT] token. Based on this, we propose the End-of-Text Pre-
check
[EoT]
Yes
No
Learn2PD
Input tokens for next step
All tokens after [Eo are discanded
diction (EoTP) approach: at each step. when [EoT] token is unmasked, dis- card tokens in subsequent positions. The shortened sequence then used as the in- update
Masked token
Unmasked token
Discarded
Figure 6: Schematic of the End-of-Text Prediction Policy. During the inference process, upon detection of an [EoT] token in a decoded block, all subsequent tokens are discarded.
put of next step. Therefore, the inference process to handle the long- generation-length challenge in Appendix B. We illustrate the details in Figure Our analysis shows that 89.59% of computational cost comes from decoding padding tokens after [EoT]. EoTP yields substantial computational savings by dynamically reducing the effective input length throughout the diffusion process. The experiments and relevant analysis are Appendix D.
4 EXPERIMENT
4.1 EXPERIMENTAL SETTINGS
Models and Datasets. We implement our methods on the representative dLLM: LLaDA-8B. Instruct [Nie et al., 2025] to measure the acceleration of the inference process across various benchmarks. To ensure the broad applicability the methods, we conducted experiments on four datasets covering three different types of problems, which GSMbe|Cobbe et al., 2021], Math

---

Published as a conference paper at ICLR 2026
Table 1: Benchmark results the L...DA-B--nstruuc suite. Each method evaluated using two generation lengths (256 and 1024) across four datasets. Performance is measured using three metries: TPS (tokens/sec), speedup, and accuracy score. The highest throughput and speedup values for each configuration are highlighted in bold.
Inference Efficiency
Performance
Task
Methods
Gen Length
TPS1
Speed (TPS)t
Score
256
3.41
1.00x
78.70
LLaDA-8B-Instun
1024
0.54
1.00x
77.60
GSM8K (5-shot)
256
14.07-10.66
4.13x
78.62
Learn2PD
1024
6..3
12.21 x
77.26
256
11.35-10.94
4.21 x
78.62
Lear22D EoTP
1024
12.26
22.58 x
79.83
256
4.70
1.00x
32,90
L.LDA-SB-Instru
1024
1.70
1.00x
35.21
Math (4-shot)
256
15.16-10.46
3.21
32.22
Learn2PD
1024
10.98
6.45
34.01
256
15.21-10.51
3.23
31.40
Leam2PD EoTP
1024
12.27-10.57
7.22x
34.60
256
3.33
1.00x
39.63
LLaDA-8B
-tatA--B-Instruct Instruct
1024
0.53
1.00x
37.21
HumanEval (0-shot)
256
11.66.48.3
3.5x
38.41
Learm2PD
1024
44.20
8.78x
37.84
256
11.88
3.57
38.41
Lear2PD ЕoTP
1024
6.63
12.55x
35.98
256
3.14
1.00x
31.22
(LaDA-8B-Instrus
1024
0.58
1.00x
10.61
MBPP (3-shot)
256
14.96
4.77x
30.84
+ Learn2PD
1024
6.96
12.08x
10.04
256
15.88-12.74
5.06x
31.03
Lear22D + ЕoTP
1024
9.89
17.16x
11.02
[Lewkowycz et al., 2022], HumanEvallChen et al., 2021], and MBPP [Austin et al., 2021b]. All experiments are conducted on 4 NVIDIA A6000 GPUs.
Filter Model fo Training. To train filter model that can be applied wide range of tasks, selected 40 samples from each of the 66 types of questions in the FLAN dataset, resulting in a total of 2,640 samples for training. In this experiment, we used the simplest two-layer MLP as our filter model. Since the dLLMs remains frozen and only fe is trained, the number of trainable parameters is extremely limited. For example, for LLaDA with block size of 32, the total number of trainable parameters is only 2.1 12. We trained fe for 5,000 epochs until the model converged. The learning rate is set 0.001, and the Adam optimizer used optimize fo-
Our training process consists of two stages. In the first stage, samples are collected by following an Extremely Greedy Parallel policy, recording the confidence scores and token selections each step during parallel decoding. This data is then used in the second stage to train a filter model fa. The data collection in the first stage was conducted on NVIDIA RTX A6000 GPUs and took approximately three hours. The subsequent training of the filter model in the second stage was deployed on T4 GPU and required only 6 minutes. The details of training are in Appendix E.
Evaluation. We evaluate the inference acceleration and generation quality of Learn2PD and ЕoTP methods by using quantitive metrics. The inference speed is quantified with Tokens Per Second (TPS), indicating the average number of tokens generated per second. And the generation quality is measured in task-specific metrics, such accuracy for GSM8K, showing the model's
8

---

Published as a conference paper at ICLR 2026
performance with acceleration methods. In addition this, set the Generation Length to 256 & 1024 and the Block Size to 32.
4.2 MAIN RESULTS
We present the inference performance and efficiency profits for Learn2PD and EoTP on the LLaDA- 8B-Instruct across four benchmarks, as shown in Table I.
In summary, Learn2PD significantly enhances inference efficiency across all tasks. Compared to the baseline model, our optimal method typically achieves a to 4 times speedup at generation length of 256 and 6 to 12 times speedup generation length of 1024. When EoTP is incorporated, the improvements become even more pronounced, particularly with a generation length of 1024. For instance, combining Learn2P and ЕoTP results in throughput increase of 22.58x (on GSM8K, 5-shot) and 17.16x (on MBPP, 3-shot) relative to the baseline. These results demonstrate that our methods not only effective individually but also highly orthogonal, resulting in compounded acceleration. More importantly, these efficiency gains have negligible impact on accuracy. The performance scores our accelerated methods remain within 1-2 points of the baseline, and in some cases, the score is even slightly improved.
4.3 COMPATIBILITY WITH KEY- VALUE CACHI
We further evaluate the compatibility of our approach with established Key-Value (KV) Cache tech- niques by integrating both Dual Cache and Prefix Cache strategies [Wu et al., 2025]. Experiments conducted on GSM8K with generation length of 1024 tokens. As summarized in Table 2, the baseline model (Learn2PD & EoTP) achieves a throughput of 12.26 TPS, a speed-up of 22.58x, and an accuracy score of 79.83. When augmented with the Dual Cache, the system attains substantially higher efficiency. reaching 31.23 TPS and 57.51x speedup. albeit with a slight decrease in accu- racy (74.0)). Similarly, incorporating the Prefix Cache also brings noticeable improvements, yield- ing 14.79 TPS and 27.23x acceleration while maintaining a competitive score of 77.71. These results confirm that our method orthogonal to and fully compatible with standard KV caching mechanisms, demonstrating its ability to leverage such strategies to enhance inference efficiency.
Table 2: A comparison of our method with and without KV Cache. The results show a signifi- cant performance improvement when augmented with both Dual and Prefix Caches, underscoring that our method is orthogonal to and fully com- patible with existing KV caching strategies.
Table 3: A comparison of the acceleration perfor- mance using filter models of varying complex- ity (represented by the number of MLP layers). The results indicate that two-layer MI.P model achieves the optimal balance by providing signif- icant speedup.
Methods
TPS
Speed
Score
# Layers
TPS
Speed
Score
Learn2DD & EoTP
12.26
22.58x
79.83
Single-layer
8.77
2.57x
78.62
+ Dual Cache
31.23
57.51x
74.00
Two-layer
14.07
4.13x
78.62
Prefix Cache
14.79
27.23x
77.71
layer
11.41
3.35x
78.85
4.4 ANALYSIS: ABLATION STUDY
Effect of Filter Model fo Complexity. To investigate the impact of the filter model's architec- tural complexity on acceleration performance, we conduct an ablation study using MLP-based fil- ter models with varying depths. As illustrated in Table 3, a two-layer MLP achieves the highest throughput (14.07 TPS) and speed-up (4. 13x), while maintaining 78.62% accuracy. comparison, the lingle-layer model yields lower efficiency with similar accuracy, suggesting limited represen- tational capacity. Although the four-layer model attains a marginally better accuracy score, it results in reduced inference speed, indicating increased computational overhead. These results demon- strate that a two-layer configuration offers the optimal trade-off between efficiency and predictive performance, effectively balancing model complexity and acceleration gain.

---

Published as a conference paper at ICLR 2026
Effect of Generation Length. To examine the impact of generation length on the performance of our methods, compare the speedup and accuracy of Learn2PD) and its enhanced vari- (Learn2PD & FoTP) across varying output lengths. As shown in Table 4, both methods achieve greater acceleration as the generation length increases, while consistently maintaining competitive scores. At a shorter length of 128, Learn2PD & EoTP reaches speed-up of 3.36x.
Table 4: Performance comparison of our methods across different generation lengths. While maintaining a comparable accuracy, both Learn2PD and To deliver substantially greater speedup at length of 1024 compared to shorter sequences.
Gen Length
Methods
Speed
Score
Leam2PD
x.29
73.92
128
Lear22PDD EoTP
3.36x
74.07
And the speed-up improves steadily with longer sequences, culminating in substantial 22.58x acceleration at a length of 1024. These results in- dicate that our approach is particularly effective for long-sequence generation, efficiently reduc- ing the unnecessary decodings maximize infer- ence speed without compromising output quality.
Leam2PD
4.18x
78.62
256
Lear2PDD & EoTP
4.21
78.62
Leam2PD
6.66x
77.71
Leam2PD EoTP
7.60x
79.08
Leam2PD
12.21 x
77.26
1024
Lear22DD EoTP
22. x
79.83
Effect of Filter Model fo threshold т. We perform an ablation study to examine the im- pact of the filtering threshold on inference ac- curacy and throughput. As shown in Figure 7, reducing the threshold improves through- put but leads to corresponding decline in ac- curacy. For example, at т = 0.99, the model achieves throughput of 4.68 TPS (vs. base- line 3.41 TPS) with 78.92% accuracy. In con-
Gea Leigh-tte
137%
GSMNK (5-she) Meersey
4X13X
1-0.90
12-
3,72x
4,49
r+0.95
trast, lowering the threshold to r 0.9 causes a more pronounced reduction in accuracy. The results indicate that a threshold of 0.96 of- fers an optimal balance, delivering both high throughput (4.13x speedup) and bese-baseline accuracy. These findings underscore the criti- cal role of the filtering threshold in achieving an effective trade-off between inference effi- ciency and output quality.
5.77X
Thrugggua (Takeenahac)
Figure 7: Impact of the filtering threshold curacy and throughout. We find that a threshold of 0.96 represents favorable balance, maintaining high accuracy and comparable inference speed.
CONCLUSION
In this work, we investigate the issue of extensive repetitive decoding during inference in Diffusion- based Large Language Models. To enable timely unmasking of correctly predicted tokens, we pro- pose Learn2PD, a parallel decoding architecture that employs a filter model to make case-specific selections. This filter model is lightweight and tre-trained, thus requiring additional training dur- ing inference. Furthermore, to address the time overhead caused by repeated encoding of the [EoT] token as the generation length increases, introduce the EoTP mechanism, which halts decoding immediately after [FoT] is generated, thereby redueing unnecessary computational cost. Exten- sive experiments across multiple benchmarks and model baselines (LLaDA) demonstrate that our approach achieves up to 22.58x speedup without sacrificing accuracy- and up to 57.51x when com- bined with KV Cache. Our proposed method offers a compelling solution for deploying diffusion- based LLMs as alternatives to autoregressive models in future applications.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We acknowledge that our acceleration techniques, by lowering the computa- tional cost of inference, could inadvertently lower the barrier for malicious use. This could enable bad actors to scale up harmful applications such as disinformation campaigns, spam, phishing, or automated malicious code generation more efficiently.
Potential for Misuse:
REPRODUCIBILITY STATEMENT
To support reproducibility, a complete anonymized code repository is provided as supplementary material. The repository encompasses all necessary components to replicate our work: the imple- mentation source code for the proposed model and algorithms, the scripts required to run the exper- iments, comprehensive hyperparameter configuration files, and detailed execution instructions.
REFERENCES
Marianne Arriola, Subham Sekhar Sahoo, Aaron Gokaslan, Zhihan Yang. Zhixuan Qi, Jiaqi Han, Justin T Chiu, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In The Thirteenth International Conference Learning Repre- sentations, 2025. URL https: hictps://pperrevirt wwtp:wiwwwwweeeeeeeeeee.eeeeeeee net www.:ood-uuutaaoo httpeeeewwwee..ttttoooooooaaaaae 67.x.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 101111991-7777 1179993, 2021a.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint Att::2100.007332. 2021b.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 55:28266-28277 2022.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked gener- ative image transformer. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray. Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert-Vosss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano. Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10.111011111000 14168, 2021.
Pieter- Tjerk De Boer, Dirk P Kroese, Shie Mannor, and Reuven Rubinstein. tutorial on the cross-entropy method. Annals of operations research, 134(1):19-67, 2005.
11

---

Published as a conference paper at ICLR 2026
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask Mask-predict: Parallel decoding of conditional masked language models. arXiv preprint arXiv: (904:0990.09444 2019.
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL https:///// https////pppeeeeee..ooom net forumoorrm?cc jitsLYKwgB.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in neural information processing systems, 34: 12444-12412455, 2021.
Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprint 2.XXx:2505.21400 2025.
Yaniv Leviathan, Matan Kalman, and Yossi Matias, Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp. 19274-19286. PMLR, 2023.
Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Son -oloo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative rea- soning problems with language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: http:///peeeeeeeeeeeeeeeeeeeeeee net
Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508. 19982, 2025.
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion- Im improves controllable text generation. Advances in neural information processing systems, 35: 4328 4343, 2022.
Zhiyuan Liu, Yieun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.066. 06290, 2025.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. 2023.
Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Asgari Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey, ArXiv, abs/2402.06190 2024. URL https:://// /ppi. www.nteessssieooe wew.mmiiiicooll...............a. org wwtg///oopssDi..oo
2676177332
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171, PMLR, 2021.
Shen Nie, Fenggi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen. and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.00990 09992, 2025.
12

---

Published as a conference paper at ICLR 2026
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions clean data. In The Thirteenth International Conference Learning Representations, 2025. URL https:
//oppnreviee ew, net. forum? i d= XP 8 Tanm.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136-1300 2024.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances neural information processing systems, 37: 103131-103161 2024.
Jaseha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265, pmlr, 2015.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit. Blockwise parallel decoding for deep autore- gressive models. Advances in Neural Information Processing Systems, 31, 2018.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Peter Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. MLP-mixer; An all-MLP architecture for vision. In A. Beygelzimer, Y. Dauphin, Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https: /ppp/eeeeeeeeeeeeeeeee/ee eew...ww.. net forum? id=E IIKoXkkk....
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Dongrui Liu, and Linfeng Zhang. Accelerating diffusion large language models with slowfast sampling: The three golden principles. CoRR, 2025.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training- free acceleration of diffusion Ilm by enabling kv cache and parallel decoding. arXiv preprint 1.Xiv:2505.22611. 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508. 15487, 2025.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du Chen Yang. Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian Yun Nic, and Ji- Rong Wen. A survey of large language models. arXiv preprint arXiv: 18223, 2023. URL http::/ /tap:///arxvv.../ orgg abs 22033 18223.
Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yo Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. arXiv preprint arXiv:2409.02900 2024.
Kun Zhou, Yifan Li, Wayne Xin Zhao, and Ji rong Wen. Diffusion-nat: Self-prompting discrete diffusion for aon-autoreeressivve text generation. In Conference of the European Chapter of the Association Computational Linguistics, 2023, URL https: / api semant icscholar. oTt//CorppuTD::2855777
13

---

Published as a conference paper at ICLR 2026
Zhuang Ziyu, Chen Qiguang, Ma Longxuan, Li Mingda, Han Yi, Qian Yushan, Bai Haopeng, Zhang Weinan, and Ting Liu. Through the lens core competency: Survey on evaluation of large language models. In Jiajun Zhang (ed.), Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 2: Frontier Forum), pp. 88-109, Harbin, China, August 2023. Chinese Information Processing Society of China. URL https: / https////aaaaaalo//o../ org/2023.ccl 1...023.cc-..8.......
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
Large language models (LLMs) were used in this work exclusively for the purpose of text polishing and refinement. Their role was strictly limited assisting with grammatical correction, improving sentence fluency, and enhancing word choice to increase the overall clarity and readability of the manuscript.
B UPDATE INFERENCE ALGORITHM
Algorithm 3 Update Inference
Require: Diffusion large language model M, filter model fo. prompt Pprompe generation length Lygen block size s, filter threshold
1: for each I; proompp do
2: 4 x (eeenggh(aa)) X concat(x;, [MASK] (mm))
Ly do 4: M + {1,2.
3:
for b
5: while M + do confi, pre, M (X), logit, = foonf)))) if logitss > r then M M (j}, X4bs+s precd
6:
7:
8:
9:
end if end while
11:
if [endoftext] in X then break 13: end if
end for
15:
response, = Xtaht-
16:
end for
17:
return response
C EXPERIMENTS ON DREAM MODEL
Table 5: Results of Learn2PD and EoTP mechanisms Dream in different generation lengths.
Generation Length
Methods
TPS
Speed
Score
Dream
2.38
1.00
74.6
256
Learn2PD
4.04
1.70x
74.5
Leam2PD EoTP
4.73
1.99
73.4
Dream
0.38
1.00x
71.9
1024
Leam2PD
2.19
5.84x
72.1
Lear2PD & EoTP
4.12
10.99
73.1
The table presents the results the Learn2PD and EoTP mechanisms applied to the Dream model across two generation lengths: 256 and 1024. For a generation length of 256, adding Learn2PD increased the TPS to 4.04 and speed to 1.70x, with a slightly higher score of 74.5. Combining Learn2PD and EoTP further improved TPS to 4.73 and speed to 1.99x. For generation length of 1024, incorporating Learn2PD significantly boosted TPS to 2.19 and speed to 5.84x, with a marginally higher score of 72.1. Combining Leam2PD and TP yielded the highest TPS of 4.12 and speed 10.99x. These results demonstrate that Learn2PD and Еo could considcrably accel- erate the Dream model with only minimal loss in performance.
15

---

Published as a conference paper at ICLR 2026
D EXPERIMENTS AND ANALYSIS ON ЕОTP MECHANISM
Table A comparison of LLaDA and EoTP mechanisms in different generation lengths.
Methods
Generation Length
TPS
Speed
Score
256
3.41
1.00x
78.70
LLaDA
512
1.67
1.00x
77.71
1024
0.54
1.00x
77.62
256
3.76
1.10x
79.23
+ EoTP
512
3.38
2.02x
78.77
1024
3.25
5.98x
79.38
This table 6 clearly demonstrates the sigmificant advantage of integrating the EoTP mechanism with the base LLaDA model, particularly for long-sequence generation. While the standalone LLaDA model exhibits substantial performance degradation as generation length increases evidenced by the sharp decline in TPS from 3.41 to 0.54-the incorporation of ЕoТP not only mitigates this degradation but also delivers considerable speedup. Most notably, at sequence length of 1024, EoTP achieves dramatic 5.98x acceleration while simultaneously improving output quality, with the Score increasing from 77.62 to 79.38. These gains can be largely attributed to the elimination of redundant computation: quantitative analysis shows that 89.59% of the baseline computational cost arises from decoding padding tokens after the [EoT] token. FoTP effectively removes this overhead by dynamically detecting sequence completion once all original non- [MASK] positions are unmasked, thereby optimizing inference efficiency without compromising performance.
16

---

Published as a conference paper at ICLR 2026
E TRAINING CURVE OF FILTER MODEL
As shown in Figure 8, this learning curve illustrates that both training (blue) and validation (red) loss fall sharply in the earliest epochs (from 0.68 0.28), then decrease much more gradually over the full 5,000-epoch rum, approaching plateau near 0.21-0.23. The validation curve remains slightly above the training curve throughout, indicating only a modest generalization gap rather than pronounced overfitting: the parallel, steady decline shows the model continues to improve on unseen data but with diminishing returns. The long, flat tail of both curves indicates the model has effectively converged.
Trining
Validation Loss
0.6
0.4
1000
2000
3000
4000
5000
Figure 8: Learning curve of filter model fe. The learning curves illustrate the progression of training and validation loss across 5,000 epochs, with the former in blue and the latter in red.
17


Reviews:
[{"review_id": "ivQjL7Qwbz", "reviewer": "Reviewer_Y2jx", "summary": "This paper proposes Learn2PD, a dynamic re-masking method that enables more efficient generation in diffusion language models. A learnable filter model is employed to achieve substantial acceleration in generation performance.", "questions": "1. The proposed filter adopts a pure MLP architecture, which can be understood as a form of confidence-based feature engineering. Have you attempted to incorporate context-aware structures such as attention mechanisms?\n\n2. There is a typo: on line 326, it should be 256 and 1024.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a clear motivation, employing deep learning techniques to design a more principled unmasking mechanism.\n\n2. The proposed Filter model is lightweight and easy to obtain.", "weaknesses": "1. The main experiments are conducted on the LLaDA model, and the effectiveness of the proposed method on causal diffusion models such as Dream remains to be further verified.\n\n2. There is concern about the method’s performance on finer-grained blocks in mainstream block-level diffusion models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0fZBZ6Ni3k", "reviewer": "Reviewer_pSk7", "summary": "This paper addresses the autoregressive decoding bottleneck in large language models (LLMs) by focusing on diffusion-based LLMs (dLLMs), which enable parallel token generation via iterative denoising. However, existing parallel decoding strategies rely on fixed heuristics (e.g., confidence thresholds) that fail to adapt to input-specific characteristics, leading to suboptimal speed–quality trade-offs. The authors propose Learn2PD (“Learning to Parallel Decode”): a lightweight filter model trained in a post-training phase (requiring only minutes of GPU time) to predict whether a token’s current prediction matches the final output, thereby approximating an ideal “Extremely Greedy Parallel” (EGP) oracle that unmasks tokens only when predictions are correct—avoiding redundant re-decoding. Additionally, they introduce an End-of-Text Prediction (EoTP) mechanism to detect sequence termination and skip decoding of padding tokens. Evaluated on the LLaDA benchmark (GSM8K, MATH, HumanEval, MBPP), Learn2PD + EoTP achieves 22.58× speedup with no performance loss, and 57.51× with KV-Cache at minimal accuracy cost.", "questions": "- **Model details**: What are the exact specifications of \\( f_\\theta \\)? Hidden dimension size? Activation function (e.g., ReLU)? Could incorporating confidence signals from more than one block improve performance? Additional experiments would help clarify design choices.\n\n- **Transferability**: How would Learn2PD adapt to non-LLaDA dLLMs like Dream? Does retraining the filter incur prohibitive costs? What is the cross-task generalization gap (e.g., train on GSM8K, test on MATH)?\n\n- **Robustness**: How does the method perform under low-resource settings (e.g., zero-shot) or with adversarial/noisy prompts? Furthermore, the “re-decoding gap” distribution in Figure 2 is based on only 10 samples—what are the mean and standard deviation across the full dataset?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The method is elegantly simple and highly efficient: the filter model uses a two-layer MLP that takes block-level confidence scores as input and outputs a binary decision (remask or not) via a sigmoid-thresholded logit . Its computational overhead is negligible (<1% of inference time), avoiding complex architectures or task-specific engineering—demonstrating the principle that “simplicity works.” Moreover, the approach is highly reproducible: pseudocode is clearly provided, and inference requires no ground-truth labels, making it well-suited for open-source release and practical deployment.", "weaknesses": "- **Limited model and architecture coverage**: The method is evaluated exclusively on LLaDA (8B-Instruct) and not tested on other dLLMs such as DiffuLLaMA (Gong et al., 2025) or Dream (Ye et al., 2025), raising concerns about generalizability and model dependency.\n  \n- **Insufficient task and baseline comparisons**: The empirical analysis (e.g., Figure 2) focuses only on subsets like GSM8K and HumanEval, lacking comprehensive evaluation across the full LLaDA benchmark suite. Moreover, comparisons are limited to the official LLaDA baselines, omitting key alternatives such as confidence-threshold-based or slow-fast sampling strategies.\n\n- **Oracle claims and training robustness**: The paper claims EGP achieves 15–20× optimal speedup, yet Figure 4 only reports a median of 2 steps per block without worst-case (lower-bound) analysis. Additionally, while the post-training phase is said to take “minutes,” there is no ablation on hyperparameter sensitivity (e.g., learning rate, threshold) or convergence behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uq3wN106aC", "reviewer": "Reviewer_yNHD", "summary": "This paper proposes Learn2PD and EoTP, two methods to accelerate inference in diffusion-based large language models (dLLMs), addressing inefficiencies in current parallel decoding approaches. Learn2PD trains a lightweight and adaptive filter model to predict whether the current prediction matches the final output. EoTP detects decoding completion at the end of the sequence, avoiding redundant decoding of padding tokens. With these methods, they achieve significant speedup without performance drop.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Decoding strategy for dllm is a very important topic. Unlike autoregressive models, dllm needs to decide which token to decode and which position to decode in each denoising step, which is much harder. This paper proposes an inspiring method by learning to predict the position that is ready to be decoded.\n\nExperimental results show that this method can accelerate the inference significantly.\n\nThe filter model only takes the confidence score as input and exhibits very good prediction results. This means that the logits contain rich information that can be leveraged.", "weaknesses": "**Section 3.1.2 is difficult to understand.**\n> we measured the amount of unnecessary and repetitive decoding, which is defined as the number of times the model continues to decode a token after that token has first matched the reference answer\n\nWhat does this sentence mean? Does this mean that the model predicts the correct token but fails to unmask it?\n\n**Figure 2 and 4 are hard to read.**\n\n**Limited Evaluation benchmarks and baselines**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VBnoumvW7V", "reviewer": "Reviewer_vfNL", "summary": "The paper proposes Learn2PD, a learnable parallel decoding framework for dLLMs that replaces fixed heuristics with a lightweight filter model that predicts, per token, whether its current prediction already matches the final output. \nThis adaptive filter enables tokens to be “unmasked” dynamically and in parallel, improving the speed–quality tradeoff. They also introduce an EoTP mechanism to detect completion and prevent redundant decoding, and experiments show significant speedups.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The problem this paper tries to address is an important one: current dLLM decoding is mainly heuristic-based. A hyperparameter needs to be used (like confidence) for unmasking. The neural predictor is indeed a plausible solution. \n\n2. The speedup achieved is impressive. \n\n3. The evaluation is qutie solid, including compatibility with KV cache methods.", "weaknesses": "1. Using a KV cache during inference can cause a misalignment with the training process, potentially altering the model's generated outputs from the training ground truth.\n\n2. As shown in Table 2, there is a big acc drop with dual cache (5.83). Is this caused by Learn2PD? \n\n3. The evaluation is mainly on LLaDA. How does the method generalize to other models like Dream?\n\n4. How much is the inference overhead is the additional predictor? \n\n5. How does the performance change with respect to the size of the predictor?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)