Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
Authors: Feng Hong
Geng Yu
Yushi Ye
Haicheng Huang
Huangjie Zheng
Ya Zhang
Yanfeng Wang
Jiangchao Yao
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion LLM
Efficient Inference
Primary area: foundation or frontier models, including LLMs

Abstract: Diffusion Large Language Models (DLLMs) have emerged as a compelling alternative to Autoregressive models, designed for fast parallel generation. However, existing DLLMs are plagued by a severe quality-speed trade-off, where faster parallel decoding leads to significant performance degradation. We attribute this to the irreversibility of standard decoding in DLLMs, which is easily polarized into the wrong decoding direction along with early error context accumulation. To resolve this, we introduce Wide-In, Narrow-Out (WINO), a training-free decoding algorithm that enables revokable decoding in DLLMs. WINO employs a parallel draft-and-verify mechanism, aggressively drafting multiple tokens while simultaneously using the model's bidirectional context to verify and re-mask suspicious ones for refinement. Verified in open-source DLLMs like LLaDA and MMaDA, WINO is shown to decisively improve the quality-speed trade-off. For instance, on the GSM8K math benchmark, it accelerates inference by 6 while improving accuracy by 2.58%; on Flickr30K captioning, it achieves a 10 speedup with higher performance. More comprehensive experiments are conducted to demonstrate the superiority and provide an in-depth understanding of WINO.

Published as a conference paper at ICLR 2026
WIDE-IN, NARROW-OUT: REVOKABLE DECODING FOR EFFICIENT AND EFFECTIVE DLLMS
Feng Hong! Geng Yu Yushi Ye' Haicheng Huang'
Huangjie Zheng
Ya Zhang Yanfeng Wang²
Jiangchao Yao
'Cooperative Medianet Innovation Center, Shanghai Jiao Tong University
School of Artificial Intelligence. Shanghai Jiao Tong University $ Institute of Artificial Intelligence for Medicine, Shanghai Jiao Tong University School of Medicine
[feng hong, Sunarker @sjju. ddsjtu.edu..co edu.
ABSTRACT
Diffusion Large Language Models (DLLMs) have emerged compelling alter- native to Autoregressive models, designed for fast parallel generation. However, existing DLLMs are plagued by severe quality-speed trade-off, where faster par- allel decoding leads to significant performance degradation. We attribute this to the irreversibility of standard decoding in DLLMs, which is easily polarized into the wrong decoding direction along with early error context accumulation. To re- solve this, we introduce Wide-In, Narrow-Out (WINO). training-free decoding algorithm that enables revokable decoding in DLLMs. WINO employs parallel traft-and-verie mechanism, aggressively drafting multiple tokens while simulta- neously using the model's bidirectional context to verify and re-mask suspicious ones for refinement. Verified in open-source DL.LMs like L.LaDA and MMaDA, WINO is shown to decisively improve the quality-speed spped trade-off. For instance, on the GSM8K math benchmark, accelerates inference by 6x while improving ac- curacy by 2.58%; on Flickr30K captioning, it achieves a 10x speedup with higher performance. More comprehensive experiments are conducted to demonstrate the superiority and provide in-depth understanding of WINO.
INTRODUCTION
Autoregressive (AR) large language models (Radford et al., 2018: 2019), such the GPT ries (OpenAI, 2022), have shown impressive performance in a ranging of language tasks. However, their foundational token-by-token generation mechanism introduces inherent limitations, including severe inference latency, susceptibility to error propagation (Stechly et al., 2023; Valmeekam et al., 2023), and challenges in maintaining global coherence (Mei et al., 2025). In response, Diffusion Large Language Models (DLLMs) have emerged as a compelling non-autoregressive alternative, architected to overcome these bottlenecks. By generating tokens simultaneously (Li et al., 2022), DLLMs theoretically enable massive inference acceleration, while their native bidirectional attention offers improved consistency. The immense potential of DLLMs has been showcased by proprictary, closed-source systems (e.g., Mercury Coder (Inception Labs, 2025) and Gemini Diffusion (Google DeepMind, 2025)), which have demonstrated astonishing speeds exceeding 1,000 tokens per second, serving as powerful proof-of-concept.
Despite this promise, the performance of open-source DLLMS has been still disappointing. One critical bottleneck is that they are caught in severe quality- trade-off dilemma. Specifically, to achieve high-quality output, these models are often forced decode slowly, generating just one token a time, which negates their primary architectural advantage. As shown in Fig. 1, attempting to accelerate inference by generating multiple tokens in parallel invariably leads to a significant degradation in output quality (Nie al., 2025). This stark trade-off has largely prevented the open- source DLLMM from becoming a viable, high-performance alternative to their AR counterparts.
Equal contribution
Code: http://giiilllooo//oooogog/////n

---

Published as a conference paper at ICLR 2026
GSM8K (LLaDA)
Flickr30k (MMaDA)
Accuracy
Decoding Steps
CIDEr
Decoding Steps
+2.58
+0.16
x.00x 6.10x
-16.52
8.00 10.05x
-8.57
64.67
5.82
53.67
37.15
$3.83
2.56
2
Standard Decoding
Naive Parallel Sampling
WINO
Figure/ I: Demonstration of speedup and performance improvement of WINO over standard decod- ing and naive parallel sampling evaluated on GSM8K with LLaDA and Flickr30K with MMaDA. The standard decoding unmasks token per decoding step, while the naive parallel sampling un- masks M(> 1) tokens per decoding step. We set M for GSM8K and M for Flickr30K.
We attribute this trade-off fundamental flaw in DLLMs (Sahoo al., 2024; Ou et al., 2025): its irreversibility of the standard decoding process. Specifically, the standard generation in diffusion steps typically begins with sequence of [MASK] tokens, which are then filled in greedily. Once a token is decoded, the decision is final and cannot be revised, even as more informative context becomes available in later steps. However, this is challenging for parallel decoding, where initial to- kens are generated with very limited information, easily causing early errors to become permanently embedded, accumulated and propagated throughout the output. Therefore, such rigid process es- sentially prevents DLLMs from using their greatest strength of bidirectional attention (Seo et al., 2017) to refine the early errors when the context progressively becomes rich.
To resolve this problem, we introduce Wide-In, Narrow-Out (WINO), novel decoding algorithm that enables revokable decoding for DLLMs. WINO employs a novel draft-and-verify procedure that operates in parallel. At each step, draft module aggressively proposes multiple new tokens based on a lenient threshold (the "Wide-In"). Concurrently, a verify module leverages the newly enriched global context re-evaluate all previously generated tokens. Any token that fails a stricter verification check is re-masked for refinement a future step (the "Narrow-Out", This mechanism breaks the irreversibility of the conventional decoding in DLLMs, allowing
brings two merits: the early error to be corrected for better performance; 2) it permits more aggressive token generation in each diffusion step for faster speedup with quality guarantee. Besides, our WINO training-free and aan--and-puug which enables the general DLLMs to be both highly efficient and effective.
Our extensive experiments show that when applied existing open-source models like LLaDA (Nie et al., 2025) and MMaDA (Yang et al., 2025), WINO achieves massive speedups, and also consis- tently improves model accuracy on both language and visual-language tasks. For instance, as shown in Fig. 1, on the GSM8K (Cobbe et al., 2021) math reasoning benchmark, WINO accelerates infer- ence by 6x while simultaneously increasing accuracy by 2.58%, and on Flickr30K (Young et al., 2014) image captioning benchmark, it speedup decoding by 10x with even higher performance. By making decoding revokable, WINO unlocks the latent power of DLLMs in this area.
2 RELATED WORK
Diffusion- based Language Models. Diffusson models "Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021; Peng et al., 2026), originally popularized in image generation (Rombach et al., 2022; Nichol et al., 2022; Saharia et al., 2022), have recently gained attention as an alternative to autoregressive language models (ARLMs) for text generation. This expansion from continuous do- main to discrete domain is first studied by Sohl-Dickstein et al. (2015). Subsequently, D3PM (Austin et al., 2021a) provides general framework which models the diffusion forward process as a dis- crete state Markov chain defined by the multiplication of specific transition matrices over discrete time steps. Campbell et al. (2022) later expands D3PM to a continuous time setting, utilizing the theory of continuous time Markov chain(CTMC). More recently, research masked diffusion
2

---

Published as a conference paper at ICLR 2026
models(MDMs)) (Shi et al., 2024) derived from the absorbing state diffusion in D3PM has shown promising results both in small-scale models (e.g., MDLM (Sahoo et al., 2024) and RADD (Ou et al., 2025)) and large-scale implementations (e.g., LLaDA (Nie et 2025) and Dream (Ye al.,
2025)). Extending this line of work, MMaDA (Yang et al., 2025) introduces a novel class of mul- timodal large diffusion models featuring shared probabilistic formulation and modality-agnostic architecture.
DLLM Acceleration Techniques. The existing acceleration study for DLLMS falls into two direc- tions: KV cache and sampling compression. The former targets to build the KV cache for DLLMs due its bidirectional full attention mechanism, unlike the causal attention of ARLMs. Typical works like Block Diffusion (Arriola et al., 2025). Fast-dLLM-cacch (Wu et al., 2025) and dLLM- cache (Liu et al., 2025) respectively explore different caching mechanisms, which shows promising performance for speedup. Note that this direction is out of the scope of our work here. The latter di- rection focuses on optimizing the sampling process itself. For the classic low-confidence remasking strategy, several works have introduced novel sampling strategies to dynamically adjust the num- ber of tokens predicted in parallel, thereby improving inference efficiency. - (Wu et al., 2025) adopts straightforward approach by selecting tokens with confidence scores exceeding a predefined threshold. Meanwhile, Ben- Hamu et al. (2025) propose an ontropy-bounded (EB) sam- pler, a drop-in replacement for conventional samplers that leverages procedure to dynamically decode multiple tokens per step while maintaining a predefined error tol- erance. Although our WINO brings the acceleration promise due to sampling compression, different from these works, we explore to address the inherent limitation of standard decoding in DLLMs. Re- cent follow-up work such COVER (Xiang et al., 2026) further optimizes our revokable decoding framework by utilizing an in-place KV cache override mechanism to mitigate redundant flip-flop oscillations during verification.
entropy- satropy-basedd unmasking
3 PRELIMINARY: DECODING PROCESS FOR DLLMS
[31. ....... yE] with pre- defined response length L. The response sequence is initialized as all special mask tokens, (
Given a prompt X, DLLM designed to generate response
[[MASK] h [MASK]] .... [MASK]]]. The decoding process iteratively refines the response sequence (k) over a total of K denoising steps. In the following, we detail the case of K L (i.e., decoding one token per step), as existing models typically achieve optimal performance under this setting (Nie et al., 2025).
At step k, the goal of decoding is to refine the sequence (*-1) into (k), Given the token vo- cabulary V and the model parameterized with 0, the model estimates the probability distribution over the response sequence pa (Ý (k)))). As a common example, in high-confidence greedy decoding, (k) is obtained by unmasking the most confident [MASK] token based on (k-1)), i.e.,
(16
max Po(ir = ((-1))) NEV
arg max (R-1)
[ASKI] }
arg max po = v/ x|x,y(k-1)) if ((b) v E (1,2,...,
V1
(R)
VEV
Vi
(k-1)
otherwise,
After completing all K decoding steps, the final generated response is Y = Y(K), Existing DLLMs, such as LLaDA (Nie et al., 2025) and MMaDA (Yang et al., 2025), can also accelerate the decoding process via naive parallel sampling by generating multiple tokens (e.g., 2 or 4) per step. However, empirical results reveal that such strategies often result in substantial performance degradation, lim- iting their practical effectiveness despite the computational speedup (Nie et al., 2025).
Semi-Autoregreesive Diffusion Decoding. This strategy is widely adopted by DLLMs like LLaDA (Nie et al., 2025) and MMaDA (Yang al., 2025), which involves splitting the response sequence into multiple blocks and decoding them sequentially from left to right. Within each block, the typical diffusion decoding strategy described above is applied.
3

---

Published as a conference paper at ICLR 2026
Veftt
curr
right
Sahad
Viuyt
Y
Fright
Yahud
Pos
[MASK]
pure yeaq
Bernass
koken
Prediction Confidence
MASK
(Stador duak)
Aatdy
MASK
(Shadow made)
Unaank
Herrank
(a) An overview of WINO.
(b) Attention Mask.
Figure 2: (a) An overview of WINO. (b) Illustration of our designed attention mask. The green squares denote I, the grey squares denote and "Pos ID" is short for position ID. Verified tokens refer to tokens in the prompt X or previously decoded blocks. Draft tokens denote tokens in the current block that are unmasked up the current decoding step. [MAAKK] (shadow draft) refer to tokens in the shadow block whose position IDs correspond to the draft tokens while [MASK] (shadow mask) refer to the remaining tokens the shadow block.
4 METHOD
4.1 KEY LIMITATION OF DECODING PROCESS FOR DILMS
While architecturally suited for parallelism, DLLMS face a critical bottleneck that hinders effective multi-token decoding. During the early generation stages, the sparse context often causes the model to produce low-quality or contradictory tokens when decoding in parallel. This issue significantly exacerbated by the standard decoding process due to its irreversibility nature. As these flawed initial predictions are permanently locked in, they inevitably propagate and degrade the final generation quality in the progressive diffusion steps, forcing undesirable trade-off between the speed of parallel decoding and the quality of serial decoding.
To improve the trade-off, standard decoding of DLLMs, and build a process of revokable decoding for progressive refinement. This principle empowers the model iteratively refine its initial parallel outputs. As more context emerges during generation, the model can correct its preliminary predictions. Such a mechanism effectively addresses the core conflict by marrying the efficiency of parallel generation with the accuracy of context-driven corrections.
critical point is to abandon the assumption of imeversibility in the
4.2 ITERATIVE REFINEMENT VIA PARALLEL DRAFT-AND-VERIRY
Motivated by the above analysis and the design intuition, propose parallel Draft-and- -erriyy framework to enable revokable decoding for more efficient and higher-quality generation in DLLMs.
Specifically, framework performs two modules in parallel at each decoding step: 1) Draft: ag- gressively unmasks multiple [MASK]] tokens into candidate meaningful tokens; 2) Verify: evaluates all currently unmasked tokens and re-masks those deemed low- low-quality for further refinement. We adopt the most common and general cemi-autoregressive decoding paradigm to present our method. When the block length equals the generation length, it becomes equivalent full diffusion decoding.
4.2.1 DRAFTING
We denote the entire sequence as = [Yens Yeur, Yigtu]. where Yea contains the prompt X and the previously decoded blocks, Your [ycur,1,.... Meur, represents the current block being decoded, and Yrigh denotes the remaining blocks to be decoded. Here, Lg is the block length. At the k-th decoding step, instead of decoding fixed number of tokens, perform aggressive multi-token
4

---

Published as a conference paper at ICLR 2026
parallel decoding based on confidence threshold 71:
(k) Yeur.1 = arg max (o = u|Y), max po (Meur, = r|Y) > 71 and Yeur,
(k-1)
[MASK]
(2)
eEV
-EV
Here, a relatively low confidence threshold Ti is adopted to allow more possible tokens to be de- coded at each step, which will achieve the acceleration if only a few tokens among them are revoked during the verification module detailed in the next section. While Eq. (2) presents greedy decod- ing via argmax, WINO readily supports stochastic sampling by introducing Gumbel noise to the logits (Gumbel Max trick) before thresholding, enabling diverse generation without altering the verification module.
4.2.2 VERIFICATION
The design principle of the verification module is to utilize the increasingly enriched semantic con- text at each decoding htep-relative to earlier steps, evaluate the quality of previously unmasked tokens. By re-mane re-masking low-quality tokens, the decoding process becomes revokable and amenable for the proper early error correction.
To realize effective quality verification about the decoded tokens, we design an auxiliary shadow block consisting entirely of [MASK]] Yshuu [MASK]] La. This block is appended to the sequence
Y. resulting in an extended sequence = [Yinn Yeur Yright Yahad]- We carefully design the position IDs and attention mask associated with Yshad to ensure that its output can effectively verify the quality of the tokens decoded at the corresponding positions in Yeur-
Position IDs. Although Yshad is appended to the right end of the sequence, we assign it the same position IDs as Yeur- Thus, the output of Yshad corresponds to the same positions as Yeur- enabling position-wise verification.
Attention Mask. As illustrated in Fig. 2(b), we carefully design the attention mask after incorpo- rating Yhha into the sequence Y. Specifically, tokens in Yen. Yeur. and Yiigt can freely attend to each other, but they are not allowed attend Yshad. In contrast, each token Yshad allowed attend to all tokens except its corresponding position in Your. Although indirect attention paths theoretically exist within Your. our ablation study (Sec. 5.3) confirms that blocking the direct path is sufficient to prevent practical information leakage during verification.
With the above design of position IDs and attention masks, we achieve the following properties:
For any token in the current block Year. appending Yshad does not affect the model's s output. Formally,
po (Dear, (Y) = Po(teur,1())
* For any token in Yauns the following properties hold. For example, consider the token Yshmd,3 in Fig. 2(a), which assigned position ID 6.
shares the same position ID as yeur, and is allowed to attend to Yee and Yightt
It is explicitly prevented from attending to Yeur,3, effectively avoiding information leakage during verification;
-For all other positions in Yeurs each position is attended by exactly one decoded token (from Your) and one [MAAK]] in Y.tad- The former provides progressively richer contex- tual semantics during decoding, while the latter serves regularize the confidence of decoded tokens in Yeur. reflecting the uncertainty and the need for potential refinement.
With the specially designed position IDs and the attention mask described above, the verification module can be formally expressed as:
Yeur!
MMASK]
if po (istad, Veur. ( < T2 and Yeur,
(k-1)
MASKI
(3)
where T2 is the confidence threshold for verification.
5

---

Published as a conference paper at ICLR 2026
4.2.3 OVERALL PROCEDURE
In summary, decoding step k our framework enables both the drafting and verification processes to be completed in a single forward pass:
arg max po ( = 1/ v))), if max Pa (Weur, = - v//) > 7i and (k-1)
Meurr
MASK
(k) Meur,1
VEV
vEV if po (ishadd - Neur,1 / 1))) < T2 and yeur,!
(k-1)
(4)
[AAS]] (k-1)
MMASK]
Neur!!
otherwise.
We iteratively refine the entire Your using the procedure in Eq. (4), until all tokens in Your are no longer [MASK]] We set the drafting threshold T1 and the verification threshold such that 71 < lower Ti accelerates the decoding process by allowing more tokens to be generated in parallel, while a higher 72 ensures the quality of the final output by enforcing stricter acceptance criteria. We refer to this design philosophy as "Wide-In, Narrow-Oud Our" and term our method as WINO in short. To facilitate implementation and reproducibility, provide the formal pseudocode of the WINO decoding algorithm in Sec. B.
5 EXPERIMENT
5.1 EXPERIMENT SETUP
Datasets and Baselines. We conduct experiment to evaluate WINO across different types of tasks and domains. Specifically, for language domain, we compare WINO with the standard decoding of LLaDA on eight tasks: GSM8K (Cobbe et al., 2021), MATH-500 (Hendrycks et al., 2021), Hu- manEval (Chen et al., 2021), MBPP (Austin et al., 2021b), Countdown (Zhao et al., 2025), Su- doku (Zhao et al., 2025), ARC-E (Clark et al., 2018), and ARC-C (Clark et al., 2018), covering four categories of textual generation tasks, including math reasoning. code generation, logical reasoning, and commonsense reasoning. For vision-languag, domain, we evaluate WINO against the standard decoding of MMaDA (Yang et al., 2025) on six multimodal understanding tasks: Flickr30k (Young et al., 2014), A12D (Kembhavi et al., 2016), MATH- (Wang al., 2024), Math Vista (Lu et al., 2024), MMMU (Yue et al., 2024) and ScienceQA (Lu et al., 2022), spanning four types of multi- modal tasks captioniin, chart understanding, math reasoning and clarity, we test on the validation set of MMMU and the official testmini subset of Math Vista.
reasoning. For
Evaluation Details. All benchmarks are evaluated in a zero-shot manner, except Sudoku, which is evaluated in a 4-shot setting. We use the CIDEr metric (Vedantam et al., 2015) for the Flickr30k benchmark and accuracy for all the remaining benchmarks. To assess the inference efficiency of the decoding method, we measure the required decoding steps and Tokens Per Second (TPS) of the baselines and WINO on every task by averaging over all the samples in benchmark. TPS is calculated as the total generated tokens divided by the total wall-clock inference time, accounting for all computational overheads (e.g., shadow block computation).
Implementation details. We adopt the open-sourced marks and sampling strategy introduced in LLaDA (Nie et al., 2025), where the output sequence is partitioned into multiple blocks and generated from left to right. our evaluation, conducted on a single NVIDIA A 100 (80GB) GPU, we set the generation length to 256 and the block length to 128, unless specified otherwise. For the hyperparameters of WINO, we set the verification threshold TO to 0.9 and tune the drafting threshold T₁ from (0.5, 0.6, 0.7).
MADD--B--Instrue for language bench- for vision-language tasks. We employ the semi-autoregressive
5.2 MAIN RESULTS
Performance and speedup text generation. We report the performance, decoding steps and throughput (TPS) of LLaDA, with and without WINO, on language benchmarks in Tab. 1. WINO achieves significantly better accuracy with far fewer decoding steps than the baseline LLaDA, except
a http:://wuguiighhccccoooooeeeeea

---

Published as a conference paper at ICLR 2026
Table 1: Performance and inference speedup comparison on diverse language benchmarks.
Step Reduction
TPS Speedup
Benchmark
Method
Accuracy
Steps
TPS
GSM8K
LLaDA
73.24
256
1.00
17.76
1.00 x
Math Reasoning
WINO
75.82 (+2.58)
41.93 (-214.07)
6.10
100.53 (+82.77)
5.66
MATH-500
LLaDA
32.00
256
1.00
17.62
1.00
Math Ressoning
WINO
34,20
74.44 (-181.56)
3.44
55.86 (+38.24)
3.17
Eval
LLaDA
37.80
256
1.00
14.52
1.00
Cooe Cenrnean
WINO
42.07 (+4.27)
33.3226162.66))
2.74 x
37.19 (+22.67)
2.56 x
MBPP
LLaDA
36.40
256
1.00
18.52
1.00 x
WINO
36.40 (40.00)
96.57 (-159,43)
2.65 x
45.39 (+26.87)
2,45 x
Countdown
LLaDA
24.21
256
1.00 x
17.22
1.00 x
WINO
33.20 (+8.99)
105.88 (-150.12) 12)
2.41
38.97 (+21.75)
2.26
Sudoku
LLaDA
14.23
256
1.00
11.61
1.00 x
Logical Recsociing
WINO
15.20 (+0.97)
131.6 (-12400)
1.94
21.11 (+9.50)
1.82
ARC-E
LLaDA
59.13
256
1.00
17.26
1.00
Comm-nameet Remaening
WINO
81.19 (+22.06)
40.19 (-215.8))))
6.37
101.61 (+84.35)
5.89
ARC-C
LLaDA
51.87
256
1.00
17.10
1.00
Commoraanne oo Rezssning
WINO
73.89 (+22.02)
47.41 (-208.59)
5.40 x
85.42 (+68.32)
5,00 x
for the MBPP task, where WINO achieves the same performance as LLaDA. For instance, WINO improves accuracy on GSM8K by 2.58% with 6. 6. step reduction and 5.66x TPS speedup. Tab. demonstrate the effectiveness of WINO in enhancing generation quality and inference efficiency.
Performance and speedup on multimodal understanding and reasoning. We assess the perfor- mance and efficiency gain of WINO incorporated into MMADA and summarize the results in Tab. 2. Compared to the vanilla MMaDA. WINO demonstrates consistent and substantial improvements in inference efficiency across all benchmarks. Notably, the speedup effect is even more pronounced than that textual domain tasks, when compared with results in Tab. 1. Furthermore, WINO greatly improves the task performance of MMaDA on A12D, MMMU and ScienceQA while maintaining comparable results on Flickr30k, MATH-Vision and Math Vistaa These results indicate that WINO consistently delivers both performance gains and accelerated inference in the multimodal domain.
Relation between speedup and task complexity. As shown in Tab. and Tab. 2, we observe a consistent pos- itive correlation between the degree of speedup and task performance across all benchmarks. For instance, WINO achieves a 10.05 step reduction on the relatively simple captioning task Flickrš0k, compared to only 5.73 step re- duction on the more challenging math reasoning benchmark MATH-Vision. This is because models can, in principle, solve tasks they are more proficient at with lower compu- tational cost, leaving greater room for acceleration under our decoding method. And since models are typically more confident when handling easier tasks, each decoding step in WINO tends to yield a larger number of effective tokens.
Difficulty Level
2 52 A
LLaDA
25
50
75
100
240
260
Decoding Steps
Figure 3: Decoding steps of WINO on subsets of the MATH benchmark with varied difficulty levels.
To further investigate this, we evaluate the decoding steps of WINO across subsets of the MATH-500 benchmark categorized by difficulty levels. As shown in Fig. 3, WINO achieves progressively greater acceleration as the difficulty de- creases, highlighting its capability to adaptively optimize inference speed based on task complexity.
5.3 ABLATION STUDY AND FURTHER ANALYSIS.
On different generation length. In Tab. 3, we evaluate the performance of WINO with experiments on different generation lengths, where the block length L, is fixed to 128 and the baselines unmask 1 token every decoding step (to achieve their best generation performance). When the generation lenghh is set to 512, WINO still achieves comparable or better task performance with significantly fewer decoding steps, demonstrating the effectiveness of WINO across different generation lengths.
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance and inference speedup comparison across diverse multi-modal understanding and reasoning benchmarks. We use CIDEr for Flickr30k and accuracy for other benchmarks.
Step
TPS Speedup
Benchmark
Method
Performance
Steps
TPS
Reduction
Flickr30k
MM3DA
53.67
256
1.00
6.41
1.00
Captosing
WINO
(+0.16))
25.47 (-230.53)
10.05 x
55.11 (+48.70)
8.60 x
A12 Undersuadiig
MMADA
54.86
256
1.00 x
6,31
1.00 x
WINO
57.19 (+2.33)
30,90 (-225.10)
8.30 x
46.04 (+39.73)
7.30 x
MATH- Vision
MMaDA
256
1.00 x
6,22
1.00 x
Mulh Ressoning
WINO
9.57 (+1.02)
44.69 (-211.31)
5.73
31.17 (+24.95)
5.01
MathVista-min
MMADA
31.10
256
1.00
6.21
1.00
Muth Reasoning
WINO
31.40 (+0.30)
33.45 (-222.55)
7.65
41.96 (+35.75)
6.76
MMMU-val
MMADA
18.56
256
1.00
6.02
1.00 x
Mili diciplice Reasoting
WINO
245.044
38.47 (-217.53)
6.65 x
36.13 (+30.11)
6,00 x
ScienceQA
MMADA
30.89
256
1.00 x
6.07
1.00 x
disciplice Reasoning
WINO
(+11..)))
28.12 (-227.88)
9.10 x
49.45 (+43.38)
8.15 x
Table Experiment results on different generation lengths and full diffusion setting, respectively.
Generation Length
Block Length
Step Reduction
TPS Speedup
Benchmark
Method
Accuracy
Steps
TPS
Different Generation Lengths
LLaDA
73.24
256
1.00
17.76
1,00
256
128
WINO
75.82 75.82 (+2.58)
41.93
6.10 x
100.53
5.66 x
GSM8K
LLaDA
74.60
512
1.00 x
11,84
1.00
512
128
WINO
79.91 (45.31)
68.53
7.47
82.64
6.98
MMaDA
18.56
256
1.00
6.02
1.00
256
128
WINO
24.00 (+5.44)
38.47
6.65 x
36.13
6.00 x
MMMU-val
MMaDA
18.44
512
1.00
1.00
512
128
WINO
23.44 (+5.00)
64.82
7.90
35.01
6.99
Full Diffusion
LLaDA
34.34
256
1.00
17.73
256
256
WINO
58.22 (+23.88)
38.77
6.60 x
93.61
5.28
GSM8K
LLaDA
58.60
128
1.00 x
23.23
1.00 x
128
128
WINO
62.32 (+3.72)
23.95
5.34
114.29
MMaDA
17.22
256
1.00x
1.00x
256
256
WINO
22.44(44520)
24.94
10.26x
50.03
8.19x
MMMU-val
MMaDA
15,33
128
1.00x
6.70
1.00 x
128
128
WINO
(+7.78)
19.14
6.69x
On full diffusion decoding (instead of semi-autoregressive decoding). In Tab. 3, we compare the baselines and WINO applying full diffusion decoding, which means the block length Ls is set equal to the generation length. Compared to results on the semi-autoregressive decoding in Tab. and Tab. 2, WINO demonstrates substantially strong accuracy gains under the full diffusion setting. Notably, while LLaDA suffers a substantial accuracy drop on GSM8K with full diffusion decoding, WINO maintains reasonable performance with far fewer decoding steps. These results indicate that WINO unlocks sigmificantly greater potential for boosting model performance and computational efficiency when applied in full diffusion decoding scenarios.
Comparison with Other Parallel Sampling Strategies. While naive parallel sampling acceler- ates decoding, it causes severe performance drops (e.g., GSM8K accuracy falls to 64.67% with tokens/step). We further compared WINO with advanced dynamic samplers, including Fast-dLI L-M parallel (Wu et al., 2025) and kens aggressively based on confidence entropy without revocation. As detailed in Sec. C, while these methods improve speed (approx. 3x), they still suffer from accuracy degradation (e.g., Fast- dLLM--pralle achieves 72.33% GSM8K). In contrast, WINO achieves 5.66x speedup while boosting accuracy to 75.82%, demonstrating that the revocation mechanism is essential for breaking the quality trade-off.
Bounded sampler (Ben-Hamu al., 2025), which draft to-
Ablation on Verification and Leakage. We validate the necessity of our verification mechanism and attention mask design in Tab. 4. First, removing verification ("Only Draft") leads to signifi-
8

---

Published as a conference paper at ICLR 2026
Table 4: Experiment results on the variant of WINO without the verification module.
Step Reduction
TPS
Benchmark
Method
Accuracy
Steps
TPS
Speedup
LLaDA
73.24
256
1.00 x
17.76
1,00 x
Only Draft (r) = 0.6)
70.28
34.79
7.36 x
130.89
7.37 x
GSM8K
Only Draft (7) = 0.9)
72.33
81.39
3.15 x
56.
3.16 x
WINO Full Leakage)
72.25
44.85
5.71
92.38
5,20
WINO
75.82
41.93
6.10
100.53
5.66
MMADA
18,56
256
1,00 x
6.02
1.00 x
Only Draft (m) 0.6)
19.89
35.63
7.18
43.22
7.18
MMMU-val
Only Draft (r) 0.9)
18.56
79.74
3.21
19.38
3.22 x
WINO (w/ Full Leakage)
18.22
44,43
5,76 X
31,54
5.24x
WINO
24.00
38.47
6.65 X
36.13
6.00 x
0.9
0.6
0.9
0.5
RefacsonOO
Stop Redocsoni))
Step Roliecin (0))))
22.5
22.5
15.0
28.0
20,0
17.5
Stap
15.0
Accurasy
Savel me Accur
-Mk- Stop Recluction
Accunacy
Baseline Assuracy
Stop Betluction
(a) WINO (LLaDA-basse based) on GSM8K.
(b) WINO (MMaDA-based) on MMMU-val.
Figure Ablation study on the drafting threshold тΙ and the verification threshold 72-
Question: family, there are brothers sisters. sisters are the same age, which is 16. One brothers years old, which the age older brother. What the total age all these siblings? Correct Answer: 84
1. We know ......
Accooding problem,
4. Ansoecing to the prbble, 12 a
We know Acconding
writ the equation: = iutliply bush
the age the older brother we
1. We know Assording age older
half the age of the older brother,
can the equation:
age older
Sulving for
by x
5. Solving for
muitiply both sides by 24.
sides by 2:x 24. Tharelore, the oláe bother 24
we can write the equation: 12 =
years old.
[MASK)..... .M......SKK)95
Solving for
multiply both
other bruthes, shus 24 years old.
years
x. The tona age of the two brothers
24 24 48.
7. The younger brocher, being the only
The tutal of the thnes sivie
Therefoe,,
[MASK] years.
years old.
sides by 2:x 25.
we can wrile the aqusioon: -
Tharefore, the older brother 24
The sotal of [MASK]
for
miltpply
8. The total of the brothers the ages
yeur
3x16 =48 years.
7. The toul age of the siners is
Therefore,
12-yeu--od years.
3x16
your
10. unal
wiliine
siblings 48 36 84 years.
age
LLaDA: step 162
LLaDA: final (step 256)
WINO: step 42
WINO: final (step 51)
Figure 5: Case Study: GSM8K Example. We compare standard decoding with LLaDA against the intermediate and final results produced by WINO. More detailed case studies are provided in Sec. A.
cant performance degradation when using aggressive thresholds (ri = 0.6), as the model lacks mechanism correct inevitable draft errors. This variant effectively represents dynamic sampler that aggressively drafts tokens without revocation. Second, we address the concern of information leakage. Theoretically, Ysnad could access Your information indirectly via self-attention within You-- To test if this compromises verification, we introduce "WINO (w/ Full Leakage)", where Yeha is allowed to directly attend to corresponding token in Y Your. shown in Tab. 4, allowing direct leakage drops accuracy significantly (e.g., -3.57% GSM8K). This confirms that the direct atten- tion path is the dominant factor credit assignment. By blocking this path, our proposed mask effectively forces the model to verify tokens based on context rather than trivial identity mapping, making the potential indirect leakage negligible in practice.
Effect of threshold tuning. In Fig. 4, present the evaluation results of WINO with varying drafting threshold t and verification threshold 72. Our experiments suggest that WINO consis- tently outperforms baselines across different benchmarks and the Ti and 72 values in terms of both

---

Published as a conference paper at ICLR 2026
task performance and inference efficiency. As the TI value decreases, more candidate tokens unmasked at each decoding step, thereby accelerating inference by reducing the required decoding steps. However, this comes the cost of introducing more unreliable predictions, which may place a greater burden on the verification module to correct errors. Empirically, we find that setting the value of drafting threshold Tı within the range of 0.5 to 0.7 achieves optimal balance, maintain- ing competitive task performance while preserving efficient generation. The verification threshold 72 controls the strictness of the verification process and thus influences decoding speed. Since the performance is relatively robust to 72, we fix = 0.9 in all experiments, while leaving open the possibility of further tuning this parameter for even better performance and speedup.
While the main experiments in this paper focus on deterministic greedy decoding, WINO is not restricted to deterministic inference and can be naturally extended to stochastic sampling for generating diverse out- puts. Specifically, during the drafting phase, we incorporate the Gumbel- Max trick to enable sampling-based token selection under a non-zero tem- perature. As reported in Tab. 5, even under stochastic decoding with temperature set to 0.5, WINO consistently outperforms its corresponding baselines across both benchmarks. In particular, WINO achieves noticeable accuracy improvements while simultaneously delivering substantial inference speedups, as reflected by the step reduction factors.
Table 5: Performance comparison using stochastic sampling (temperature 0.5). We report the mean and standard devi- ation over 3 runs.
Benchmark
Method
Accuracy (%)
Step Reduction (x)
LLaDA
73.01 0.21
GSM8K
WINO
76.06 0.16
6.05 0.06
MMaDA
18.32 = 0.43
MMMU-val
WINO
23.89 = 0.29
6.69 0.09
GPU memory usage. WINO introduces an auxiliary shadow
30
block of size Ls, which theoretically scales the memory over- head linearly with the block length. However, in practice, this cost remains minimal. In the standard semi-autoreeressivee setting (LB- = 128, L = 256), WINO increases usage by only 2.4% (16. 18GB vs. 16.57GB GSM8K, see Fig. 6).
CPU Merooyy Coeuge
25
MMaDA
20
WINO
15
10
To assess the upper bound, we further evaluated the most memory-intensive full diffusion setting (L, = L 512),
GSMSK
ARC-C MMMMU-vl Benchmark
A/2D
where the shadow block effectively doubles the processed se- quence length. Even in this extreme case, the overhead re- mains below 8% (rising from 20.96GB to 22.65GB on MMaDA), demonstrating WINO's efficiency for longer contexts.
Figure 6: GPU memory usage.
Decoding Dynamics. To conduct a fine-grained examination of the decoding pro- cesses of WINO, we present an example from GSM8K in Fig. 5. As shown, the baseline may produce erroneous tokens at the early decoding stages. Since the generated tokens by the base- line remain unchanged in subsequent decoding steps, the false contextual information propagates throughout the whole generation process, eventually leading to low-quality generation results. In contrast, WINO enables dynamical refinement of generated tokens via an iterative draft-and-verify mechanism, which mitigates error accumulation and facilitates high-quality decoded outputs.
Case Study:
6 CONCLUSION
In this work, we introduce Wide-In, Narrow-Out (WINO), training-free decoding algorithm that resolves the critical quality-speed trade-off in Diffusion Large Language Models (DLLMs) by mak- ing their generation process revokable. WINO overcomes the limitations of irrevokable standard decoding by employing a parallel draft-and-verify mechanism, allowing the model to aggressively generate tokens and iteratively correct errors using its full bidirectional context. Our experiments on existing open-source models like LLaDA and MMaDA demonstrate that WINO simultaneously accelerates inference by up to 10x while significantly improving accuracy across a diverse set of lan- guage and assion-language tasks. While leddownediing areas for future architectural improvements, WINO fundamentally enhances the practicality of DLLMs by rethinking the decoding process itself, establishing them truly efficient and high-quality alternative autoregressive systems.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
WINO is training-free decoding algorithm that improves the efficiency and output quality of exist- ing Diffusion Large Language Models (DLLMs). Our method operates on pre-trained models and does not involve new training data or modifications to the models' underlying parameters. Therefore, WINO does not introduce new ethical concerns beyond those already associated with the foundation models it is applied to. Any potential for bias or malicious use is characteristic of the base model, not the decoding process we have developed.
ACKNOWLEDGEMENT
This work is supported by National Natural Science Foundation of China (No. 62306178), STCSM (No. 22DZ2229005), and the 111 Plan (No. BP0719010).
REFERENCES
Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autore- gressive and diffusion language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025.
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural In- formation Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 1798 1-17993, 2021a.
Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Do- han, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. CoRR, abs/2108. 087327 2021b.
Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, and Brian Karrer. Accelerated sampling from masked diffusion models via entropy bounded unmasking. CoRR, abs/2505. 248557, 2025.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural In- formation Processing Systems 35: Anmual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such. Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss,, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained code. CoRR, abs/2107.03374. 2021.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the A12 reasoning challenge. CoRR, abs/1803 803.05457, 2018
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reñchiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/21 10. 14168, 2021.
11

---

Published as a conference paper at ICLR 2026
Google DeepMind. Gemini diffusion. Google DeepMind Models, 2025. URL https:// deepmind. ndd google /models -ttp:///eedeeeee//nneeeiiiiieeee /tt.//ginndifiisiiooooooooee
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In NeurIPS Datasets and Benchmarks, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Inception Labs. Introducing mercury:
The first commercial diffusion- language
model. Inception Labs Blog, 2025.
URL https: /wwp///wwww...... inception labs.alis
introducing--erre mercury...
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Min Joon Seo, Hannaneh Hajishirzi, and Ali Farhadi. diagram is worth dozen images. CoRR, abs/1603.07396, 2016.
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. Diffusion-lm improves controllable text generation. In Sanmi Koyejo, S. Mohamed, A. Agar- wal, Danielle Belgrave, Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Zhiyuan Liu, Yieun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dilm-cache: Accelerating diffusion large language models with adaptive caching. CoRR, 0os/2506.06299 2025.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang. Song- Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In NeurIPS, 2022.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In ICLR, 2024.
Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu. A survey of context engineering for large language models, 2025. URL htttt:ttt///aatt org/ abg//bs// 7507..... 13334.
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: towards photorealistic image generation and editing with text-guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 16784-16804. PMLR, 2022.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. CoRR, abs/2502.09992, 2025.
OpenAI. Chatgpt: Optimizing language models for dialogue. OpenAl Blog, November 2022. URL https: //pp//ai...a.////////ee// cww/coooo//////oo blog///og//// log/ chattpt//
Jingyang Ou. Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025.
Xinyu Peng, Han Li, Yuyang Huang. Ziyang Zheng, Yaoming Wang. Xin Chen, Wenrui Dai, Chenglin Li, Junni Zou, and Hongkai Xiong. Towards holistic modeling for video frame in- terpolation with auto-regressive diffusion transformers. arXiv preprint 2xXx:26011119950 2026.
12

---

Published as a conference paper at ICLR 2026
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre- vreetraining.... 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9 2019.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10674-10685. IEEE, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, Cho, and Oh, editors, Advances in Neural Information Processing Systems 35; Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Or- leans, LA, USA, November 28- December 9, 2022, 2022.
Subham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language mod- els. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: An- nual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. In ICLR, 2017.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis K. Titsias. Simplified and gener- alized masked diffusion for discrete data. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet. Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural In- formation Processing Systems 38: Anmual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. CoRR, abs/1503.035855 2015.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Viriual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Kaya Stechly, Matthew Marquez, and Subbarao Kambhampati. GPT-4 doesn't know it's wrong; An analysis of iterative prompting for reasoning problems. CoRR, abs/2310. 2023.
Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. Can large language models really improve by self-critiquing their own plans? CoRR, abs/2310.08 18, 2023.
Ramakrishna Vedantam, C. Lawrence Zitnick. and Devi Parikh. Cider: Consensus-based image description evaluation. In CVPR, pages 4566-4575, 2015.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In NeurlPS, 2024.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao. Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-fres acceleration of diffusion L.MM by enabling KV cache and parallel decoding. CoRR, abs/2505.22618 8, 2025.
Yanzheng Xiang, Lan Wei, Yizhen Yao, Qinglin Zhu, Hanqi Yan, Chen Jin, Philip Alexander Teare, Dandan Zhang, Lin Gui, Amrutha Saseendran, et al. Stop the flip-flop: Context-preserveee veri- fication for fast revocable diffusion decoding. arXiv preprint 26Xi::2602.06111, 2026.
13

---

Published as a conference paper at ICLR 2026
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. CoRR, abs/2505.15809, 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. URL https: / hetes://hhkuuu /hkunlp gittub.......................... io htobll///////oooooo.ooooooooooot 20////225///eeeee
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions visual denotations: New similarity metrics for semantic inference over event descriptions. Trans. Assoc. Comput, Linguistics, 2:67-78, 2014.
Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. In CVPR, pages 9556-9567. IEEE, 2024.
Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. dI: Scaling reasoning in diffusion large language models via reinforcement learning. CoRR, abs/2504. 12216, 2025.
14

---

Published as a conference paper at ICLR 2026
USE OF LARGE LANGUAGE MODELS (LLMS)
Large language models (LLMs) (e.g. GPT-5 and Gemini 2.5 Pro) were employed editing tool during the final preparation this manuscript. The model's s use was confined to language polishing, grammar correction, and formatting. It played no part in the substantive scientific work; all research concepts, methodologies, analyses, and conclusions were generated exclusively by the authors.
A
CASE STUDY
Decoding Dynamies. To conduct fine-grained examination of the decoding pro- cesses of the baselines and WINO, we present two generation examples-one one from GSM8K and one from MMMU-val below. As shown, the baseline may produce erroneous tokens the early de- coding stages. Since the generated tokens by the baseline remain unchanged in subsequent decoding steps, the false contextual information propagates throughout the whole generation process, eventu- ally leading to low-quality generation results. In contrast, WINO enables dynamical refinement of generated tokens via an iterative draft-and -ed-rrf mechanism, which mitigates error accumulation and facilitates high-quality decoded outputs.
Case Study:
Case: GSM8K, Geneation Length 256, Block Length 128, blacks in total
Question: a family, there brothers and sisters. All sisters are the same age, which 16. One of the brothers 12 years old, which is half the age of the older brother. What is the total age of all these siblings?
GT: 84.
LLaDA:
Step:
128 (Ist block decoding completed). block: <ressoning>
sisters are years old, 2. One of the brothers is 12 years old, which ia half the age of the older brother.
Letts denote the age the oleer brother as a. According to the problem, 12 half the of the older brother, write
the equation: 12 = 5
Solving æ, sultiply both sides by Therefore, the
Step: 162 (early overconfident error). 2nd block:
o_der brother is 24 years old,
[_ (mdd......dddnmssssi. </reasoning> <onswer> 96
</answer>< itt_idppp
Final (error accumulation):
<ressoning>
1. Re knew that all three sisters are 16 years cld.
2. One the brothers is 12 years old, which half the age of the older brother.
Let's denote the age the older brother
4. Azcording to the problem, 12 is half the age of the clder brother, so we can grite equation: 12
5. Solving for æ, we multiply both sides by 2: == 24.
brother 7. the younger brother, being the only other brother, is also 24 years old. The total age of the two brothers is 21 21 18,
Therefore,
years
The tetal of the three sisters 3x 16 16 48.
10. The total age of all the siblings is 48 96.
i/reasonings
<answer>
96
WINO:
Step: 35 (Ist block decoding completed). block:
15

---

Published as a conference paper at ICLR 2026
<ressoning>
know
sisters years
2. One of the brothers is 12 years old, which is half the age of the clder brother. age
o dar brother as a. According to the problem, 12 La ha_f the of the older brother, write the equation: 12 5-
Let's a denota
8olving 2, multiply both sides therefore,
Step: 37 (draft error), 2nd blok: brother
years old.
mddinna 12 <lidd__maak 964 mdm_nnaki [aaiiinnnnssss>>______ makk s/zeasoning<| Imd <answer>
9< 9c|adn_madd mack|>
</answer>< not _dl>< endoftext]]
Step: (refinement: still wrong). 2nd block:
o'derr brother is 24 years old.
7. The total age of [mcdi_maakk sisteres Imdin maakl Imdi_ maak > 16 1411mdn_mask ........ ..r...snnae <Inén_maek totsl of all thes Indn_nask] is mdd__aak| [><] md
A) years.
</reasoning>
<answer>
86
</answer>< leo__________o Idd
Final (all correct through iterative refinement): cressoning>
are years
2. One of the brothers is 12 years old, which is half the age of the cldez brother.
3. Let's denote the age the older brother as a.
According
the problem, 12 half the of the clder brother, 12= 3-
write
the equation:
Solving for 2, multiply both sides
24.
6. Therefore, the older brother la 24 yeazs old.
sisters 8. the tetal of the brothers is the sum of the of the 12-year-old and the
age
years.
24 year old brother: + 36 years.
the tetal age of all the siblings is 48 + 36 84 years.
</reasoning>
84
</answer>< not sot_id/>__diooooooo endoftext]
Case: MMMU-val, Geneation Length 256, Block Length 128, 2 blocks in total
f(x)
Question: The graph of y f(x) is shown in the figure above. The shaded region A has area and the shaded region B has area b. If g(x) f(x + 3, what is the average value of the interval
[-2, 4]?
A. b+ 3)/6
B. (-a b-a+b43)) 3)/6
C. (- b) 4
D. (a 33
GT C.
MMaDA:
16

---

shaded Published as a conference paper at ICLR 2026
Step: 70 (early overconfident error). 1st block:
<think>
1. To find the average value of g(a) on the interval [-2, 4), we need to calculaze the area graph g(x) over this interval.
2. The area of the graph of g(2) is given by the sum of the areas of the two shaded zegions.
mdn__nas >... P............................... mack
Step: 128 (error accumulation). 1st block:
<think>
avarage value g(a) interval [-2,4), we to calculate the area of the graph of a(a) over this interval.
2. The area of the graph of g(z) is given by the sum of the aceas of the two shaded regicns.
3. The area region is and the area region B b. Therefore, the
a(x) a
5.
The average value ((a) the interval
Final (error accumulation):
<think>
1. To find the average value of g(x)) the interval [-2, need calculate the arez of the graph of g(x) over this interval.
The
the graph g(z) given by the the
regions.
The area region
and area region
4. Therefore, the total area of g(a) is a
5. The avarage value of g(x) on the interval [-2,4] is atb </think>
6
3x a<lendoitex:
WINO:
Step: (draft error). 1st block:
<hhink>>
1. To find the average value of 9(ar) on the interval [-2, 4), we need to use
the madd_naskil
ther (adelladm_mmas mask ((x) over<|ndn_nask]
2. The average value
given by \Erac(1) madn_mask \times \<|mdn_maak maak_ jas Indn_maak maak 12) :4x [add_mmmamlaamaaa_eeaaaaaaa maak/........./oon......
Step: (refinement: correct). Ist block:
<think>
interval 12, to formula for the average of f(x) over an interval.
find average value g(x)
2. The average value of f(x) is given by f(x)
Civen that g(x) f(x) 3, Therefore, montm_mask/>.........
find f(z). averoge (()) -0th
Final
through
refinement):
<think>
1. To find the average value of g(x) on the interval [-2, 4], we need to use the formula interval.
for the ovarage f(x)
2. The average value of f(x) is given by } f.f2, ((x)dz
3. Given that g(r) = f(x) +3, 3, we need to find f(x). The averoge value of f(a) is -0th
4. Therefore, the avezage value of g(x) on the interval [-2,4] is meth +3. correct answer is option C.
</think>
(<lendoftext
B WINO DECODING ALGORITHM
We present the formal algorithm for the proposed Wide-In, Narrow-Our (WINO) decoding in Algo- rithm 1. This algorithm details the iterative draft-and-verify loop within single diffusion block.
C COMPARISON WITH ADVANCED DYNAMIC SAMPLERS
To further demonstrate that the performance gains of WINO stem from the revokable mechanism rather than merely aggressive dynamic sampling, compared WINO against state-of-the-arf dy-
17

---

Published as a conference paper at ICLR 2026
Algorithm WINO Decoding for a Single Block
Require: Prompt context Yen Current block length Lb. Drafting threshold 71, Verification thresh- old 72, Model Po
Initialize current block Yeur [MASK]' los
2: Initialize shadow block Yshad <- [MASK] Lb
3: Construct extended sequence Y (ee,, Yeurs Yrights Sshma)
4: while Yau contains [MASK] do
5: Forward pass to get logits for Y
6: for ( = 1 to Lu do
7: // Parallel Draft (Wide-In)
8: if Neur,1 is [MASK] then
9:
Get candidate token v arg max, Po (Weur. 111
10:
Get confidence Cdraft = max, po (Weur. = 1 ( if Cdroft T1 then
11:
12:
Your. + v [Unmask token)
13:
end if
14:
else
Parallel Verify (Narrow-Out)
Get verification confidence Cuerify polssood, = Yourr! 8)
17:
Cverify < T2 then
Year.1 [MASK] (Re-mask (Revoke))
19: end if
end if'
21: end for
22: Update Y with new Your
23:
end while
24:
return Yeur
namic samplers: Fast-dLLMMparallle (Wu et al., 2025) and Entropy-Boundee (EB) sampler (Ben- Hamu et al., 2025). As shown Tab. 6, while these methods achieve speedups over the base- line, they suffer from accuracy degradation (e.g., Fast-dLLM drops 3.9% GSM8K compared to WINO). WINO, utilizing the proposed verification and revocation mechanism, achieves superior speedups (approx. 6x) while significantly improving accuracy.
Table 6: Comparison with advanced dynamic samplers (Fast-LLLM-parallee) and EB Sampler) on GSM8K and MMMU-val. WINO achieves the best trade-off between quality and speed.
Method
Accuracy
Steps
Step Red.
TPS
Speedup
GSM8K
LLaDA (Baseline)
73.24
256.00
1.00x
17.76
1,00x
Fast-dl M--paalll
72.33
81.39
3.15x
56.12
3.16x
EB Sampler
70.37
95.74
2,76x
46.71
2,63x
WINO (Ours)
75.82
41.93
6.10x
100.53
5.66x
MMMU-val
MMADA (Baseline)
18.56
256.00
1.00x
6.02
1.00x
Fast-dLLM Marralllll
19.89
35.63
7.18x
43.22
7.18x
EB Sampler
17.87
61.04
4.20x
24.89
4.13x
WINO (Ours)
24.00
38.47
6.65x
36.13
6.00x
D LATENCY ABLATION STUDY
To verify the per-step cost of WINO's custom attention mask, we measure the wall-clock time for a single forward pass on LLaDA-8B. As shown in Tab. 7, when controlling for the total sequence length (Prompt Gen + Shadow), the WINO custom mask ineurs additional latency compared to standard full attention. This confimms that the overhead WINO comes solely from the increased sequence length of the shadow block, not the masking mechanism itself.
18

---

Published as a conference paper at ICLR 2026
Table 7: Per-step wall-clock latency ablation (ms). length processed by the GPU.
"Total Length" indicates the actual sequence
Config
Gen Len
Total Len
Attn. Type
Latency (ms)
Standard
512
1024
Full
121.36 + 0.37
Standard (Pad)
512
1152
Full
132.65 + 0.60
WINO
512
1152
Custom Mask
132.49 0.89
E LIMITATIONS AND FUTURE DIRECTIONS
Our work establishes WINO as a training-free framework that makes the decoding process Diffu- sion Large Language Models (DLLMs) revokable, effectively addressing their quality-speed trade- off. The degree of acceleration, however, is inherently linked to the base model's capabilities; as our experiments show, more proficient models produce better drafts that require fewer refinement steps, leading to greater speedups. This insight points to a promising future direction: integrating the con- cept of revokable sampling directly into the training phase. model trained with awareness of draft-and-verify mechanism could learn to generate more robust initial predictions and self-correct more efficiently, potentially unlocking even greater gains in performance and speed.
19


Reviews:
[{"review_id": "7NJuRpWc42", "reviewer": "Reviewer_qPHN", "summary": "This paper introduces Wide-In, Narrow-Out (WINO), a novel training-free decoding algorithm for Diffusion Large Language Models (DLLMs) that enables revokable decoding. WINO's core contribution is a parallel draft-and-verify mechanism. It first aggressively generates a large set of candidate tokens, and then uses the model's bidirectional context to verify all currently unmasked tokens (including historical ones), re-masking any that are identified as low-quality. The experiments show that WINO improves the quality-speed trade-off, reducing inference steps while improving accuracy.", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is training-free, allowing it to function as a plug-and-play module for existing pre-trained DLLMs.\n2. The algorithm demonstrates strong performance gains while accelerating the inference.", "weaknesses": "1. The proposed solution to prevent information leakage during verification seems insufficient. The paper states that a token in the Y_shad cannot attend to its corresponding token in Y_cur. However, it can attend to other tokens within Y_cur. Since full self-attention is applied within Y_cur, these other tokens already contain contextual information about the token being verified. Therefore, information leakage still exists. It causes doubt on the mechanism's claimed effectiveness, as the model is not re-evaluating the token with truly independent context.\n\n2. The applicability of WINO to the full class of diffusion language models is unclear. It is not obvious how this method would apply to or benefit other diffusion frameworks, such as SEDD[1], which utilize different sampling schemes (e.g., τ-leaping) that are inherently iterative and corrective.\n\n3. The decoding algorithm appears to be deterministic. The paper does not address how WINO could be adapted for stochastic sampling to produce diverse outputs for a single prompt.\n\n[1] Lou et al., Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MFVVB0iZRT", "reviewer": "Reviewer_ommK", "summary": "This paper identifies the \"irreversibility\" of standard decoding in dLLMs as a primary cause for their poor quality-speed trade-off. To address this, the authors propose a training-free decoding algorithm named WINO: Wide-In, Narrow-Out. WINO introduces a revokable generation process through a parallel draft-and-verify mechanism. At each step, it uses a lenient threshold to aggressively draft multiple tokens (\"Wide-In\") and, in the same forward pass, leverages an auxiliary \"shadow block\" to re-evaluate previously generated tokens. Tokens that fail a stricter verification check are re-masked for later refinement (\"Narrow-Out\"). The method is evaluated on the LLaDA and MMaDA models, where it is shown to significantly reduce the number of decoding steps while often improving task performance. \n\n\nWhile I am not against accepting the paper, I have a some concerns regarding the empirical evaluation and a lack of detail on the practical overheads. My primary reservations is on baselines. The paper's central claim of superiority is substantially weakened by the lack of comparison to other sophisticated, recently proposed dLLM acceleration techniques(but maybe they are concurrent work? depending on the definition of ICLR on this term). The experiments primarily compare against \"standard\" (one token at a time) and \"naive parallel\" decoding, which are weak baselines. Without benchmarking against other dynamic samplers, it is difficult to assess the true contribution of WINO over the state-of-the-art.", "questions": "- My most critical point is the comparison to other dynamic samplers. Can you provide results comparing WINO against Fast-dLLM-parallel and/or an entropy-bounded sampler on key benchmarks like GSM8K and Flickr30k? This would be essential to isolate the benefit of your novel verification/revocation mechanism.\n\n- Could you provide a more detailed breakdown of the practical overheads? Specifically, what is the wall-clock time for a single forward pass with WINO compared to a standard forward pass on a sequence of equivalent length (i.e., `L_orig + Lb`)? This would clarify the true per-step latency cost of the custom attention mechanism.\n\n- Additionally,  a formal pseudocode algorithm for the WINO decoding loop would be highly beneficial for clarity and reproducibility.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- **Clear and Intuitive Motivation:** The identification of \"irreversibility\" as a core problem is a strong conceptual contribution. The proposed draft-and-verify solution is a logical and elegant response to this problem. However, this seems to be aligning with many recent work, which might be worth including to related work, and the idea is also not super novel in AR LLMs community. Being honest with that will not hurt the novelty of the paper but establishes a great baseline. \n\n- **Novel Mechanism:** The \"shadow block\" trick for enabling verification within a single forward pass is a clever and efficient design.\n\n- **Promising Initial Results:** The ability to simultaneously reduce decoding steps by a large margin and improve accuracy on challenging benchmarks like GSM8K and ARC is a noteworthy and promising result. (but I have another concern that GSM8K seems to be a very contaminated dataset...)\n\n- **Training-Free and Model-Agnostic:** The method's applicability to existing, pre-trained dLLMs without any fine-tuning is a major practical advantage.", "weaknesses": "- **Missing Key Baseline Comparisons:** This is the most significant weakness. The paper compares WINO to generating 1 token/step or a fixed M tokens/step. However, the related work section itself mentions other advanced samplers like Fast-dLLM-parallel and the entropy-bounded (EB) sampler, which also perform dynamic, confidence-based parallel decoding. To convincingly demonstrate the superiority of the WINO mechanism (specifically, the benefit of *revocation*), it is crucial to compare against these state-of-the-art dynamic samplers that perform aggressive drafting but lack a verification/revocation step. Without this, the reported gains might stem from simply being a better dynamic sampler, rather than from the novel revocation mechanism.\n\n- **Lack of Analysis on Computational and Memory Overhead:**\n     -  **Latency:** The speedup is reported in terms of step reduction and TPS. However, WINO increases the sequence length by adding a shadow block and requires a non-standard attention mask. This could slow down the per-step computation, especially if it prevents the use of highly optimized kernels like FlashAttention. A direct wall-clock time ablation comparing a forward pass with and without the WINO modifications (on sequences of the same effective length) is needed to understand the true per-step cost.\n\n    - **Memory:** The paper claims the memory cost is \"marginal.\" While likely true for the tested block length of 128, this cost is `(batch_size * Lb * d_model)`, which is non-trivial and scales with the block length `Lb`. A more rigorous analysis of how this memory overhead scales and its implications for longer-context decoding would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T91FiUnHS6", "reviewer": "Reviewer_hTSB", "summary": "The paper proposes Wide-In, Narrow-Out (WINO), a decoding method for accelerating inference in DLLMs. WINO appends a shadow block to the end of the context so that, after a single forward update, the model can write new tokens in parallel at several positions within the current block and also verify whether tokens written earlier in that block are reasonable, re-masking those deemed incorrect. Two hyperparameters, a drafting threshold and a verification threshold, allow a balance between throughput and accuracy. Because WINO is training-free and plug-and-play, it can be adopted by other DLLMs. The method is validated across multiple benchmarks and baselines.", "questions": "1. Why did the authors choose to append an additional auxiliary block at the end of the context to run the verification step on the freshly drafted block? Were alternative designs explored to achieve the same goal, for example adding trainable layers or parameters inside the model? \n2. The draft-verify workflow can be interpreted as explicit self-refinement during DLLM generation, where the decision to re-mask a position is the refinement step. Under this view, allowing more reflection steps during inference could trade speed for higher accuracy. However, in the threshold-tuning ablations (Fig. 4), accuracy often rises at first and then declines as the number of steps increases. Do the authors have a plausible explanation for this pattern?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The work introduces a novel draft-verify procedure that replaces the naive write-many-at-once update used in prior DLLM decoding. Compared with earlier approaches, WINO maintains high speedups without sacrificing performance and in many benchmarks even improves accuracy. \n2. The study evaluates both text-only and vision-language settings over 14 tasks, which strengthens the credibility of the conclusions. \n3. WINO does not require retraining and is easy to deploy as a plug-and-play component in other DLLMs.", "weaknesses": "1. As described in Section 4.2, during verification each position in y_shad conditions on the corresponding position in y_curr from the previous step. It is unclear whether this one-step lag negatively affects performance. \n2. The paper lacks detailed hardware information for the experiments, especially which GPU models and how many were used to obtain the main TPS results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FHRSUjHWur", "reviewer": "Reviewer_Rjq2", "summary": "Identified problem: DLLMs have to deal with quality speed trade off and it is attributed to irreversiblity of standard DLLM decoding.\n\nThe paper introduces WINO (wide in, narrow out), a training free algorithm that enables revocable decoding.\n\nWINO's parallel draft-and verify strategy includes:\n- Drafting (Wide in): aggressively draft multiple new tokens based on a lenient confidence threshold.\n- Verification (Narrow out): A verification module that re-evalutes all unmasked tokens with a bit stricter verification check, remasking the ones that fail the check.\n\nResults show good efficiency and quality gains on GSM8k and Flickr30k.", "questions": "see weakness section", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Revokable Decoding: The paper’s main idea is to make the decoding process \"revokable.\" This means the model can undo an early mistake and fix it right away, instead of being stuck with it (which is what happens in standard DLLMs).\n\n- No Extra Training Needed: The WINO method is a simple algorithm change, meaning it works immediately with existing DLLMs like LLaDA and MMaDA without having to retrain them.\n\nImproved Speed and Quality: WINO solves the trade-off between speed and accuracy. It makes the models much faster while also making them more accurate. The method provides huge gains in real-world tasks: GSM8K and Flickr30K.", "weaknesses": "- Please include some comparison on latency instead of decoding steps. \n- Additionally add a comparison of the number of parameters in the baseline model vs combined number of params in the drafter and verifier.\n- It is a bit unclear, but IIUC, the baselines do not use remasking based on low confidence, which is pretty standard and also mentioned in the related work section. Do your baselines use remasking?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)