WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning
Authors: Haojin Yang
Rui Hu
Zequn Sun
Rui Zhou
Yujun Cai
Yiwei Wang
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion language models
dynamic decoding
Primary area: generative models
TL;DR: WavefrontDiffusion introduces a dynamic decoding schedule for diffusion language models that adaptively expands a local wavefront to preserve semantic units, improving reasoning coherence without increasing compute cost.

Abstract: Diffusion Language Models (DLMs) have shown strong potential for text generation and are becoming a competitive alternative to autoregressive models. 
The denoising strategy plays an important role in determining the quality of their outputs.
Mainstream denoising strategies include Standard Diffusion and BlockDiffusion. 
Standard Diffusion performs global denoising without restricting the update range, often finalizing incomplete context and causing premature end-of-sequence predictions. 
BlockDiffusion updates fixed-size blocks in a preset order, but its rigid structure can break apart coherent semantic units and disrupt reasoning. 
We present WavefrontDiffusion, a dynamic decoding approach that expands a wavefront of active tokens outward from finalized positions. 
This adaptive process follows the natural flow of semantic structure while keeping computational cost equal to block-based methods. 
Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves state-of-the-art performance while producing outputs with higher semantic fidelity, showing the value of adaptive scheduling for more coherent and efficient generation.

Published as a conference paper at ICLR 2026
WAVEFRONTDIFFUSION:
DYNAMIC SCHEDULE FOR IMPROVED REASONING
DECODING
Haojin Yang', Rui Zhou', Rui Hu², Zequn Sun?, Yujun Cai 1 School of Software and Microelectronics, Peking University
Yiwei Wang
State Key Laboratory for Novel Software Technology, Nanjing University
"The University of Queensland, *University of California, Merced yh wwt.stt.ppueeu..coo cn; siweiwanger cue rced. edu https://githie http////iithubb.c..aaaeeaee 1ink
ABSTRACT
Diffusion Language Models (DLMs) have shown strong potential for text gener- ation and are becoming competitive alternative autoregressive models. The denoising strategy plays an important role in determining the quality of their out- puts. Mainstream denoising strategies include Standard Diffusion and BlockD- iffusion. Standard Diffusion performs global denoising without restricting the update range, often finalizing incomplete context and causing premature end-of- sequence predictions. BlockDiffusion updates fixed-size blocks in a preset or- der, but its rigid structure can break apart coherent semantic units and disrupt reasoning. We present WavefrontDifffsion expands a wavefront of active tokens outward from finalized positions. This adap- tive process follows the natural flow of semantic structure while keeping compu- tational cost equal to block-based methods. Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves oaate-of-the-ar performance while producing outputs with higher semantic fidelity, showing the value of adap- tive scheduling for more coherent and efficient generation.
dynamic decoding approach that
INTRODUCTION
Recent advances in large language models (LLMs) have achieved remarkable progress in complex reasoning and structured generation tasks such as mathematical problem solving and code synthesis (OpenAl et al. 2025] DeepSeck-AI et al. 2025). Autoregressive (AR) models remain the dominant paradigm for these tasks due to their stepwise logical consistency (Deletang et al. 2024). However, their strictly sequential nature introduces latency and limits flexibility, which can be problematic in settings that demand both accuracy and responsiveness, such as interactive assistants or real-time code generation. These limitations have motivated the exploration of alternative decoding paradigms that can balance quality, efficiency, and adaptability (Leviathan et al. 2023).
Diffusion Language Models (DLMs) have recently emerged as a promising alternative by framing text generation as an iterative denoising process (Gong et al., 2025 Song et al. 2025). Unlike AR models, which finalize one token per step, DLMs update multiple tokens in parallel over series of iterations (Schiff et al., [022 Lou et al., 2024). This design enables more flexible generation while maintaining global coherence, offering potential gains in both efficiency and representational Sapacity(Sahove et al. 2024).
The capabilities of DLMMs partly rely on the design of their denoising schedule, yet this aspect has received limited exploration. Current mainstream strategies include Standard Diffusion(Chang et al.) 2022) and BlockDiffision(AAriiaa al., (0025). Standard Diffusion applies global and uniform denoising policy without restricting the update range, which often causes overconfidence in end- of-sequence tokens(Nie et al., 2025) and insufficient context for denoised tokens, leading to early errors that cannot be corrected. Block Diffusion improves stability by processing fixed-size blocks in preset order, but it does not account for the natural semantic boundaries of the output, causing the denoising of coherent sequences to be artificially fragmented.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Standard Diffission Generation
Difeaiont Generation
Stop
Seep
Step
Step
Slep
Step
Candidate scape is global during denoising process
Step
BlockDirftsiin Generation
Step
Step
Step
Candidate scope expands from denoised token ench step
Step
Cundidate scope won't 't Rlw until current scope denoised
Prourpt
Masic
Completico
Predict
Candideres
Non candidoees
Figure I: Illustratoon of scheduling strategies in diffusion language models. A Diffusion Language Model predicts all masked tokens. At each step, a scheduling strategy decides which tokens to denoise. There are three strategies: Standard Diffusion uses global scheduling. It always selects tokens from all masked positions. Biffuusnn limits the choice to candidate scope. The scope is fixed blocks, similar to semi-autorege auni-auuoregressionn. WavefrontDiffusion also uses a candidate scope. Unlike BifcsDiff the scope grows dynamically during decoding.
To address these limitations, a new denoising strategy should offer several key benefits. First, should support adaptive scheduling, dynamically adjusting the denoising process to follow the evolving generation context rather than relying on fixed update patterns. Second, should provide each token with more complete contextual information during denoising, enabling smoother and more coherent outputs. Finally, it should preserve computational efficiency, matching the cost of block-based methods while delivering higher quality generation.
We introduce WavefrontDiffusion, a dynamic scheduling strategy for DLMs designed to overcome the limitations of prior approaches. It adaptively expands wavefront of active positions, allowing the denoising process to follow the natural structure of the generation context. Each token is updated with richer and more complete contextual information, producing smoother and more coherent out- puts. At the same time. WavefrontDiffusion maintains the computational efficiency of block-based methods, delivering higher quality generation without additional cost.
Contributions. Our contributions are fourfold:
1. We identify the key limitations of existing denoising schedules, including insufficient con- text in Standard Diffusion and the rigid boundaries of Block Diffusion that fragment coher- ent sequences.
2. We propose WavefrontDiffusion, a training free dynamic scheduling method that adapts the generation context without introducing any additional computational cost.
3. We conduct comprehensive experiments across four reasoning and code generation bench- marks, demonstrating clear improvements in output correctness and generation quality compared to strong baselines.
4. We perform a detailed hyperparameter analysis to validate the reliability and robustness of WavefromtDiffussonn under different settings.
BACKGROUND
This section provides background on Diffusion Language Models (DLMs) and two decoding sched- ules: Standard Diffusion and BlockDiffusion. These methods serve as baselines for comparison with our proposed WavefrontDiffussion.
2.1 PRELIMINARIES: DIFFUSION LANGUAGE MODELS
Diffusion Language Models (DLMs) treat text generation as an iterative denoising process over a sequence of discrete tokens (Austin al. 2021a). Let Xa ( denote clean token
2

---

Published as a conference paper at ICLR 2026
sequence of length N. The forward corruption process gradually replaces tokens with special mask token [MASK], following a masking probability t E [0, 1]:
N (xx xb) = II (1 t) 6(z1, z0,1) + ( = [MASK])]
==1
where 5() is the Dirac delta function. Intuitively, each token is independently masked with proba- bility t. The reverse process parameterized by model pe. which learns to iteratively denoise the corrupted sequence by predicting less-noised states:
N II ( Xx, c),
(x X1, c)
where с represents the conditioning context, such as a prompt or input sequence. Training minimizes the cross-entropy loss over masked positions under variational lower-bound (VLB) objective. At inference time, generation starts from fully masked sequence X1 [MASK] N and iteratively applies po until clean sequence produced.
2.2 STANDARD DIFFUSION DECODING
Standard Diffusion follows mask- mask-prediction process similar to MaskGIT. At each step, the model predicts tokens for all masked positions in parallel:
P: = (Xmasked X,, c),
where с is the conditioning context and x; is the current partially generated sequence. From these predictions, only subset of tokens with the highest confidence is selected and finalized, while the remaining positions stay masked for future updates.
This process repeats until every position is finalized. Standard Diffusion allows flexible, sequential generation without fixed update order. However, because selection relies solely on local confidence, early
may be locked in results in premature end-of-sequence (EOS) tokens and logically inconsistent outputs, especially in reasoning tasks that require multi-step coherence.
and propagate through later steps. This often
Standard Diffusion predicts all masked positions in parallel but updates only the most confident ones each step. It is simple and efficient but prone to cascading errors due the lack of global structural guidance.
2.3 BL.CK--WIS DIFFUSION SCHEDULING
BlockDiffusion limits the search space for each denoising step by dividing the sequence into K contiguous, non-overlapping blocks {B),..., Bx The model generates tokens block by block in fixed left-to-right order, where is the initial prompt context.:
p(Br p, B,,)),,,BBB.),,,
Only tokens inside the current block are considered for updating each step. This restriction helps control error propagation and provides more stable decoding process than Standard Diffusion. However, the fixed block boundaries and update order introduce two key problems: Semantic frag- mentation: Block boundaries can split coherent structures such function signatures, equations, or reasoning steps, which forces the model to finalize incomplete semantic units. Lack of adaptiv- ity: The update order cannot change according to context token-level confidence, which reduces flexibility during generation.
BlockDiffusion improves stability but remains limited for complex reasoning tasks. Semantic units often vary in length and dependencies can span multiple blocks, making a fixed schedule suboptimal. This limitation motivates the adaptive scheduling strategy introduce in Section
3

---

Published as a conference paper at ICLR 2026
3 WAVEFRONTDIFFUSION METHOD
This section introduces WavefrontDiffusion, new adaptive scheduling strategy for discrete diffu- sion language models. Unlike exed-block decoding, which relies on pre-defined partitions, Wave- frontDiffusion dynamically expands a localized frontier of candidate positions. This approach better aligns with the semantic structure of the output while maintaining identical computational cost.
3.1 METHOD THEORY
The decoding process of diffusion language models (DLMs) relies heavily on the scheduling pol- icy, which determines the order in which masked tokens are finalized. A well-designed schedule should align with the natural semantic flow of generation while maintaining computational effi- ciency. However, existing methods fall short in this regard. Standard Diffusion updates all positions simultaneously, often finalizing incomplete or incorrect context. BlockDiffusion improves stability by partitioning the sequence into fixed-size blocks, but its rigid structure artificially fragments se- mantic units such as reasoning steps or code segments. This misalignment between block boundaries and true semantic boundaries leads to premature commitments and cascading errors.
Intuition: generation as spreading wave. To address these limitations, we introduce the concept of wavefront-a dynamic set of candidate positions that expands outward from already finalized tokens. Instead of processing fixed contiguous blocks, the decoding process grows like a wave, gradually extending into surrounding masked regions as local context becomes available. This de- sign ensures that each token is updated only when has access to sufficient contextual information, thereby respecting semantic locality and preventing incomplete updates. Unlike static schedules, wavefront adapts to the evolving structure of the sequence, allowing the model to prioritize seman- tically relevant regions each step.
Theoretical Grounding. We formalize this intuition through the Information Gradient Hypoth- esis, which posits that the conditional entropy of token increases monotonically with its distance from the finalized context. While BlockDiffusion restricts the search space to rigid pre-defined blocks, WavefrontDiffusion performs an optimal restricted search within "entropy isosurfaces" de- fined by this distance. We prove in Appendix [ that our dynamic boundary is mathematically guaranteed to contain higher density low-entropy candidates compared to static blocks, thereby minimizing the probability of semantic mismatch.
Mathematical definition. Let N be the total length of the output sequence and C, denote the set of finalized tokens at iteration f. We define the wavefront set W, as:
Wi = {i dist(i, C.) S R}.
(1)
where dist(i, C.) measures the minimum distance between position i and any finalized position, and R is user-defined expansion radius. Intuitively, the wavefront contains all masked tokens that are within R steps of a completed token, ensuring that updates focus on tokens with nearly complete local context.
At the beginning of decoding, C. contains only the prompt context, and Wo is initialized as the first F positions following the prompt, where F is the maximum wavefront size. As generation proceeds, W; expands dynamically based on the evolving structure of C. This expansion process adaptively shifts computational focus to regions that are semantically important, while maintaining strict limits on total computational cost.
This dynamic frontier design provides two key benefits:
1. Context completeness: Tokens are only finalized when surrounded by sufficient context, mitigating the boundary truncation problem in BlockDiffusion.
2. Compute parity: By capping the wavefront size at F, the overall number of token updates remains identical to
methods, ensuring no increase computational cost.
In the next subsection, we will provide a step-by-step algorithmic description of how WavefrontD- iffusion operationalizes this theory into practical decoding strategy.
4

---

Published as a conference paper at ICLR 2026
3.2 METHOD: ALGORITHMIC DETAILS
Building on the theoretical foundation in Section 3.1 we now describe the concrete implementation of WavefrontDiffusion. The goal to operationalize the dynamic wavefront mechanism into practical, step-by-step decoding strategy. At a high level, WavefrontDiffusion iteratively updates subset masked positions, allowing the denoising frontier to expand outward from previously finalized tokens while strictly controlling computational cost. This ensures that tokens are finalized context-aware manner without increasing the total number of forward passes compared block- based schedules.
Core iterative process. Each iteration of WavefrontDiffusion consists of four sequential steps, summarized below and illustrated in Algorithm D
1. Score. Perform a single forward pass of the model to compute logits for all masked posi- tions. For each masked position j. compute a confidence score 8, as the maximum softmax probability:
maxxplzz u De. c). N
(2)
where V is the vocabulary and e the conditioning context (e.g., prompt). These scores are cached for later use.
2. Select & Denoise. From the current wavefront W1-1, select the top-k, positions with the highest confidence scores and finalize them by replacing their mask tokens with the predicted values:
he = libuse + iit S extra].
where kbuse [N/T] evenly divides the total N tokens over T steps, and extra dis- tributes the remaining tokens. If [M/-111 < k2, additional high-confidence positions outside the frontier are included to meet the per-step budget.
3. Expand. For each finalized position i, add all masked neighbors within radius R to the next wavefront W, This adaptively expands the active frontier around newly completed regions:
W. = UG dist(j, & R, Ij = [MASK]] iEC.
(4)
4. Prune. If [W:] > F. retain only the top- positions by confidence score. This strictly bounds the per-step computational footprint, ensuring parity with bosk-based methods.
This four-step cycle repeats until all tokens are finalized or the step budget T' is exhausted.
The full WavefrontDiffusion procedure is formalized in Algorithm Here, denotes the maximum wavefront size, and R controls the expansion distance of the frontier.
Algorithm WavefrontDiffusioon Decoding
1: Input: model 8, prompt context c, generation length N, total steps T, max wave size F, expan- sion radius R
2: Initialize masked sequence of length N and wavefront Wo with the first positions
3: hhuse N//T,, extra N mod! T
4: for = to d'
5: K; hrase + 1 extra]
Per-step budget
6: Run one forward pass to compute logits for all masked positions
7: Compute confidence scores s, for each masked position
8: Finalize these positions by replacing masks with arg max predictions
Expand W, with neighbors within radius of finalized positions 10: Prune W, to size F' using cached confidence scores
9:
11:
end for
The total number of token updates performed by WavefrontDiffusion is strictly bounded by the product F T, which matches the compute budget of BlockDiffusion. The only difference lies in the location of updates, not their quantity. This design guarantees that improvements in quality arise solely from better scheduling rather than increased computational resources.
5

---

Published as a conference paper at ICLR 2026
WavefrontDiffusion achieves dynamic, context-aware generation by repeatedly scoring, selecting, expanding, and pruning candidate tokens. By strictly limiting the wavefront size and per-step bud- get, the method maintains identical computational cost to block-based baselines while producing semantically coherent outputs.
4 EXPERIMENTS
We conduct series of experiments to rigorously evaluate the effectiveness WavefrontDiffusion and to answer the following key research questions:
1. RQ1: Performance. Does WavefrondDiffusion achieve higher accuracy challenging
reasoning and code generation benchmarks compared to compute-matched baselines?
2. RQ2: Semantic Fidelity. Does WavefrontDiffusion produce outputs that more seman- tically coherent and faithful to ground truth, as measured by semantic similarity metrics?
3. RQ3: Qualitative Behavior. Can we qualitatively demonstrate that WavefrontDifftssion better respects semantic boundaries during generation, avoiding the forced segmentation observed rigid block scheduling?
The following sections are organized to address these questions in order: Section 4.2 presents bench- mark performance results (RQ1), Section 4.3 evaluates semantic fidelity using BERTScore (RQ2), and Section 4. presents quantitative analysis of MHCO to demonstrate how WavefrontDiffusion better respects semantic boundaries during generation (RQ3).
4.1 EXPERIMENTAL SETUP
Tasks and Datasets. We evaluate WavefrontDiffusion on five benchmarks that test both mathemat- ical reasoning and program synthesis: GSMSK(Cobbe al. 2021), MATH(Saxton al. 2019). HumanEval et al. 202Ip. MBPP et al., 20211)) and Big Bench Hard (BBH)Suzgun
al., (0022). These datasets cover diverse domains and represent challenging evaluation scenarios for diffusion-based decoding.
All experiments are performed in a strict zero-shot setting without chain-of-thoon off-thugtht prompting or few-shot examples. Any intermediate reasoning traces are generated by the model itself rather than injected hints. For reasoning benchmarks (GSM&K, MATH, BBH) we report accuracy using exact match (EM), and for HumanEval and MBPP report pass@ I accuracy. For semantic quality evaluation (Section 4.3) we use the WikiText dataset. We randomly sample 1,000 sentences and repeat evaluations several times, then report the averaged scores.
Model and Baselines. We use three discrete diffusion language models: -LLDA--B-Inssie LLaDA-1.5, and Dream-7B (Ye et al., 2025). Keeping the same backbone across all methods en- sures that performance differences come only from the decoding schedule. We compare Wavefront- Diffusion with two baselines: 1) Standard Diffusion, which updates all masked tokens in parallel at every step. This highlights the effect of structured scheduling, 2) BlockDiffusion which denoises fixed, non-overlapping blocks in rigid order and is the current state-of-the-ee among block-based schedules.
Evaluation Protocol. All experiments run in a deterministic environment with temperature set to 0.0. This setting removes randomness and guarantees reproducibility, so we do not report standard deviations or seed-based variation in the main tables. The computational budget is controlled by fixing the number of forward steps to 1024 for every method. We measure cost as the wall-clock runtime needed to complete evaluation on each dataset. Because runtime can vary slightly across runs, the reported cost may differ by small margin even when the number of forward steps is identical. All core hyperparameters (wavefront size F, expansion radius R, and step budget T) are provided in the Appendix to ensure transparency and reproducibility.
4.2 MAIN RESULTS
We first evaluate whether WavefrontDiffusion improves reasoning and code generation accuracy when the compute budget is fixed. Table shows results across five benchmarks. WavefrontDif-

---

Published as a conference paper at ICLR 2026
Table 1: Main Performance Comparison. Each dataset reports accuracy (Acc) and computation cost time (Cost), Highlighted rows indicate our proposed WavefrontDifftsion, which consistently outperforms Block- Diffusion under identical compute budgets. MBPP and Dream-7B 7B results are added to demonstrate generaliza- tion.
GSM8K
MATH
HumanEval
MBPP
BBH
Strategy
Ace
Cost
Acc
Cost
Acc
Cost
Acc
Cost
Acc
Cost
MaaDA--B-Instruuee
Standard Diffusion
23.15
110.1
26.60
108.8
17.68
115.8
13.50
109.1
11.30
122.7
BlockDiffusion
80.74
113.6
40.62
109.4
45.73
107.2
41.17
109.2
43.23
116.6
WavefrontDiffusion (Ours)
82.03
112.2
41.04
109.9
47.56
109.3
42.40
107.8
44.30
115.1
LLaDA-1.5
Standard Diffusion
30.17
111.9
27.04
111.4
34.76
118.0
17.04
114.4
17.12
127.2
BlockDiffusion
82.33
111.6
41.64
107.4
46.34
110.1
44.04
118.1
44.56
118.6
WavelrontDiffusion (Ours)
82.94
109.3
41.96
110.9
48.17
112.9
46.20
112.3
45.26
114.1
Dream-7B
Standard Diffusion
35.03
99.8
29.98
100.8
20.12
99.5
25.05
99.1
16.27
103.2
BifuuDin
78.92
99.2
43.60
103.5
53.05
100.6
58.52
95.9
45.13
100.5
WavefrontDiffusion (Ours)
80.66
101.7
44.00
98.0
54.27
101.4
59.03
95.7
46.91
105.7
fusion achieves the best performance on all tasks and for all three model families. It outperforms Standard Diffusion by large margin and also provides consistent gains BlockDiffusion.
Analysis. For the larger LLaDA-8B-Instree -8--Inst WavefrondDiffusion improves accuracy by +1.27 on GSM8K, +0.42 MATH, +1.83 on HumanEval, +1.23 on MBPP, and +1.07 BBH compared to BlockDiffusion. For the smaller LLaDA-1.55 the improvements are +0.61 on GSM8K, +0.32 on MATH, +1.83 HvmanEval, +2.16 MBPP, and +0.70 BBH. We observe similar consistent gains on the Dream-7B model (e.g., +1.74 on GSMSK, +1.22 on HumanEval, and +0.51 on MBPP), confirming that the benefits of our method generalize across different architectures.
These gains are obtained with the same number of forward steps (1024) and under the same runtime budget. Although the reported wall-clock cost may differ slightly due to measurement noise, the computational load is identical across methods.
The results highlight key points. First, adaptive scheduling helps the model finalize tokens only when sufficient context is available. This reduces premature commitments that often occur in BlockDiffusion and improves answer correctness. Second, the improvements appear consistently across both mathematical reasoning and program synthesis, and across different model sizes and families, showing that the benefits of dynamic scheduling are general and not tied to scale. Together, these findings confirm that the performance gains come from better scheduling rather than additional compute.
4.3 SEMANTIC QUALITY EVALUATION
To address RQ2, we test whether WavefrontDiffusion produces outputs that are more faithful and coherent than BlockDiffusion. We use BERTScore (Zhang et al.) 2220),, which compares system outputs with human references through contextual embeddings. Recall (R) reflects completeness of the output, Precision (P) reflects avoidance of irrelevant tokens, and F1 balances both.
Analysis. WavefromtDiffusion reaches the highest scores across all metrics. The gain in Precision shows that it avoids inserting irrelevant or noisy tokens, while the gain Recall shows that it com- pletes sequences more fully. Together these improvements lead to a stronger Fl. These results confirm that the wavefront strategy helps the model wait until enough context available before finalizing tokens. This behavior preserves semantic boundaries and reduces the fragmentation that common in block-based schedules. The findings give direct evidence that adaptive scheduling improves not only task accuracy but also the quality and coherence of generated text.

---

Published as a conference paper at ICLR 2026
Table 2: BERTScore results on WikiText. Higher better. WavefrontDiffusion achieves consistent gains Precision, Recall, and FI.
Strategy
F1
Standard Diffusion
0.7885 + 0.0045
0.7664 + 0.0064
0.7913 0.0018
BlockDiffusion
0.7946 + 0.0035
0.7663 + 0.0055
0.8142 + 0.0020
WavefrontDifftsson (Ours)
0.8094 + 0.0033
0.7749 + 0.0052
0.8236 + 0.0018
0.200
0.20
0.175
0.175
Awrrgg MHCO
0.125
0.100
0.000
0.775
0.075
0.055
0.02
0.000
GSMK
MATH
Homoeaal
2000
0.002
GSMAK
MATH
Homantaal
R00
(a) LLLADA-BB-InSS 8B-Itrructt
(b) LLaDA-1.5
Figure 2: Comparison MHCO values across strategies. Lower values mean fewer priority viola- tions and closer alignment with semantic order.
4.4 MEASURING PRIORITY VIOLATIONS VIA MHCO
To address RQ3, we introduce the Masked Higher-Confidennc Outside (MHCO) metric. This metric measures how often the decoding process finalizes low-confidence token before nearby token with higher confidence. A lower MHCO score means that the schedule respects confidence ordering and more consistent with semantic boundaries.
Definition. Let Sy be the set of tokens selected for denoising at step t. Nout be the set of masked tokens within the expansion radius R of the active frontier. We define:
MHCO,
Σ1
Nous cels) > aiii).
iES.
where (i) is the confidence assigned to token i. A smaller value indicates that higher-confidence tokens are prioritized more consistently.
Results. Figure 2 shows that WavefrontDiffusion produces consistently lower MHCO values than BlockDiffusion on all datasets and both model scales. This means that the dynamic wavefront more often respects local confidence ordering.
These results give quantitative support for the intuition behind WavefrontDiffusion. The method finalizes tokens only when they are well supported by context and by confidence. This prevents the rigid boundary effects seen BlockDiffusion, where tokens inside block may be forced ahead of higher-confidence tokens just outside. Lower MHCO correlates with the accuracy gains in Table [ showing that reducing priority mismatches leads to more coherent reasoning and better answers.
4.5 HYPERPARAMETER ANALYSIS
WavefrontDiffusion depends on two main hyperparameters: the maximum wavefront size F and the expansion radius R. The parameter F controls how many candidate tokens considered in each step. The parameter R controls how far the frontier can expand beyond finalized tokens. We study their impact while keeping all other settings fixed.
Effeet of F. We fix R 2 and test F E {4,8, 16). Table 3 shows that accuracy increases when F grows from to 8. A larger frontier allows the model consider candidate positions and
8

---

Published as a conference paper at ICLR 2026
finalize tokens that are better supported by context. However, aceuracy does not improve further
F = 16. This indicates diminishing returns and suggests that very large frontiers may add redundant candidates without adding useful information.
Effeet of R. We fix = 8 and test {2, 8,). As Table shows, expanding the radius from 2 to 4 yields small gains by letting the frontier reach semantically relevant positions earlier. Increasing R further to does not bring consistent improvements and can reduce accuracy on HumanEval. This suggests that an overly wide frontier may weaken local focus and introduce noise.
The results show that WavefrontDiffusion is stable across a wide range of values. Moderate settings such F and R 2 provide good balance between local precision and global coverage. These settings also match our default configuration. Because the method is not highly sensitive to either parameter, it requires little tuning and can be deployed without major adjustments.
Table 3: Performance of WavefronDDifussion under different hyperparameter settings. Accuracy remains stable across a wide range of R and F, showing that the method is robust.
Varying F with R
(b) Varying R with
Dataset
F=4 4
F 8
F = 16
Dataset
R R= 2
R 4
R 8
MATH
41.02
41.04
41.22
MATH GSM8K
41.04
40.98
41.00
GSM8K
82.71
82.03
82.03
82.03
82.03
82.09
HumanEval
45.12
47.56
45.12
HumanEval
47.56
46.34
42.07
Discussion. In practice, we recommend using a conservative (e.g., F' 8) and adjusting It based on task complexity and compute resources. This keeps tuning overhead low and ensures reliable performance. Despite these advantages, WavefrontDiffusion shares limitations with other diffusion- based decoders. It depends internal confidence scores, which can be miscalibrated, especially out of domain. also cannot fully avoid error propagation when early mistakes occur in long reasoning reversible decoding reduce these risks,
chains. Future work may explore gelayed-finalizatin and improved calibration methods to enhance robustness. Extending the approach to multi-modal or structured domains, such code or graphs, another promising direction.
RELATED WORK
This work relates closely to several areas: Diffusion Language Models (DLMs), decoding strategies for text generation, and confidence-based generation methods.
Diffusion Language Models Diffusion models originated in image generation (Ho et al. 2020
Song & Ermon 2020 Recent research extends these principles to text, leading to Diffusion Lan-
guage Models (DLMs) (Li et al, 2022). DLMs generate text through an iterative denoising process, showing promise in various text generation tasks (Ye al. 2025 Xie et al., 20255). Early DLMS em- ployed global masking strategies, predicting all masked tokens in parallel at each denoising step
al. 2023). DLMs offer advantages in long text generation and mitigating exposure bias Zeng
et al.
2025).
Decoding Strategies for DLM Decoding strategies in text generation optimize output quality. Au- toregressive models commonly use greedy decoding, beam search, or sampling methods (Meister al. 2022). These methods typically generate tokens sequentially. For non-autoregressive mod- els. particularly DLMs, decoding strategies are crucial. Existing DLM decoding strategies include: Standard Diffusion: This method repredicts all masked tokens at each timestep, lacking exploita- tion of already finalized tokens. Block Diffusion: This strategy partitions the sequence into fixed blocks for parallel generation. However, can disrupt semantic coherence due to rigid boundaries. These strategies often involve trade-offs between generation quality and computational efficiency.
Confidence- Generation Confidence-bast generation methods leverage a model's cer- tainty in its predictions to guide the generation process. Such approaches are common in autoregressive models, enhancing both generation efficiency and quality (Gulrajani & Hashimoto,
2023 et al. 2025). For instance, some methods select high-confidence tokens for finalization

---

Published as a conference paper at ICLR 2026
based prediction certainty, focusing subsequent steps on lower-confidenee regions. This strategy prioritizes stabilizing accurate predictions, thereby reducing error accumulation.
CONCLUSION
We introduced WavefrontDiffusion, dynamic scheduling method for diffusion language models. The method balances semantic coherence and local accuracy in an adaptive way. Unlike static schedules, uses token-level confidence to guide the expansion process. This design allows more effective exploration of the output space.
Experiments on mathematical reasoning and code generation benchmarks show that WavefrontD- iffusion improves accuracy while preserving the computational cost of block- based decoding. We also proposed the Multi-Hop Coherence (MHCO) metric. This metric evaluates semantic consis- tency across generations and offers insights that standard accuracy cannot capture.
WavefrontDiffusion improves semantic coherence but depends on reliable token-level confidence. Poor calibration can reduce performance. Current experiments focus zero-shot settings. Extend- ing the method to few-shot or chain-of-thought prompting is an open direction. The computational cost remains similar block decoding, but very long contexts may cause efficiency issues.
Future work includes three directions. First, improve confidence estimation to increase the reliabil- ity of dynamic scheduling. Second, explore alternative guidance signals for the denoising path, such as entropy or other uncertainty-basee measures. Third, apply the method to domains such as pro- gram synthesis and long-form text generation. We expect WavefrontDiffusion encourage broader research on adaptive decoding for diffusion-bars based models.
ACKNOWLEDGEMENTS
We thank the anonymous reviewers for their constructive comments and suggestions, which helped improve the clarity and quality of this work. The work is partially supported by the U.S. National Science Foundation (NSF) Grant CRII 2451683, an NVIDIA Academic Grants Program, a U.S. Bank Academic Research Award, the University of California, Merced, and a UC Merced Faculty Research Award. The views and conclusions are those of the authors and do not necessarily reflect the official policy or position of the U.S. Government.
ETHICAL CONSIDERATIONS
This research employs large language models for text generation. While our method primarily fo- cuses on improving the technical performance of diffusion language models for tasks like reasoning and code generation, it crucial to acknowledge the broader ethical implications associated with such technologies.
Our work, like all research involving generative AI, may contribute to the development of models that could potentially be used for generating misinformation. harmful content, or facilitating auto- mated unethical activities. This includes, but is not limited to, the creation of deceptive text, biased narratives, or malicious code. We emphasize that the responsible development and deployment of these models require careful consideration of societal impact.
We adhere to the principles of responsible Al research. Our objective is to advance the capabilities of language models in beneficial domains, such as assisting in complex problem-solving and code development. We strongly advocate for the use of our findings for positive and ethical applications, and we urge future researchers and practitioners to critically assess and mitigate potential harms arising from the misuse of generative AI technologies.
Furthermore, any generative model may inadvertently perpetuate biases present in its training data. While our method is decoding strategy and does not introduce new biases into the base model itself, is dependent on the underlying model's 's inherent characteristics. Future work should continue to explore methods for detecting and mitigating such biases in the complete generation pipeline.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We confirm that all experiments reported in this paper can be reproduced using the provided code and environment. The exact training and evaluation procedures, including dataset preprocessing, hyperparameter settings, and evaluation scripts, are included our submission. We will release the full codebase and necessary instructions the public after the review process to ensure transparency and facilitate future research.
LANGUAGE MODEL USAGE STATEMENT
This paper used large language model (LLM) solely as tool assist in code refinement and writing polishing. All core ideas, experiments, results, and analyses originate from the authors. The authors fully reviewed and validated every change introduced by the LLM ensure correctness and avoid hallucinations.
No section of the research, including theoretical contributions or experimental design, was authored or invented by the LLM. Responsibility for all content rests entirely with the authors, who confirm that no misleading claims, fabricated data, or misrepresentations are present due to the use of the LLM.
REFERENCES
Anonymous. Diffusion with truncated blocks: Towards fast and high-quality text generation using truncated block generation. In Submitted The Fourteenth International Conference on Learn- ing Representations, 2025. under review.
.nt. .nt.tt/Eoorum.d-BvyyTlleee
Marianne Arriola, Subham Sekhar Sahoo, Aaron Gokaslan, Zhihan Yang. Zhixuan Qi, Jiaqi Han, Justin T Chiu, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In The Thirteenth International Conference Learning Repre- sentations, 2025. URL htttp:://pperreiiiiii:
net wow.e///ooo.ddoy//rrrr.
Jacob Austin, Daniel Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021a. URL https::/cpenneett
net forum h XiXPCAL
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021b. URL hwt.::ttp//// arxiv. org/ abs 2108 07732
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked gener- ative image transformer. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2022.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code, 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jemy Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL
arg//bbs/111 14168
11

---

Published as a conference paper at ICLR 2026
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng. Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma Yiyuan Liu, Yongqiang Guo, Yuan Ou Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang. Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan. Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang. Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li. Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforce- ment learning, 2025.
org/ abs 2501 12948
Gregoire Deletang, Anian Ruoss, Amuloimbooise Duquenne, Elliot Catt, Tim Genewein, Christo- pher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. Language modeling is compression. In The Twelfth International Confer- ence on Learning Representations, 2024. URL
net cenet/foruune fwr.mmirrmm
jzrbbi ynus
Yifeng Gao, Ziang Ji, Yuxuan Wang, Biqing Qi, Hanlin Xu, and Linfeng Zhang. Self speculatie de- coding for diffusion large language models, 2025. URL
04147
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration, 2025. URL
20639
Ishaan Guirajani and Tatsunori B. Hashimoto. Likelihood based diffusion language models, 2023. URL httpthttts////////eee .rgg
Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. Mdpo: Overcoming the training-inference
divide of masked diffusion language models, 2025.
13148
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL attts:://arxn org/ abs 2006 11239
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. 2023. URL htttps://aaaiiii.. http://wwwxiv..g.abbb//a/aoa
17192
Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A survey on diffusion language models, 2025. URL http:////aaaaaa.oo///////...... 10875
Xiang Lisa Li, John Thickstun, Ishaan Guirajani, Perey Liang, and Tatsunori B. Hashimoto. Diffusion- improves controllable text generation, 2022. httpst:s: // /ttp///axvvv./////////// www.//oorg/aaa /bs/ 2205. 74217
12

---

Published as a conference paper at ICLR 2026
Yifan Li, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. Diffusion models for non autorrgresssvev text generation: A survey, 2023. [
Aaron Lou, Chenlin Meng. and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In Forty first Internattonnal Conference on Machine Learning, 2024. URL
httts:///peereet
net/
Clara Meister, Tim Vieira, and Ryan Cotterell. Best-first beam search, 2022. URL https: //
Arxiv.org/ar abcpg//bb//0e/ 2007.0000000 03909
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025. URL https://
a mrxiv.org/ale /20...50222. 09992
OpenAI, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, Kai Chen, Mark Chen, Enoch Cheung, Aidan Clark, Dan Cook, Marat Dukhan, Casey Dvorak, Kevin Fives, Vlad Fomenko. Timur Garipov, Kristian Georgiev, Mia Glaese, Tarun Gogineni, Adam Goucher, Lukas Gross, Katia Gil Guzman, John Hallman, Jackie Hehir, Johannes Heidecke, Alec Helyar, Haitang Hu, Romain Huet, Jacob Huh, Saachi Jain, Zach Johnson, Chris Koch, Irina Kofiman, Dominik Kundel, Jason Kwon, Volodymyr Kyrylov, Elaine Ya Le, Guillaume Leclerc, James Park Lennon, Scott Lessans, Mario Lezcano-Casado, Yuanzhi Li, Zhuohan Li, Lin, Jordan Liss, Lily, Liu, Jiancheng Liu, Kevin Lu, Chris Lu, Zoran Martinovic, Lindsay McCal- lum, Josh McGrath, Scott McKinney, Aidan McI.aughlin, Song Mei, Steve Mostovoy, Tong Mu, Gideon Myles, Alexander Neitz, Alex Nichol, Jakub Pachocki, Alex Paino, Dana Palmie, Ash- ley Pantuliano, Giambattista Parascandolo, Jongsoo Park, Leher Pathak, Carolina Paz, Ludovic Peran, Dmitry Pimenov, Michelle Pokrass, Elizabeth Proehl, Huida Qiu, Gaby Raila, Filippo Raso, Hongyu Ren, Kimmy Richardson, David Robinson, Bob Rotsted, Hadi Salman, Suvansh Sanjeev, Max Schwarzer. D. Sculley, Harshit Sikchi, Kendal Simon, Karan Singhal, Yang Song, Dane Stuckey, Zhiqing Sun, Philippe Tillet, Sam Toizer, Foivos Tsimpourlas, Nikhil Vyas, Eric Wallace. Xin Wang, Miles Wang, Olivia Watkins, Kevin Weil, Amy Wendling, Kevin Whinnery, Cedric Whitney, Hannah Wong, Lin Yang. Yu Yang, Michihiro Yasunaga, Kristen Ying, Wojeiech Zaremba, Wenting Zhan, Cyril Zhang, Brian Zhang, Eddie Zhang, and Shengjia Zhao. gpt-oss- 120b & gpt-oss-20b model card, 2025. URL wtt.:htts://aratto org abs 2508. 2508.1099900 10925
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:1 30133-130088888 2024.
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical rea- soning abilities of neural models, 2019. URL
Yair Schiff, Subham Sekhar Sahoo, Hao Phung. Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P. de Almeida, Alexander Rush, Thomas Pierrot, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models, 2025. UtLhtts::// orgba/// 1312210199
Yang Song and Stefano Ermon. Generative modelngg by estimating gradients of the data distribution, 2020. URL https: http:///aaxxv..or/bbbao 907. 05600
Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, Yuwei Fu, Jing Su, Ge Zhang, Wenhao Huang, Mingxuan Wang, Lin Yan, Xiaoying Jia, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Yonghui Wu, and Hao Zhou. Seed diffusion: A large-scat scale diffusion language model with high-speed inference, 2025. URL
https:: //arviv// org/ abs 2508 02193
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big- bench tasks and whether chain-of thiin-of-thuuhht can solve them. arXiv preprint arX1v:2210.09261, 2022.
13

---

Published as a conference paper at ICLR 2026
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao. Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training- free acceleration of diffusion Ilm by enabling kv cache and parallel decoding, 2025. ULL/Ettss//aaxx..rrrrr/////////
2505. 22618
Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream-coder 7b: An open diffusion language model for code, 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models, 2025. URL htttps://axiv..o www. abs//
22588.1483
Yiming Zeng. Jinghan Cao, Zexin Li, Yiming Chen, Tao Ren, Dawei Xiang, Xidong Wu, Shangqian Gao, and Tingting Yu. Treediff: Ast-guided code generation with diffusion llms, 2025. URL
https://arxiv.ortt wwg. abs /2008.258. 01473
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi Bertscore: Evalu- ating text generation with bert, 2020. URL httpt:t//axiiiii/ org. hwlgg/ab///ie /tt.ppss//axxii..rr///////////// 09675
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A HYPERPARAMETER INTRODUCTION
This section provides a comprehensive overview of all hyperparameters utilized in the Wavefront Diffusion algorithm and its experimental setup.
A.1 WAVEFRONT DIFFUSION SPECIFIC PARAMETERS
Parameter
Description
The maximum size of the wavefront. This integer parameter defines the maximum number of masked tokens within the wavefront that be simultaneously considered for denoising, A larger / allows for broader contextual consideration but may increase computational overhead per step.
R
The expansion radius. This integer parameter determines how many to- kens outwards from a newly denoised (finalized) token the wavefront expand. It controls the locality of the wavefront's growth. A larger R facilitates faster wavefront propagation.
A.2 GENERAL DIFFUSION LANGUAGE MODEL PARAMETERS
Parameter
Description
The total number of diffusion timesteps. This integer parameter defines the maximum number of iterative denoising steps allowed for generating the complete sequence.
Noise schedule parameter. This parameter, if applicable, controls the degree of stochasticity or the magnitude of noise introduced during the reverse (denoising) process. Specific values depend on the chosen noise schedule function.
T
Temperature for sampling. This real- valued parameter influences the ran- domness of token sampling during generation. Higher values increase randomness, while lower values promote more deterministic (greedy- like) selections.
B EXPERIMENTAL DESIGN
This section details the datasets, experimental parameters, and computational environment emvironmentemployed for evaluating Wavefront Diffusion.
B.1 DATASETS
Our experiments utilize standard benchmarks for reasoning and code generation tasks.
dataset for mathematical word problems requiring multi-step reasoning. We use the official test set for evaluation.
GSM8K:
MATH: challenging dataset comprising competition-style math problems across various difficulty levels. We report results on the test set.
HumanFyal: dataset designed for evaluating code generation capabilities, consisting of Python programming problems with test cases. We follow the standard evaluation protocol.
Big Bench Hard (BBH): A challenging suite of reasoning tasks covering diverse domains such as logic puzzles and multi-step reasoning.
15

---

Published as a conference paper at ICLR 2026
For all datasets, input prompts formatted according to the specifications of the base Diffusion Language Model.
B.2
EXPERIMENTAL PARAMETERS
All experiments are conducted using two pre-trained diffusion-basek language models as the core denoising backbone.
Base Model: LLaDA-8B 8B-ntstuutt,, 1L
Sequence Length: All generated sequences are fixed to a maximum length of 1024 tokens.
Number of Diffusion Steps (7"): Consistent across all experiments T 1024 steps.
Wavefront Diffusion Parameters:
For all benchmarks, the wavefront size is set to 8 and expansion radius R to 2.
For Standard Diffusion and Block Diffusion baselines, parameters are match the respective original implementations or optimized for best performance on these tasks, ensuring a fair comparison. (e.g., 'Block Diffusion block size B 8*).
Baselines Parameters:
B.3 EXPERIMENTAL ENVIRONMENT
Experiments were performed on a cluster equipped with NVIDIA A100 GPUs. Each GPU pos- sesses 80 GB of memory. The operating system Ubuntu 20.04.2 LTS, running Python 3,9.16 with PyTorch 2.7.0. All code was implemented using Hugging Face Transformers library.
B.4 EXAMPLE TEST PROMPTS
To enhance reproducibility and clarity, this subsection provides representative test prompts used for each dataset. These examples illustrate the exact input format provided to our Diffusion Language Model.
Table 4: Example test prompts for GSM8K, MATH, and HumanEval datasets. The full set of prompts is included in the released code.
Dataset
Example Prompt
You're re an expert solving elementary math problems involving additien, subtraction, and multiplication... ####
MACH
Let'ss think step and output the Final ansver within boxed:).
0: (query)
RumanBval
def has_eicse, ements (numbers: List (flost), threshold; float) >> bool: "Civen 1ist of numbers.... #44 True False
B.5 PROMPT TEMPLATES FOR BBH SUBSETS
We provide representative prompt templates for subsets of BBH. The templates follow standardized reasoning and answer formats, ensuring reproducibility across tasks.
16

---

Published as a conference paper at ICLR 2026
Table 5: Prompt templates for BBH subsets. Each template enforces explicit reasoning before pro- ducing the final answer.
BBH Subset
Prompt Template
causel judgement. ga
You are deciding YYea or \No*,
r.nndestannding web.of.lies
Think step by step... trt#cyes(Ncc>
boolean.ex expressions
You are deciding VTrue" or \False", Think step by step... t#ttcurue ralss>
Formal fallacies
You are deciding Walld" Miavalid" Think step by step... intrevalidiinvalidd invalids
date.unde ratardinn disambigus ion.ge, seometsicsshape. hyperbation,
You expert at answering multiple choice chiipee questions...
t#t (A) (BI
logi ca _deduct fonut, stte.eecooeedaallc
penguins
ruin.names,
snarks,
Lemporai..equerness
stackkingshuffflee.oojj...
dyck.languages,
tistep ari thmetio two, e jecte.connntingg
Solve the problem step by step... ####ctnswer>
worc_aorting
Solve the problem step step... ####woord word2 word3 ...>
B.6 DETAILED ACCURACY OF BBH
report detailed accuracy across BBH subsets for both LLLDA-8B- and LLaDA-1.5. Base Accuracy refers to standard decoding, Wave Accuracy refers to WavefrontDiffusion decoding, and A denotes Wave Base.
B.7 CASE STUDY: DYNAMIC WAVEFRONT vs. STATIC BLOCK
To complement the quantitative MHCO analysis in Section 4.4 we present a qualitative case study visually compare decoding behaviors of BlockDiffusion and WavefrontDiffusionn This pro- vides intuitive understanding of how adaptive scheduling enables better alignment with semantic structure, directly addressing RQ3.
Setup. evaluate both strategies on a code generation task from the HumanEval benchmark using the following prompt:
"Write Python function compute Fibonacci numbers recursively."
All experiments use the L.LDDA-BBInstrruc model under strictly matched compute budget, with deterministic decoding (temperature 0). No chain-of-thougg prompting or auxiliary signals are provided, ensuring that differences arise solely from the decoding schedule.
Observations.
Table shows snapshots of the decoding process.
17

---

Published as a conference paper at ICLR 2026
Table 6: Accuracy of LLaDA-8B-Instruct across BBH subsets, A denotes Wave Base.
Task
Base Accuracy
Wave Accuracy
A
causal. juusel.
0,484
0,488
+0.004
temporal_sege sequences
0,276
0.280
+0.004
geometric_shapes
0,176
0.288
+0.112
wwaiiiinnssuffeddd.oo -ojjects___tteseeeeeee fiveeliveollllle -objects
0.156
0.156
0.000
0,452
0.428
-0.024
woolcan.expression.
0,832
0.844
+0.012
web of lies
0.544
0.548
+0.004
formal fallacies
0,548
0.576
+0.028
Boakking_shuflld_ooll _objects_sevenn seeeesssseeeeeeeeeeeeeee ceven_objects
0.112
0.128
+0.016
wwwpoeettetteeeoopppooooooooo .bbects
0,296
0.328
+0.032
ruin_names
0,400
0.420
+0.020
dyck
0,012
0.008
-0,004
movie recommendation
0,440
0.408
-0.322
snarks
0.412
0.356
-0.556
navigate
0.684
0.672
-0.012
disanbiguation.ga
0,408
0.420
+0.012
hyperbaton
0,516
0,500
-0.016
countiin
0.704
0,660
-0.044
reasoning_ wwwore.. objjccts
0,452
0.424
-0.028
logical deduucton five .bjjects
0.348
0.380
+0.032
date understanding
0.424
0.412
-0.012
penguins in_a_table
0.300
0.344
+0.044
logical www.dddddddddoseeeeeeeeeeee obbects
0,436
0.448
+0.012
multistep m ceatistep.aritmmeticc warithaeticc.ni logical deduction three objects
0,476
0.568
+0.092
0,500
0.556
+0.056
Table 7: Accuracy of LLaDA-1.5 across BBH subsets. A denotes Wave Base.
Task
Base Accuracy
Wave Accuracy
mooieerecommendeee
0,416
0.392
-0,024
reasoning about colored objects disambiguation qa
0,496
0.508
+0.012
0.428
0.428
0.000
web_of_lies
0.556
0.524
-0.032
fallaiiflllaciss date date unteessderst
0.556
0.568
+0.012
0,420
0,380
-0.040
ruin ruin_names
0.364
0.444
+0.080
temporal_sequencee
0,280
0,288
+0.008
geometric shapes
0.252
0.356
+0.104
tracking shuffled objects seven objjccs
0.124
0.124
0.000
logical www.ddducttioseeveeneooee
0.476
0.468
-0.008
dyck_language
0,024
0.028
+0.004
salient translation error detection
0,460
0.496
+0.036
logical deduction five objects
0.408
0.364
-0.044
logical sewraalddduuttoonttttee wwwpetttrreaaeeeeeeeee
thheeteeeeeeeessss .bjects
0.516
0.596
+0.080
snarks
0.368
0.384
+0.016
penguins a.talle
0.348
0.348
0.000
tracking-shie wwwffhhedfoohpoppee obbjects_trree.ottl
0.344
0.396
+0.052
object.counting
0.684
0.668
-0.016
multistep arithmetic two
0,424
0.484
+0.060
tracking shuffled objects five objects
0.180
0.124
-0.056
boolean vvolean.expressions
0.852
0.836
-0.016
hyperbaton
0,512
0.496
-0.016
navigate
0.724
0.676
-0.048
causal,
causal_judgenee judgement
0.484
0.488
+0.004
18

---

Published as a conference paper at ICLR 2026
Table Case Study: Decoding snapshots for same task at matched compute. Column 1: decoding step Column 2: BifckDiffusion outputs, progressing strictly left teft-to-right. Column 3: WavefrontDiftusion outputs, dynamically expanding to semantically relevant regions.
Step
BlockDiffusion
WavefrontDiffusion
t
1
def fib
def Eib partial if clause
t 2
def fib (n) if
def fib (n) if <= 1: + partial return def (ib(n): if n <= 1: return n
t == 3
def fib (n) if n <= 1:
t = 4
return fib(n-1) + fib(n-2)
return fib (n-1) (n-(2
At the initial step 1), BlockDiffusion begins strictly from the left, decoding only the initial function signature. In contrast, WavefrontDiffusion simultaneously expands outward, partially de- coding both the signature and the beginning of the if clause.
By t 2, BlockDiffusion still has not completed the if condition, resulting in an incomplete logical structure. WavefrondDifussio, however, has already produced a nearly complete condition and begun generating the return statement.
At t 3, WavefrontDiffusion fully resolves the core recursive structure (return n or recursive calls), while BlockDiffusion has just finished the condition.
Finally, at 4, both methods converge to the same final implementation, but WavdfroniDiffussio reached a semantically coherent intermediate state much earlier.
Implication. This case study demonstrates how Wivefron avoids the artificial barriers imposed by fixed blocks. By flexibly expanding to semantically related regions, it maintains co- herent partial outputs throughout the generation process. This visual evidence complements our MHCO analysis and explains the accuracy improvements observed in Table [I] highlighting how adaptive scheduling aligns decoding with the natural structure of reasoning and code.
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 COMPARISON WITH ADVANCED DYNAMIC BASELINES
To further validate the effectiveness of WavefrontDiffusion, extend our comparison include three recent advanced decoding strategies that also incorporate dynamic or adaptive mechanisms. These methods represent the otate-of-the-art in optimizing diffusion decoding beyond fixed sched- ules:
Truncated-BloccDiiiee BlockDiffusion (TBD) (Anonymous 2025): : A concurrent approach that adap- tively adjusts the block size during generation based local confidence, aiming solve the rigidity of fixed blocks.
Rumning Confidence Remasking (RCR) HH et al.. 2025 : A training-free free strategy pro- posed in MDPO that identifies and re-masks cowcconideece tokens during the decoding process, allowing the model to revise potential errors.
Self-Speculative Decoding (SSD) (Gao et al. 2025): A method designed primarily for acceleration via hierarchical verification, which effectively acts as a dynamic acceptance policy for generated tokens.
Results. We conduct the comparison using (LLDA--B-Innsreen with the total step budget fixed at T = 1024. As shown in Table 9 DivefrontDiffusionn consistently outperforms these advanced baselines across all four benchmarks. While TBD and RCR improve upon Standard Diffusion, they still lag behind WavefromtDiffusion in complex reasoning tasks (e.g.. MATH and GSM8K). This suggests that while adaptive block sizing (TBD) and remasking (RCR) are beneficial, the wave- front expansion strategy- which aligns update order with semantic dependencies proviidddeess a more optimal decoding path for reasoning-heat generation.
19

---

Published as a conference paper at ICLR 2026
Table 9: Comparison with Advanced Dynamic Baselines. Accuracy on LLaDA-8B-Instrut (T 1024). WavefrontDiffusion achieves state-of-the-art performance among training-free dy- namic strategies.
Method
GSM8K
MATH
HumanEval
BBH
Standard Diffusion
23.15
26.60
17.68
11.30
BlockDiffusion
80.74
40.62
45.73
43.23
Truncated- Sumeaated-loocDifffssse (TBD)
80.90
40.84
45.73
39.41
Running Confidence Remasking (RCR)
80.60
40.02
43.90
43.10
Self-Speculative Decoding (SSD)
79.15
38.88
44.51
41.25
WavefrontDiiffusion (Ours)
82.03
41.04
47.56
44.30
C.2 EFFICIENCY AND COMPUTATIONAL PARITY ANALYSIS
key concern in iterative decoding is whether performance gains come the cost of increased latency. We clarify that WavefrontDiffusion is scheduling strategy, not an acceleration method, designed to align strictly with the computational budget of standard block-based decoding.
Computational Parity. As detailed in Table 10 our method shares the identical number of denois- ing steps (T 1024) and model configuration as BlockDiffusion. WavefrontDiffusion alters the order of token updates but not the quantity of forward passes. Consequently, the total FL.OPs per sample remain equivalent (2.517 10t5 for L.L.DA-8B). Observed wall-clock time differences are within -2, attributed solely to system variance rather than algorithmic overhead.
Table 10: Compute-Parity Comparison, We report FLOPs, total inference time (s), and through- put (Tokens Per Second, TPS) on LLaDA-8B and LLaDA-1.5 (T = 1024). WavefrontDiffusion maintains identical computational cost to BifekDiffision while achieving higher accuracy.
FLOPs
GSMSK
MATH
HumanEval Time
BBH
Model
Strategy
(x10'5)
Time
TPS
Time
TPS
TPS
Time
TPS
Standard Diffusion
2.517
110.1
9.30
108.8
9.41
115.8
8.84
122.7
8.35
LLaDA-8B
BlockDiffusion
2.517
113.6
9.02
109.4
9.36
107.2
9.55
116.6
8.78
WavefrontDiflusion
2.517
9.13 112.2
9.32 109.9
109.3
9.37
115.1
8.89
Standard Diffusion
2.517
111.9
9.15
111.4
9.19 107.4
118.0
8.68
127.2
8.05
L..
BifckDiffusion
2.517
111.6
9.17
9.53
110.1
9.30
118.6
8.63
WavefrontDiffusion
2.517
109.3
9.37
110.9
9.23
112.9
9.07
114.1
8.97
Compatibility with Acceleration. Although WavefrontDiffusion itself does not aim to accelerate inference, it is fully compatible with existing acceleration frameworks. Specifically:
KV-Cache Optimization: Since method only changes the masking schedule, it inte-
grates seamlessly with memory reuse techniques like Fast-dLLM (Wu et al. 2025).
Speculative Decoding: The adaptive wavefront expansion is orthogonal to sampling com- pression and hierarchical verification methods (Gao et al., (2025). allowing for combined use to achieve both quality improvements and latency reduction.
C.3 ABLATION STUDY DENOISING STEPS
To investigate how the number of denoising steps affects generation quality, we performed trolled ablation on LLaDA-8B-Instruue using the GSM8K benchmark. We varied the total diffusion steps T E (256, 512, 1024), which correspond to finalize quotas k, E {4, 2, 1} respectively.
As shown Table II WavefrontDiffusion remains competitive even at lower denoising steps and consistently surpasses BlockDiffusion as increases. The relative improvement becomes more
20

---

Published as a conference paper at ICLR 2026
pronounced at higher T, indicating that the adaptive wavefront scheduling effectively leverages extended denoising horizons to propagate semantic dependencies.
Table 11: Ablation on Denoising Steps (7). Accuracy on GSM8K using LLaDA--B-Instruut. WavefrontDiffusion scales effectively with denoising depth.
Strategy
Steps 256
Steps 512
Steps 1024
Standard Diffusion
12.96
17.05
23.15
BlockDiffusion
43.14
67.02
80.74
WavefrontDiftusion (Ours)
42.38
73.79
82.03
C.4 ROBUSTNESS ANALYSIS: TEMPERATURE AND RANDOM SEEDS
We further evaluated the robustness of WavefrontDiffusion under stochastic sampling conditions. We conducted experiments GSM8K (LLaDA-8B)) varying the sampling temperature from 0.0 to 0.8. Each value represents the mean accuracy = standard deviation over three random seeds.
Table T2 demonstrates that WavefrontDiiffssion maintains superior performance and low variance across all temperature regimes. The consistent advantage over BlockDiffusion (approx. + +1.6% 2.0%) indicates that our dynamic frontier scheduling is robust to sampling randomness.
Table 12: Temperature Robustness Analysis. Mean accuracy + standard deviation GSM8K (LLaDA-8B).
Strategy
Temp = 0.0
Temp 0.3
Temp 0.6
Temp 0.8
Standard Diffusion
23.15 + 0.00
23.42 0.15
22.97
20.35 + 0.22
BlockDiffusion
80.74 + 0.00
80.83 0.14
81.09 0.20
78.42 + 0.37
WavefrontDifffssonn
82.03 + 0.00
81.47 0.12
81.97 1 0.26
78.86 + 0.41
C.5 EFFECTIVENESS WITH ALTERNATIVE TOKEN-PRIORIT METRICS
To verify that our method's success not dependent solely on confidence scores, we evaluated WavefrondDiffusion using two alternative fronticr-priont metrics on the Dream-7B model:
1. Entropy: Lower entropy indicates higher priority.
2. Margin:
Larger gap between top-1 and top-2 probabilities indicates higher priority.
WavefrondDiffusion consistently outperforms baselines regardless of the priority metric used, confirming the universality of the wavefront scheduling mechanism.
As shown in Table 13
Table 13:
Performance under Alternative Token-Priority Metrics. Experiments on Dream-7B
with equal FLOPPs 1024).
Strategy
Metrie
GSM8K
MATH HumanEval
BBH
Standard Diffusion
Entropy
34.80
29,56
20.13
15.90
BlockDiffusion
Entropy
81.96
42.26
50.61
44.63
WivefrontDifftusion
Entropy
82.79
42.90
52.43
44.71
Standard Diffusion
Margin
34.90
29.28
20.73
16.10
Biffussin
Margin
78.85
41,16
50.00
44.90
WavefrontDifftsion
Margin
79.38
42.08
50.61
45.35
21

---

Published as a conference paper at ICLR 2026
C.6 MATCHED-LATENGE COMPARISON WITH AUTOREGRESSIVE MODELS
Finally, we provide latency-matched comparison against Autoregressive (AR) greedy decoding and Spee-ueeeuteie Decoding (SSD). Table [4 shows that WavefrontDiffusion achieves comparable latency to AR decoding while delivering superior accuracy, demonstrating its practical viability.
Table 14: Matched-Latency Comparison. LLaDA-8BB -anD---B-Instruct GSM8K.
Method
Latency (ms/token)
Accuracy
Autoregressive Greedy
9.2
80.67
Speculaccee Decoding
5.8
79.30
WavefrontDiffusion (Ours)
9.1
82.03
C.7 CALIBRATION ANALYSIS
Since WavefronnDiffusion relies on confidence scores to determine the decoding frontier, validating that these scores are well-calibrated critical. We conducted rigorous calibration analysis to ensure that the model's internal confidence serves as reliable signal for decision-making.
Experimental Setup. To measure intrinsic calibration without the confounding factor of divergent reasoning paths, we employed standard Teacher Forcing protocol. We fed the model the ground truth prefix each step and recorded the confidence of tokens selected by WavefrondDiffusion versus BlockDiffusion. We report the Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) on GSM8K, MATH, and HumanFyal.
Quantitative Results. Table 15 summarizes the calibration metrics. WavefrontDiffusion achieves consistently lower ECE scores across all domains, with a reduction of approximately 40% on GSM8K (0.1093 0.0659). Furthermore, it significantly reduces the Maximum Calibration Er- ror (MCE) on complex tasks like MATH and HumanEval (0.7051 + 0.4502). This indicates that adaptive wavefront effectively filters out poorly calibrated tokens that static block schedules forced to commit to.
Table 15: Calibration Metries. Comparison of Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) under Teacher Forcing. Lower is better.
Dataset
Method
ECE (L)
MCE (1)
BlockDiffusion
0.1093
0.4727
GSM8K
WavefrontDiffussio (Ours)
0.0659
0.4824
BlockDiffusion
0.1025
0.3457
MATH
WavefrontDiffusion (Ours)
0.0901
0.2393
BlockDiffusion WavefrontDiflusion (Ours)
0.0920
0.7051
HumanEval
0.0853
0.4502
Reliability Diagrams. Figure 3 visualizes the reliability diagrams for both methods across the three datasets. Ideally, the calibration curve should align with the diagonal (dashed line). Compar- ing the left column (BlockDiffusiin)) and the right column (WavefrontDiffusion)) we observe that WavefronDDiffusio's curves are consistently closer to the diagonal, particularly in high-confidence regions. This confirms that the tokens selected by the wavefront are fundamentally more "trustwor- thy," validating the reliability of our adaptive scheduling mechanism.
C.8 DECODING DYNAMICS ANALYSIS
To explicitly address the concern regarding frontier evolution and stability, visualize the step- wise decoding dynamics in Figure g This analysis captures the "process" of generation, contrasting the rigid scheduling of BifckDii with the adaptive flow of WavefrontDifftsiinn.
22

---

Published as a conference paper at ICLR 2026
Reliabiity Diagram 0,0443, 0.1953
Reliabllit Diagram
ECE: 0.1631
Mice: pCD- cusscl)
VJaee LSC 2411)
0.8
of
02
0.2
24
Garffdance
Confidence
(a) GSMSK BlockDiffusion
(b) GSM8K WavefrontDiffusion
Reliabilit
Reliability Diagram
ECE: 0.045, MCE: 0.029
ECE: MCE: 0.1021
Parfecty callordees
Serfectlyy cal brited I5CE-0 5422)
1000-6004001
CO-
0.
ACcurccy
0.5
Contdence
Confidence
(c) MATH BlockDiffusion
(d) MATH WavefronnDiffusion
Reliablltt Diagram
Reliability Diagram
FCF: 0.0809, MCF: 0.7817
FCF: 0.0752, MCF: 3,2364
10-
Perlecty
1.3
"ertects frrtel
Accursoy
of
6.2
20
02
9.5
2.0
3.0
0.
0.3
24
0.3
1.5
Confidance
Canfidence
(c) HumanEval- - BlockDiffusion
(I) HumanEval Wavefron
Figure 3: Reliability Diagrams Comparison. Calibration curves for BlockDiffusion (Left) and WavefrondDiffusion (Right) across GSM8K, MATH. and HumanEval. The dashed diagonal line represents perfect calibration. WavefronDDiffusion exhibits curves that are consistently tighter the diagonal, indicating superior calibration.
1. Frontier Evolution (Smoothness vs. Stagnation). As shown in Figure 4a
BlockDiffusion ex- patterr, where the frontier freezes within a block until all tokens are
hibits a characteristic "staircase" pattern, finalized, regardless of their difficulty. In contrast, WavefrontDiffusion (green curve) demonstrates a smooth, continuous progression. The absence of horizontal plateaus confirms that our method avoids premature freezing and adapts the generation pace naturally the sequence structure.
23

---

Published as a conference paper at ICLR 2026
Frontter Evolution
Confidence Stabiiity (Pat--Finliiatioo
Wevefrent Confidence Hestmap
Noafta
DITAK
205
410
D'Pcutan
2013
(a) Frontier Evolution
(b) Wavefront Confidence Heatmap
(c) Confidence Stability
Figure 4: Decoding Dynamies Analysis. (a) WavefronDDiffusion shows smooth, continuous pro- gression (green), avoiding the rigid "staircase" artifacts of BlockDiffusion (red). (b) The heatmap reveals a clear "diagonal frontier" where the wavefront adapts its velocity to token difficulty. (c) WavefrontDiffusion maintains consistently high post-finalzzation confidence, indicating minimal "regret," whereas BlockDiffusion suffers from confidence drops due to insufficient lookahead con- text.
The "Wave" Mechanism (Adaptivity). Figure visualizes the confidence heatmap of Wave- frontDiffusion. The bright diagonal band represents the active wavefront. Notably, the boundary is not a perfect straight line but exhibits local fluctuations. This visualizes the adaptivity of the algo- rithm: the wavefront accelerates through high-confidence (easy) regions and slows down to accu- mulate context for low-confidence (hard) tokens, effectively implementing a dynamic computation allocation.
fost-Finalization Stability (Regret Analysis). Figure 4c we track the average confidence of tokens after they finalized. Ideally, model should not "regret" its decisions (i.e., confidence should not drop after finalizing). BlockDiffusion (red) shows jitter and drops, implying that forced block commits often lead sub-optimal decisions that the model later finds inconsistent. Wavefront- Diffusion (green) maintains a stable, near-1. 1.0 confidence trajectory, proving that its context-aware expansion significantly reduces finalization regret.
D THEORETICAL JUSTIFICATION
In this section, provide the formal theoretical framework that justifies Wavefron the optimal expansion of eestrette-scaach decoding strategies. Our derivation rests on two core pillars: the Information Gradient Hypothesis and the optimality of dynamic restricted search.
D.1 PREMISE 1: THE INFORMATION GRADIENT HYPOTHESIS
Let C, be the set of finalized tokens at step t. For any masked token i, let d(i, C.) be the minimum distance the finalized context (as defined in Eq. 1). We introduce the Information Gradient Assumption, which states that the conditional entropy / of a masked token æ; is monotonically non-decreasing with respect its distance from the finalized context:
H(o.(ac.) S H(z,lzc). = d(i,C,) d(j.c,)
(5)
This formalizes the intuition that tokens strictly adjacent to the known context possess the highest "information density" and thus the highest probability of being correctly predicted.
D.2 PREMISE THE VALIDITY OF RESTRICTED SEARCH
The global optimization objective at step is select a subset of tokens S, (where |S, K) to finalize such that we maximize joint confidence. BlockDiffusion (Arrioa et al.) 2025) demonstrated key theoretical contribution: Global search is unnecessary. It proved that limiting the search space to a constrained subset Ω C Van yields efficient results, provided the subset contains valid candidates.
24

---

Published as a conference paper at ICLR 2026
BlockDiffusion's Constraint: BBBoock {i E Block})
Limitation: Because Block is static, there exists non-zero probability of "semantic mismatch," " where true high-confidence tokens outside the block. That is, 3j $ Block such that (xy) < miniggRaoock H (2:4)
D.3 THEOREM: WAVEFRONTDIFFUSION AS OPTIMAL RESTRICTED SEARCII
Our method aligns the restriction boundary Ω with the Information Gradient defined Premise 1. We define the Wavefront search space ?wave as:
Owave {i d(i,c)) R}
(6)
Theorem. Under the Information Gradient Assumption, the optimal subset S (the set set K tokens with lowest entropy) satisfies S* Swaav for sufficient radius whereas S llBlock is not guaranteed.
Proof Sketch. Since entropy H (z;) increases with distance d(i, Ct), the "lowest entropy isosur- faces" form expanding contours around C,:
1. BlockDiffusion forces a search within a rectangular window Block₁- This window in- evitably intersects with high-entropy regions (far from context) while potentially excluding low-entropy regions (near context but the next block) due to rigid segmentation.
2. WavefronDDiffusion strictly defines Ωwave to encapsulate the lowest distance values. Therefore, Wae is mathematically guaranteed to contain the highest density of cor- rectable tokens.
3. Consequently, WavefrontDiffusion ensures that the local optimization is performed within the theoretical upper bound of candidate quality, extending the efficiency logic of BlockD- iffusion to optimal semantic conclusion.
Empirical Verification. This theoretical proof is empirically supported by our MHCO analysis in Section 4.4. The significantly lower MHCO scores for WavefrontDiffusion confirm that our dynamic boundary successfully captures the optimal local search space defined by this framework, whereas BlockDiffusion's 's static boundary frequently violates it.
25


Reviews:
[{"review_id": "Vg09GH9cTr", "reviewer": "Reviewer_HnuB", "summary": "The paper introduces WavefrontDiffusion, a dynamic decoding strategy for Diffusion Language Models that adaptively expands a “wavefront” of active tokens during generation. This approach preserves semantic coherence and contextual completeness while keeping the same computational cost as block-based methods. Experiments on reasoning and code generation benchmarks show that WavefrontDiffusion consistently improves accuracy and output quality over existing diffusion decoding strategies.", "questions": "1. Will this method slow down the inference process? How does its speed compare to other methods?\n\n2. Could you run the results on MBPP?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. WavefrontDiffusion dynamically adjusts the denoising process to follow the evolving semantic structure, preventing premature or fragmented token generation.\n\n2. By expanding from finalized tokens outward, it ensures each token is generated with sufficient context, leading to smoother and more logically consistent outputs.\n\n3. The method matches the computational cost of block-based decoding while delivering higher accuracy and better output quality.", "weaknesses": "1. The paper lacks a baseline for its method. It needs to compare its approach with current decoding methods in the DLM field to demonstrate its advantages.\n\n2. Experiments were only conducted on one model category. Similar experiments need to be performed on Dream for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KKT5gTW6B9", "reviewer": "Reviewer_skjz", "summary": "This paper introduces a dynamic decoding scheduling strategy named WavefrontDiffusion, designed to address the issues of semantic coherence and computational efficiency in text generation with diffusion language models. Whereas traditional decoding strategies like Standard Diffusion and BlockDiffusion have inherent limitations, WavefrontDiffusion dynamically expands a \"wavefront\" region of active tokens. This allows the denoising process to align with the natural flow of semantic structure while maintaining the same computational cost as block-based methods. Experiments demonstrate that this approach achieves state-of-the-art performance across multiple benchmarks and generates outputs with higher semantic fidelity. This research presents a new paradigm for applying diffusion models to complex reasoning and code generation tasks.", "questions": "1. Are there any unique application scenarios where this method can demonstrate a more significant advantage over BlockDiffusion?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The method is intuitive and addresses the limitations of hard boundaries in BlockDiffusion.\n\n2. The research methodology is well-structured, with clear explanations of the wavefront theory, a four-step algorithm, and mathematical definitions. The experimental design covers four benchmark tests and evaluates the method using multiple metrics such as accuracy, BERTScore, and the MHCO indicator.\n\n3. The experimental analysis is thorough and provides insights into parameter selection.", "weaknesses": "1. The method is an incremental improvement over BlockDiffusion; both the methodology and the experimental results are incremental in nature.\n\n2. Regarding the writing, Figure 1 is not sufficiently intuitive and requires further revision.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X2A2VGkjwb", "reviewer": "Reviewer_my5v", "summary": "WavefrontDiffusion dynamically expands a confidence-guided frontier in diffusion LMs, preserving context, matching block compute, and improving reasoning/code accuracy and semantic fidelity across benchmarks under compute-parity.", "questions": "1. Add equal FLOPs and equal token updates tables, and ablate k_tallocation strategies.\n2. Report mean/σ over multiple seeds, and vary context length, temperature, and prompting (few-shot, CoT).\n3. Test entropy/energy or calibrated confidence to reduce dependence on raw max-softmax.\n4. Include longer-context reasoning and additional code sets (e.g., MBPP, DS-1000).\n5. Provide matched-latency/throughput, end-to-end comparisons vs. speculative decoding to map advantage boundaries.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method avoids premature EOS and half-baked spans by not locking in locally high confidence tokens too early.\n2. It completes semantically “ready” regions first (e.g., function signatures, reasoning steps) and is not hostage to rigid chunk boundaries.", "weaknesses": "1. Only F and R are studied; there is no analysis of the per-step finalize quota k_t, nor strict equal FLOPs / equal token updates controls.\n2. The setup is mostly zero-shot with T=1024 and temperature 0; it lacks length/temperature sweeps and multi-seed variance.\n3. The very long context engineering story is unclear; the overhead of frontier maintenance and cache policies at extreme lengths is not evidenced.\n4. Autoregressive baselines at matched latency are missing; there is no head-to-head against speculative decoding under equal delay/throughput.\n5. Early errors can still propagate; once an incorrect span is finalized, downstream reasoning may be constrained by that commitment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DSyNel63zW", "reviewer": "Reviewer_M3nR", "summary": "The paper proposes a wavefront-style dynamic decoding schedule for discrete, mask-based diffusion language models. Instead of global synchronous denoising or fixed-order block decoding, the model maintains a frontier of finalized tokens and iteratively performs scoring, denoising, expansion, and pruning. By capping both the per-step update size $F$ and total steps $T$, it enforces compute parity with block decoding, ensuring that improvements stem from scheduling rather than added computation. The approach is training-free and integrates easily with existing diffusion LMs. However, despite the elegant design, the paper’s experimental analysis remains limited. The evaluation is confined to the LLaDA family (e.g., 8B and 1.5 variants) rather than a broader set of diffusion LMs, and it lacks calibration robustness checks. The MHCO metric shows improved consistency, yet its dependence on the frontier radius $R$ is unexamined; a short theoretical or empirical study would clarify interpretability and stability, and several ablations (e.g., local scoring efficiency, calibration impact) are missing. While the reported gains are consistent, they are modest, and the absence of broader baselines or theoretical grounding weakens the generality of the conclusions.", "questions": "1. **Confidence Stability:**\n   Have the authors evaluated how WavefrontDiffusion’s performance changes under temperature scaling or calibration shifts? If confidence scores become over- or under-confident, does the frontier expansion remain stable? Would entropy-based or learnable temperature gating improve robustness?\n\n2. **Baseline Scope:**\n   Could the authors include comparisons with other dynamic decoding approaches (e.g., remasking-based or truncated-block methods) under the same $F×T$ compute constraint? This would help isolate the benefits of the scheduling mechanism itself from those of confidence gating.\n\n3. **Long-Sequence Dynamics:**\n   For long reasoning or code-generation tasks, how does the frontier expand and finalize over time? Are there cases of premature convergence or late instability? A quantitative or visual analysis would be insightful.\n\n4. **Theoretical Insight:**\n   Can the authors provide intuition or approximate analysis explaining why expanding around finalized tokens with radius $R$ improves semantic fidelity? Does this mechanism implicitly encourage smoother denoising trajectories or faster entropy reduction?\n\n5. **Calibration Ablation:**\n   Have the authors tested whether applying calibration techniques (e.g., temperature scaling or isotonic regression) changes the MHCO metric or final accuracy? This would clarify the sensitivity of the method to calibration errors.\n\n6. **Model Dependency:**\n   The method is evaluated only on LLaDA. Does the approach depend on LLaDA’s specific sequence-level noise or architecture? Would it generalize to other diffusion language models, such as those using token-level noise or trained with reinforcement learning? Any preliminary evidence or reasoning supporting model-agnostic applicability would be valuable.\n\n7. **MHCO Sensitivity:**\n   Since MHCO depends on the radius $R$, have the authors examined how different $R$ values affect MHCO scores? Does the metric remain consistent across reasonable $R$ choices, or is it highly sensitive to this hyperparameter?\n\n8. **Localized Scoring Efficiency:**\n   In the core iterative process, the model first scores *all* masked positions before selecting tokens from the current wavefront $W_{t−1}$. Have the authors tested a variant where scoring is restricted to $W_{t−1}$ and its neighboring masked positions? Would this local scoring reduce computational cost without hurting performance?\n9. **Long-sequence dynamics and stability:** \n   On longer reasoning/generation sequences, does the wavefront exhibit premature freezing or oscillation? Could you report curves/statistics of frontier radius $R$, finalized-token ratio, and “finalization regret” over steps and sequence length?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The work is well organized and clearly presented, with strong empirical grounding and sensible ablations. It introduces a novel scheduling perspective for diffusion decoding that reallocates compute dynamically rather than increasing it, preserving $F×T$ parity. The design is practical and model-agnostic—requiring no retraining and the safeguards (frontier pruning, capped updates) are thoughtfully engineered. Experimental reporting is transparent, and the improvements, while modest, are consistent across settings. The paper’s clarity and practicality make it an appealing addition to the diffusion decoding literature.", "weaknesses": "1. **Dependence on Confidence Calibration**\n   The proposed approach relies heavily on token-level confidence scores to determine which tokens are finalized. The approach relies on token-level confidence scores to finalize tokens but does not evaluate robustness under different temperature settings or calibration shifts, although the discussion section acknowledges calibration as a general limitation. While Section 4.5 acknowledges calibration as a general limitation of diffusion-based decoders, there is no empirical analysis showing how WavefrontDiffusion behaves under miscalibrated conditions. If the confidence estimates are unstable, the wavefront expansion could prematurely freeze or miss critical tokens, potentially affecting reliability.\n\n2. **Narrow Baseline Coverage**\n   The experiments compare only with Standard Diffusion and BlockDiffusion. More recent dynamic decoding methods—such as remasking-based or truncated-block strategies are not included. Since these methods also dynamically adjust the denoising schedule, evaluating under a matched $F×T$ compute budget would clarify whether the observed gains come from the proposed scheduling design itself or from confidence-based gating.\n\n3. **Limited Theoretical or Consistency Analysis**\n   While the “wavefront” intuition is compelling, the paper does not formalize why this adaptive frontier expansion should outperform fixed or global schedules. The MHCO metric shows improved consistency, yet the connection between this measure and denoising optimality remains informal. Moreover, the sensitivity of MHCO to its hyperparameter $R$ (the frontier radius) is not analyzed, leaving unclear whether its interpretability or stability depends on this choice. A brief theoretical or empirical study of MHCO’s dependence on $R$ would improve clarity.\n\n4. **Lack of Analysis on Long-Sequence Behavior**\n   Although the paper reports results on reasoning and code benchmarks, it does not visualize or analyze how the frontier evolves for long sequences. For instance, it is unclear whether the wavefront ever converges prematurely or oscillates during extended decoding. A step-wise visualization or “finalization regret” analysis would clarify how the dynamic schedule behaves as sequence length grows.\n\n5. **Efficiency and Scoring Computation**\n   In the core iterative process, the model scores all masked positions before selecting the top-$k$ candidates from the current frontier $W_{t−1}$. This full scoring procedure may increase computational cost. The paper does not discuss whether a *localized* scoring strategy—scoring only masked tokens near the current wavefront—has been tested, nor whether it could reduce compute while maintaining performance.\n\n6. **Presentation and Appendix Details**\n   Some hyperparameter details and appendix cross-references could be clearer. For instance, the main text does not specify the exact ranges or adaptation rules for $R$ and $F$ across tasks. Additionally, an explicit scope note that this work focuses solely on *discrete* diffusion (rather than continuous-space diffusion) would help prevent confusion.\n\n7. **Evaluation Limited to a Single Model**\n   All experiments are conducted on LLaDA-8B. Although the method is described as “model-agnostic,” it is not validated on other diffusion language models with different masking or training policies. Since diffusion LMs vary in their noise formulation, reweighting, and decoding dynamics, results based solely on one model may not fully establish generality. Demonstrating consistent improvements across multiple diffusion models would further strengthen the claim.\n   \n8. **Missing long-sequence scalability analysis**\n   The paper does not analyze wavefront dynamics on long sequences (e.g., premature convergence or oscillation). Visualizing the frontier trajectory and reporting a “finalization regret” diagnostic would clarify stability as sequence length grows.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)