Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models
Authors: Wen Wang
Bozhen Fang
Chenchen Jing
Yongliang Shen
Yangyi Shen
Qiuyu Wang
Hao Ouyang
Hao Chen
Chunhua Shen
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion Language Models
Semantic Entropy
Self-Consistency
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs
TL;DR: We find that diffusion language models hide useful answers mid-generation and introduce simple voting and reinforcement learning methods that exploit the temporal dynamics to boost accuracy.

Abstract: Diffusion large language models (dLLMs) generate text through iterative denoising, yet current decoding strategies discard rich intermediate predictions in favor of the final output. Our work here reveals a critical phenomenon, temporal oscillation, where correct answers often emerge in the middle process, but are overwritten in later denoising steps. To address this issue, we introduce two complementary methods that exploit temporal consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time decoding strategy that aggregates predictions across denoising steps to select the most consistent output; and 2) a post-training method termed Temporal Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a measure of semantic stability across intermediate predictions, as a reward signal to encourage stable generations. Empirical results across multiple benchmarks demonstrate the effectiveness of our approach. Using the negative TSE reward alone, we observe a remarkable average improvement of 24.7% on the Countdown dataset over an existing dLLM. Combined with the accuracy reward, we achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and 25.3% on Countdown, respectively. Our findings underscore the untapped potential of temporal dynamics in dLLMs and offer two simple yet effective tools to harness them.

Published as a conference paper at ICLR 2026
TIME IS A FEATURE: EXPLOITING TEMPORAL DYNAMICS IN DIFFUSION LANGUAGE MODELS
Wen Wang 1.2+ Bozhen Fang" Chenchen Jing Yongliang Shen', Yangyi Shen',
Qiuyu Wang?. Hao Ouyang".
Hao Chen', Chunhua Shen .2,33
Zhejiang University, State Key Laboratory of CAD & Computer Graphics 2 Ant Group 3 Zhejiang University Technology Stanford University, USA
ABSTRACT
Diffusion large language models (dLLMs) generate text through iterative denois- ing, yet current decoding strategies discard rich intermediate predictions in favor of the final output. Our work here reveals a critical phenomenon, temporal oscillation, where correct answers often emerge in the middle process, but are overwritten later denoising steps. To address this issue, we introduce two complementary methods that exploit temporal consistency: 1) Temporal Self- Consistency Voting, a training-free free. test-time decoding strategy that aggregates predictions across denoising steps select the most consistent output; and 2) a post-training method termed Temporal Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), measure of semantic stability across intermediate predictions, as reward signal to encourage stable generations. Empirical results across multiple benchmarks demonstrate the effectiveness of our approach. Using the negative TSE reward alone, we observe a remarkable average improvement of 24.7% % on the Countdown dataset over an existing dLLMM. Com- bined with the accuracy reward, we achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH5000, 6.6% on SVAMP, and 25.3% on Countdown, respectively. Our findings underscore the untapped potential of temporal dynamics in dLLMss and offer two simple yet effective tools to harness them.
Project webpage: https: : htt///mmmmooo////// . github. 10/ LLL///LLMMMMMMMM
INTRODUCTION
Diffusion large language models (dLLMs) (Nie et al., 2025; Zhu et al., 2025; Ye et al., 2025) have recently emerged a promising alternative the auto-regressive (AR) large language models, garnering significant attention for their competitive performance and potential for faster inference. In contrast to AR models, which generate text in a strictly sequential manner by predicting one token at time, dLLMs operate through iterative cycles of denoising and remasking, predicting all masked tokens in parallel each step. A small subset of the predicted tokens, typically those with high confidence (Nie et al., 2025), are retained, while the remaining tokens are remasked and refined in subsequent steps. Despite their drastic architectural differences, current dLLMMs typically adopt decoding strategy that mirrors AR models: solely relying the sequence predicted in the final denoising step as the final answer, and discarding all the intermediate predictions.
In this work, we challenge this convention by uncovering a critical phenomenon that we term temporal oscillation: correct answers often appear during intermediate denoising steps but overwritten in later iterations. This discrepancy between the final output and intermediate correctness suggests that dLLMs possess rich temporal dynamics that are largely under-utilized.
As depicted in Fig. 1, analyze two key metrics across four widely used benchmark datasets using two representative models: LLaDA-8P -8------ntstt (Nie et al., 2025) and LLaDA-1.5 (Zhu et al., 2025). The first metric, final pass rate, measures the accuracy of the final output, while the second, ever-pass rate, captures whether a correct answer appears at any point during the decoding process. In Fig. la,
WW and BF contributed equally. CS is the corresponding author.

---

Published as a conference paper at ICLR 2026
MATH5000
Question: There plants Mrs. Smith's garden. One-fourth of her plants indoor plants. Two-thirds the remaining are outdoor plants while the are flowering plants. What percent of plants are flowering plants?
Acomayy (h)
225 SVAMP
312
13
Counldown
$12
Ground Truth: 25
103
Extracted Answer: 25
Aconcc
Correctness
Sampling Step
57
64
125
225
312
212
Sampling Process
LL.DA-NB
LLaiMA-155 Passij
LL4DAA-8--lncceet EverPassgol
1L4DA--15 EverPasg [I
b
Illustration of temporal oscillation during sampling. (a) Across four datasets, significant gap is observed between the final answer's pass rate (denoted as Pass @1) and the ever- pass rate any intermediate step (denoted EverPass @1 (). This gap reveals the phenomenon we refer to as temporal oscillation, where correct intermediate answers are sometimes overwritten as the generation proceeds. (b) Example of temporal oscillation: For given math problem. the model initially gives the correct answer, 25, at an intermediate step (e.g., step 55), aligning with the ground truth. However, by the final step, this correct answer replaced with an incorrect one:
Figure 1:
consistent and significant discrepancy exists between these metrics. This gap reveals critical phenomenon: models often generate correct answers during intermediate steps but subsequently overwrite them with incorrect
Fig. 1b illustrates this concretely-in math problem, the model produces the correct answer "25" at sampling step 55, only to replace it with an incorrect "2" by the final step 64. More examples on temporal oscillation are presented in Appendix E.6.
To better understand this behavior, we analyze dLLMs from the entropy perspective and introduce new metric: Temporal Semantic Entropy (TSE), which captures the distribution of semantic variation across intermediate outputs during decoding. Specifically, we collect the sequence of intermediate answers across denoising steps and group them into clusters based on semantic equivalence. TSE then quantifies the degree of uncertainty in the semantic content of these answers. higher TSE indicates greater semantic fluctuation throughout the trajectory, i.e., the model changes its answer frequently, while lower TSE suggests convergence toward stable meaning.
To harness the latent signals embedded in dLLMs decoding, we treat temporal oscillation as an informative feature and develop two complementary methods that exploit the temporal dynamics:
Temporal Self-Consistency Voting: training-free, test-time decoding strategy that aggregates predictions across multiple denoising steps and selects the most temporally consistent output. This simple yet effective method improves accuracy while introducing only negligible computational overhead, making it practical for real-world deployment.
post-training method based on reinforcement learning that uses negative TSE a reward signal encourage stable and consistent generations explicitly. Notably, leveraging negative TSE as the reward enables reliable performance improvements without requiring ground-truth labels for reward computation. Furthermore, when ground-truth labels are available, combining them with TSE-based rewards can provide richer and more complementary supervision, ultimately leading to even greater and more robust improvements in generation quality.
Temporal Consistency Reinforcement:
Experiments across multiple datasets validate the effectiveness both our decoding-time strategy and our RL-based post-training method. Specifically, Temporal Self-Consistency Voting brings an average improvement of 1.5% over the LLaDA -L-DA-BB-Iia In-Irutt baseline with negligible overhead. In terms of Temporal Consistency Reinforcement, fine-tuning using the negative TSE reward alone, observe a substantial average improvement of 24.7 % on the Countdown dataset. When combined with the accuracy reward derived from ground truth, approach yields notable improvements
2

---

Published as a conference paper at ICLR 2026
across diverse datasets: 2.0% GSM8K, 4.3% MATH500, 6.6% SVAMP, and impressive 25.3% on Countdown, respectively. By quantifying and leveraging temporal consistency. we offer a new perspective dLLM decoding and introduce practical tools to unlock their potential. We hope that this study inspires further research into the temporal characteristics of diffusion decoding.
RELATED WORK
Diffusion language models. Building on the success of diffusion image and video genera- tion (Song et al., 2020; Ho et al., 2020; 2022), diffusion methods have been extended to text. Early continuous approaches (Han et al., 2022; Li et al., 2022) operate in continuous space, while others map text to the probability simplex (Avdeyev et al., 2023; Stark et al., 2024). More recent work applies flow matching on the simplex to learn categorical distributions (Davis et al., 2024; Cheng et al., 2024), but remains limited to simpler sequences. Discrete diffusion models, pioneered by D3PM (Austin et al., 2021), advanced through masked token frameworks (Shi et al., 2024; Sahoo al., 2024; Nie et al., 2024) and scaling efforts. Lightweight variants like Plaid (Gulrajani & Hashimoto, 2023) and SEDD (Lou et al., 2023) rival GPT-2 (Radford et al., 2019), yet lag autoregressive models in scalability. To bridge this gap, BD3-LMs (Arriola al., 2025) and Eso- LMs (Sahoo et al., 2025) interpolate between autoregressive and diffusion paradigms, enabling parallel sampling with competitive performance. Recent efforts scale up dLLMM: Dream (Ye al., 2025) converts pretrained autoregressive models into diffusion models, while LLaDA (Nie et al., 2025) trains strong models from scratch. Another line of work studies sampling in dLLMs. For instance, Kim et al. (2025) show token ordering affects performance and propose adaptive inference, while ReMDM (Wang et al., 2025) uses inference- time remasking to boost generation. Rather than modifying local sampling heuristics, we study dLLMs through the underexplored lens of temporal stability across the entire denoising trajectory.
Several concurrent work also investigates temporal behaviors in diffusion language models. Li et al. (2025a) shows that dLLMMs often generate correct intermediate answers that are later overwritten, and propose stopping generation early when such signals appear. He et al. (2025) further demonstrates that temporal cues along the denoising trajectory can be directly exploited to improve reasoning. Xie al. (2025) introduces step-aware RL method that aligns denoising steps with hierarchical reasoning to avoid unstructured refinement. Differently, our work takes a complementary direction by developing both a lightweight test-time voting method and an unsupervised reinforcement signal grounded in trajectory-level temporal consistency.
Test-time Strategy, Test-time strategies (Wei et al., 2022; Madaan et al., 2023; Snell et al., 2024; Yao et al., 2023; Liu et al., 2025a) are widely used to improve LLM accuracy, consistency, and reliability. A simple yet effective method is Self-Coosistency (Wang et al., 2022), which selects the most consistent answer from multiple outputs via majority voting. Building on this idea, we propose a temporal self-consistency strategy tailored for dLLMs, which adds negligible inference overhead and integrates seamlessly into existing frameworks. Another important technique is semantic entropy (Farquhar et al., 2024; Kuhn et al., 2023), an uncertainty metric that clusters semantically equivalent outputs before computing entropy. While previously applied to uncertainty estimation and hallucination detection, we extend it to dLLMs by introducing Temporal Semantic Entropy, capturing stability and confidence throughout the denoising process.
Dost-Training using Reinforcement Learning. Group Relative Policy Optimization (Shao et al., 2024; Guo et al., 2025) (GRPO), a variant of Proximal Policy Optimization (Schulman al., 2017), computes advantages directly from group rewards, removing the need for separately trained value function. GRPO has shown strong performance in reasoning tasks like mathematics and code generation, and promising results across broader modalities (Huang et al., 2025; Shen et al., 2025; Qi et al., 2025; Zhong et al., 2025; Li al., 2025b; Damani et al., 2025). Building on this, refinmenns such as DAPO (Yu et al., 2025) introduce dynamic sampling to balance training batches, while entropy-based methods (Zhang et al., 2025; Prabhudesai et al., 2025; Cui et al., 2025; Agarwal et al., 2025) further enhance RL. For example, EMPO (Zhang et al., 2025) derives rewards from semantic entropy, and Seed-GRPO (Chen et al., 2025) improves advantage estimation. Recent adaptations to dLLMs include diffu-GRPO (Zhao al., 2025), UniGRPO (Yang et al., 2025), and coupled- GRPO (Gong et al., 2025). which still rely on ground-truth rewards. By contrast, our method is fully unsupervised, enhancing temporal consistency without ground-truth supervision.
3

---

Published as a conference paper at ICLR 2026
3 EXPLORATIONS ON DLLMS
3.1 PRELIMINARIES ON DLLMS
DLLMs formulate text generation as a process of iteratively denoising text sequences across different time steps. Let Xp Pdata (Xo) denote the original clean input sequence. For each diffusion step i [0, T), let x E V2 denote the corresponding noisy token-sequence tensor of length L.. The noisy sequence x, is generated via masking- based forward corruption process, in which subset of tokens is stochastically masked at each step. The forward noising process is defined as a Markov chain (XL:TT xo) = II- g(x; xt-1), which progressively adds noise to Xo over time steps. This process incrementally transforms the clean sequence X into highly noisy version Xy through a series of conditional transitions. In contrast, the reverse (generative) process is modeled as:
po(Xer)) = pe(xe) IT Po(x-- xx) = pe(xr) IT=, [Ex; 9(x--1 x²) pa(x) xx)].
(I)
where the summation Ex; enumerates all possible decoded clean sequences x at step t, making explicit that the sampling procedure first predicts clean intermediate sequence and then re- re-applies noise via (xx-1 xx])
This decomposition reveals that each reverse step consists of two sub-operations aligned with the marginalization in Eq. (1). First, the model predicts a clean latent sequence through po xt). yielding an intermediate hypothesis xf. Second, the next state Xt-1 is generated by sampling from (x,-11 x6). which re-applies noise to x following the forward corruption process. This remasking mechanism is realized in practice through strategies such as random low-confidence re-masking. as adopted in (Nie et al., 2025).
3.2 TEMPORAL OSCILLATION
In the reverse process of dLLMs, predictions px( xx) at a single step are often inaccurate, especially under high noise when t large. Existing models perform iterative denoising according to the generative process framework, where the final output is determined by the prediction x) ~ po (x) x1) at the last denoising step, while neglecting all intermediate predictions xx ~ po xx)) ) generated during the iterative process. In this work, conduct in-depth investigation into these intermediate-ston results, revealing a critical phenomenon in diffusion-base text generation.
To formalize our analysis, let 6i, denote the Pass @1 rate (Chen et al., 2021) for the prediction generated at the k-th noise step the i-th question in the evaluation benchmark. Building on this, we introduce the ever pass rate, denoted as EverPass @1 | t. which measures the proportion of questions in the dataset for which the model produces correct answer any timestep along the sampling trajectory. Formally, it is defined as:
EverPass @1 1 E max ei
(ke().....)
This metric captures the cumulative correctness across all sampling steps, reflecting the overall fraction of questions for which the model arrives at a correct solution at least once, even that solution is later discarded in the final output.
Experiment Setup. We compare the final pass rate, i.e., Pass @1 the last step with EverPass @1 I on two representative dL.MMs: -LLDA--B--nntrrue and LLaDA-1.5, evaluated across different answer lengths and four reasoning benchmarks: GSM8K (Cobbe et al., 2021), MATH500 (Lightman et al., 2023), SVAMP (Patel et al., 2021), and Countdown (Pan et al., 2025).
Observations. As shown in Fig. and Table 1, there is a notable gap between the final pass rate and the ever pass rate For instance, on GSM8K with length 128, LLaDA-SB-Instree final pass rate versus 80.5% ever pass rate a gap of 12.0%. This gap shows that many questions correctly solved at intermediate steps but later revised incorrect answers during refinement. reveals an instability in iterative decoding, where correct paths can be overwritten as generation proceeds. We term this phenomenon temporal oscillation, with examples in Appendix E.6.
achieves 68.5%
4

---

Published as a conference paper at ICLR 2026
GSMSK
Countdown
0.8
GSMBX MATH SWAMP Costitins
0.5
Ivetheag |+
Ivcraaggll
Accuracy
Pregi
Pawgi
Accurucy
0.2
0.4
0.1
0.4
0.0
0.0
Sampling Step
Sampling Step
Sampling Step
Figure 2: Patterns of accuracy evolution over diffusion sampling steps. Responses of length 128 are generated with 64 steps using ALaDA--B-Instte steps across datasets; SVAMP starts high, while harder ones like Countdown start low but improve steadily. Middle/Right: We compare the final pass rate, Pass @1, with cumulative EverPass @1 over steps. A clear gap persists between them, shown by the green shaded area.
Left: Accuracy generally rises with more
GSMSK
Countdown
Average Entropy
Coundown
Answe Linti Satropp
Finally-
Fisally-CCoreee Intemadiiue-Cornes Incemm-
Internelinne
Always Incureet
64
16
52
16 Sampling Step
64
Sampling Step
Sampling Step
Figure 3: Patterns of entropy evolution over difflusion sampling steps. Responses are generated with length 128 using 64 diffusion steps from the LLaDA-8B-Instrus model. Left: Average token- level entropy decreases steadily during sampling. GSM8K shows lower entropy than Countdown, aligning with its higher accuracy. Middle and Right: Both Intermediate- Correct and Always- Incorrect questions exhibit higher overall entropy compared Finally-Core Correct ones. On GSM8K, Intermediate-Coree Correct questions display lower entropy in the early steps than Always-Incorrect, indicating initial confidence, whereas on Countdown the entropy trend is less stable.
Takeaway 1: Correct intermediate answers may be lost during sampling
During sampling, answers may oscillate between correct and incorrect states across diffusion steps. A notable portion of questions achieve correct answers in the intermediate steps, but ultimately yield incorrect results in the final step.
3.3
ANALYSES
To gain deeper understanding of the temporal oscillation phenomenon, we conduct comprehensive analyses from multiple dimensions: accuracy, entropy, and semantic stability across decoding steps.
Accuracy Across Sampling Steps. As shown Fig. 2a, accuracy generally improves with more sampling steps. Simpler datasets like SVAMP start high and remain stable, while harder ones like Countdown begin low but benefit from iterative refinement. To probe further, we compare Pass @1 and EverPass @1 1 on GSMSK and Countdown. Early correct predictions appear sooner on GSM8K but later on Countdown, and growing gap between the two metrics reveals that early correctness does not ensure stable reasoning, This underscores the importance of preserving correct intermediate states. Additional results for SVAMP and MATH500 are in Appendix C.I.
To quantify this, we analyze the average token-level entropy. As shown in Fig. 3, entropy decreases steadily and approaches
Entropy Across Sampling Steps. Temporal oscillations reflect model uncertainty.
the final step, with GSM8K starting lower, indicating higher initial confidence. We categorize questions into three groups: Finally-Correct, Always-Incorres and Intermediate-Correct where Intermediate-Corree means least one intermediate step is correct
5

---

Published as a conference paper at ICLR 2026
but the final answer is incorrect. For GSM8K and Countdown, incorrect final answers show consistently higher entropy. On GSM8K, Intermediate- Correct questions begin with lower entropy than Always-IIcorree suggesting initially confident but unstable predictions. Considering the semi- autoregressive sampling strategy (Nie et al., 2025), we also measure the average entropy of the currently generated block, detailed in Appendix C.3.
Temporal Semantic Entropy. Token-level entropy reflects local uncertainty, but we also need a measure of semantic consistency across the decoding trajectory. We therefore introduce Temporal Semantic Entropy (TSE), which cap- tures the semantic variations of answers during sampling. During decoding, we obtain se- quence of intermediate answers, denoted by {x)) We cluster them by semantic mean- ing into e = (C,...., ... J. where each cluster Cs groups answers with equivalent semantics. We define the probability mass a semantic Ex; p(x) E C&) |CI//T, where |C&| counts how many interme- diate answers fall into cluster Cs- Based this, the TSE of a sampling trajectory is defined as:
Temponl Semantic Entropy
FaalpComat Crome Alagg lncemee
1>
GSMSK
MATH500
SVAMP
Countdown
Figure 4: Temporal semantic entropy across four benchmarks. Error bars denote the mean + standard deviation of TSE, clipped to respect its non-negative range. This metric measures the uncertainty in the semantic content of answers across decoding steps. Statistically, correctly answered questions exhibit lower entropy.
cluster as P(Cx)
K
TSE TSE( = Σ P(Ck) log P(C,);
1=1
which quantifies the uncertainty in semantic content semantic variation, while lower TSE implies convergence to a consistent meaning.
steps: higher TSE indicates more
As shown in Fig. 4, TSE offers insight into model behavior during generation. In datasets like Countdown and MATH, where performance is weaker, we observe higher entropy than in GSM8K and SVAMP, reflecting greater semantic instability. Moreover, questions ultimately answered correctly generally exhibit lower entropy than incorrect ones, including both "finally correct" and "intermediate correct" cases, indicating that stable, semantically consistent trajectories align with better performance. Thus, high TSE may signal model uncertainty and highlight samples for further improvement. More results on TSE are provided in Appendix C.2.
Takeaway Correct answers statistically exhibit lower temporal semantic entropy
Temporal semantic entropy, computed over intermediate predictions during the decoding trajectory, reflects the semantic stability of the model's outputs. Statistically, correctly an- swered questions tend to have lower entropy, indicating greater consistency and confidence throughout the generation process.
4 METHOD
4.1 TEMPORAL SELFF-CONSISTENCY VOTING
We propose a temporal self-consistency decoding strategy for dLLMs, which leverages intermediate predictions to improve final outputs. As discussed in Sec. 3.3, while the last timestep usually yields the best result, the correct answer may also appear earlier, so relying only on the final prediction risks discarding better outputs. To address this, aggregate predictions across timesteps using weighted voting mechanism. Formally, given a diffusion sampling trajectory {xi]<_) our method selects the final answer a" according to weighted vote over all timesteps:
(4) Here, 1(.) indicates whether x decodes to and f(t) is timestep weighting function. Since accuracy generally increases with later steps, we design ftt) as monotonically decreasing function of diffusion step. We experiment with constant, linear, and exponential weighting, in Sec. 5.2.
a" arg max, Σr, f(t) 1 mcaning (xx) a).

---

Published as a conference paper at ICLR 2026
Discussion. Our method is conceptually related to self-consistency decoding (Wang al., 2022), which improves reasoning in autoregressive LLMs by sampling diverse reasoning paths and selecting the most consistent answer via majority voting. However, self-consistency requires multiple full-length forward passes with high cost. contrast, our approach requires only a single sampling trajectory. By exploiting the temporal nature of diffusion inference indicated by Eq. (I), we obtain series of intermediate predictions without additional model evaluations. This makes our method both efficient and effective for boosting accuracy through temporal aggregation.
4.2 TEMPORAL CONSISTENCY REINFORCEMENT
Motivated by observation in Sec. 3.3 that correct answers generally exhibit lower Temporal Semantic Entropy (TSE) than incorrect ones, reffecting stronger semantic consistency over time, we propose post-training approach designed encourage temporal consistency in model outputs. Specifically, we adopt Group Relative Policy Optimization (GRPO) (Shao et al., 2024; Guo et al., 2025) as our reinforcement learning framework and use TSE as self-suue reward signal.
Negative TSE as the Reward. Following GRPO, for each question sampled from the dataset D. we draw group of G responses {01, o; receives a scalar reward ri TSE(o:), where TSE(o,) is computed using Eq. (3) from Sec. 3.3. This reward encourages the model to produce responses whose intermediate predictions remain semantically consistent throughout the decoding process. Based on this, we define the unnormalized advantage (Liu et al., 2025b) for tokens k l...., |o;| as A (TT = r;(T)
oG) from the old policy T0wc- Each response
mcan (1))()))==)) The training objective follows the standard GRPO formulation with our TSE- based reward. We optimize the policy model by maximizing the following objective:
{
G
loil min (pt clip (of.1 1----- 1,,1--,,)) +)) 4t) -DDa (x0((la) (5)
(0)
01nn (la)
where &ee is the reference policy, pf ma (o
To 10, is the importance sampling ratio, e is the clipping threshold, and B controls the strength of the KL penalty. To compute the token-level probabilities used in p², we follow the diffu-GRPO method (Zhao et al., 2025), which estimates these probabilities by averaging outputs from multiple randomly masked versions of the prompt.
Combining TSE with Accuracy Reward. When ground-truth answers are available during training. we combine TSE with an accuracy reward further enhance performance. Specifically, We design composite reward function that integrates correctness and temporal consistency.
The accuracy reward follows a binary scheme: assigns when the model's prediction correct (o, = o²), and 0 otherwise. To measure temporal consistency, we derive normalized confidence score from TSE: c(o.) Kmae-TSEi TSE((o.) H(max log T. Here, T denotes the total sampling steps.
Kmax
This normalization yields c [0, 1], with larger values indicating stronger consistency.
We further transform c(o:) using the spherical scoring rule (Gneiting & Raftery, 2007), which has shown strong empirical performance across tasks. The final reward defined as:
c(o,)
ri lo;=o' Vic(o))) (1-co.))
(6)
Here, the first term enforces correctness while the second term encourages higher temporal consistency. This formulation makes the reward sensitive not only to prediction accuracy, but also to the stability of the underlying generation process. A detailed comparison of various scoring rules and their empirical performance in the Appendix E.4.
Discussion. Unlike prior reinforcement learning post-training methods for dL.MMs, such as diffu- GRPO, which relies on ground-truth answers for reward computation, our approach operates without any labeled data. Instead, we harness the model's internal temporal dynamics as self-supervised signal, employing negative TSE to assess answer quality. This enables our method to be broadly applicable, particularly in unsupervised settings, and offers novel direction for improving dLLMs. Furthermore, we show that combining the negative TSE reward with the accuracy reward based on ground-truth answers yields notably better performance than using the accuracy reward alone.

---

Published as a conference paper at ICLR 2026
Table 1: Performance of temporal majority voting. We compare three strategies: fixed, linear, and exponential weighting, on four datasets using Innnstuuctttt and LLaDA-1.5. Bold numbers mark group bests, and green values show gains over baseline. For reference, we report the oracle EverPass @1 as an upper bound.
GSM8K
MATH500
SVAMP
Countdown
Method Seq Len
128
256
512
128
256
512
128
256
512
128
256
512
LLaDA-SB-Iustre
baseline
68.5
76.3
78.2
27.4
33.4
35.8
84.0
83.3
84.7
20.3
21.5
Fixod Weighting
68.0
73.4
78.3
26.6
30.8
34.2
87.0
84.3
84.3
22.7
18.8
11.3
+ Temporal Voting
Linear Weighting
70.0
78.0
78.8
28.0
34.4
34.6
87.0
84.3
84.3
24.2
21.9
16.0
70.1
78.9
28.4
35.6
36.2
84.3
84.7
25.0
23.4
Exp. Weighing
+1.6
+2.4
+0.7
+1.0
+2.2
+0.4
+2.0
+1.0
+0.0
+4.7
+1.9
+0.0
EverPass @1l |
80.5
85.2
80.3
40.2
45.6
47.2
91.3
89.3
86.7
28.1
27.7
21.1
LLaDA-1.5
baseline
69.8
79.4
81.1
29.0
32.4
35.4
85.3
86.3
83.3
21.5
21.1
20.7
Fixed Weightng
68.8
75.7
80.3
27.3
30.8
34.6
87.3
85.3
84.0
23.4
22.3
18.8
Temporal Voting
Lincar Weighting
71.0
79.8
81.0
29.2
32.8
35.8
86.0
87.0
84.0
24.2
23.4
19.1
Exp. Weighting
70.7
79.8
81.1
29.0
33.2
36.2
85.7
87.7
84.3
26.2
25,0
21.1
43.9
Ever unes R81
81.5
88.9
83.6
39.8
47.4
49.2
90.7
90.3
86.0
30.5
27.0
25.4
5 EXPERIMENTS
5.1 IMPLEMENTATION DETAILS
Experimental Setup. We use -LADA-8B-Instrrr (Nie et al., 2025) and LLaDA-1.5 (Zhu et al., 2025), evaluating on four math benchmarks: GSM8K (Cobbe et al., 2021), MATH500 (Hendrycks et al., 2021), SVAMP (Patel et al., 2021), and Countdown (Pan et al., 2025). Following dl (Zhao et al, 2025), we report performance under different output lengths. Temporal self-consistency voting applics exponential weights to sampling steps. All of our experiments adopt semi- autorersoreee decoding with low-confidence remasking. using a block size of 32. The diffusion step is set to half of the target output length. For post-training, fine-tuning (SFT) on s1K (Muennighoff al., 2025) for 20 epochs with 4,096 token sequences, followed by reinforcement fine-tuning (RFT). For LLaDA-1.5, we omit SFT because it often results in performance degradation, likely due the model having already undergone sophisticated post- training. All training uses 8 H800 GPUs. Further details are in Appendix B.
undergoes supervised
Answer Extraction for Voting and Semantic Clustering. At every diffusion step, LLaDA first generates a fully decoded token sequence before applying the remasking operation. Although some positions may be remasked and re-predicted in later steps, this pre-remask sequence is always complete, unmasked prediction. We use these fully decoded sequences as the intermediate outputs for all temporal analyses. This design guarantees that each intermediate prediction provides a syntactically complete candidate answer. Answer extraction itself is performed by parsing the <answer>, ........//ataa /aasswrr> block. If a pre-remask sequence does not contain a valid answer span at particular step, that step is excluded from temporal self-consistency voting semantic clustering. Only steps containing valid, fully parsed answers contribute to our temporal statistics.
5.2 TEMPORAL SELF-CONSISTENCY VOTING
Voting Strategies. We apply weighted voting across denoising steps using three schemes: fixed, linear, and exponential. Each uses weighting function f((). where is the current diffusion step. The fixed scheme assigns equal weight to steps with f(t) 1. The linear weighting takes the form f(t exponential schemes prioritize early diffusion time steps, i.e., latter sampling steps.
t/T,, and exponential uses f(t) exp(a(1 - t/T") with 5. Both linear and
Ablations on Voting Strategies. As shown in Table 1, linear and exponential weighting improve inference performance. with exponential yielding the largest gains, e.g., LLaDA-8B-Instmun improves by 1.6%, 1.2%, 1.0%, and 2.2% GSM8K. MATH500, SVAMP, and Countdown, respectively. Fixed weighting performs slightly worse, likely because equal weights amplify inaccurate early predictions. We therefore adopt exponential weighting by default. We futher ablate
8

---

Published as a conference paper at ICLR 2026
00-
GSMER
MATH500
1.3
-0.25
-0.
Averago Improvemett
144
40
1.3
-0.75
-1.00
1.2
0.5
103
223
480
100
1550
2000
100
300
400
506
1000
1200
2000
1.1
SVAMP
Counsdown
1.0
0.9
1.0
2.
Training Step
Training Supp
(a) Ablations on cevalue for temporal voting
(b) Reward ourve during RFT
Figure 5: (a) Ablations on value selection in temporal voting with exponential weighting. (b) Negative temporal semantic entropy reward curve during reinforcement fine-tuning.
Performance of reinforcement fine-tuning. Unlike dI (Zhao et al., 2025). which requires ground-truti answers, our method negative Temporal Semantic Entropy (TSE) the reward without labels. Combining TSE with accuracy-basee rewards yields further gains across benchmarks. Green numbers denote improvements over baseline.
Table 2:
GSM8K
MATH500
SVAMP
Countdown
Method /Seg
128
256
512
128
128
256
512
LLaDA
baseline
70.2
78.7
80.1
23.8
34.4
36.8
81.7
83.3
82.7
21.5
19.9
21.5
sccuracy reward (dl)
71.7
78.3
82.3
31.0
36,0
40.4
85.7
88.0
88.7
34.8
35,5
37,9
72.2
78.8
80.2
30.6
34.6
38.0
84.3
89.0
88.7
38.6
$3.5
44,9
+ RFT
negative reward (ours)
+2.0
+0.1
+0.1
+6.8
+.22
+.22
+2.6
+5.7
+6.0
+17.1
+33.6
+23.4
72.1
80.0
83.0
31.2
35.4
41.4
85.0
90.3
92.3
41.5
42.6
54.7
combining both (ours)
+1.9
+1.3
+2.9
+7.4
+1.0
+4.6
+3.3
+7.0
+9.6
+20.0
+22.7
+33.2
LLaDA-1.5
haseline
69.8
79.4
81.1
29.0
32.4
35.4
85.3
86.3
83.3
21.5
21.1
20.7
accuracy reward (dl)
73.0
78.9
83.1
29.8
36.2
40.2
84.7
89.3
88.0
38.7
29.7
39.1
+ RFT
negative reward (ours)
+2.2
72.0
+1.4
80.8
+1.5
82.6
+1.2
30.2
+2.6
35.0
40.0
+4.6
+1.0
86.3
+2.0
88.3
87.3
+4.0
+28.5
50.0
+34.8
55.9
+32.4
53.1
combining both (ours)
73.2
80.5
84,0
29.6
35.4
41.4
86.3
90.3
89.0
44.5
46.9
63.3
+3.4
+1.1
+2.9
+0.6
+3.0
+6.0
+1.0
+4.0
+5.7
+23.0
+25.8
+42.6
the exponential hyperparameter a. As shown in Fig. 5a, a values ranging from 1 to consistently improve accuracy, peaking at a = 5 with an average gain of 1.5%. Thus, we set a = 5 by default.
5.3 TEMPORAL CONSISTENCY REINFORCEMENT
Main Results. Table 2 reports results of incorporating temporal consistency into RFT. We have the following observations. (1) Using TSE reward alone consistently improves performance across lengths and datasets. (2) TSE reward matches or surpasses accuracy reward despite not using ground truth, e.g. on Countdown, LLaDA-SE -B---nstruct improves 24.7% vs. 15. 1% with dl. (3) Combining TSE with accuracy reward further boosts results, with absolute gains of 0.9% (GSM8K), 0.2% (MATH500), 1.7% (SVAMP), and 10.2% (Countdown) over dl. Overall, our method achieves average improvements of 2.0%, 4.3%, 6.6%, and 25.3% over the SFT baseline, confirming the benefit encouraging temporal consistency.
Training Dynamics. We visualize the reward curves during training using LLLDD--B-IIntrruu Instrruttt as an example, as shown in Fig. 5b. The curves demonstrate a consistent upward trend in rewards across different datasets as training progresses, indicating effective learning and stable optimization.
Model Attributes After RFT. We analyze LLDA--BB-Instrul fine-tuned with the negative TSE reward, with generation length 128, evaluating its behavior across several dimensions: TSE, the ever pass rate and the number of effective tokens (defined as the average count of non-padding, non- EOS tokens per generation). We have the following observations. (1) As shown in Fig. 6a, temporal semantic entropy consistently decreases across various datasets after RFT, reflecting enhanced temporal consistency in the model's outputs-an anticipated result of reinforcement learning: (2)

---

Published as a conference paper at ICLR 2026
130
Dutas FFF
Raa
H
120
Num Elociive baa9
110
Ronporn Semonts
100
50
80
40
80
30
20
8
GSMBK
MATH500
SHAMP
Coundown
GSMaK
MATH500
SVAMP
Counsdown
50
65MBK
MATH500
SHAMP
Countdown
(a) Temporal Semantic Entropy Comparison
(b) Final Ever Pass Rate After RET
(c) Effective Tokens Number Comparison
Figure 6: Model attributes after reinforcement finetuning (RFT), (a) Temporal semantic entropy decreases after RFT, showing improved semantic consisteney in outputs.
(b) The ever pass rate remains above the final pass rate, leaving room for further gains. (c) Effective tokens per generation drop after RFT, yielding more concise outputs.
Fig. 6b demonstrates that the ever pass rate remains higher than the final pass rate after RFT, suggesting there is still potential for further improvement. (3) In Fig. 6c, effective tokens decline after RFT, implying more concise outputs. We suspect shorter generations may reduce temporal oscillations, though this requires further study. Further analysis of model attributes after RFT and detailed discussion of limitations provided in Appendix E.5 and Appendix D.
CONCLUSION
This work uncovers a critical yet overlooked aspect of diffusion large language models: their recurring issue in dLLM rich temporal dynamics. By identifying temporal oscillation predictions. Our proposed decoding. we challenge the convention relying solely on final-step methods-TTemporal Self-Consistency Voting and Temporal Consistency Reinforcement demonn- strate that intermediate predictions are not noise, but signal. These strategies improve accuracy and stability without requiring additional inference passes ground-truth supervision. Through extensive experiments, we show that temporal consistency is not just desirable property-it is powerful lever for performance. We hope that this study inspires future research treat intermediate denoising time steps not as nuisance, but as feature in diffusion-based text generation.
ACKNOWLEDGMENT
This work partially supported by the National Key R&D Program of China (No. 2022ZD0162 60101), Ningbo Science and Technology Bureau (No. 2024Z291] 1) and the National Natural Science Foundation of China (No. 62206244). The authors would also like thank Muzhi Zhu, Canyu Zhao, and Linhao Zhong at Zhejiang University for their valuable discussions and insightful feedback.
ETHICS STATEMENT
This work does not involve human subjects, personal data, sensitive information. All datasets used in our experiments (GSM8K, MATH500, SVAMP, and Countdown) are publicly available benchmark datasets designed for evaluating mathematical reasoning large language models. We strictly adhered to ethical research practices and did not conduct any data collection that could raise privacy, security, or fairness concerns. Our SachodssTTempore Temporal Consistency Reinforcement -occo on improving the robustness and accuracy of diffusion language models, without introducing risks of harmful applications. To the best of our knowledge, this research complies with the ICLR Code of Ethics and poses foreseeable ethical concerns.
Crr-CCnsistency Voting and
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility our work. Detailed dataset descriptions are provided in Appendix B. and training configurations and hyperparameters
10

---

Published as a conference paper at ICLR 2026
reported in Appendix B.2. The sampling and evaluation procedures outlined in Appendix B.3. To further promote transparency, we provide mathematical formulations and thorough descriptions of our proposed Segorithms-Temppee Temporal Self-Consistency Voting and Temporal Consistency Rein- forcement- direetlyy the main paper. Upon acceptance, we will release our models, together with training and inference code, facilitate replication and further research.
REFERENCES
Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effectiveness of entropy minimization in IIm reasoning. arXiv preprint arXiv:2505.15734 2025.
Marianne Arriola, Aaron Gokaslan, Justin Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. arXiv preprint arXiv:2503.09573, 2025.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34:1 191999,,, 2021.
Pavel Avdeyev, Chenlai Shi, Yuhao Tan, Kseniia Dudnyk, and Jian Zhou Dirichlet diffusion score model for biological sequence generation. In International Conference on Machine Learning, pp. 1276-1301 PMLR, 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
Minghan Chen, Guikun Chen, Wenguan Wang, and Yi Yang. Sced-grpo: Semantic entropy enhanced grpo for uncertainty-awar policy optimization. arXiv preprint arXiv:2505. /2346, 2025.
Chaoran Cheng, Jiahan Li, Jian Peng, and Ge Liu. Categorical flow matching on statistical manifolds. Advances in Neural Information Processing Systems, 37:54787-54819, 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05455711, 2018
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 / 10. /4 468, 2021.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo. Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22611 2025.
Mehul Damani, Isha Puri, Stewart Slocum, Idan Shenfeld, Leshem Choshen, Yoon Kim, and Jacob Andreas. Beyond binary rewards: Training Ims reason about their uncertainty. arXiv preprint arXiv:2507.16906 2025.
Oscar Davis, Samuel Kessler, Mircea Petrache, Ismail Ccylan, Michael Bronstein, and Jocy Bose. Fisher flow matching for generative modeling over discrete data. Advances Neural Information Processing Systems, 0011139554-199900 2024.
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 63008017):625-633 630, 2024.
Tilmann Gneiting and Adrian E Raftery. Strietly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):35-337) 2007.
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu. Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code generation. arXiv preprint arXiv:2506.20633 2025.

---

Published as a conference paper at ICLR 2026
Ishaan Guirajani and Tatsunori B Hashimoto. Likelihood diffusion language models. Advances in Neural Information Processing Systems, 36: 16693-16715, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. Ssd-Im: Semi-autoregressive simplex- based diffusion language model for text generation and modular control. arXiv preprint arXiv:2210.17433: 2022.
Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. Mdpo: Overcoming the training-inferenee divide of masked diffusion language models. arXiv preprint arXiv:2508. (tti:2558..11488, 2025.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 1851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)3, 2022.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXhv:2503.06749, 2025.
Sakaguchi Keisuke, Le Bras Ronan, Bhagavatula Chandra, and Choi Yejin. adversarial winograd schema challenge at scale. 2019.
Winogrande: An
Jacyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. arXiv preprint wrxiv:2502206668, 2025.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09666 2023.
Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508. 19982, 2025a.
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion- Im improves controllable text generation. Advances neural information processing systems, 35: 4328 4343-4343,400 2022.
Yuetai Li, Zhangchen Xu, Fengqing Jiang, Bhaskar Ramasubramanian Luyao Niu, Bill Yuchen Lin, Xiang Yue, and Radha Poovendran. Temporal sampling for forgotten reasoning in 1lms. arXiv preprint arXiv:2505.20 20/66. 2025b.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint 2XXi:22100007477, 2022.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint wrXiv:2209.03003 2022.
12

---

Published as a conference paper at ICLR 2026
Yexiang Liu, Zekun Li, Zhi Fang, Nan Xu, Ran He, and Tieniu Tan. Rethinking the role of prompting strategies in Ilm test-time scaling: A perspective of probability theory. arXiv preprint arXiv:2505.10981, 2025a.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503. 20783, 2025b.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310.168. 2023.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing 3 36:46534-46594 2023.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsumori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501. 2025.
Shen Nie, Fengqi Zhu, Chao Du Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410.188 18514, 2024.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint 20X2:22502.09992, 2025.
Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. http:///gitubbbcomm//aaaaaaaaaii Pan/TinyZeoo,, 2025. Accessed: 2025-01-24.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word problems? arXiv preprint arXiv:2103.07191, 2021.
Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. arXiv preprint arXiv:2505.22660 2025.
Zhangyang Qi, Zhixiong Zhang, Yizhou Yu, Jiaqi Wang. and Hengshuang Zhao. Vln-rl: Vision-
language navigation via reinforcement fine-tuning. arXiv preprint arXiv:2506..722// 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:1301366 17:113013-131888 2024.
Subham Sekhar Sahoo, Zhihan Yang, Yash Akhauri, Johnna Liu, Deepansha Singh, Zhoujun Cheng, Zhengzhong Liu, Eric Xing, John Thickstun, and Arash Vahdat. Esoteric language models. arXiv preprint 2rXiv:2506.01928. 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 17X1v:1007.06347, 2017.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. VIm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504.076
2025.
Jiaxin Shi, Kehang Han, Zhe Wang, Amaud Doucet, and Michalis K. Titsias. Simplified and generalized masked diffusion for discrete data. In Advances Neural Information Processing Systems, 2024.
13

---

Published as a conference paper at ICLR 2026
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint 0rXiv:2408.03314. 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020.
Hannes Stark, Bowen Jing, Chenyu Wang, Gabriele Corso, Bonnie Berger, Regina Barzilay, and Tommi Jaakkola. Dirichlet flow matching with applications to dna sequence design. arXiv preprint arXiv:2402.058/ 002.4444,.4,,, 2024.
Guanghan Wang. Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. arXiv preprint arXiv:2503.00307, 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.1117,, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24877 2022.
Shaoan Xie, Lingjing Kong, Xiangchen Song, Xinshuai Dong, Guangyi Chen, Eric P Xing, and Kun Zhang. Step-aware policy optimization for reasoning in diffusion large language models. arXiv preprint arXiv:2510.0154: 2025.
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505. 15809, 2025.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36: 11111180-1100 1122, 2023.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. URL https: /twknlkkppppp///p/ github 1o /bwgg/log/////// 2025/ dream.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. /4476, 2025.
Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. Right question is already half the answer: Fully unsupervised Ilm reasoning incentivization. arXiv preprint arXiv:2504.05812. 2025.
Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. dl: Scaling reasoning in diffusion large language models via reinforcement learning. arXiv preprint arXiv:2504. /22 2025.
Hao Zhong, Muzhi Zhu, Zongze Du, Zheng Huang, Canyu Zhao, Mingyu Liu, Wen Wang, Hao Chen, and Chunhua Shen. Omni-rl: Reinforcement learning for omnimodal reasoning via two- system collaboration. arXiv preprint arXiv:2505.200 20256, 2025.
Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, al. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint 2rXi:2555..19233 2025.
14

---

Published as a conference paper at ICLR 2026
LLM USAGE
In this section, we clarify the role of large language models (LLMs) in preparing this work. The model was used exclusively for language polishing, such as refining grammar, style, and readability, without contributing to the research design, analysis, conclusions.
A APPENDIX OVERVIEW
This appendix provides additional implementation details, empirical analysis, and extended results to supplement the main paper. It organized as follows:
Appendix B: More Implementation Details
Provides further implementation information, including:
Appendix B.I: Detailed descriptions of the datasets used
Appendix B.2: Training configurations and hyperparameters
Appendix B.3: Sampling strategies and evaluation procedures
Appendix C: More Analysis
Presents extended analyses, including:
Appendix C.I: Accuracy and entropy analysis on the MATH500 and SVAMP datasets
Appendix C.2: Temporal semantic entropy across varying generated lengths
Appendix C.3: Block-level token entropy analysis
Appendix C.4: Time analysis for temporal consistency voting
Appendix C.5: Investigating the Causes of Time Oscillation
Appendix D: Limitations
Diseuss limitations and analyze failure cases, including:
Appendix D.I: Discuss potential limitations of our method
Appendix D.2: Analysis of failure cases the Sudoku dataset
Appendix E: More Experimental Results Includes additional experimental findings, such as:
Appendix E.2: Training a unified model across multiple tasks
Appendix E.3: Possible alternatives to temporal semantic entropy
Appendix E.4: Ablation studies on different scoring rules for combining TSE with accuracy reward
Appendix E.5: Performance of temporal self-consistency voting on reinforcement
fine-tuned models
Appendix E.6: Detailed examples illustrating temporal oscillation
B MORE IMPLEMENTATION DETAILS
B.1 DATASETS
We provided detailed descriptions of the datasets follows:
GSMSK (Cobbe et al., 2021) comprises 8.5K linguistically diverse grade school math word problems (7.5K training. 1K test), solvable by bright middle school students via 2-8 steps of basic arithmetic, suited for multi-step mathematical reasoning.
MATH500 (Lightman al., 2023) is curated subset of 500 problems selected from the broader MATH dataset (Hendrycks et al., 2021), featuring high-school-level competition math problems.
SVAMP (Patel al., 2021) serves benchmark for elementary-level Math Word Problems (MWPs), where each MWP short natural language narrative describing a scenario and asking questions about unknown quantities.
15

---

Published as a conference paper at ICLR 2026
MATH500
SVAMP
04
EvaPao@I Paggi
Pangl
Accuracy
0.3
0.6
0.2
0.4
16
32.
48
64
32
48
54
Answer Entropy
Fisslly--onn
Finslly-Cornet
Intemmdd Comrect
Intermediate C Correct
Alwags-nncorret
Always-
Sampling Step
Sampling Step
Figure $1: Top Row: Pass @1 and Pass @1 /. for MATH500 and SVAMP are provided as supplementary results Fig. 2, using the experimental settings. A noticeable gap similar to that observed in GSM8K and Countdown between Pass @ and Pass @1 t is present across all sampling steps. Bottom Row: Answer Entropy for MATH500 and SVAMP are provided as supplementary results to Fig. 3, using the same experimental setups. MATH500 and SVAMP exhibit answer entropy patterns similar to those of GSM8K and Countdown.
Countdown (Pan al., 2025) involves combinatorial arithmetic game with three numbers, requiring models to reach target numbers using basic arithmetic operations on a given set of numbers.
B.2 TRAINING
During reinforcement fine-tuning, we train our model using sequences of 256 tokens, with batch size per GPU and gradient accumulation over steps. Low-Rank Adaptation (LoRA) (Hu al., 2022) is applied with rank of 128 and scaling factor of 64. During reward computation, answers are parsed from generated text sequences for semantic clustering. When answer parsing fails due to an inaccurate format, we simply discard the answer for temporal semantic entropy computation. Moreover, since answers generated in the first half the sampling steps tend to be rough and less reliable, exclude them from consideration. Only answers from the second half of the sampling steps are used to calculate the temporal semantic entropy.
B.3 SAMPLING AND EVALUATION
Unified Decoding Configuration. During sampling, we adopt the semi-autoregreesive sampling approach following LLaDA (Nie et al., 2025). Specifically, the sequence is split into multiple blocks, which are generated in left-to-right manner. For each individual block, we employ the low- confidence remasking strategy during the sampling process. We use block size of 32, following LLLDA (Nie al., 2025) and dl (Zhao et al., 2025). Following the practice d1 (Zhao et al., 2025), evaluate the model every 100 steps, starting from step 600 to 8,000 steps, and report the best results.
Generation Lengths. (1) Evaluation. For test-time self-consistency voting and all RL evaluations, we evaluate three output lengths: 128, 256, and 512 tokens, corresponding to 64, 128, and 256 diffusion steps, respectively (again following d1 (Zhao et al., 2025)). (2) Analysis experiments. In the exploratory study in Sec. 3.2 Fig. 1, we analyze output lengths of 128 256, and 512, and observe temporal oscillation under all settings. Sec. 3.3 uses the default 128-
16

---

Published as a conference paper at ICLR 2026
Generated Length 256
Generated Length
Temporal Semantic Entropy
Centul Inceract
Censal
Iooma:
GSM8K
MATH500
SVAMP
Countdown
GSMSK
MATH500
SVAMP
Countdown
Figure S2: Temporal semantic entropy with varying generation lengths. Average temporal
semantic entropy across datasets with generation lengths of 256 and 512, showing consistent patters with Fig. 4. Higher entropy generally correlates with lower accuracy.
token 64-step configuration. Appendix Sec. C.2 / Fig, S2 further reports temporal semantic entropy under 256- and 512-length generations to demonstrate robustness.
C MORE ANALYSIS
C.I ANALYSIS ON MATH500 AND SVAMP DATASETS
Accuracy Analysis. As shown in the first row in Fig. S1, MATH500 and SVAMP exhibit similar pattern to that observed Fig. 2 in Sec. 3.3, where a noticeable gap emerges between Pass @1 and Pass @1 In MATH500 and SVAMP, correct answers start appear early in the sampling process(near 4% and 39% at first step. respectively), and continue to improve over subsequent iterations. Interestingly, on the SVAMP dataset, distinct pattern emerged in the model's Pass @1 accuracy across different sampling steps. Between steps 3 and 20, performance declined noticeably, followed by recovery after step 20. This distinctive fluctuation trajectory represents another manifestation of the temporal oscillation phenomenon.
shown the second row Fig. S1, for MATH500 and SVAMP, the answer entropy of Finally- Cartle- questions remains the lowest throughout the sampling process. Intermediate Correct questions consistently exhibit lower entropy in the early sampling steps compared Always-Incorrect ones, pattern observed across all four datasets. However, unlike GSM8K, MATH500, and Countdown, where the final entropy of Intermediate-Com Correct and Always- Incorrect questions is similar, SVAMP displays relatively high entropy in the final sampling step for Intermediate-Corin Correct questions.
Entropy Analysis.
C.2 TEMPORAL SEMANTIC ENTROPY OVER VARYING GENERATED LENGTH
To further validate the generalizability of our findings regarding temporal semantic entropy, we extended the experiments beyond those presented in Fig. 4, which used a generation length of 128 and 64 diffusion steps. Specifically, we tested generation lengths of 256 and 512, with diffusion steps half the generation length.
As shown Fig. S2, the pattern of temporal semantic entropy observed here aligns with the conclusions drawn in Sec. 3.3: questions that are eventually answered correctly tend display consistently low temporal semantic entropy, indicating that their intermediate predictions remain stable and semantically coherent throughout the decoding process.
C.3 ANALYSIS ON THE AVERAGE ENTROPY IN BLOCKS
In addition to the token-level entropy considered in Sec. 3.3 and the answer-level entropy considered in temporal semantic entropy, we additionally introduce a block-level entropy. This is motivated by the fact that dLLMs typically adopts cemi-autoregresssvve sampling strategy. In this sampling strategy, the entire generated sequence is divided into multiple fixed-length blocks, with each block allocated specific number of sampling steps. During these steps, remasking and unmasking
17

---

Published as a conference paper at ICLR 2026
GSM8K
MATH500
SVAMP
Countdown
Blook Entropy
Sampling Step
Sampling Step
Sampling Step
Sampling Step
Figure S3:
Block-level entropy dynamics under semi-autoregrrssive sampling, During sampling, sequences are partitioned into fixed-length blocks, processed in a left-to-right order, with remasking and unmasking operations restricted to the current block. Average token entropy within block decreases with more sampling steps. A sharp entropy spike occurs when shifting new block, likely due to simultaneous decoding of multiple masked tokens increasing initial uncertainty.
Table $1: Runtime comparison (in seconds) for temporal self-consisteney voting. For the voting setting, we report end-to-end latency from receiving the input to producing the final voted output. For the non-voting setting, we measure the time required to generate an output the same final length. Across all datasets, the additional cost introduced by voting is minimal.
GSM8K
MATH500
SVAMP
Countdown
Method Seq Len
128
256
512
128
256
512
128
256
512
LLaDA-SB DIDD--8-Insmmet
w/o voting (baseline)
4.48
10.76
29.24
4.47
10.87
29.45
4.23
10.31
28.45
4.59
10.99
29.58
vosing
4.53
10.86
29.76
4.54
10.91
29.61
4.29
10.41
28.59
4.64
11.12
29.81
operations are performed exclusively on the current block. Starting with the first block, the process steps to the next block only after all tokens in the current block have been unmasked.
We denote the current block as containing tokens indexed from a start s to an end e (inclusive), such that the block spans indices i {s, + e, 1}. The block entropy is then calculated
H Hlock
Σ H(i
i=s
where (i) represents the entropy of the i-th generated token, and e - denotes the total number of tokens the block.
As illustrated in Fig. S3, the average token entropy within a single block exhibits consistent downward trend as sampling steps accumulate. This pattern intuitive: as more tokens in the current block are decoded, they collectively form a richer contextual foundation, thereby mitigating model's uncertainty about subsequent tokens the block. Interestingly, when decoding shifts to new block, the entropy rises sharply. This phenomenon is likely due to the need to decode multiple masked tokens simultaneously at the start of a new block, which increases the model's 's uncertainty.
C.4 TIME ANALYSIS FOR TEMPORAL CONSISCNNYS VOTING
To assess the efficiency our temporal seff-consistency voting approach, we compare the per- sample runtime with and without voting. We use LLaDA-8B-Instaa with semi-autor comi-autoregressive decoding (block size 32), a diffusion step set to half of target output length, and batch size of 1. All timing experiments are performed on four NVIDIA 4090-24GB GPUs.
For the voting setting, we measure the end-to-end runtime from receiving the input to producing the final voted answer. For the non-voting setting, measure the time to generate output of the same final length. The difference reflects the additional operations performed by voting- -nnnely answer parsing, clustering, and selecting the final answer. We report the average time to generate one response of the fixed output length for each dataset.
As shown in Table S1, temporal self-consistency voting introduces essentially extra runtime. Across all datasets, the overhead is about 1% per cunple-negligiblee relative to the total computation time.
18

---

Published as a conference paper at ICLR 2026
Table S2: Evaluation results under different randomness levels and remasking strategies. Black numbers denote Pass @1, while gray numbers denote EverPass @1lt. As shown, the gap between Pass @1l and EverPass @1 persists across all seeds, temperature settings, and remasking strategies, indicating that the phenomenon of time oscillation is not sensitive to these configuration choices.
Setting
GSM8K
MATH500
SVAMP
Countdown
Seed
Temp.
Remasking
128
256
512
128
256
512
128
256
512
128
256
512
67.9
76.5
79.2
27.4
33.4
35.8
83.3
84.0
84.7
18.8
20.3
15.6
42
0.0
Low Confidence
80.3
84.8
81.0
40.2
45.6
47.2
90.7
89.7
87.3
25.8
27.7
19.9
218
0.0
Low Confidence
68.2
76.5
79.2
27.2
33.4
35.8
83.0
84.0
84.7
20.3
20.3
16.0
81.5
84.8
81.0
40.8
45.6
47.2
91.0
89.7
87.3
29.3
27.0
20.7
69.6
76.3
78.2
27.2
33.4
35.8
83.3
83.3
84.7
20.3
22.3
18.8
66
0.0
Low Confidence
80.3
80.3
45.6
47.2
90.7
89.3
27.7
29.3
22.7
67.4
76.6
78.5
25.6
33.8
33.8
84.7
85.0
84.7
20.3
22.3
18.8
42
0.6
Low Confidence
82.6
86.7
81.5
43.2
49.0
50.8
91.7
90.7
87.0
29.3
20.7
22.7
66.1
75.2
77.0
23.8
30.6
30.8
85.0
84.3
85.0
19.5
17.6
16.8
42
1.0
Low Confidence
78.8
81.9
79.2
37.6
42.0
42.2
89.7
87.0
87.3
22.3
19.5
21.1
57.2
64.5
62.1
19.8
24.4
23.4
73.0
73.3
72.7
10.9
8.2
10.5
42
0.0
Random
76.9
76.6
66.8
35.8
38.6
33.4
86.0
82.0
78.0
22.3
12.5
14.5
59.4
61.6
62.1
19.8
24.6
21.2
75.0
73.3
72.7
12.1
9.8
10.5
218
0.0
Random
75.3
75.7
67.2
35.2
42.2
33.6
87.3
79.0
77.7
23.8
14.1
15.6
C.5 INVESTIGATING THE CAUSES OF TIME OSCILLATION
First, we investigate the effects of randomness and temperature time oscillation. As shown in Table S2, although both Pass @1 and EverPass @1|t exhibit slight fluctuations across different seeds and temperature settings, the gap between them consistently persists. We further examine the impact of applying a random remasking strategy. While this strategy noticeably reduces both Pass @1 and EverPass @ 1 |t., the gap remains. These observations suggest that time oscillation is an inherent and pervasive phenomenon originating from the model itself.
We hypothesize that time oscillation may arise from the training dynamics, where the same input signal can he associated with multiple valid outputs, which is analogous to diffusion models in the vision domain (Liu al., 2022; Lipman et al., 2022) Specifically, consider sequence seqi "abed", possible that another sequence seq2 = "abce" also appears in the training data with the same masked pattern, and the model is likewise supervised predict from "<MASK >bc <MASK >" As a result, the model receives conflicting supervision signals, causing it to oscillate between "abed' and "abce" during sampling.
Moreover, even if no alternative sequence like seq2 exists, the model may still suffer from underfitting under certain masking patterns. such cases, given the prompt "<MASK>bc <MASK>", the model might incorrectly predict "abex" instead of the intended sequence. This behavior can likewise induce time oscillation during sampling.
D LIMITATIONS
D.1 DISCUSSIONS ON LIMITATIONS
While our temporal self-consistency voting and post-training approach demonstrates effectiveness in many scenarios, exhibits significant limitations when applied to tasks where the model's intermediate predictions are consistently inaccurate. As discussed in Appendix D.2, for the Sudoku dataset, the average correctness across all intermediate generation steps remains exceedingly low (below 5%), making it difficult reliably vote for the correct answer. Similarly, RFT relies on the model already achieving reasonably good performance to produce meaningful reward signals (Prabhudesai et al., 2025; Agarwal et al., 2025), though combining TSE with the accuracy reward may alleviate this issue to some extent. This underscores that our approach depends on the model's inherent ability to generate correct or near-correct answers in the sampling trajectory.
19

---

Published as a conference paper at ICLR 2026
Table S3:
Performance of temporal self-consistency voting and temporal consistency
reinforcement on the Sudoku dataset. The baseline model is obtained by applying supervised using the sIK dataset.
fine-tuning Model
Sudoku
Method Seq
256
512
baseline
5.5
temporal voting
12.5
2.8
accuracy reward (d1)
23.2
12.7
RFT
negative TSE reward (ours)
3.3
combining both (ours)
27.5
27.8
16.6
Sukiku
MATH
40
Temporal Accuracy (%)
30
20
10
128
256
512
Generated Length
Figure S4: Temporal accuracy across datasets. Temporal Accuracy Sudoku remains consistently low (all below 5%) across different settings, indicating that the model rarely generates correct answers during the sampling process. This scarcity of valid candidates severely limits the effectiveness of voting mechanisms, as there are insufficient correct outputs to reliably converge on the correct answer.
D.2 FAILURE CASE ANALYSIS
As discussed in Appendix D.I, our method may rely the model's initial performance achieve further improvements. To investigate this limitation, we take the challenging Sudoku dataset as case study. As shown in Table $3, directly applying our proposed temporal self-consistency voting and temporal consistency reinforcement with the negative TSE reward alone results noticeable performance drop on the Sudoku dataset. For example, the original accuracy of the base model for the generation length of 512 is 5.5%, while the voting method achieves 2.8%, reflecting a decline of 2.7%.
To better understand this problem, we conduct a deeper analysis of the model's behavior during generation. We define a metric called Temporal Accuracy, which quantities the average correctness across all intermediate sampling steps in the generation process. Formally, let 6 represent the correctness indicator for the i-th question at the f-th sampling step, where 6 = if the answer is correct, and 0 otherwise. Then, the Temporal Accuracy for a dataset with N examples and T sampling steps computed as:
N
T
TemporalAccuracy N.
CiP
(S1)
As shown in Fig. $4, the Temporal Accuracy of the Sudoku is much lower than the other 4 datasets, where temporal self-consistency voting proved effective. On the Sudoku dataset, we observe that the Temporal Accuracy remains exceedingly below 5% average-acrov all intermediate
20

---

Published as a conference paper at ICLR 2026
Table S4: Temporal self-consisteney voting results on ARC-C and Winogrande using the semi-autoregresivee setup (block size 32). Exponential and linear weighting schemes yield consistent aceuracy improvements without QA-specific tuning, demonstrating robust gains across both datasets.
ARC-C
Winogrande
Method / Sey Len
128
256
512
128
256
512
MalDA-8U-IInsruuue
baseline
81.4
83.3
83.1
53.1
53.6
52.5
Fixed Weighting
82.6
83.8
83.4
54.4
55.1
55.0
Temporal Voting
Linear Weighting
82.6
83.8
83.4
56.1
57.7
58.4
82.6
83.8
83.4
55.1
55.0
Exp. Weighting
+1.2
+0.5
+0.3
+1.3
+1.5
+2.5
EverPass XX1 |
86.1
86.9
85.5
61.5
61.6
62.0
Table S5: Temporal Consistency Reinforcement results ARC-C and Winogrande. Both onE- and combined rewards consistently outperform the baseline across sequence lengths, with the combined reward yielding the largest gains.
ARC-C
Winogrande
Method Seq Len
128
256
512
128
256
512
LLaDA
bascline
81.6
82.8
83.4
54.5
56.6
56.5
negative TSE reward (ours)
84.5
84.8
84.8
59.1
60.6
61.3
+2.9
+2.0
+1.4
+4.6
+4.0
44.8
RET
combining both (ours)
84.5
85.8
86.3
66.6
69.0
68.5
+2.9
+3.0
+2.9
+12.1
+12.4
+12.0
steps. This low signal makes it difficult for temporal voting mechanisms to reliably identify the correct answer and for reinforcement learning signals to guide the model effectively.
Interestingly, while RFT with only negative TSE reward leads to poorer results, combining TSE with accuracy reward can achieve better performance than using accuracy reward alone. We hypothesize that this is because the integration TSE allows the model to receive more fine-grained rewards, rather than just binary outcomes of correct or incorrect.
E MORE EXPERIMENTAL RESULTS
E. EXPERIMENTS ON NATURAL LANGUAGE QUESTION-ANSWERING DATASETS
To verify that methods generalize beyond math reasoning, added experiments two natural language QA benchmarks, ARC-C (Clark et al., 2018) and Winogrande (Keisuke et al., 2019), evaluating both Temporal Self-Consistency Voting and Temporal Consistency Reinforcement.
Temporal Self-Consistency Voting. We adopt exactly the same semi-autoregressive decoding configuration described in the main paper, using block size of 32. For the exponential weighting scheme, we directly reuse the hyperparameter a = 1.5, which was originally tuned on mathematical reasoning datasets, and perform no additional tuning on QA tasks. As shown in Table $4, on ARC- C, exponential temporal voting yields an average improvement of approximately +0.7 accuracy points over the baseline. On Winogrande, the untuned exponential weighting achieves an average gain of +1.8 points, and applying linear weighting leads to further improvements, reaching 58.4 4 accuracy (compared to a 52.5 baseline length 512). These findings demonstrate that temporal voting provides consistent benefits on natural-language QA tasks and remains robust under different weighting schemes.
Temporal Consistency Reinforcement. We train on ARC-C and Winogrande using the same GRPO configuration as in the main experiments, adopting either (i) negative TSE reward alone (ii) combined TSE + accuracy reward. As shown in Table S7, both configurations consistently outperform the bascline across all sequence lengths on ARC-C, yielding improvements of approximately 2-3 accuracy points (e.g., 81.6 to 84.5 128 tokens and 82.8 85.8 at 256
21

---

Published as a conference paper at ICLR 2026
Table S6: Unified Model Performance Across Multiple Tasks. single model is trained jointly on GSM8K, MATH500, SVAMP, and Countdown.
GSM8K
MATH500
SVAMP
Countdown
Method Seq Len
128
256
512
128
256
512
128
256
512
128
256
512
LLLDD
haseline
70.2
78.7
80.1
23.8
34.4
36.8
81.7
83.3
82.7
21.5
19.9
21.5
accuracy reward (dl)
37.1
RFF
negative TSE reward (ours)
71.5
77.2
81.2
29.0
33.8
39.0
86.3
84.7
87.0
36.7
25.4
46.1
combined reward (ours)
72.4
78.8
80.4
28.8
35.2
40.2
88.7
87.0
86.0
37.1
27.7
46.9
LLaDA-1.5
baseline
69.8
79.4
29.0
32.4
35.4
85.3
86.3
83.3
21.5
21.1
20.7
accuracy reward (d1)
72.5
79.2
81.7
28.6
35,0
39.0
89.0
87.7
85.3
30.5
24.6
41.8
+ RFT
negative TSE reward (ours)
72.5
79.2
81.4
29.0
32.6
38.2
89.3
85.3
86.7
37.3
44.5
34.0
combined reward (ours)
73.7
79.5
82.5
28.2
35.0
41.6
88.3
90.0
88.0
37.9
29.7
50.4
Generated Length 128
Generuted Length 256
Generated Length
0.8
0.6
Incomect
Incomeett
Incormat
Change Rate
0.6
0.6
0.4
0.4
0.2
0.0
0.0
0.0
GSMSK MATIL500 SVAMPCoundown.
GSM8K MATHI00 SVAMP Counidown
GSM8K MATH500 SVAMP Countdown
Figure S5: ChangeRate of correct answers across different output lengths for each dataset. Lower ChangeRate indicates more stable predictions over the sampling trajectory.
tokens). On Winogrande, the combined reward yields the largest gains, with improvements of up to +12.4 points (e.g., 56.6 to 69.0 at 256 tokens).
Overall, these additional experiments on ARC-C and Winogrande demonstrate that both our test- time voting and reinforcement learning approaches generalize beyond mathematical reasoning to natural-language QA. This further supports our claim that leveraging temporal dynamics constitutes broadly applicable principle for improving diffusion language models.
E.2 TRAINING A UNIFIED MODEL ACROSS MULTIPLE TASKS
To evaluate the generalization ability of our approach, we validate the proposed method in a multi- task setting. Specifically, we jointly train single model on four datasets: GSM8K, MATH500, SVAMP, and Countdown. To ensure balanced training across tasks, we subsample each dataset so that the number of training examples is equal for all tasks.
Table S6 summarizes the results. Across all four benchmarks, our combined reward method consistently outperforms the supervised fine-tuning baselines. On GSM8K, we observe clear gains at shorter sequence lengths, for example, with LLLDA-1155 on length 128, accuracy improves from 69.8 to 73.7, yielding +3.9 points. SVAMP, similar short-sequence gains are observed, from 81.7 to 88.7. a +7.0 points increase. For the more challenging MATH500, the best improvement occurs at LLaDA-1.5, length 512, where performance increases from 35.4 to 41.6 (+6.2 points). Finally, on Countdown, method achieves dramatic improvements, for instance, with LLaDA-1.5 at length 512, accuracy climbs from 20.7 to 50.4, a striking +29.7 points.
These results demonstrate that our method not only scales well across different reasoning domains but also enhances robustness under varying sequence lengths, confirming its effectiveness as a unified reward design for multi-task learning.
22

---

Published as a conference paper at ICLR 2026
Table S7: Ablation results replacing TSE with alternative metrics, including negative average token entropy (ATE) and pairwise agreement (PA) between consecutive diffusion steps. Bolded values indicate the best performance among the three methods.
GSM8K
MATH500
SVAMP
Countdown
Method / Seq Len
512
128
256
512
128
256
512
baseline
70.2
78.7
80.1
23.8
34.4
36.8
81.7
83.3
82.7
21.5
19.9
21.5
negative TSE reward
72.2
78.8
80.2
30.6
34.6
38.0
84.3
89.0
88.7
38.6
53.5
44.9
negative ATE reward
69.1
77.7
79.5
27.4
32.2
37.2
82.7
85.3
85.0
22.9
23.4
30.5
PA reward
71.3
78.9
79.7
31.0
33.0
37.6
83.0
87.3
86.7
32.0
44.7
41.3
TSE & Accuracy reward
72.1
80.0
83.0
31.2
35.4
41.4
85.0
90.3
92.3
41.5
42.6
54.7
ATE Accuracy reward
70.2
77.2
34.8
38.4
84.7
85.3
28.1
29.7
35.9
PA & Accuracy reward
71.0
79.2
81.4
32.0
33.8
39.6
85.0
89.7
88.7
36.9
43.4
50.5
E.3 POSSIBLE ALTERNATIVES TO TEMPORAL SEMANTIC ENTROPY
Our method primarily adopts Temperal Semantic (TTr as the underlying mechanism for temporal consistency modeling. However, TSE is not the only viable choice. To contextualize our design and clarify the landscape of alternatives, we examine two additional possible approaches: pairwise agreement and token entropy.
To measure temporal consistency, natural idea is to quantify how frequently the predicted answer changes along the sampling trajectory. Motivated by this intuition, introduce ChangeRate as metric for assessing the model's 's confidence and stability.
During decoding, we obtain sequence of intermediate predictions, denoted as {x]]T= Based on this sequence, we define ChangeRate as the fraction of consecutive steps that produce the different answer:
ChangeRate T Σ
1 = x
(S2)
A higher ChangeRate indicates that the model's predictions vary more frequently across successive steps, signaling lower temporal consistency. For completeness, we define pairwise agreement (PA)
as the complement of ChangeRate, PA 1 ChangeRate, and employ it as an RL reward to promote temporally consistent predictions.
Similar to TSE, we compute the ChangeRate for correct answers across different output lengths, with the results shown in Fig. S5. We observe that, for correctly answered questions, the ChangeRate tends to be lower. However, unlike TSE, ChangeRate does not exhibit significant variation across datasets. That is, whether the model performs well or poorly on a given dataset, the ChangeRate does show clear pattern. In contrast, TSE is noticeably lower on easier datasets such as GSM8K and SVAMP compared to more challenging ones like MATH500 and Countdown, as shown Sec. 3.3.
In addition, we propose using the answer entropy, which is computed from the token distribution of the predicted answer, as a supervisory signal. Although this entropy measure cannot directly supervise temporal consistency in the answer trajectory. it still provides a meaningful indication of how confident the model in its prediction. As shown Fig. 3, higher entropy corresponds to lower confidence, while lower entropy suggests that the model is more certain about the generated answer.
As shown in Table S7, observe that: (1) Using negative token entropy the reward yields moderate improvement, indicating that local confidence is useful but insufficient. Gains are consistently smaller than with TSE; (2) The pairwise agreement reward performs better than token entropy and comparable to TSE.
Intuitively, PA captures dynamic stability; for example, it can tell that "010101" is less stable than "000111," but it cannot differentiate between semantically diverse outputs such as "010203" and "010101." In contrast, TSE measures the semantic dispersion of intermediate predictions, capturing
23

---

Published as a conference paper at ICLR 2026
Table S8: Ablations on scoring rules for combining TSE with accuracy reward. We compare different scoring rules, including entropy scoring, quadratic scoring, spherical scoring, and logistic scoring.
Model/ Dataset
GSM8K
MATH500
SVAMP
Countdown
Method Seq
128
256
512
128
256
512
128
256
512
128
256
512
LLaDA
haseline
70.2
78.7
80.1
23.8
34.4
36.8
81.7
83.3
82.7
21.5
19.9
21.5
entropy
71.7
78.5
82.3
31.6
38.2
39.2
89.3
47.6
50.0
53.1
quadratic
71.3
79.9
82.0
31.0
37.6
89.3
50.0
34.4
48.1
+RFT
logistic
73.0
79.5
81.1
31.6
36.8
39.4
87.7
87.3
90.7
46.5
37.9
51.2
spherical
72.1
80.0
83.0
31.2
35.4
41.4
85.0
90.3
92.3
41.5
42.6
54.7
LLaDA-1.5
buseline
79.4
81.1
29.0
32.4
35.4
21.5
21.1
20.7
entropy
71.2
79.2
83.4
27.8
36,6
41.2
86.7
90.3
89.0
44.1
45.3
58.6
quadratic
73.4
79.1
83.2
29.2
33.2
42.4
86.0
86.7
90.0
43.4
45.3
57.4
+RIT
logistic
71.2
79.2
83.1
30.2
33.8
41.0
88.3
89.0
91.0
48.8
46.9
61.3
spherical
73.2
80.5
84.0
29.6
35.4
41.4
90.3
89.0
44.5
46,9
63.3
how the meaning of partial answers drifts along the trajectory, but it is less sensitive to fine-grained dynamical fluctuations among semantically similar states.
We believe an ideal compstel-cnsistteeyyy metric should integrate the complementary strengths of PA and TSE, leveraging PA's sensitivity dynamics and TSE's sensitivity semantic variation, and we plan to explore such hybrid rewards future work.
E.4 ABLATIONS ON SCORING RULES FOR COMBINING TSE WITH ACCURACY REWARD
In Sec. 4.2, we combine TSE with the accuracy reward using the proper scoring rule (Gneiting & Raftery, 2007). The purpose of these scoring rules is to promote truthful confidence assessments: they attain their minimum value when the predicted confidence e precisely mirrors the actual probability that the model's output corresponds to the correct answer o². An exception is the spherical version we use, which consistently encourages both conrectness and higher certainty, without penalizing overconfidence in incorrect predictions. Here, we conduct ablation study on different scoring rules. Specifically, we consider the following four forms:
Entropy scoring:
ent 10 c(oi))
Quadratic scoring:
quad = 1 10 - (c(o.) - 1 12==0)²² rs
Logistic scoring:
los 1a 1a. log(c(os)) + 1o,, -- log(1 c(o:))
c(oi)
Spherical scoring: sphe 10,
c(o:))
All four proposed reward functions aim to jointly encourage correctness and temporal self- consistency. Notably, the y function gives a reward for incorrect answers, while for correct answers, the reward is given by c(o.) Kmay-TSB(o.) where Hmux log T. This reward reaches maximum value of when all sampling steps yield the correct answer. The remaining three functions, yuuad r and sphe correspond the commonly used quadratic scoring, logarithmic scoring, and spherical scoring in proper scoring rules (Gneiting & Raftery, 2007), respectively. We report the performance of all four reward combination methods in Table S8. By default, we use the spherical scoring rule because it demonstrates more superior results compared to the alternatives.
Haaa
E.5 TEMPORAL SELF-CONSISTENCY VOTING AFTER RFT
is worthwhile to investigate whether Temporal Self-consistency Voting continues to provide performance benefits after Temporal Consistency Reinforcement. To this end, conducted experiments using two models: the first derived from the LLaDA-SB -B-Inrttruct model trained with the negative TSE reward, and the second is trained using the accuracy reward combined with TSE.
24

---

Published as a conference paper at ICLR 2026
Table S9: Performance of temporal majority voting after reinforcement learning. Temporal self-consistency voting was applied to the model fine-tuned via temporal consistency reinforcement. The upper part is derived from the LLaDA-8B-Instru reward, whereas the lower part is based on the model trained with a combination of TSE and aceuracy reward. For reference, we include the oracle EverPass @1 as a performance upper bound.
model trained using the Negative TSE
GSM8K
MATH150
SVAMP
Countdown
Method / Seq Len
128
256
512
128
256
512
128
256
512
128
256
512
Negative TSE reward
After RFT
72.2
78.8
80.2
30.6
34.6
38.0
84.3
89.0
88.7
38.6
53.5
Fixed Weighting
70.4
77.6
80.2
30.8
34.4
37.6
85.0
88.3
88.7
39.5
53.5
45.7
+ Temmorrl Voting
Weighting
72.3
78.8
35.4
Exp. Weighting
72.6
79.2
80.6
30.8
35.0
38.0
85.3
89.0
88.7
39.5
53.9
45.7
+0.4
+0.9
EverPoss 01l
80.5
81.8
81.3
33.4
37.4
40.0
88.7
90.7
89.7
44.1
59.8
55.5
TSE & accuracy reward
Afer RFT
72.1
80.0
83.0
31.2
35.4
41.4
85.0
90.3
92.3
41.5
42.6
54.7
Fixed Weighting
71.2
79.6
82.8
31.2
35.6
41.2
85.3
90.7
92.3
40.6
41.5
53.2
Temporal Voting
Lincar Weighting
73.0
81.9
83.0
36.0
41.0
92.7
40.9
42.8
54.2
Exp. Weighting
72.6
81.1
83.3
31.6
35.8
41.4
86.3
90.7
92.3
41.5
42.7
55.1
+1.1
+0,4
+0.0
+0.0
EverPass R1 |
84.0
91.4
87.9
43.6
49.2
52.0
90.7
91.7
92.7
54.3
68.0
70.3
We applied Temporal Self-Consisee Coff-Consistency Voting both models, and the results are summarized in Table $9, The findings indicate that both models still exhibit performance improvements when temporal voting is applied, even after undergoing reinforcement learning. This suggests that Temporal Self-Consistency Voting and Temporal Consistency Reinforcement are complementary techniques, and can be effectively combined further enhance model performance.
E.6 EXAMPLES OF TIME OSCILLATION
We present representative examples of temporal oscillation from GSM8K, using LLaDA-8B-Innntane as the backbone model. Note that all examples shown below were ultimately incorrect but were classified as correct answers by our voting method, as described in Sec. 4.1. Correct answers are in green, incorrect answers in red. We use blue to highlight the key segments in each example.
We observe that many questions become incorrect during temporal oscillation due to flawed reasoning paths. Although the model may initially provide the correct answer and follow a logical reasoning path, an incorrect trajectory can ultimately lead to an erroncous response.
A notable case is seen in Example 3. Initially, the model correctly calculates the total cups, per-hour revenue, and the per-hour cost deduction to derive the right profit per hour (step 54). However, during reasoning, it introduces erroncous masking regarding key calculation elements the per- hour cost to subtract. Even though the answer remains correct at first (step 61), the flawed reasoning trajectory eventually results in miscalculations and an incorrect final response (step 62).
25

---

Published as a conference paper at ICLR 2026
Example
Ruby is 6 times older than Sam. years, Ruby will be times as old Sam. How old is Sam now?
Problem:
Cornertness
16 70
24
28
36
40
44
48
56
Sampling Step
Step 59:
<reasoning>>
Let Sam's age be x years. Then Ruby's age is 62 years. In 9 years, Ruby's age will be 62 + and Sam's 's age will be According the problem, in 9 years, Ruby will be 3 times as old as Sam, so we have the equation 6z + 9 3(x 9). Simplifying this equation, we get 3z 0 so
</reasoning> <answer>
Janswer>>
Step 60:
<reasoning>>
Let Sam's age be years. Then Ruby's age is 6ær years. In 9 years, Ruby's age will be 6z 9 and Sam's s age will be According to the problem, in 9 years, Ruby will be 3 times as old Sam,
have the equation 6z 3(x 9). Simplifying this equation, get
3z = 0 x==
</reasoning>
Kanswer>
</answer>
Example
Problem:
Shiela bought five cell phones for $150 each for 3-month installment. A 2% interest will be charged for each unit. How much will Shiela pay each month for months
Conrectuess
28
48
56
Sampling Step
Step 59:
<reasoning>>
Shiela bought five cell phones for $150 each, so the total cost is $5 x 150 = $750. The
interest for each unit is 2%. the total interest is $750 0.02 $15. SincelELa will pay this amount over 3 months, the monthly payment is $750 $15 = =<<< >> 750) 255. </reasoning>
<answer>>
255
</answer>
Step 60:
<reasoning>>
Shiela bought five cell phones for $150 each, so the total cost $5 x 150 = $750. The
interest for each unit is 2%, so the total interest is $7.0 $15. SincelELa will pay this amount over 3 months, the monthly payment is $750 + $15 = $<< 750/3 =>> 250. </reasoning>>
<answer>>
250
</answer>>
26

---

Published as a conference paper at ICLR 2026
Example
Millic decides to open lemonade stand. She spends $18 to buy enough supplies make 3 pitchers of lemonade. Each pitcher holds 12 eups of lemonade. She sells each cup of lemonade for S1. She sells an average cups per hour that her lemonade stand open. If Millie sells all of the lemonade, how much profit will she make per hour that she spends running the lemonade stand?
Problem:
Sampling Step
Step 54:
<reasoning> Millie spends $18 on supplies to make 3 pitchers of lemonade. Each pitcher holds 12 cups, so she has a total of 12 = 36 cups of lemonade. She sells each cup for $1 and and sells average of 4 cups per hour. Therefore, her revenue per hour is $ S1 $4. To find per hour profit, we subtract the cost of supplies from the revenue: $4 - $2 $2 per hour.
</reasoning>
<answer>
</answer>
Step 61:
<reasoning>>
Millie spends $18 on supplies to make 3 pitchers of lemonade. Each pitcher holds 12 cups, so she has total of 12 = 36 cups of lemonade. She sells each cup for $1 and and sells an average of 4 cups per hour. Therefore, her revenue per hour is 4 S1 $4. To find per hour profit, we subtract cost supplies from the revenue: $4 $3 $2 per hour. </reasoning>
<answer>
<Janswer>
Step 62:
<reasoning>>
Millie spends $18 on supplies make 3 pitchers of lemonade. Each pitcher holds 12 cups, so she has a total of 3 12 = 36 cups of lemonade. She sells each cup for $1 and and sells an average of cups per hour. Therefore, her revenue per hour * S1 $4. To find per hour profit, we subtract the cost of supplies from the revenue: $4 - $3 $1 per hour. </reasoning>
<answer>
< Zanswer>>
27


Reviews:
[{"review_id": "cfQ1Ak1htO", "reviewer": "Reviewer_CBgE", "summary": "This paper investigates temporal dynamics in diffusion language models (dLLMs), highlighting that correct intermediate predictions are often lost during later denoising steps. The authors identify a “temporal oscillation” phenomenon and propose two complementary methods: (1) Temporal Self-Consistency Voting, a test-time decoding strategy that aggregates predictions across steps, and (2) Temporal Consistency Reinforcement, a post-training method using Temporal Semantic Entropy (TSE) as a reward to encourage stable generations. Experiments show consistent improvements across reasoning benchmarks, suggesting that temporal signals within diffusion decoding are a valuable yet underexplored source of information.", "questions": "1. In Figure 3, is the average token-level entropy computed over the last-step tokens only or all tokens across trajectories?\n\n2. In Figure 4, why not separately report metrics for \"finally correct\" versus \"intermediate correct\" cases to better illustrate temporal oscillation behavior?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a new metric, Temporal Semantic Entropy (TSE), to quantify semantic fluctuation across denoising steps, revealing meaningful patterns in dLLMs.\n\n2. Proposes both training-free (voting-based decoding) and training-based (RL with TSE reward) methods, offering complementary ways to leverage temporal consistency.\n\n3. Demonstrates that TSE can serve as a soft reward signal for RL even in unlabeled scenarios, expanding applicability to broader settings.", "weaknesses": "1. In Table 2, combining accuracy reward and TSE sometimes leads to negative effects under RFT, indicating potential instability in multi-reward optimization.\n\n2. After RFT, the model’s everPass@1 performance decreases for some tasks (MATH500 and SVAMP), as shown in Table 1 and S4.\n\n3. Using TSE as an RL reward requires semantic clustering, which introduces extra computation overhead not discussed in the paper.\n\n4. The paper lacks clarity about which decoding strategy is used in the analysis experiments (e.g., random remarking, low-confidence remasking, or block decoding). It appears analysis assumes full diffusion (len=128, steps=64), whereas RL experiments adopt semi-autoregressive decoding—raising concerns about experimental consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PhZ2bDSSZO", "reviewer": "Reviewer_TojT", "summary": "This paper investigates temporal oscillation in diffusion language models—where correct intermediate answers are later overwritten—and proposes two methods, Temporal Self-Consistency Voting and Temporal Consistency Reinforcement, to leverage this phenomenon. The work is well-motivated and empirically strong, showing consistent improvements across benchmarks, with large gains on Countdown. However, methodological novelty is limited: TSCV mainly adapts prior self-consistency ideas, and the proposed TSE metric does not directly capture temporal dynamics but instead reflects semantic stability. The related work section could be more focused on studies addressing similar temporal behaviors. Overall, solid experiments and clear presentation justify a weak accept.", "questions": "Q1 The claimed “temporal oscillation” phenomenon requires more systematic experimental validation to support its research significance. Autoregressive models can also produce correct intermediate answers but incorrect final outputs, typically due to decoding uncertainty or local likelihood fluctuations. Similarly, oscillations observed in diffusion models may simply result from randomness or resampling noise during denoising. To justify the proposed methods, the authors should provide additional evidence showing that the phenomenon is structural and exploitable rather than a random artifact.\nQ2 The strong performance of TSE-based reinforcement is primarily observed on the Countdown dataset, while its effects on GSM8K, MATH500, and SVAMP are comparable to those of TSCV. If TSE is only effective for specific tasks or distributions, its generalizability remains questionable. The authors are encouraged to extend their experiments to a broader range of datasets and task types to better demonstrate the method’s robustness and applicability.\nQ3 The Related Work section could be revised to better connect with the paper’s central theme. In particular, the authors are encouraged to compare their study with more recent works that explicitly discuss similar temporal phenomena in diffusion models. For example, Diffusion Language Models Know the Answer Before Decoding also reports that correct answers often appear in intermediate denoising steps but are overwritten in later iterations—a phenomenon conceptually similar to the “temporal oscillation” described in this paper.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1 This paper proposes leveraging the rich intermediate hidden states generated during the diffusion process—not only for inference-time aggregation but also for post-training reinforcement. This breaks away from the conventional paradigm that focuses solely on the final denoised output and opens a new avenue for understanding and enhancing dLLMs.\nS2 The authors define Temporal Semantic Entropy to quantify semantic stability during the generation process and further incorporate it as a self-supervised reward signal in reinforcement learning. This method consistently improves performance across datasets, with especially notable results on the Countdown task, validating the effectiveness of temporal consistency signals in improving both stability and accuracy.\nS3 The experimental results are strong and convincing. The proposed methods deliver clear and consistent gains across multiple reasoning benchmarks (GSM8K, MATH500, SVAMP, Countdown) and across different model scales (LLaDA-8B, LLaDA-1.5). In particular, the Temporal Consistency Reinforcement achieves up to 25 percentage-point improvement on the Countdown dataset, substantially outperforming prior diffusion-based baselines.\nS4 The evaluation is comprehensive and well-controlled, including ablations over weighting schemes, reinforcement reward formulations, and entropy metrics. The improvements are not marginal but significant and stable across output lengths and datasets, demonstrating the general applicability of the proposed approach.", "weaknesses": "W1 The proposed Temporal Self-Consistency Voting essentially adapts the self-consistency idea from Self-Consistency Improves Chain-of-Thought Reasoning in Language Models (Wang et al., 2022) to diffusion models. This adaptation lacks substantial algorithmic innovation, and the observed performance gains are smaller than those achieved in autoregressive models—raising doubts about the method’s unique contribution within the diffusion framework.\nW2 The proposed TSE metric is based on semantic clustering to measure semantic variation across denoising steps. However, this metric reflects the stability of latent semantic distributions rather than true temporal dynamics. Since the clustering process depends on embedding space similarity rather than the evolution of diffusion timesteps, TSE may not adequately represent temporal consistency.\nW3 In Section 4.1, the paper attributes the efficiency and scalability of Temporal Self-Consistency Voting to its design. However, these “efficient” properties largely arise from the inherent parallel generation architecture of diffusion models rather than the proposed method itself. In other words, the ability to obtain multiple intermediate outputs within a single sampling trajectory is a natural feature of diffusion mechanisms, not a unique contribution of this approach.\nW4 The Related Work section contains a substantial amount of content describing the general development of diffusion models and reinforcement learning variants, but these discussions are only tangentially related to the paper’s specific focus on temporal oscillation and consistency. This makes the literature review feel somewhat disconnected from the central contribution and could be tightened to highlight works directly addressing intermediate-step dynamics or temporal consistency.\nW5 While the experimental results are quantitatively strong, the paper provides limited qualitative or diagnostic analysis of why temporal oscillation occurs and under what conditions correct intermediate answers are lost. Without a clearer causal explanation or visualization of representative trajectories, it remains difficult to interpret whether the proposed improvements stem from genuine temporal stabilization or simply from additional ensembling and regularization effects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PoxtMByb8U", "reviewer": "Reviewer_B4qA", "summary": "This paper reveals a critical phenomenon for diffusion language models: temporal oscillation. The authors observed that when answering a question, diffusion language models may generate a correct answer in the middle but replace it with an incorrect answer in the final output. The authors also conducted an empirical study to quantify this phenomenon. Based on this observation, the authors proposed two methods to enhance performance for diffusion language models: temporal self-consistency voting, a training-free majority voting method with minimal computational overhead; and temporal consistency reinforcement, a GRPO reinforcement learning algorithm that improves semantic consistency during post-training. For both methods, the authors observe improvements in accuracy across four reasoning benchmarks.", "questions": "* My biggest question is how the authors extract answers for intermediate token sequences. This is confusing because for LLaDA, once a token is decoded, it does not change throughout the sampling process, unless predictor-corrector samplers such as ReMDM are used. Thus, it is counterintuitive that the time oscillation has such a significant influence on the model's performance, as shown in Figure 1(a). Besides, it is not clear how to extract an answer from a partially masked token sequence such as \"[MASK] [MASK] the [MASK]\". Can the authors explain this in more detail, because it is clear that this is vital for both temporal self-consistency voting and the computation of TSE.\n* Can the authors explain why TSE is chosen as the metric to use? The current explanation is Figure 4, but such a phenomenon holds true for many other metrics. For temporal consistency reinforcement, how will the performance be if TSE is replaced with token entropy? If they perform similarly, it is not clear why we should choose TSE. If they perform differently, can the authors provide some justification for why TSE is a good metric?\n* What algorithms do the authors use for clustering when computing TSE? Is TSE robust to different clustering algorithms? \n* Does temporal consistency reinforcement improve ever pass as well? What are the results if we apply temporal self-consistency voting to both the original checkpoint and the checkpoint after consistency reinforcement?\n* In previous works such as d1, and wd1, Sudoku is also a commonly used benchmark. Can the authors provide results on the Sudoku benchmark to further justify the method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* This paper is well-written and easy to follow.\n* This paper handles an interesting phenomenon that is distinct for diffusion language models. The authors conducted a quantitative analysis of temporal oscillations and proposed practical guidelines for leveraging them to improve the reasoning performance of diffusion language models.\n* The experimental section of this paper is relatively abundant, with many ablations and extra analyses.", "weaknesses": "* The mathematical notation system is not rigorous and self-consistent. In equation 1, the authors tried to demonstrate the sampling algorithm of LLaDA-like diffusion language models. Although I understand what the authors were trying to convey, Eq. 1 is incorrect and confusing from a probabilistic perspective, and I strongly advise the authors to revise its presentation. Throughout the paper, the authors made no distinction between scalar values and tensors. For example, the $x_t$ in Eq. 1 represents a token sequence and is neither bolded nor indicated as a tensor. In Eq.3, it is not clear what is meant by $p(x)$, despite its importance. In Eq. 5, a minus sign is missing when it means a loss, but not when it means the policy gradient.\n* In Figure 4, it was not indicated what the error bars meant. I do not understand why the error bar for the correct column for SVAMP dips below 0, since the entropy values should not be negative.\n* In Table 1, the improvements of temporal majority voting seem marginal. I think more justifications, such as error bars, are needed to prove that these improvements are real but not random noise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MTJftSQtnV", "reviewer": "Reviewer_39aQ", "summary": "The paper addresses temporal oscillation in dLLMs, where correct answers often appear during intermediate denoising steps but are later overwritten. Building on insights from self-consistency and semantic entropy, the authors propose two complementary approaches: (1) Temporal Self-Consistency Voting, a training-free decoding method that aggregates intermediate predictions, and (2) Temporal Consistency Reinforcement, a post-training method using GRPO with Temporal Semantic Entropy as a reward to promote stable generations. Experiments on mathematical reasoning benchmarks demonstrate consistent improvements, with particularly strong gains when TSE is combined with accuracy-based rewards.", "questions": "- Since temporal oscillation arises from the combination of remasking and incorrect denoising, it could be in principle be mitigated by a modified diffusion schedule. How much of the observed oscillation is inherent to dLLMs (and thus requires methods like those proposed here), versus being reducible by scheduling or inference design choices?\n- The use of TSE in this work is empirically motivated, and it would be possible to consider other types of temporal consistency metrics or rewards, for example, averaging the pairwise agreement between consecutive timesteps of a (later) denoising trajectory. What is the empirical advantage of using entropy-based methods over such alternatives?\n\nMinor typos/comments\n- p. 8 Paragraph \"Ablations on Voting Strategies.\" Fig. 5b -> Fig. 5a\n- p. 4 Eq (1). The use of $x_0$ on the RHS is inconsistent with the LHS. Given the later definition of $x_0^t$, it would be more precise to write $\\sum_{x_0^t} q(x_{t-1}|x_0^t) p_\\theta (x_0^t | x_t)$? Also, the later decoding step would be $x_0^t \\sim p_\\theta(x_0^t | x_t)$.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The framing of temporal oscillation as a central issue in dLLM decoding and devising a way of extracting signal from the intermediate outputs is both novel and timely given the emerging importance of dLLMs.\n- This work presents strong empirical evidence, including ablation studies on reward design and weighting strategies.\n- Newly introduced metrics, such as EverPass and TSE, are intuitive and well-motivated, with their significance demonstrated through clear and quantitative analysis.", "weaknesses": "- While the evaluation spans four datasets (GSM8K, MATH500, SVAMP, Countdown), this focuses primarily on mathematical reasoning tasks. Extending the analysis to other domains (e.g., natural language QA) would strengthen the case for the generalizability of this approach.\n- The success of the introduced strategies, inference-time voting as well as RL fine-tuning, appears to rely heavily on the baseline model's accuracy, which may suggest limited gains for harder reasoning tasks to expand the applicability of dLLMs broadly (although this point is mentioned in Appendix D.1).\n\nI also note the existence of a concurrent work (https://arxiv.org/abs/2508.19982) on closely related problem and method, while it assumes early stabilization of the prediction and proposes stopping criteria for training-free approach. This is not considered in this review, since they are concurrent works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)