TEST-TIME SCALING IN DIFFUSION LLMS VIA HIDDEN SEMI-AUTOREGRESSIVE EXPERTS

## Abstract

Abstract Diffusion-based large language models (dLLMs) are trained flexibly to model extreme dependence in the data distribution; however, how to best utilize this information at
inference time remains an open problem. In this work, we uncover an interesting
property of these models: dLLMs trained on textual data implicitly learn a
mixture of semi-autoregressive experts, where different generation orders reveal
different specialized behaviors. We show that committing to any single, fixed inference
time schedule, a common practice, collapses performance by failing to
leverage this latent ensemble. To address this, we introduce HEX (Hidden semiautoregressive
EXperts for test-time scaling), a training-free inference method that
ensembles across heterogeneous block schedules. By doing a majority vote over
diverse block-sized generation paths, HEX robustly avoids failure modes associated
with any single fixed schedule. On reasoning benchmarks such as GSM8K,
it boosts accuracy by up to 3.56× (from 24.72% to 88.10%), outperforming top-K
margin inference and specialized fine-tuned methods like GRPO, without additional
training. HEX even yields significant gains on MATH benchmark from
16.40% to 40.00%, scientific reasoning on ARC-C from 54.18% to 87.80%, and
TruthfulQA from 28.36% to 57.46%. Our results establish a new paradigm for test-time scaling in diffusion-based LLMs (dLLMs), revealing that the sequence in which masking is performed plays a critical role in determining performance during inference.

## 1 Introduction

Diffusion-based large language models (dLLMs) are (^†^†footnotetext: ^∗Equal contribution.) rapidly emerging as a promising alternative to traditional autoregressive LLMs generalizing beyond the next token prediction [2]. Unlike autoregressive models, dLLMs generate text via an iterative mask-and-unmask process, allowing them to decode tokens in essentially arbitrary order [1]. This fundamental change in the generation mechanism during training grants dLLMs remarkable flexibility at inference time. In fact, recent dLLMs have already demonstrated competitive (and sometimes superior) performance compared to their autoregressive counterparts on a similar scale [3]. These early successes indicate that the masking strategy during inference plays a crucial role.

Gaps in our understanding about dLLMs.
The freedom to choose the generation order, the masking strategy, is the central advantage of dLLMs. Recent works [1, 2] have tried to harness this flexibility by relying on prediction confidence, progressively unmasking high-confidence tokens (top-K margin in Figure [1](https://arxiv.org/html/2510.05040v1#S0.F1)).

However, such an approach often leads to inherently biased solutions as they overlook the crucial sequential structure present in the language training data, which induces implicit biases in the learned masking strategies. As a result, these methods might perform worse than random unmasking, as shown in Figure [1](https://arxiv.org/html/2510.05040v1#S0.F1). Why this happens and what we can do to avoid such a failure remains an open question in the literature. In this work, we take a first step towards resolving this.

Our key finding: hidden semi-autoregressive experts. We uncover a new dimension of test-time scaling for diffusion LLMs, centered on the masking strategy. We find that the effective design of a masking strategy is influenced by the structural biases encoded in the training data. The sequential nature of language data, for instance, causes dLLMs to implicitly learn a mixture of semi-autoregressive experts during training. Each of these hidden “experts” is biased toward distinct masking distributions, often favoring those that reflect left-to-right or autoregressive-like generation orders.
We, for the first time, demonstrate that this latent mixture can be deliberately accessed during inference. By varying the block size used in semi-autoregressive decoding, we can activate different experts, mirroring the conditions the model saw during training. This insight unlocks a novel method for test-time scaling in dLLMs. By marginalizing across these block schedules, we can exploit the latent ensemble of experts, resulting in significantly more robust and optimal inference as shown in Figure [5](https://arxiv.org/html/2510.05040v1#S6.F5).

Hence, we propose HEX (Hidden semi-autoregressive EXperts), a training-free inference method that uncovers a new dimension of test-time scaling for dLLMs. HEX marginalizes across block schedules, treating block size and order as latent variables that define an additional scaling dimension, and aggregates predictions via majority voting. In doing so, it robustly avoids the pitfalls of committing to any single decoding path, turning dLLMs’ hidden flexibility into a principled mechanism for test-time scaling. We summarize our contributions as follows.

(i) Highlighting a Key Limitation in dLLM Inference. We identify and analyze a key limitation of existing inference strategies for dLLMs. We show that current methods can lead to suboptimal performance and generation instability because they overlook implicit structural biases that the model learns during training. This pinpoints a crucial, unaddressed aspect of dLLM generation (Section [4](https://arxiv.org/html/2510.05040v1#S4)).

(ii) HEX: New dimension of test-time scaling in dLLMs. Our key insight is that dLLMs implicitly learn a mixture of semi-autoregressive experts, and block scheduling helps to uncover this latent structure. Based on this finding, we introduce HEX, a novel, training-free inference algorithm specifically designed to leverage this latent structure by ensembling diverse semi-autoregressive schedules with a majority vote aggregation (Algorithm [2](https://arxiv.org/html/2510.05040v1#alg2)), turning ordering into a new test-time scaling dimension for dLLMs. (Section [5](https://arxiv.org/html/2510.05040v1#S5)).

(iii) Comprehensive experimental analysis: matching GRPO-level performance.
HEX achieves GRPO-level results on GSM8K, MATH, ARC-C, and TruthfulQA, without retraining, establishing test-time scaling as a powerful new paradigm for diffusion LLMs. HEX outperforms existing state-of-the-art inference methods [1] on reasoning tasks, boosting accuracy by up to 3.56× (from 24.72% to 88.10%.)
HEX even produces massive gains on more challenging tasks, including MATH [4] (from 16.40% to
40.00%), ARC-C [5] (from 54.18% to 87.80%),
and TruthfulQA [6] (from 28.36% to 57.46%). (Section [6](https://arxiv.org/html/2510.05040v1#S6)).

## 2 Related Work

Diffusion Large Language Models.
Diffusion models have achieved state-of-the-art performance in image generation [7, 8], and recent advances extend them to the discrete domain of language. The early approaches applied continuous diffusion to latent text representations [9, 10, 11], but faced challenges with scalability and discretization. A masked diffusion paradigm soon emerged as a more tractable discrete alternative [12, 13], with large-scale implementations such as DiffuLLaMA [14], Dream [15] and LLaDA [2] demonstrating that diffusion LLMs (dLLMs) can rival similarly sized autoregressive models, even on complex reasoning [3, 16]. This potential extends even to multimodal understanding [17, 18, 19, 20, 21].

Inference-Time Methods for dLLMs.
In autoregressive models, inference-time scaling has been extensively studied, ranging from chain-of-thought prompting [22] and self-consistency [23] to scaling the allocation of test-time compute [24]. In contrast, inference-time methods for dLLMs remain sparse [25, 26]. Most gains in dLLM performance so far have come from training-time improvements, such as applying GRPO [3, 16] or post-training method such as temporal consistency reinforcement [26]. Additional discussion is provided in Appendix [B.1](https://arxiv.org/html/2510.05040v1#A2.SS1).

## 3 Problem Formulation

Masked Diffusion Language Models (MDM).
Let $x=(x_{1},\ldots,x_{n})\in\mathcal{V}^{n}$ be a length-$n$ token sequence over vocabulary $\mathcal{V}$. A masked diffusion large language model (dLLM) specifies a conditional denoiser $p_{\theta}(x[M]\mid x[M^{c}])$ for any mask $M\subseteq[n]$, where $M^{c}=[n]\setminus M$. The notation $x[M]$ is defined as the subsequence of tokens from $x$ on the indices of $M$, i.e. $x[M]=(x_{i})_{i\in M}$, where $x[M]$ denotes the masked tokens in $x$. In the forward (corruption) process, a random subset of tokens $M\subseteq{[n]}$ is masked (replaced by a special symbol [MASK], and model $p_{\theta}$ is tasked with recovering the original tokens in $M$ given the unmasked tokens in the complement ${M^{c}}:=[n]\setminus M$. Formally, for a random mask pattern $M$, the model produces a conditional distribution $p_{\theta}(x[M]\mid x{[{M^{c}}]})$ on the masked tokens. The training objective is to maximize the likelihood of ground-truth tokens in these masked positions. Hence, the training problem can be written as

$$ $\theta^{*}\in\arg\min_{\theta}\mathcal{L}_{\text{mask}}(\theta):=\mathbb{E}_{x\sim\mathcal{D}}\mathbb{E}_{\ell\sim\text{Unif}([n])}\,\mathbb{E}_{M\subseteq[n],|M|=\ell}\Big[-\sum_{i\in M}\log p_{\theta}(x_{i}\mid x{[{M^{c}}]})\Big],$ (1) $$

where $\mathcal{D}$ is the data distribution, $\ell\sim\text{Unif}([n])$ is a uniformly sampled number of masked tokens $\ell\in\{1,\dots,n\}$ and $M\subseteq[n]$ is the randomly selected subset of length $|M|=\ell$. The summation in ([1](https://arxiv.org/html/2510.05040v1#S3.E1)) runs over all masked

Figure: Algorithm 1 Vanilla MDM Inference

token positions $i\in M$, and the loss on each such position is the negative log-likelihood of the true token $x_{i}$ given the remaining context (unmasked) $x{[{M^{c}}]}$. The objective in ([1](https://arxiv.org/html/2510.05040v1#S3.E1)) trains the model to predict randomly masked-out tokens, and can be viewed as averaging next-token losses over all token permutations, i.e., an any-order objective [2].

Inference as the Core Challenge.
After learning $\theta^{*}$ from ([1](https://arxiv.org/html/2510.05040v1#S3.E1)), generation happens step by step. For instance, for a given prompt $x_{\text{prompt}}$, it starts by selecting the number of tokens to be generated (say $L$), and then requires choosing *how* to reveal tokens. Let a *decoding trajectory* be a sequence of masks $\tau=(M_{1},\ldots,M_{T})$ that partition $[n]$ (such that $\bigsqcup_{t=1}^{T}M_{t}^{c}=[n]$), with per-step sizes $\ell_{t}=|M_{t}|$. At step $t$, the model predicts all masked tokens conditioned on the currently revealed context $x\big[\cup_{s=1}^{t-1}M_{s}^{c}\big]$. For a fixed trajectory $\tau$ and prompt $x_{\text{prompt}}$, a functional of natural log-likelihood is

$$ $\mathcal{J}(\tau;\theta\mid x_{\text{prompt}})=\sum_{t=1}^{T}\ \sum_{i\in M_{t}}\log p_{\theta^{*}}\!\left(x_{i}\mid x_{\text{prompt}},x\Big[\,\bigcup_{s=1}^{t-1}M_{s}^{c}\Big]\right).$ (2) $$

We summarize the vanilla inference procedure in Algorithm [1](https://arxiv.org/html/2510.05040v1#alg1). The ideal (but empirically intractable(^0^00Because simply making consecutive greedy choices does not guarantee that the overall $J$ is maximized, and performing a global search would require evaluating all possible trajectories, which is practically infeasible.)) goal is to choose $\tau$ such that we obtain a sample which maximizes $\mathcal{J}(\tau;\theta\mid x_{\text{prompt}})$.

Because ([1](https://arxiv.org/html/2510.05040v1#S3.E1)) trains on *all* mask patterns, many training subproblems are intrinsically ill-posed (e.g., extremely large masks with scant context) due to the implicit sequential bias in the language training distribution. For example, some conditionals are rarely observed or provide little meaningful context, making them effectively unsolvable. As a consequence, the model ends up learning only a subset of subproblems or conditionals, while others remain poorly learned or ignored. A uniform masking objective forces the model to put equal weight on every subproblem, including those that confound the masking sequence, resulting in a suboptimal masking strategy overall. This mismatch creates a gap at inference time: the model’s behavior becomes highly sensitive to the masking schedule, and strong performance depends on selecting strategies that align with the sequential biases implicitly learned during training.

Key Open Question.
Thus, the central question becomes: *how can we design an inference strategy that faithfully reflects what the model has learned during training, given that dLLMs leave the process fundamentally under-specified and require us to decide the optimal masking trajectory.*

## 4 Limitations of SoTA and Our Key Insight

Failure of existing inference methods for reasoning tasks. Existing dLLMs rely on heuristic inference-time strategies to choose which tokens to unmask at each denoising step. Common approaches include random sampling (randomly picking masked positions to predict) and confidence-based selection (choosing the token positions with high model confidence or probability). For example, Kim et al. [1] showed that for Sudoku puzzles, a simple confidence-based top-K margin method can boost accuracy from 7% to nearly 90%. Intuitively, one might expect a similar benefit for reasoning tasks. Surprisingly, we find the opposite in reasoning benchmarks. In our experiments (see Figure [2](https://arxiv.org/html/2510.05040v1#S4.F2)) on GSM8K math problems, random unmasking far outperforms top‑K margin (high-confidence) decoding.
Instead of guiding generation, high confidence derails the unmasking trajectory into producing degenerate outputs. In Figure [2](https://arxiv.org/html/2510.05040v1#S4.F2), the top‑K margin strategy consistently unmasked the [AfterEoT] ($\langle\text{endoftext}\rangle$) token at all positions, proceeding backward from the end to the front.
This leads to degenerate outputs (shown in red text) in Figure [2](https://arxiv.org/html/2510.05040v1#S4.F2). This surprising result challenges our intuition built through studying prior work.

Figure: Figure 2: Random vs. Top-K margin inference on GSM8K. Left: Random decoding achieves 50.87% accuracy, while Right: Top‑K margin only 24.72%. For each method, the text box shows the result at the last unmasking step. Top-K margin generates output tokens in reverse, from the end toward the beginning, and exhibits a catastrophic collapse in which all tokens are [AfterEoT] (shown in red). Over 55.5% of top‑K margin runs suffered this collapse, yielding very low accuracy. These failures cast doubt on methods that rely solely on token confidence.
Refer to caption: x3.png

Unexpected reversal of intuition.
Our findings highlight a key limitation of relying on common heuristics from autoregressive models and prior work [1]. While one would expect that “follow the model’s own highest-probability tokens” is a reliable strategy, our results show that methods relying solely on token confidence are not sufficient for strong performance in complex reasoning tasks. Our results thus raise basic questions: Why does random sampling outperform top‑K margin? Why does the model overconfidently pick the [AfterEoT] so early? To answer these questions, we propose a new perspective on the dLLM’s internal structure. Our key insight is that the dLLM can be viewed as an implicit mixture of experts, which allows us to mitigate the risk of overconfidently predicting [AfterEoT] tokens too early. By aggregating predictions from experts conditioned on different subsets of tokens, effectively marginalizing over contexts, our approach avoids prematurely committing to high-confidence tokens like the [AfterEoT] token.

From failure to mechanism. The surprising failure of confidence-based schedules suggests that local token probabilities are unreliable under many masked contexts induced at inference. Because the dLLM objective (in ([1](https://arxiv.org/html/2510.05040v1#S3.E1))) averages over a wide variety of maskings, including ill-posed ones, some conditionals are poorly learned and disproportionately favor special tokens such as [AfterEoT]. Our view is to treat each semi-AR block schedule as querying a different conditional expert, then marginalize across experts to recover robust predictions. This replaces brittle confidence-following with consensus-seeking and is the core rationale behind HEX.

Our Key Insight: dLLM is an implicit mixture of experts.
From ([2](https://arxiv.org/html/2510.05040v1#S3.E2)), it is evident that the diffusion LLM training leads to a model with a family of masked–token conditionals

$$ $\big\{\,p_{\theta}(x_{i}\mid[x_{\text{prompt}},x[U]]):i\in[n],\ U\subseteq[n]\setminus\{i\}\,\big\},$ $$

which we can view as implicit “experts” indexed by the visible/unmasked set of tokens $U$. For a fixed target index $i$ at test-time, the goal is to aggregate the relevant experts $\{p_{\theta}(x_{i}\mid[x_{\text{prompt}},x[U]])\}_{U\subseteq[n]\setminus\{i\}}$ into a single prediction rule.
A principled aggregation strategy is the mixture-of-experts predictor given by

$$ $\displaystyle p_{\text{mix}}(x_{i}=a\mid x_{\text{prompt}})\;=\;\sum_{U}\pi(U\mid x_{\text{prompt}})\,p_{\theta}(x_{i}=a\mid[x_{\text{prompt}},x[U]]),$ (3) $$

where $\pi(U\mid x_{\text{prompt}})$ denotes the weight or trust placed on expert $U$ for prompt $x_{\text{prompt}}$.

A Toy Example. In our toy example (Figure [3](https://arxiv.org/html/2510.05040v1#S4.F3)), we examine the model’s answer to the question “Who invented telephone?” (ground truth: “The inventor was Bell.”). We treat each latent conditional context (or expert) as a separate setting that produces a distribution to predict masked tokens. Figure [3](https://arxiv.org/html/2510.05040v1#S4.F3) plots these distributions for a particular position, $i=4$ (the token ‘Bell’). As the plot makes clear, most experts concentrate probability on “Bell” (the correct token), but a few contexts produce flat or incorrect distributions that never predict “Bell”.

$\bullet$ Correct-Answer Contexts (Experts): The majority of conditionals yield a distribution that peaked at the correct answer token Bell. These contexts effectively act as “experts” on this question.

$\bullet$ Non-Expert Contexts: Some conditionals do not produce a clear peak in ‘Bell’.
These contexts fail to predict the correct answer.

Figure: Figure 3: The distribution of the 4th token ‘Bell’ in the output sequence changes significantly depending on the $2^{3}$ masking conditions applied to the previous three tokens: ‘The’, ‘inventor’, ‘was’. The star mark indicates the highest confidence for each distribution generated by $U$. Some masking conditions (violet and green) produce collapsed distribution: "Bell Bell was invented." (ungrammatical sentence), "The telephone was invented." (missing target information), respectively.
Refer to caption: x4.png

We note that even though the model as a whole can answer correctly, not all conditionals are experts. The figure shows that only a subset of experts “know” the answer, while others do not. Hence, our toy example shows that it is difficult to identify which context is the right expert. We do not know a priori which conditional context (or expert) will yield the correct token. The hidden gating distribution $\pi(U)$ that governs how likely each context knows the answer is unknown. The dLLMs does not learn the underlying gating distribution.

## 5 Hidden Semi-autoregressive Experts for Test-time Scaling

However, in order to estimate $p_{\text{mix}}$, we would need to estimate the likelihood for dLLM, which is extremely challenging, as also highlighted in [16, 3, 2]. Ideally, we would compute the Bayes‐optimal conditional probability $p_{\text{mix}}(x_{i}=a\mid x_{\text{prompt}})$ by fully marginalizing over all possible sequences of tokens in $U$. In practice, this is infeasible for diffusion LLMs, since their likelihood is intractable and must be approximated. To sidestep this, we approximate the ideal mixture by ensemble‐averaging over a small set ${\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}B}$ of “semi-autoregressive,” each defined by a particular block-schedule $b\in B$. Concretely, we sample several semi-AR decoding schedules $b$, each of which queries exactly one expert $U_{b}$, and then averaging:

$$ $\displaystyle p_{\text{mix}}(x_{i}=a\mid x_{\text{prompt}})\;\approx\;\mathbb{E}_{{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}b\sim B}}\big[p_{\theta}(x_{i}=a\mid[x_{\text{prompt}},x[U_{b}]])\big].$ (4) $$

The final prediction can be made using the Bayes rule $\hat{a}=\arg\max_{a}\;p_{\text{mix}}(x_{i}=a\mid x_{\text{prompt}}).$
A simple Monte Carlo approximation of this decision rule is majority vote: draw one sample $\hat{a}_{b}$ from each queried expert $U_{b}$, and return the mode of the sampled values. Thus, either by mixture averaging or majority vote, test-time aggregation amplifies the correct prediction by combining the specific conditionals the model actually learned. Based on this insight, we summarize our proposed approach in Algorithm [2](https://arxiv.org/html/2510.05040v1#alg2).

Figure: Algorithm 2 Hidden semi-autoregressive EXperts for test-time scaling

Why semi-autoregressive?
Diffusion LLMs allow all trajectories to reveal masked tokens, but uniformly random orders are suboptimal for language: they create unnatural partial contexts that
the model was

**Table 1: Semi-AR based decoding eliminates [AfterEoT] collapse and improves accuracy.**
| Dataset | Collapsed (↓, %) | Accuracy (↑, %) |
| --- | --- | --- |
| Baseline (non–semi-AR) |  |  |
| GSM8K | $55.80$ | $22.52$ |
| MATH | $29.80$ | $16.60$ |
| Semi-AR |  |  |
| GSM8K | $0.00$ | $76.27$ |
| MATH | $0.00$ | $32.80$ |

never intended to generate at test time. A practical restriction is semi-autoregressive left-to-right decoding (semi-AR): fix a block size $b\in\{1,\dots,B_{\max}\}$ (where $B_{\max}=n$) and partition $[n]$ into consecutive blocks

$$ $M_{t}=\{(t-1)b+1,\dots,\min(tb,n)\},\quad$ $$

for $t=1,\dots,T(b)=\lceil n/b\rceil$, revealing blocks left-to-right while denoising within each block via diffusion. This preserves a strong prefix structure (natural for language), yet allows parallel denoising inside a block. Empirically, we find (see Table [1](https://arxiv.org/html/2510.05040v1#S5.T1)) that semi-AR decoding avoids pathologies seen in fully parallel decoding. In particular, when using a single large block (i.e., non-semi-AR parallel decode), we often observe an [AfterEoT] collapse: the model erroneously floods the tail with [AfterEoT] tokens or repeats (Figure [8](https://arxiv.org/html/2510.05040v1#A1.F8)). By contrast, constraining to moderate block sizes (decoding left-to-right) eliminates this collapse and dramatically improves accuracy. (See Table [1](https://arxiv.org/html/2510.05040v1#S5.T1): semi-AR has 0% collapse and much higher accuracy than non-AR decoding.) Intuitively, focusing first on the left part of the output prevents the model from prematurely committing to a length or drifting with high-confidence tail tokens. In addition, the semi-AR setting (Figure [4](https://arxiv.org/html/2510.05040v1#S5.F4)) follows the pattern observed in the toy example (Figure [3](https://arxiv.org/html/2510.05040v1#S4.F3)) we presented above.

Figure: Figure 4: When asked about the 2024 Turing Award winners, names other than the actual recipients (such as Michael or David) might be generated due to different block sizes, which in turn risks producing incorrect information in the subsequent token sequence. However, if we generate outputs with various block sizes and then select the most frequently produced answer, that answer is more likely to be correct, since it was probably derived through a valid reasoning (Andrew) during the inference process.
Refer to caption: x5.png

## 6 Experiments

In this section, we empirically validate our claims. (i) Effectiveness: We first demonstrate that HEX significantly outperforms existing training-free and fine-tuned methods on a suite of reasoning benchmarks. (ii) Scaling behavior: We then analyze the performance-computation trade-off, showing how accuracy scales with more diverse generation paths. (iii) Working mechanism: Finally, through a series of ablations and qualitative examples, we explore the mechanisms behind HEX’s success, confirming that its gain comes from ensembling a latent mixture of semi-AR experts rather than relying on heuristics like model confidence.

### 6.1 Setup

Datasets and Metrics. We follow standard reasoning benchmarks: GSM8K [27] consisting of high-quality problems with diverse linguistic expressions, MATH [4] is a more challenging math benchmark that includes competition-level math problems, ARC-C [5] is the Challenge Set from AI2’s ARC dataset, consisting of science knowledge-based questions that are difficult to solve with simple keyword matching or retrieval, and TruthfulQA [6] which evaluates the tendency of language models to generate false information by following human misconceptions or false beliefs.(^1^11We use official evaluation scripts; numeric parsing strips LaTeX wrappers/whitespace/commas.)
Primary metric is task accuracy.

Models and Baselines.
All experiments with inference methods were performed using the LLaDA-8B-Instruct model [2], and the application of d1 (GPRO) [3] is subsequently based on this model. For all methods, when the output length is $256$ tokens, the number of unmasking steps is $128$. At each step, two masked tokens are unmasked, and this process is repeated until all tokens are revealed. $Random$ unmasks two randomly chosen masked tokens per step. Top-$k$ margin unmasks, at each step, the two masked tokens with the highest margin defined as $(\text{top-$1$ confidence}-\text{top-$2$ confidence})$
at their positions. d1 (GRPO) row uses the reported best value [3] for GSM8K and MATH, and for ARC-C we report a value reproduced after 1 epoch of training. TruthfulQA trained on d1 (GRPO) is excluded because there is no training data available, and neither were checkpoints released.
HEX draws five samples at temperature = $0.9$ for each of the block sizes $[8,16,32,64,128]$, yielding $25$ samples in total. If a tie occurs for the most frequent value, the value generated with the smallest block size is selected (Algorithm [2](https://arxiv.org/html/2510.05040v1#alg2)).

### 6.2 Main Results: HEX Establishes a New State-of-the-Art

Figure: Figure 5: HEX improves reasoning accuracy. On LLaDA-8B-Instruct, HEX outperforms training-free baselines (Random, Top-$k$, Top-$k$-margin) on GSM8K, MATH, ARC-C, and TruthfulQA. In GSM8K, MATH, ARC-C, it even outperforms the model trained with GRPO without any training.
Refer to caption: x6.png

Overall performance. Figure [5](https://arxiv.org/html/2510.05040v1#S6.F5) shows that HEX achieves the strongest results across all four reasoning benchmarks, outperforming both training-free and fine-tuned baselines. Compared to existing decoding strategies [2, 1], HEX delivers large and consistent gains. In GSM8K, for example, HEX reaches 88.10% accuracy, far higher than Random decoding (50.87%) and Top-$k$ margin (24.72%). These results show that confidence-based heuristics are unreliable in diffusion LLMs, whereas consensus-based voting in HEX is robust (Figure [7](https://arxiv.org/html/2510.05040v1#A1.F7)).

Comparison with GRPO fine-tuned models. Perhaps most strikingly, HEX also surpasses d1 (GRPO), which requires costly reinforcement learning fine-tuning. On GSM8K (88.10% vs. 79.80%), MATH (40.00% vs. 37.20%), and ARC-C (87.80% vs. 82.68%), HEX sets a new state of the art without updating model parameters.

Intuitively, fixed inference scheduled in existing techniques sometimes asks the model to guess hard tokens too early, which leads to mistakes. In contrast, HEX tries several semi-autoregressive schedules and then picks the answer that many schedules agree on. In practice, answers that show up across schedules are more reliable than answers from any single schedule.

Takeaway. These results suggest that the reasoning ability of a diffusion LLM remains latent and can be unlocked at inference time through block-marginalized ensembling, without any fine-tuning.

### 6.3 Analysis of Scaling and Compute Trade-off

Figure [6](https://arxiv.org/html/2510.05040v1#S6.F6) shows that HEX’s accuracy improves monotonically as the number of voting samples increases, while the tie rate, an indicator of ambiguity, steadily declines. Intuitively, different semi-AR schedules make different mistakes but tend to agree on the correct answer; adding schedules cancels schedule-specific errors and strengthens consensus, so ties resolve and accuracy improves. This trend holds consistently across all four benchmarks. Because sampling more trajectories linearly increases compute cost, HEX effectively exposes a tunable accuracy, compute knob: practitioners can trade inference cost for accuracy in a predictable way, without retraining.

Figure: Figure 6: As the number of majority voting samples in HEX increases, accuracy improves and the tie rate decreases. The block sizes used are [8, 16, 32, 64, 128], and sampling was performed while increasing the number of seeds (1-6).
Refer to caption: x7.png

Takeaway. HEX not only establishes state-of-the-art performance but also provides a principled mechanism for test-time scaling, ensuring accuracy improves with more inference budget.

### 6.4 Ablation Studies

Next, we analyze the mechanisms behind the HEX improvements, focusing on two key factors: the role of block diversity and the role of likelihood versus frequency in candidate selection.

Effect of block diversity.
Beyond using a fixed set of block sizes, we test whether ensembling over more varied (and even randomly generated) block schedules further boosts performance. As shown in Table [2](https://arxiv.org/html/2510.05040v1#S6.T2), increasing the number of dynamic trajectories from 5 to 30 on GSM8K improves accuracy from 81.96% to 84.15% while

**Table 2: HEX dynamic block size results. Accuracy and tie rate (%) on GSM8K across dynamic block size. See Figure [10](https://arxiv.org/html/2510.05040v1#A2.F10) for details.**
| Size | Accuracy (↑ %) | Tie (↓ %) |
| --- | --- | --- |
| 5 | 81.96 | 3.87 |
| 10 | 82.34 | 3.18 |
| 15 | 82.49 | 1.59 |
| 20 | 82.79 | 1.59 |
| 25 | 83.47 | 1.52 |
| 30 | 84.15 | 1.06 |

reducing the tie rate to less than half.
This reinforces our hypothesis that performance gains come from aggregating diverse “semi-AR experts.” We note that diversity matters, but structured diversity (fixed block set with multiple seeds) is even stronger (as in Table [3](https://arxiv.org/html/2510.05040v1#S6.T3)), yielding the highest overall gains.

Frequency vs. likelihood.
We then examine whether HEX’s gains could simply come from likelihood-based re-ranking. Table [3](https://arxiv.org/html/2510.05040v1#S6.T3) shows that the selection of the lowest negative log-likelihood candidate (NLL) performs poorly, in some cases worse than Random decoding (e.g., ARC-C: 70.05% vs. 60.84%). In contrast, HEX’s frequency-based majority vote achieves much higher accuracy (74.57%), confirming that consensus among diverse trajectories is more reliable than model confidence scores. This shows that the key driver of HEX’s success is ensemble agreement.

Tie break and Latency.

HEX defaults to the smallest block size in tie situations, as Table [5](https://arxiv.org/html/2510.05040v1#A2.T5) indicates that jointly considering frequency and log-likelihood does not bring a clear advantage. In addition, we present the wall-time latency of HEX and the baseline inference methods in Table [6](https://arxiv.org/html/2510.05040v1#A2.T6).

**Table 3: Ablations across datasets. NLL selects the candidate with the lowest NLL. HEX’s tie issue diminishes as the number of samples increases. Block sizes: [8, 16, 32, 64, 128].**
|  | GSM8K | MATH | ARC-C | TruthfulQA |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) |
| Baselines |  |  |  |  |  |  |  |  |
| Random | 50.87 | – | 16.80 | – | 70.05 | – | 42.40 | – |
| top-$k$ | 22.52 | – | 16.60 | – | 47.87 | – | 29.82 | – |
| top-$k$ margin | 24.72 | – | 16.40 | – | 54.18 | – | 28.36 | – |
| d1 (GRPO) | 79.80 | – | 37.20 | – | 82.68 | – | – | – |
| Likelihood-based |  |  |  |  |  |  |  |  |
| NLL | 76.72 | 4.09 | 34.40 | 16.00 | 60.84 | 2.99 | 28.07 | 4.24 |
| HEX |  |  |  |  |  |  |  |  |
| HEX | 82.18 | 4.09 | 38.40 | 16.00 | 74.57 | 2.99 | 45.91 | 4.24 |
| HEX ×5 seeds | 88.10 | 1.36 | 40.00 | 10.20 | 87.80 | 1.11 | 57.46 | 2.78 |

## 7 Conclusion and Limitation

In this work, we study how diffusion-based language models (dLLMs) generate text. We found that their performance is fundamentally tied to the decoding schedule, the order in which tokens are generated. This is because dLLMs implicitly learn a "set" of semi-autoregressive experts during training. Different schedules activate different experts, and choosing the right one is crucial for getting a high-quality answer. This single insight helps explain common dLLM issues, such as why they sometimes stop generating text too early or fail even when they seem confident.
Based on this insight, we introduced HEX (Hidden semi-autoregressive EXperts), a powerful inference method that requires no extra training. Instead of relying on a single schedule, HEX tries many different schedules at once and lets the experts "vote" on the best final answer. By combining the strengths of the entire hidden team, HEX turns the model’s flexibility into a reliable tool for boosting performance. On challenging reasoning benchmarks, HEX doesn’t just beat standard methods; it even surpasses models fine-tuned with costly techniques like reinforcement learning (GRPO).

HEX has some limitations. It requires more computation at test time, and we have mainly evaluated it on reasoning tasks. Applying this method to more creative areas like open-ended stories, image generation, or long conversations remains a promising area for future work. Further, we have not established any theoretical understanding of HEX, which is a valid scope of future work.

## References

- [1]
Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen.
Train for the worst, plan for the best: Understanding token ordering in masked diffusions.
arXiv preprint arXiv:2502.06768, 2025.
- [2]
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li.
Large language diffusion models.
arXiv preprint arXiv:2502.09992, 2025.
- [3]
Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover.
Scaling reasoning in diffusion large language models via reinforcement learning.
arXiv preprint arXiv:2504.12216, 2025.
- [4]
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe.
Let’s verify step by step.
arXiv preprint arXiv:2305.20050, 2023.
- [5]
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.
Think you have solved question answering? try arc, the ai2 reasoning challenge.
arXiv preprint arXiv:1803.05457, 2018.
- [6]
Stephanie Lin, Jacob Hilton, and Owain Evans.
Truthfulqa: Measuring how models mimic human falsehoods.
arXiv preprint arXiv:2109.07958, 2021.
- [7]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
Advances in neural information processing systems, 33:6840–6851, 2020.
- [8]
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
arXiv preprint arXiv:2010.02502, 2020.
- [9]
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg.
Structured denoising diffusion models in discrete state-spaces.
Advances in neural information processing systems, 34:17981–17993, 2021.
- [10]
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto.
Diffusion-lm improves controllable text generation.
Advances in neural information processing systems, 35:4328–4343, 2022.
- [11]
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, and Jonas Adler.
Continuous diffusion for categorical data.
arXiv preprint arXiv:2211.15089, 2022.
- [12]
Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li.
Scaling up masked diffusion models on text.
arXiv preprint arXiv:2410.18514, 2024.
- [13]
Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov.
Block diffusion: Interpolating between autoregressive and diffusion language models.
arXiv preprint arXiv:2503.09573, 2025.
- [14]
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, et al.
Scaling diffusion language models via adaptation from autoregressive models.
arXiv preprint arXiv:2410.17891, 2024.
- [15]
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong.
Dream 7b: Diffusion large language models.
arXiv preprint arXiv:2508.15487, 2025.
- [16]
Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic.
Weighted policy optimization for reasoning in diffusion language models.
arXiv preprint arXiv:2507.08838, 2025.
- [17]
Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li.
Llada-v: Large language diffusion models with visual instruction tuning.
arXiv preprint arXiv:2505.16933, 2025.
- [18]
Yuqing Wen, Hebei Li, Kefan Gu, Yucheng Zhao, Tiancai Wang, and Xiaoyan Sun.
Llada-vla: Vision language diffusion action models.
arXiv preprint arXiv:2509.06932, 2025.
- [19]
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang.
Mmada: Multimodal large diffusion language models.
arXiv preprint arXiv:2505.15809, 2025.
- [20]
Jiaming Zhou, Hongjie Chen, Shiwan Zhao, Jian Kang, Jie Li, Enzhi Wang, Yujie Guo, Haoqin Sun, Hui Wang, Aobo Kong, Yong Qin, and Xuelong Li.
Diffa: Large language diffusion models can listen and understand.
arXiv preprint arXiv:2507.18452, 2025.
- [21]
Xuanzhao Dong, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Peijie Qiu, Shao Tang, Xin Li, and Yalin Wang.
Llada-medv: Exploring large language diffusion models for biomedical image understanding.
arXiv preprint arXiv:2508.01617, 2025.
- [22]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language models.
Advances in neural information processing systems, 35:24824–24837, 2022.
- [23]
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.
Self-consistency improves chain of thought reasoning in language models.
arXiv preprint arXiv:2203.11171, 2022.
- [24]
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar.
Scaling llm test-time compute optimally can be more effective than scaling model parameters.
arXiv preprint arXiv:2408.03314, 2024.
- [25]
Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhenguo Li, Wei Bi, and Lingpeng Kong.
Diffusion of thoughts: Chain-of-thought reasoning in diffusion language models.
arXiv preprint arXiv:2402.07754, 2024.
- [26]
Wen Wang, Bozhen Fang, Chenchen Jing, Yongliang Shen, Yangyi Shen, Qiuyu Wang, Hao Ouyang, Hao Chen, and Chunhua Shen.
Time is a feature: Exploiting temporal dynamics in diffusion language models.
arXiv preprint arXiv:2508.09138, 2025.
- [27]
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.
Training verifiers to solve math word problems.
arXiv preprint arXiv:2110.14168, 2021.
- [28]
GSAI-ML.
LLaDA-8B-Instruct.
[https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct).
MIT License, accessed 2025-09-18.
- [29]
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto.
s1: Simple test-time scaling.
arXiv preprint arXiv:2501.19393, 2025.
- [30]
Learning to reason with llms.
2024.
- [31]
Pranjal Aggarwal and Sean Welleck.
L1: Controlling how long a reasoning model thinks with reinforcement learning.
arXiv preprint arXiv:2503.04697, 2025.
- [32]
Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei.
Towards thinking-optimal scaling of test-time compute for llm reasoning, 2025.
- [33]
Daman Arora and Andrea Zanette.
Training language models to reason efficiently.
2025.
- [34]
Gongfan Fang, Xinyin Ma, and Xinchao Wang.
Thinkless: Llm learns when to think.
arXiv preprint arXiv:2505.13379, 2025.
- [35]
Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Haodong Zhao, Hao Li, Jiansong Chen, Ke Zeng, and Xunliang Cai.
When to continue thinking: Adaptive thinking mode switching for efficient reasoning.
arXiv preprint arXiv:2505.15400, 2025.
- [36]
Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei.
Think only when you need with large hybrid-reasoning models.
arXiv preprint arXiv:2505.14631, 2025.
- [37]
Guosheng Liang, Longguang Zhong, Ziyi Yang, and Xiaojun Quan.
Thinkswitcher: When to think hard, when to think fast.
arXiv preprint arXiv:2505.14183, 2025.
- [38]
Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li.
Adaptthink: Reasoning models can learn when to think.
arXiv preprint arXiv:2505.13417, 2025.
- [39]
Shijue Huang, Hongru Wang, Wanjun Zhong, Zhaochen Su, Jiazhan Feng, Bowen Cao, and Yi R Fung.
Adactrl: Towards adaptive and controllable reasoning via difficulty-aware budgeting.
arXiv preprint arXiv:2505.18822, 2025.
- [40]
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano.
Learning to summarize with human feedback.
Advances in Neural Information Processing Systems, 33:3008–3021, 2020.
- [41]
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al.
Webgpt: Browser-assisted question-answering with human feedback.
arXiv preprint arXiv:2112.09332, 2021.
- [42]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.
Llama 2: Open foundation and fine-tuned chat models.
arXiv preprint arXiv:2307.09288, 2023.
- [43]
Maxim Khanov, Jirayu Burapacheep, and Yixuan Li.
Args: Alignment as reward-guided search.
arXiv preprint arXiv:2402.01694, 2024.
- [44]
James Y Huang, Sailik Sengupta, Daniele Bonadiman, Yi-an Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchhoff, and Dan Roth.
Deal: Decoding-time alignment for large language models.
arXiv preprint arXiv:2402.06147, 2024.
- [45]
Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Chen, Heng-Tze Cheng, Michael Collins, Trevor Strohman, et al.
Controlled decoding from language models.
arXiv preprint arXiv:2310.17022, 2023.
- [46]
Souradip Chakraborty, Soumya Suvra Ghosal, Ming Yin, Dinesh Manocha, Mengdi Wang, Amrit Singh Bedi, and Furong Huang.
Transfer q star: Principled decoding for llm alignment.
arXiv preprint arXiv:2405.20495, 2024.
- [47]
Ruizhe Shi, Yifang Chen, Yushi Hu, Alisa Liu, Hannaneh Hajishirzi, Noah A. Smith, and Simon S. Du.
Decoding-time language model alignment with multiple objectives.
The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.

## Appendix

## Appendix A Qualitative results

### A.1 Qualitative analysis of Baselines vs. HEX

Figure: Figure 7: An instance of generated text responses of different decoding strategies.
Refer to caption: x8.png

### A.2 Qualitative analysis of semi-AR vs. non-semi-AR

As shown on the right side of Figure [8](https://arxiv.org/html/2510.05040v1#A1.F8) in confidence-based non-semi-AR decoding, the phenomenon where [AfterEoT] tokens accumulate from the end of the output towards the front indicates that the model is assigning high confidence to [AfterEoT] tokens throughout the unmasking steps.
The input to the dLLM consists of the number of tokens that make up the prompt and the number of tokens in the desired output sequence, and during training it is subject to limits on the input sequence length for parallel computation.
For LLaDA-8B-Instruct [28], this limit is 4,096 tokens. However, in the training of reasoning tasks, most of the output finishes within 256 tokens.
In other words, the majority of ground truth tokens in the output sequence (more than 93.75%) are [AfterEoT]: Given that the training objective is to maximize the average likelihood, we can infer that the dLLM is most strongly taught to generate the [AfterEoT] token.

This suggests that confidence-only decoding is fundamentally limited in its ability to prevent such phenomena during inference, and highlights why the positions of tokens to be unmasked should not be selected based solely on confidence.

Figure: Figure 8: Blue denotes mask tokens, red denotes [AfterEoT] tokens, white denotes text tokens, and green denotes [EoT] tokens (note that in the LLaDA-8-Instruct model, [EoT] and [AfterEoT] are represented as $
$ and $
$, respectively [28]). As unmasking proceeds, two mask tokens are unmasked at each step (output length = 256, unmasking steps = 128). Under a semi-AR regime with block size = 32, positional constraints force reasoning to progress left-to-right while still allowing diffusion-style generation within each block. By contrast, when the positional constraint is removed with block size = 256 (non-semi-AR), the model starts from the last token with the highest confidence—[AfterEoT]—and, due to the inertia of repeatedly generating the same token backward, ultimately collapses into a catastrophic output in which all tokens become [AfterEoT].
Refer to caption: x9.png

Figure: Figure 9: Few cases (2.96%) where the block size of output length hits and semi-AR fails. However, in those cases, the unmasking order converges from both ends toward the center. It means the model commits to an answer before engaging in a full reasoning process—i.e., it states the answer first and only then provides a derivation. This is not only unintuitive; [26] shows that the answer the dLLM settles on can flip repeatedly during the reasoning process. Consequently, when the model locks in an answer first, it is hard to claim that it truly “knows” the answer with certainty.
Refer to caption: x10.png

Furthermore, confidence-only decoding can lead to issues such as those shown in Figure [9](https://arxiv.org/html/2510.05040v1#A1.F9), even when the answer is correct. This arises in reasoning tasks where the correct answer should be derived through step-by-step justification. Instead, the model often generates the answer first without providing supporting reasoning (with the answer tokens generated on the right first), and only afterward begins to generate the reasoning (by unmasking tokens in the middle). Fundamentally, unless the already-generated answer tokens are revised, this generation pattern means that the reasoning cannot actually influence the answer.

Besides, as observed in [26], the probability distribution corresponding to the answer tokens, whether they are correct or incorrect, continues to undergo substantial changes during the reasoning process. This indicates that the model’s confidence in the produced early answer cannot be considered reliable.

## Appendix B Additional Details, Examples, and Results Which Can be Useful

### B.1 Additional context of related work

Test-Time Scaling in Autoregressive LLMs. A growing body of work in autoregressive LLMs has begun to formalize the phenomenon of test-time scaling, the empirical observation that allocating more inference-time compute (e.g., longer reasoning traces or more sampled candidates) can significantly improve model performance without retraining. [29] introduced budget forcing to emulate the compute-dependent reasoning behaviors observed in o1-style models [30], while [31] proposed length-controlled policy optimization to explicitly regulate the reasoning horizon. There are also complementary efforts, such as thinking-optimal scaling [32] and adaptive reasoning policies [33, 34, 35, 36, 37, 38, 39], that highlight the goal of dynamically adjusting cognitive effort based on task complexity and available compute budgets.

In parallel, researchers have explored how controlled sampling and decoding strategies can steer model behavior toward desired objectives without fine-tuning. Best-of-$N$ selection [40, 41, 42] provides a simple form of test-time search (which requires a reward signal access) over diverse outputs. We also include an ablation where we compare an instance of Best-of-$N$ with majority vote as the reward signal in Table [4](https://arxiv.org/html/2510.05040v1#A2.T4). We note that this is lower than random sampling (cf. Figure [5](https://arxiv.org/html/2510.05040v1#S6.F5)(a)). On the other hand, more structured methods recast decoding as a reward-guided search [43, 44, 45, 46]. Recent multi-objective decoding frameworks [47] further generalize this idea, balancing several alignment criteria simultaneously.

**Table 4: Comparison of majority voting between another non–semi-AR (BoN) and semi-AR experts on GSM8K. The number preceding [ ] represents the output length, and the numbers inside [ ] correspond to the block sizes used.**
| Method | Accuracy (↑ %) | Tie (↓ %) |
| --- | --- | --- |
| 256 [256, 256, 256, 256, 256], BoN with majority voting | 44.35 | 44.58 |
| 256 [8, 16, 32, 64, 128], HEX | 82.18 | 4.09 |

### B.2 Details of dynamic HEX block settings

Following the semi-AR left-right generation scheme while dynamically adjusting block size induces more diverse trajectories, thereby increasing candidate diversity in majority voting. However, as shown in
Table [2](https://arxiv.org/html/2510.05040v1#S6.T2) and Table [3](https://arxiv.org/html/2510.05040v1#S6.T3), it does not empirically outperform structured diversity (a fixed block set with multiple seeds).

Figure: Figure 10: Examples of the block sizes and counts used in the dynamic HEX block settings. Block sizes and counts were randomly chosen and adjusted to match the total output length. The output length is 256 and the number of unmasking steps is 128, meaning that each step unmasks 2 tokens. Accordingly, all block sizes are multiples of 2, and decoding was performed in a semi-autoregressive manner.
Refer to caption: x11.png

### B.3 Experimental results of HEX’s tie-breaking methods

Frequency-based majority voting has the characteristic that ties can occur within a finite set of candidates. To determine a single final output, we experimented with various tie-breaking rules. We found that likelihood-based selection did not provide any significant benefit. As shown in Figure [11](https://arxiv.org/html/2510.05040v1#A2.F11), since we observed that performance tends to degrade as the block size approaches the output length, we adopt selecting the output with the smallest block size in tie situations as our default setting.

**Table 5: Evaluation on tie breaking methods. If the most frequent output is in a tie situation, TIED: NLL selects the result with the lowest negative log-likelihood in tie situations, TIED: first selects the result generated from the smallest block size when tied, and TIED: any treats the case as correct if a correct option exists among the tied candidates. The results of TIED: any clearly highlight that majority voting of HEX works well across datasets.**
|  | GSM8K | MATH | ARC-C | TruthfulQA |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) | Acc (↑%) | Tie (↓%) |
| HEX (tie-breaking rules) |  |  |  |  |  |  |  |  |
| HEX, TIED: NLL | 82.18 | 4.09 | 38.00 | 16.00 | 74.49 | 2.99 | 46.20 | 4.24 |
| HEX, TIED: first | 82.18 | 4.09 | 38.40 | 16.00 | 74.57 | 2.99 | 45.91 | 4.24 |
| HEX, TIED: any | 83.09 | 4.09 | 41.00 | 16.00 | 76.11 | 2.99 | 47.66 | 4.24 |

Figure: Figure 11: Accuracy across tasks when the block size (B) is increased in fixed increments of 16. Here, block count = (256 // B) when 256 % B is 0 otherwise (256 // B) + 1. Despite the uniform increase in block size, the performance drops drastically in the 128–256 range.
Refer to caption: x12.png

### B.4 Analysis of decoding latency across inference methods

We measured the latency of each inference method through wall-time evaluation. As shown in Figure [6](https://arxiv.org/html/2510.05040v1#S6.F6), increasing the number of candidate samples used in HEX’s majority voting leads to improved performance and a lower tie rate. This indicates that the increase in inference cost for HEX is accompanied by a clear and consistent improvement in performance.

**Table 6: Decoding latency across different inference methods. We report the generation time (in seconds) for GSM8K, along with the corresponding scaling factor.**
| Method | GSM8K test dataset (1319) | per batch (8) | per datapoint (1) | ratio |
| --- | --- | --- | --- | --- |
| random | 2775.73 | 16.76 | 2.09 | $\times 1.00$ |
| top-k | 2921.70 | 17.64 | 2.20 | $\times 1.05$ |
| top-k margin | 3187.56 | 19.25 | 2.41 | $\times 1.15$ |
| HEX | 14613.72 | 88.23 | 11.03 | $\times 5.26$ |

### B.5 Further semi-AR analysis

Figure [12](https://arxiv.org/html/2510.05040v1#A2.F12) shows that it is rare for outputs generated with different block sizes under the same prompt to all fail simultaneously. It also reveals that no particular block size consistently dominates in performance. Through qualitative analysis, we identified the existence and patterns of block sizes that successfully lead to correct answers for specific prompts, namely, their frequency of agreement, as illustrated in Figure [13](https://arxiv.org/html/2510.05040v1#A2.F13).

Figure: Figure 12: Visualization of the correct (blue) and incorrect (white) answers for each data point in GSM8K when reasoning with various block sizes. Apart from the case where the block size is equal to the output size, i.e., non–semi-AR generation (256), there is no consistent pattern across block sizes in this figure that would allow us to conclude which block size is most suitable for semi-AR reasoning for given tasks. However, the observation that it is rare for all semi-AR outputs to fail on a given data point suggests that the model has the potential to arrive at the correct answer even without additional training, simply by selecting the appropriate decoding policy for the question.
Refer to caption: x13.png

Figure: Figure 13: For the same input prompt (blue), the output (orange) varies depending on the block size, because the accumulated context changes at each unmasking step of the reasoning process. Refer to Figure [4](https://arxiv.org/html/2510.05040v1#S5.F4).
Refer to caption: x14.png

### B.6 Ablation study of HEX across different output lengths

Across various output lengths, HEX consistently proves to be an effective method. As shown in Table [7](https://arxiv.org/html/2510.05040v1#A2.T7), for output lengths of 128, 256, and 512, HEX consistently surpasses both other inference methods and GRPO trained models.

**Table 7: Ablation study of HEX across output lengths of 128, 256. The results demonstrate that HEX consistently exhibits robust performance irrespective of output length. Block sizes used are the same as Table [8](https://arxiv.org/html/2510.05040v1#A2.T8). Refer to Table [5](https://arxiv.org/html/2510.05040v1#A2.T5) for detailed explanation about (b), (c), (d).**
| Output Length 128 |  |  |  |  |
| --- | --- | --- | --- | --- |
| Method | GSM8K (acc, tied) | MATH (acc, tied) | ARC-C (acc, tied) | TruthfulQA (acc, tied) |
| Random | 48.82%, – | 20.60%, – | 66.21%, – | 41.96%, – |
| top-$k$ | 53.30%, – | 20.60%, – | 56.74%, – | 36.40%, – |
| top-$k$ margin | 55.57%, – | 22.60%, – | 61.35%, – | 38.30%, – |
| d1 (GRPO) | 72.60%, – | 33.20%, – | 82.68%, – | –, – |
| NLL (a) | 70.81%, 7.28% | 29.80%, 20.20% | 78.41%, 1.54% | 47.95%, 4.24% |
| HEX, TIED: NLL (b) | 73.69%, 7.28% | 32.00%, 20.20% | 85.24%, 1.54% | 52.63%, 4.24% |
| HEX, TIED: first (c) | 74.22%, 7.28% | 30.60%, 20.20% | 85.41%, 1.54% | 53.36%, 4.24% |
| HEX, TIED: any (d) | 75.82%, 7.28% | 35.60%, 20.20% | 85.92%, 1.54% | 54.39%, 4.24% |
| HEX $\times$ 5 seeds, TIED: first (e) | 81.05%, 3.64% | 33.40%, 15.60% | 88.40%, 0.60% | 53.80%, 2.78% |
| Output Length 256 |  |  |  |  |
| Method | GSM8K (acc, tied) | MATH (acc, tied) | ARC-C (acc, tied) | TruthfulQA (acc, tied) |
| Random | 50.87%, – | 16.80%, – | 70.05%, – | 42.40%, – |
| top-$k$ | 22.52%, – | 16.60%, – | 47.87%, – | 29.82%, – |
| top-$k$ margin | 24.72%, – | 16.40%, – | 54.18%, – | 28.36%, – |
| d1 (GRPO) | 79.80%, – | 37.20%, – | 82.68%, – | –, – |
| NLL (a) | 76.72%, 4.09% | 34.40%, 16.00% | 60.84%, 2.99% | 28.07%, 4.24% |
| HEX, TIED: NLL (b) | 82.18%, 4.09% | 38.00%, 16.00% | 74.49%, 2.99% | 46.20%, 4.24% |
| HEX, TIED: first (c) | 82.18%, 4.09% | 38.40%, 16.00% | 74.57%, 2.99% | 45.91%, 4.24% |
| HEX, TIED: any (d) | 83.09%, 4.09% | 41.00%, 16.00% | 76.11%, 2.99% | 47.66%, 4.24% |
| HEX $\times$ 5 seeds, TIED: first (e) | 88.10%, 1.36% | 40.00%, 10.20% | 87.80%, 1.11% | 57.46%, 2.78% |
| Output Length 512 |  |  |  |  |
| Method | GSM8K (acc, tied) | MATH (acc, tied) | ARC-C (acc, tied) | TruthfulQA (acc, tied) |
| Random | 51.71%, – | 16.60%, – | 73.29%, – | 43.86%, – |
| top-$k$ | 9.10%, – | 10.00%, – | 50.17%, – | 29.68%, – |
| top-$k$ margin | 10.08%, – | 9.40%, – | 54.35%, – | 31.14%, – |
| d1 (GRPO) | 81.90%, – | 39.20%, – | 83.28%, – | –, – |
| NLL (a) | 78.85%, 4.25% | 25.00%, 20.80% | 63.82%, 5.03% | 34.50%, 7.75% |
| HEX, TIED: NLL (b) | 82.11%, 4.25% | 40.20%, 20.80% | 83.11%, 5.03% | 54.97%, 7.75% |
| HEX, TIED: first (c) | 82.03%, 4.25% | 40.20%, 20.80% | 84.13%, 5.03% | 55.85%, 7.75% |
| HEX, TIED: any (d) | 83.47%, 4.25% | 44.80%, 20.80% | 84.81%, 5.03% | 57.75%, 7.75% |
| HEX $\times$ 5 seeds, TIED: first (e) | 88.40%, 1.14% | 47.60%, 10.40% | 87.12%, 0.85% | 59.80%, 2.63% |

Table [8](https://arxiv.org/html/2510.05040v1#A2.T8) illustrates that the average performance of the samples used in HEX’s majority voting—that is, the individual semi-AR experts—consistently falls short of HEX across all output lengths (128, 256, and 512). This indicates that dLLMs have implicitly learned a diverse set of inference behaviors from multiple semi-AR experts. Therefore, uncovering and leveraging the correct judgment among these hidden experts is essential—an ability that HEX demonstrates with remarkable effectiveness.

**Table 8: HEX (the bottom three rows) consistently surpasses the mean accuracy (the top three rows) of samples used in majority voting across various output length settings. The number preceding [ ] represents the output length, and the numbers inside [ ] correspond to the block sizes used.**
| Method | GSM8K | MATH | ARC-C | TruthfulQA |
| --- | --- | --- | --- | --- |
| 128 [4, 8, 16, 32, 64] $\times$ 1 seed | 68.57 | 28.00 | 79.44 | 48.74 |
| 256 [8, 16, 32, 64, 128] $\times$ 1 seed | 75.39 | 33.16 | 67.27 | 39.59 |
| 512 [16, 32, 64, 128, 256] $\times$ 1 seed | 66.52 | 32.88 | 66.35 | 44.82 |
| 128 [4, 8, 16, 32, 64], HEX $\times$ 1 seed | 74.22 | 30.60 | 85.41 | 53.36 |
| 256 [8, 16, 32, 64, 128], HEX $\times$ 1 seed | 82.18 | 38.40 | 74.57 | 45.91 |
| 512 [16, 32, 64, 128, 256], HEX $\times$ 1 seed | 83.24 | 39.60 | 84.81 | 57.02 |

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Please refer to weakness 2. 

2. Have authors think about how to incorporate HEX into the post-training of dllm, in order to get either faster decoding or better performance?
soundness: 3
strengths: 1. The paper clearly illustrates that previous methods such as confidence-based sampling fail in the context of long reasoning. This provides a clear motivation for designing new algorithms.

2. The method proposed is very intuitive and easy to apply.

3. The potential mechanism behind the empirical improvement is clearly outlined in Sections 3 and 4.
summary: This paper shows that by ensembling over multiple semi-autoregressive decoding orders, the current dLLMs can get a significant improvement in capability. They empirically show that confidence-based decoding fails to beat the random order baseline. With autoregressive sampling, the performance jumps more than 3x on benchmarks including GSM8k. Ensembling over block-size and randomness further improve the performance.
weaknesses: 1. It should be noted that in the abstract, the 24.72% performance is not using semi-autoregressive baseline, which is widely adopted.

2. In section 5.4, regarding the importance of block-diversity, the author shows that ensembling 30 different block schedules can improve performance but is worse than 'structural diversity' with a fixed set of 5 block schedules and 6 random seeds. However, it remains unclear whether the proposed algorithm can beat the plain method with only 1 block schedule and 30 seeds.

---

confidence: 2
contribution: 4
initial_rating: 8
presentation: 3
questions: - Can you clarify the impact of your proposed methodology on computational efficiency? Please compare both to regular sampling from diffusion LLMs and to autoregressive sampling from transformer-based LLMs.

See the section on weaknesses for additional questions.
soundness: 2
strengths: - Massive performance gains
- Interesting interpretation of block schedules as experts
summary: The authors propose a specific choice of block schedule for sampling from diffusion language models which is “semi-autoregressive”, meaning that parallel sampling is carried out in blocks that are autoregressively revealed left to right. The authors associate different choices of block schedule with different “expert” components of the dLLM and reveal large gains in accuracy by taking the majority vote over predictions by such experts (in effect marginalizing over different choices of block schedules).
weaknesses: - Identification of block schedules as experts is tenuous - do these block schedules really encode specific knowledge/expertise in the way that, say, the experts in an MoE model do? Can you provide more evidence of this?
- If semi-autoregressive decoding uses all block schedules down to very small granularity (b=1), can performance gains be explained in terms of increased granularity (and higher computational cost)? The b=1 case in particular seems to perfectly mimic standard auto-regressive sampling in transformer-based LLMs.

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 2
questions: Please see Weaknesses.
soundness: 3
strengths: 1. The paper shows dLLMs are sensitive to semi-autoregressive schedules, which provides confirmed motivation for HEX to ensemble diverse schedules for robust inference.

2. HEX is a training-free plug-and-play method, enabling a trade-off between accuracy and computation via the number of voting trajectories, and is competitive with (or exceeds) RL-fine-tuned variants under the reported settings.
summary: The paper argues that dLLMs trained with masked-token objectives implicitly encode a mixture of semi-autoregressive experts: different left-to-right block sizes activate different conditional distributions learned during training. Committing to a single decoding schedule can collapse performance. The proposed training-free method HEX ensembles across heterogeneous semi-AR block schedules and aggregates final answers via majority vote. On LLaDA-8B-Instruct, HEX reports strong gains while avoiding collapse modes.
weaknesses: 1. Although the paper treats the number of voting trajectories as a tunable inference time knob, the marginal gains from adding schedules are modest relative to the near linear growth in compute (for example, GSM8K improves only from 81.96% to 84.15%). Moreover, while HEX can surpass RL fine tuned baselines, the cost profiles differ: RL incurs a one time training expense and then serves at single pass inference cost, whereas HEX must sustain a $k$ times inference overhead on every request to remain competitive, which is an unfavorable trade at deployment scale.

2. While HEX’s wall-clock latency is reported for GSM8K, the paper does not provide budget-matched comparisons across all datasets, which limits a fair cost-effectiveness assessment.

3. Despite a persuasive mixture-of-experts interpretation, the method remains theoretically under-specified: there are no conditions or guarantees for the consistency/convergence of majority voting, sample-complexity requirements, or optimal schedule priors, weakening the theoretical closure of the contribution.

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 1
questions: What are the differences and connections between the proposed method and beam search?
soundness: 2
strengths: 1. Introducing the proposed method through a simple toy example makes the paper intuitive and easy to follow.

2. The experimental results are impressive, showing substantial improvements across multiple reasoning benchmarks.
summary: The paper proposes HEX (Hidden semi-autoregressive EXperts), a training-free inference method for diffusion-based large language models (dLLMs). Through a series of intuitive experimental observations, it reveals that dLLMs implicitly learn a mixture of semi-autoregressive experts, where different masking schedules activate distinct expert behaviors. By ensembling multiple decoding paths with majority voting, HEX achieves remarkable performance on reasoning benchmarks such as GSM8K and MATH.
weaknesses: 1. The authors repeatedly emphasize (for example, in Sec. 1 and Sec. 3) that using sampling strategies such as top-K margin can harm performance and that this issue has not been sufficiently discussed. However, previous literature has already provided explanations for this phenomenon, such as Appendix B.4 of [1].

2. The presentation is poor. For example, Eq. (1) is incorrect because it lacks the necessary reweighted coefficient. The description of the main method—such as how the majority vote is performed—is neither clearly introduced in the text nor reflected in the algorithmic formulas.

3. The evaluation is neither fair nor comprehensive. For instance, in the abstract the authors state that LLaDA-8B-Instruct achieves 16.4 % and 54.2 % accuracy on MATH and ARC-C, which HEX improves to 40.0 % and 87.8 %. However, the original LLaDA paper reports 42.2 % and 88.5 % for these datasets. The authors should not use the widely known underperforming random remasking as the baseline. In addition, the paper seems to lack a comparison of inference efficiency.

[1] Nie et al. Large Language Diffusion Models.


Decision: Accept (Poster)