Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Authors: Pengxiang Li
Shilin Yan
Jiayin Cai
Renrui Zhang
Ruichuan An
Ziyu Guo
Xiaowei Gao
Venue: NeurIPS 2025 poster ()
Keywords: diffusion
absorting kernel
language model
Primary area: deep_learning
TL;DR: We propose Adaptive Classifier-Free Guidance (A-CFG), which dynamically re-masks low-confidence tokens for a more targeted unconditional input in iterative models.

Abstract: Classifier-Free Guidance (CFG) significantly enhances controllability in generative models by interpolating conditional and unconditional predictions. However, standard CFG often employs a static unconditional input, which can be suboptimal for iterative generation processes where model uncertainty varies dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel method that tailors the unconditional input by leveraging the model's instantaneous predictive confidence. At each step of an iterative (masked) diffusion language model, A-CFG identifies tokens in the currently generated sequence for which the model exhibits low confidence. These tokens are temporarily re-masked to create a dynamic, localized unconditional input. This focuses CFG's corrective influence precisely on areas of ambiguity, leading to more effective guidance. We integrate A-CFG into a state-of-the-art masked diffusion language model and demonstrate its efficacy. Experiments on diverse language generation benchmarks show that A-CFG yields substantial improvements over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work highlights the benefit of dynamically adapting guidance mechanisms to model uncertainty in iterative generation.

Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Pengxiang Li Shilin Yan *2 Joey Tsai Renrui Zhang
Ruichuan An², Ziyu Guo Xiaowei Gao i6
PPolyU 2 Alibaba
THH 4CUHK *PKU ICLL
{2040gis, tattoo battoo.ysl)o @ @gmail.com
"Equal Contribution
Project Leader
'Corresponding Author
Abstract
Classifier-Free Guidance (CFG) significantly enhances controllability in generative models by interpolating conditional and unconditional predictions. However, standard CFG often employs static unconditional input, which can be suboptimal for iterative generation processes where model uncertainty varies dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), novel method that tailors the unconditional input by leveraging the model's instantaneous predictive confidence. At each step of an iterative (masked) diffusion language model, A-CFG identifies tokens in the currently generated sequence for which the model exhibits low confidence. These tokens are temporarily re-masked to create a dynamic, localized unconditional input. This focuses CFG's corrective influence precisely on areas of ambiguity, leading to more effective guidance. We integrate A-CFG into a state-of-the-art masked diffusion language model and demonstrate its efficacy. Experiments on diverse language generation benchmarks show that A-CFG yields substantial improvements over standard CFG, achieving, for instance, 3.9 point gain on GPQA. Our work highlights the benefit of dynamically adapting guidance mechanisms to model uncertainty in iterative generation. Code is available at https://gitt com www.//owwp//////////c///
Introduction
Diffusion models [33, 14] have recently revolutionized generative modeling, demonstrating remark- able capabilities in synthesizing high-fidelity data in continuous domains such image and audio [9, 29]. This success has ignited a surge of interest in extending their power to discrete data, with natural language generation standing as a particularly compelling frontier [2, 20, 11]. Among these efforts, Masked Diffusion Models (MDMs), exemplified by frameworks like LLaDA [26], have emerged as promising direction. These models learn reverse gradual masking process, itera- tively infilling masked tokens to construct coherent text, offering principled and flexible alternative traditional autoregressive language generation.
pivotal advancement that significantly amplified the practical utility of diffusion models, especially in conditional settings, is Classifier-Free Guidance (CFG) [15]. Originally conceived for continuous models, CFG provides an elegant way to steer the generation process towards a desired conditioning signal (e.g., textual prompt) by interpolating between conditional and unconditional model predic- tions during the reverse diffusion (denoising) phase. This is achieved without the need for an auxiliary classifier, making CFG versatile and widely adopted mechanism for enhancing sample quality and
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

[n] Taken-level Confidence During Generation
Confidence Netrics per Generation Step
Avllag Carldace
Curfi Wendenee Scor
Confitence Score
64
Figure 1: Overview of model confidence dynamics during iterative generation. (a) Token-level confidence heatmap across token positions and generation steps (darker shades indicate higher confidence). (b) Average and minimum confidence scores per generation step. This visualization highlights the dynamic and non-uniform nature of model confidence that A-CFG aims to leverage.
controllability. Naturally, the application of CFG has extended to textual diffusion models, where plays a similar role in guiding text generation.
However, the conventional application of CFG within iterative (masked) diffusion language models often encounters a subtle yet significant limitation: the "unconditional" prediction typically relies on static generic construct. This often involves using a null prompt sequence where all target tokens are uniformly masked to simulate an unconditional state. While straightforward, such a fixed approach unconditioning may not fully harness CFG's potential in the dynamic context of iterative text refinement. As an MLM progressively fills in sequence, its internal state of certainty can vary considerably across different tokens and denoising steps. A static unconditional baseline fails to adapt to these nuances, potentially leading to guidance that is either too weak, too diffuse, or misaligned with the model's 's specific points of ambiguity a given step.
This observation sparks a crucial question: can the "unconditional" component of CFG, when applied to iterative diffusion language models, be rendered more intelligent and responsive to the model's s own evolving understanding of the sequence? We posit that the model's s instantaneous predictive confidence during the iterative denoising process, which, as visualized in Figure 1, can fluctuate significantly across tokens and generation steps, offers a rich, yet largely untapped signal. Instead of blanket, context-agnostic unconditioning, what if we could dynamically shape the unconditional input to reflect and address the model's current uncertainties? This would allow the guidance mechanism to concentrate its corrective influence precisely where it most needed.
In this paper, we introduce Adaptive Classifier-Free Guidance (A-CFG), a novel framework designed to realize this vision for iterative (masked) diffusion language models. A-CFG dynamically synthesizes the input for the unconditional prediction by identifying and temporarily re- re-masking tokens for which the conditional diffusion model exhibits low predictive confidence during given denoising step. By doing so, A-CFG creates a localized "unconditional" state that compels the model reconsider its predictions at these specific points of ambiguity. The standard CFG formula is then applied, leveraging this adaptively constructed unconditional state to steer the generation with greater precision and efficacy.
We integrate and evaluate A-CFG within the LLaDA [26] framework. Our extensive experiments on range of standard language generation benchmarks demonstrate that A-CFG yields substantial improvements in complex reasoning accuracy and adherence to conditional prompts over both baseline LLaDA without CFG and LLaDA employing traditional CFG with static unconditional inputs. Specifically, A-CFG achieves up to a 3.9 point absolute improvement on the GPQA benchmark and enhances Sudoku task success by 8.0 points when compared to standard CFG.
Our contributions are thus threefold:
We identify and articulate the limitations of static unconditioning in standard CFG when applied iterative masked language models.
We propose Adaptive Classifier- Free Guidance (A-CFG), novel method that dynamically constructs the unconditional input based on the model's predictive confidence, enabling more targeted and effective guidance.

---

We demonstrate through comprehensive experiments that A-CFG significantly enhances the performance of the LLaDA model on various generation tasks, outperforming standard CFG.
2 Related Work
2.1 Diffusion Models for Language Generation
Autoregressive (AR) models, such as large language models (LLMs) like GPT-style architec- tures [27, 5] and more recent powerful open-source models including LLaMA [35, 36], Qwen [6], and Mistral [18], have become the dominant paradigm in natural language generation. These models generate text token by token, conditioning each new token on the previously generated sequence, and have demonstrated remarkable capabilities across a wide array of tasks. Their success has also spurred extensions into traditional multimodal domains [17, 41, 39, 40, 37], combining language understanding with other modalities like vision [19, 16, 24,3, 3, 10, 1, 22, 38]. However, the sequen- tial nature of AR generation can lead to challenges such as error propagation and limitations in bidirectional context modeling for certain tasks.
In response to these and other considerations, diffusion models [33, 14] have emerged as powerful alternative. While initially demonstrating success in continuous domains like images [9, 29]. signifi- cant effort has been dedicated to adapting them for discrete data, particularly text [2, 20, 11]. Early
approaches explored discrete state-space diffusion [2] or continuous diffusion in embedding spaces
(e.g., Diffusion-LM [20]. DiffuSeq [1]]), showcasing potential for controllability but often lagging behind AR models likelihood or efficiency.
A particularly relevant and successful direction has been the development of Masked Diffusion Models (MDMs) [30, 32]. These models formulate text generation as an iterative mask-infilling process, learning to reverse gradual masking procedure. Prominent examples like LLaDA [26] have demonstrated that MDMs can achieve competitive performance with strong AR models on various language tasks, even at scale. These models operate by iteratively refining a sequence, making them a prime candidate for ine-grainet guidance techniques. Our work focuses specifically on enhancing conditional generation within such iterative, masked diffusion frameworks like LLaDA.
2.2 Classifier-Free Guidance in Generative Models
Classifier-Free Guidance (CFG) [15] has become a cornerstone technique for improving sample quality and conditional control in diffusion models, initially popularized in image synthesis [29]. elegantly steers generation towards condition by interpolating between conditional and unconditional model predictions during the reverse process, avoiding the need for separate classifier training. This is typically achieved by training the diffusion model with occasional dropout of the conditioning signal (e.g., null text prompt), enabling to produce both conditional and unconditional outputs.
The adaptation of CFG to language diffusion models [23, 25] presents unique considerations. A common practice is simulate the unconditional prediction by providing a static input, such as fully masked target sequence. While effective, this static unconditioning strategy poses a limitation, particularly for iterative MDMs. As the model refines the text sequence over multiple steps, its internal state of certainty varies across different token positions and time steps. A fixed unconditional baseline fails to adapt these dynamics, potentially leading to suboptimal or misaligned guidance.
3 Methodology
Our work introduces Adaptive Classifier-Free Guidance (A-CFG), a novel enhancement to the Classifier-Free Guidance (CFG) paradigm. A-CFG is specifically designed for iterative masked language models (MLMs) and aims to improve generative control by dynamically constructing the unconditional input required for CFG. This achieved by leveraging the model's instantaneous predictive confidence regarding its current non- [MASK] tokens, allowing guidance to be more precisely targeted towards regions of the sequence where the model exhibits uncertainty. Figure provides high-level comparison of standard CFG with our proposed A-CFG.
3

---

Nut Prompt Chaositen Free Guidance
Adeptive Clasatier-Fre Guidance (A-CFG)
Mite")
Castiiedd
Your tio
Saaee Level Ourd Lossuatee
jecbor Furm!
Uncanotiond Loght Computidao
Modd
La Mex
Los Our dince Raaking Cled dddes
wheeee
Go dingss
Computateo
Al
2i0
Figure 2: Overview of (left) standard Null Prompt Classifier-Free Guidance and (right) our proposed Adaptive Classifier-Free Guidanee (A-CFG) at a single generation step k. In standard CFG, the unconditional input often involves masking the entire prompt or using null prompt. In A-CFG, after computing conditional logits from x(), token-level confidences for all non- [MASK] tokens in x*) are assessed. Tokens with low confidence (orange/red in illustration) are temporarily (k) re-masked to [MASK] to create the dynamic unconditional input Xuncond' This allows the CFG mechanism to focus guidance on areas of model uncertainty within the current sequence.
3.1 Preliminaries
Before detailing A-CFG, we briefly review the foundational concepts: iterative masked language modeling and standard classifier-free guidance.
Iterative Masked Language Models (MLMs). Our A-CFG framework operates within the context of iterative generation, characteristic of many masked language models like LLaDA. Text generation commences with an input sequence x that is either partially or entirely populated with special [MASK] tokens. The generation unfolds over series of steps. At each step k. the model Mo predicts replacement tokens for a subset (or all) of the extant [MASK] tokens. This iterative process progressively refines the sequence x. (R) until a complete output x0) is achieved (where typically decreases from initial number of steps down to 0). The predictive mechanism involves the
model M& (x) producing logits over the vocabulary for the positions designated for infilling.
Classifier-Fret Guidance (CFG). Classifier-Free Guidance [15] a widely adopted technique for enhancing sample quality and controllability in conditional generative models. CFG operates by linearly interpolating the outputs derived from a conditional model prediction, Lecond (8) c). and an unconditional model prediction, Tuncona (A) 6). Here, represents the conditioning information (e.g., textual prompt), and ( signifies null broadly unconditional context. The construction of this unconditional input can vary; for instance, some approaches derive an unconditional--ike term from masked version of the conditioning prompt itself [25]. The guided logits, Lguided, are computed as:
Lguided (x c) Luncond (x), 0) + (w + 1) -LL))((oon (x
Luncoma (A), ()),
(1)
where L denotes the model's 's output logits, and w is the guidance scale. A guidance scale w >> 0 amplifies the influence of the conditioning signal c. A central challenge in applying CFG, particularly in iterative MLM frameworks, is the effective definition and derivation of the unconditional logits
Luncona (x 0), an issue directly addressed by our A-CFG approach.
3.2 Adaptive Classifier-Free Guidance (A-CFG)
Standard CFG, while effective, often relies static generic definition for the unconditional prediction (x 0) when applied to iterative MLMs. Typically, this involves using null

---

Algorithm Adaptive Classifier-Freee Guidance (A-CFG) for one generation step k:
1: Input: Current sequence x(*), conditioning c, model Mo, guidance w, re-masking proportion p.
2: Output: Guided logits (4)
guided"
3: cood & M)(x(())
(A)
Compute conditional logits
4: reemaskable f- (i (xx()), + [MASK]]
c(k)
Identify all non- [MASK] token indices
5: CONF(A) x- 0
6: for j E ((k)
rremaskable do
7:
3 max, (I (b) )),
D Assess confidence for remaskable tokens
Add (c/ .J) CONF((
8:
9: end for
10: S low-coof 0
((A)
11: if [C
> then
rreaakaallel
12:
N
[p- (c)
remaskable
13:
Nactual
<- min( Natarge (k(k)
14:
if' N m
Aractual > 0 then
(k) in ascending order. Sort CONT(R) by confidence values of
15:
((k) Spow indices j of the first Nactual elements in sorted CONFL))
16:
17: end
18: end if
19: x uuncond f- (()
(k:
20: for E s(k)
cow- do
(k)
21:
(Xuncond);
[MAKK]
Create dynamic unconditional input
22: end for
23: L (A)
ungondd Mg uncond
(k)
Compute unconditional logits Apply CFG formula
24: I. (k)
"guided uncond + (w + 1) (I. - I, (R)
uuncond)
25: return guided
prompt masking all prompt tokens to simulate unconditional state. In complex generation scenarios, the model's uncertainty can fluctuate significantly. A static or predefined unconditioning strategy might therefore apply guidance indiscriminately, potentially misdirecting the generation process or failing to provide sufficient correction where it is most needed. This observation motivates A-CFG. Our core intuition is that the "unconditional" component of CFG can be made and targeted if it is dynamically informed by the model's own state uncertainty regarding its current, non-masked tokens. Instead of global, context-agnostic unconditioning, A-CFG focuses the guidance mechanism specific token positions within the sequence (4)) where the conditional model currently exhibits the greatest predictive ambiguity. By temporarily re-masking these low- confidence non- [MASK] tokens to form the input for Luncond, we compel the model to reconsider its predictions at these critical junetures. This adaptive unconditioning aims to make the guidance signal (Trond Tumeomut) more discriminative and effective, leading more nuanced and efficient control over the generation process.
potent
3.2.1 A-CFG Process
The A-CFG process executed each iterative generation step k. A detailed algorithmic description of this process is provided in Algorithm 1. Given the current sequence (() (which includes the prompt and partially generated text), A-CFG involves the following operations:
Conditional Logit Computation. The base model Mo first computes the standard conditional
logits based on the current full input (k);
(k) L
cond Mx(x(1))
(2)
5

---

These logits represent the model's initial predictions under full conditioning by and any already filled tokens in (8))
Token-Leevel Confidence Assessment. From & we assess the model's confidence in its predic- tions for all non- [MASK] token positions within the current sequence x(*), Let c(k) rremaskable be the set of
indices of all token positions j such that (x(1)) [MASK]. For each position j E C(A) rrmaskablee"
We compute the softmax probability distribution cond (L over the vocabu- lary.
The confidence score for position j defined the maximum probability in this distribution:
(k) max maxx((P j... This corresponds to the probability the token that the model
(k) would predict with highest likelihood for position j based on ((() A low Cj suggests the model is uncertain about the token (x(*)) or its alternatives that position.
While other confidence metries (e.g., entropy of coodd.) could be considered, find that the maximum softmax probability provides a simple yet effective measure.
Identification of Low-Confidence Tokens for Re-masking. Based the assessed confidences
(k) for tokens at positions j E c(k)
+remaskable: a subset Stowcomt low-conf C c(k)
rremasaable positions exhibiting the
lówest confidence is selected for adaptive re-masking. The extent of this adaptive intervention is controlled by a re-masking proportion hyperparameter, p. The target number of tokens to re-mask,
N is calculated as proportion of the total number of non- [MASK] tokens within ((k) step
remaskable at
N targge m
= [p [
remaaskable 17.
(3)
This heuristic scales the intensity of A-CFG intervention with the amount of non-masked content available for re-evaluation. The actual number of tokens selected for re- re-masking, Nactual is Nactull
min(N (urgee [c(&) scremaskabll)) ) If (c(k) remaskablel = or Nacuual 0, no re-masking occurs for A-CFG, and c(k)
Sww.conf empty. Otherwise, S(ii) www.conf contains the indices of these Naacuaal tokens with the lowest
confidence scores.
Construction of the Dynamic Unconditional Input. A localized "unconditional" input sequence, (k)
unoond is synthesized by modifying the current sequence (k), Specifically, the non- [MASK] tokens low-conl are replaced with the special [MASK] token:
at positions identified in ((k)
[MASK] (x(()))
if E (kk low-conf'
(4)
otherwise.
If low-cont empty (i.e., no tokens were selected for e--making), then x is identical (A)
(k)
(k)
When re-masking occurs, this transformation yields an input where the model is explicitly prompted to reconsider its predictions for positions it was previously uncertain about, effectively creating
more challenging or "less informed" context for these specific tokens by erasing its prior commitment those positions.
Unconditional Logit Computation. Mş computes the "unconditional" logits:
Using this dynamically constructed input x uncond the model
(b)
I (k)
ungond Me M
(k)
If no adaptive re-masking occurred (i.c., X = (k)), then (M) will be identical to (()) These logits, (4) u reflect the model's predictions when key points of prior uncertainty are deliberately obscured (or not, if no such points met the criteria), providing targeted bascline for guidance.

---

Application of CFG Formula for Guided Logits. Finally, the guided logits L I 'guided for the current
step k are computed using the standard CFG formula (Equation 1), now employing the adaptively derived L ung uncond and the original L (k)
cond
L unnzond
+ (w + 1) (I
(R)
(6)
"uided
T (k)
( (e.g.. due to no adaptive re-masking), then (k)
[ implying that A-CFG
uneend
"guided
applics no effective guidance shift in this specific scenario. These L guided are then used to sample or select the tokens to infill the [MASK] positions for the next iteration x (
(k)
Experiments
In this section, we empirically evaluate the effectiveness of Adaptive Classifier-Free Guidance (A-CFG). We first describe our experimental setup, including datasets, baseline models, evaluation metrics, and key implementation details. We then present quantitative results from Table 1, comparing LLaDA with A-CFG against LLaDA with standard CFG, LLaDA without guidance, and other state- of-the-art models. Subsequently, we conduct ablation studies to analyze the impact of A-CFG's core hyperparameter. Finally, we provide qualitative examples to illustrate the behavior and benefits of our proposed method.
4.1 Experimental Setup
4.1.1 Datasets and Metries
We evaluate A-CFG on a diverse suite of standard benchmarks covering general language understand- ing, mathematical and scientific reasoning, and planning tasks.
General Language Understanding: MMLU (Massive Multitask Language Understanding) [12],
BBH (Big- Beng-Bench Hard) [34], ARC-C (AI2 Reasoning Challenge Challenge Set) [7], Hellaswag [44].
TruthfulQA [21], WinoGrande [31] I, and PIQA (Physical Interaction QA) [4].
Mathematics & Science Reasoning: GSM8K (Grade School Math 8K) [8], MATH [13], and GPQA (Graduate-Level Google-Proof Q&A) [28].
Planning Tasks: Countdown [42] and Sudoku [42].
Evaluation mode. Closed-form tasks supply prompt with finite set of candidate answers; we compute each candidate's conditional log-likelihood and select the most likely. Open-ended tasks require free-form generation; we sample responses and score them with task-specific metrics such as exact-match accuracy.
Likelihood estimation. For likelihood-basse evaluations we approximate the conditional perplexity bound with Monte-Carlo sampling. single sample suffices when only one target token is queried (e.g. MMLU). We adopt the same setting as LLaDA, for all other multiple-toke tasks we draw 128 samples, which we found stabilise variance without adding prohibitive cost.
Generation hyper- parameters. Unless otherwise stated, we set the answer length to 256 tokens and the reverse diffusion process for 256 steps (one token revealed per step).
4.1.2 Baseline Models and Methods
Our primary evaluation centers on the LLaDA 8B model, assessed under three guidance scenarios:
No Guidance (base LLaDA), 2) Standard CFG (Std CFG), where conventional Classifier Free Guidance [15] uses fully masked target sequence for unconditioning, and 3) our proposed Adaptive CFG (A-CFG). For both Std CFG and A-CFG, guidance scale w is tuned. To investigate A-CFG's broader applicability, we also evaluate on the Dream-7B diffusion model [43] against its baseline. All results are contextualized against publicly reported scores from comparable autoregressive (AR) models like LLaMA3 8B [35], LLaMA2 7B [36], and Qwen2 7B [6], as detailed in Table 1.
7

---

Table 1: Benchmark Results of Pre-trained LLMs. LLaDA and Dream-7B diffusion models.
Baseline scores for LLaDA 8B and Dream-7B reflect our own re-evaluation under a consistent experimental protocol. Results indicated by are sourced from [6]. The numbers in parentheses represent the number of shots used for evaluation. "_" indicates unknown data or data not applicable.
Benchmark
LlaDA 6B
LLabi 00 (8td CPG)
LLabA 08 (A CP-)))
Dreax- 70
Drean 70 (A (L-FF))
LLah63 6B
LLaht2 7B
Qwea2 70'
Model
Diffasion
Diffusion
Diffusion
Diffasion
Diffusion
AR
AR
AR
Gleneral Tasks
65.9
65.8
66.1 (5)
69.5 (5)
(9)
65.4
45.9
ARC-C
45.5
46.3
47.8
(0)
60.6
Hellaswag
70.8 (0)
71.4 (0)
72.6 (0)
73.3 (0)
74.4
79.1 (0)
76.0 (0)
80.7
TruthfuQQA
45.5
45.1 (0)
46.2 (0)
43.9 (0)
45.1 (0)
44.0 (0)
39.0 (0)
54.2 (0)
WinoGrande
74.5 (5)
(5)
75.9 (5)
73.3 (5)
72.5
77.3
72.5 (5)
77.0
PIQA
74.4 (0)
(0)
80.6
(0)
Mathenaties Science
CSM8K
70.7 (4)
70.8
73.5 (4)
76,9 (4)
77.9 (4)
53.1
14.3
80.
GPQA
(5)
29.4 (5)
33.3 (5)
36.6 (5)
36.8 (5)
25.9
25.7 (5)
3088
Planning Tasks
Countdown
15.3 (8)
14.2 (8)
15.8 (8)
14.6 (8)
15.2 (8)
3.7 (8)
(8)
(8)
0.0 (8)
4.1.3 Implementation Details
For LLaDA's iterative generation, we use 256 sampling steps with low-confidence remasking. For Standard CFG, the guidance scale was selected from {0.5, 1.0, 1.5, 2.0} based on performance on the validation set each respective task. For our A-CFG, the guidance scale w was similarly tuned. Once value of w is chosen for given model, the same is kept fixed across all downstream benchmarks for that model. The adaptive re-masking proportion o (determining the fraction of previously generated tokens to re-mask based on low confidence, as defined in Section 3,2.1) was set to 0.7. The confidence for token selection A-CFG is based on the softmax probability of the predicted token at each masked position. All experiments conducted using NVIDIA H800 GPUs.
4.2 Benchmark Results
The efficacy of Adaptive Classifier-Free Guidance demonstrated in Table 1, which presents comprehensive comparison of LLaDA 8B equipped with A-CFG against its counterparts using no guidance and standard CFG, alongside other leading diffusion and autoregressive models.
A-CFG Enhances LLaDA Performance: Our results clearly indicate that A-CFG substantially elevates the performance of LLaDA 8B. Crucially, A-CFG consistently outperforms LLaDA 8B with Standard CFG, underscoring the benefits of its dynamic, confidence-aware unconditioning mecha- nism. The advantages are particularly pronounced on complex reasoning and planning benchmarks; for instance, on GPQA, A-CFG achieves score of 33.3, a +3.9 point improvement over Standard CFG (29.4), 2), and on the Sudoku planning task, A-CFG (42.0) surpasses Standard CFG (34.0) by a sig- nificant +8.0 points. This trend of superior performance over Standard CFG extends to mathematical reasoning (e.g.. +2.7 points on GSM8K) and across general language understanding tasks such as ARC-C, Hellaswag, and WinoGrande. When compared to LLaDA 8B with No Guidance, A-CFG also yields substantial gains, for example, +7.2 points on GPQA and +7.0 points on Sudoku. These findings highlight A A-CFG's capability more effectively steer the iterative generation process in LLaDA, leading to improved task adherence and overall output quality compared to both unguided generation and conventional CFG.
Generalizability Other Diffusion Models: To assess whether the principles of A-CFG extend beyond LLaDA, we integrated it into the Dream-7B model. Preliminary results in Table 1 suggest that A-CFG brings similar benefits, for instance, improving Sudoku performance by +8.0 points (80.0 vs. 72.0) and ARC-C by +1.0 point (60.8 vs. 59.8) for Dream-7B. These observations suggest that A-CFG's adaptive unconditioning is a promising method for enhancing other iterative masked diffusion models.

---

Competitive Standing Against Autoregressive Models: Equipped with A-CFG, the diffusion-
LLaDA 8B demonstrates a strong competitive posture against contemporary autoregressive (AR) models of comparable scale. LLLaDA 8B (A-CFG) particularly excels in mathematical reasoning. with a GSM8K score of 73.5 that surpasses several listed AR counterparts like L.LaMA3 8B (53.1). On the challenging GPQA benchmark, its score of 33.3 is notably higher than LLLMA3 8B (25.9) and competitive with Qwen2 7B (30.8). The Sudoku planning task further showcases this strength, where LLaDA 8B (A-CFG) achieves 42.0, markedly outperforming LLaMA3 8B (0.0). While leading AR models such as Qwen2 7B still exhibit an advantage on some general language understanding benchmarks, A-CFG significantly narrows the performance gap and, in specific domains demanding complex reasoning or planning, positions LLaDA as a compelling alternative.
In summary, the empirical results affirm A- as a potent enhancement for iterative diffusion language models. It not only improves upon standard CFG techniques but also enables diffusion models like LLaDA to achieve highly competitive, and in some cases superior, performance compared to strong AR baselines, especially in tasks requiring sophisticated reasoning.
4.3 Ablation Studies
To elucidate the contributions of A-CFG's core components and assess its sensitivity to key hyperpa- rameters, we conducted targeted ablation studies. This section focuses on the impact of the adaptive re-masking proportion, critical parameter in A-CFG.
4.3.1 Impact of the Adaptive Re-masking Proportion (p)
We investigated the influence of p on the ARC-C test set, chosen as a representative benchmark where A-CFG demonstrated clear benefits and sensitivity to guidance parameters. The main LLaDA 8B (A-CFG) result for ARC-C (47.8 accuracy) reported in Table 1 employed p 0.7.
Table 2a presents the performance on ARC-C as p is varied across the range [0.1, The results show a clear trend: ARC-C accuracy improves steadily as p increases from 0.1 (45,9%) to 0.3 (46.5%) 0.5 (46.8%) and culminates 0.7 (47.8%). This suggests that for a task like ARC-C, more substantial re-masking of low-confidence generated tokens is beneficial, allowing A-CFG to exert stronger corrective influence. However, increasing p further to 0.9 leads to a decline in performance, indicating that excessively aggressive re-masking can become counterproductive, potentially by erasing too much valuable context from the already generated sequence.
4.3.2 Impact of the Guidance Scale (w:)
Beyond the re-masking proportion p. the guidance scale w is a critical hyperparameter for any CFG-based method. We varied w across the set {0.5, 1.0, 1.5, 2.0). the same range used for tuning in our main experiments. Table 2b illustrates the performance observe that A-CFG performance is sensitive to the guidance scale. Specifically. a small w = 0.0 (equivalent to CFG guidance beyond the adaptive masking) yields a baseline accuracy of 45.5%. As w increases, accuracy improves, reaching a peak of 47.8% at to = 0.5 and w = 1.0. This suggests that a moderate guidance strength effectively leverages the dynamically constructed unconditional input from A-CFG. However, further increasing wo to 1.5 and 2.0 leads to a slight degradation in performance (47.5% and 47.6%, respectively). This indicates that an overly strong guidance scale might overemphasize the conditional signal at the expense of fluency or correctness, even with A-CFG's targeted unconditioning, The optimal performance at w 0.5 aligns with the value used for ARC-C in our main results (Table 1).
ARC-C w is adjusted. We
4.4 Qualitative Analysis
To provide further insight into A-CFG's dynamic mechanism, Table 3 visualizes the iterative re- finement process for mathematical reasoning examples from the GSM8K dataset. These examples illustrate how A-CFG navigates the generation process. For instance, the "Natalia's clips" problem, one can observe that while foundational elements are established in early steps (c.g., Natalia, sold), crucial components of the arithmetic reasoning, such as operators, intermediate results, or the final sum, are often resolved or corrected in later iterations. This behavior aligns with A-CFG's core principle: by identifying tokens or positions where the model exhibits low predictive confidence during the iterative process (potentially due to incomplete inconsistent intermediate reasoning

---

Table 2: Ablation studies on ARC-C. (a) Impact of guidance scale (w). (b) Impact of adaptive re- re-masking proportion (p). The main result for ARC-C in Table used p = 0.7 and w = 0.5. Scores are Accuracy (%).
(a) Re- masking Proportion (p)
(b) Guidance Scale (w)
Benchmark
Re-masking Proportion (p)
Benchmark
Guidance Scale (w)
0.1
0.3
0.5
0.7
0.9
0.0
0.5
1.0
1.5
2.0
ARC-C
45.9
46.5
46.8
47.8
46.0
ARC-C
45.5
47.8
47.8
47.5
47.6
Table Visualization A-CFG's iterative refinement process math reasoning tasks. Darker shades indicate tokens that were filled or corrected in later stages of the adaptive generation, often representing points of initial uncertainty that A-CFG helped resolve.
Task
A-CFG Refinement Process
Prompt:
Natalia sold clips to of her friends. She sold clips to each friend. Then she bought more clips. How many clips does Natalia have now?
Matalia
2o1d
4
Irienda.
clips
each.
So,
4
8
32
clips.
Then
bought
more.
Natalia
has
32
15
clips.
Anever:
47
Prompt:
John has 12 apples. gives half to Mary. Then Mary buys twice as many apples as she received from John. How many apples does Mary have now?
John
has
12
apples.
Gives
half
to
Mary-
So
Mary
geta
12
2
apples.
Marry
buya
many
Mary
apples.
steps), A-CFG dynamically re-masks these specific points. This targeted re-masking compels the model to reconsider and refine its predictions in these areas of ambiguity, thereby facilitating the construction a coherent and accurate multi-step reasoning chain. Similarly, in the "John's apples" example, later steps refine the calculation, ensuring the intermediate and final quantities derived (e.g., 6+12=18). These qualitative examples underscore A-CFG's ability to leverage its adaptive unconditioning focus guidance evolving points of uncertainty, thereby enhancing the model's capacity to resolve errors and improve the fidelity of complex, multi-step generations.
correctly
5
Conclusion
This paper introduced Adaptive Classifier-Free Guidance (A-CFG), a novel method to enhance conditional generation in iterative masked language models. By dynamically constructing the unconditional input for CFG based on the model's instantaneous predictive confidence in its already generated tokens, A-CFG offers experiments, particularly within the L.LaDA framework, demonstrate that A-CFG significantly outperforms standard CFG approaches and unguided baselines, yielding substantial improvements on diverse benchmarks, especially in complex reasoning and planning tasks. The results also highlight A-CFG's potential bolster the competitiveness of diffusion-based language models against autoregressive counterparts. This work underscores the value of leveraging model uncertainty for more nuanced control discrete diffusion, opening promising avenues for future research into adaptive generation strategies.
targeted and responsive guidance mechanism. Our extensive
References
[1] R. An, S. Yang. M. Lu, R. Zhang, K. Zeng, Y. Luo, J. Cao, H. Liang, Y. Chen, Q. She, et al. Mc-llava: Multi-concept personalized vision-language model. arXiv preprint arXiv:241 241.::44111117000 2024.
10

---

[2] J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981-17993, 2021.
[3] A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y. Hanafy, W. Zhu, K. Marathe, Y. Bitton, S. Gadre, S. Sagawa, et al. Openflamingo: An open-source framework for training large autoregressive lision-language models. arXiv preprint arX0v:2308.013 2023.
[4] Y. Bisk, R. Zellers, Gao, Y. Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, 2020.
[5] T. B. Brown. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
[6] Y. Chu, J. Xu, Q Yang, H. Wei, X. Wei, Z. Guo, Y. Leng, Y. Lv, He, J. Lin, al. Qwen2-audio technical report. arXiv preprint arXiv22407.10754 2024.
[7] P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: arXiv:1803.0550 05457, 2018.
[8] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. 2021.
[9] Dhariwal and A. Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780- 44:8700-8794, 2021.
[10] R. Fang, S. Yan, Z. Huang, J. Zhou, H. Tian, J. Dai, and H. Li. Instructseq: Unifying vision tasks with instruction-condiioone multi-modal sequence generation. arXiv preprint vrXiv:2311.18833 2023.
[11] S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong. Diffuseq: Sequence sequence text generation with diffusion models. arXiv preprint I00:22200008933, 2022.
[12] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Mcasuring massive multitask language understanding. arXiv preprint (xXv:2009.033000 2020.
[13] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, Basart, Tang, D. Song, and J. Stein- hardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.0 038744, 2021.
[14] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6833 6851, 2020.
[15] and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12595 2022.
[16] J. Hong, S. Yan, J. Cai, X. Jiang, Y. Hu, and W. Xie. Worldsense: Evaluating real-world omnimodal understanding for multimodal 1mss. arXiv preprint arXiv:2502.04322, 2025.
[17] L. Hong, S. Yan, R. Zhang, W. Li, X. Zhou, P. Guo, K. Jiang, Y. Chen, J. Li, Z. Chen, et al. Onetracker: Unifying visual object tracking with foundation models and efficient tuning. In Proceedings of the IEEE//VV conference on computer vision and pattern recognition, pages 10079-19091, 2024.
[18] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. L. Casas, Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arXiv preprint 2:.00:2310006625, 2023.
[19] B. Li, Y. Zhang, D. Guo, Zhang, F. Li, H. Zhang, Zhang, P. Zhang, Y. Li. Z. Liu, al. Llava-onevision: Easy visual task transfer. arXiv preprint SXXI::2408.03322 2024.
[20] X. Li, J. Thickstun, I. Gulrajani, P. S. Liang, and T. B. Hashimoto. Diffusion-Im improves controllable text generation. Advances Neural Information Processing Systems, 35:4328- 4343, 2022.
11

---

[21] S. Lin, J. Hilton, and O. Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.0799 00900, 2021.
[22] W. Lin, X. Wei, R. An, Gao, B. Zou, Y. Luo, S. Huang. S. Zhang, and H. Li. Draw-and- understand: Leveraging visual prompts to enable MLLMs to comprehend what you want. In The Thirteenth International Conference on Learning Representations, 2025.
[23] Lovelace, V. Kishore, Chen, and K. Q. Weinberger. Diffusion guided language modeling. arXiv preprint arXiv:2408. 2400:022000 2024.
[24] F. Ma, Y. Zhou, Z. Zhang, S. Yan, H. Li, Z. He, S. Wu, F. Rao, Y. Zhang, and X. Sun. Ee- milm: A data-efficient and compute-efficien multimodal large language model. arXiv preprint arXiv:2408. / /795, 2024.
[25] S. Nie, F. Zhu, C. Du, T. Pang. Q. Liu, G. Zeng, M. Lin, and C. Li. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410. 18514, 2024.
[26] S. Nie, Zhu, Z. You, X. Zhang, Ou, J. Hu, J. Zhou, Y. Lin, J.-R. Wen, and C. Li. Large language diffusion models. arXiv preprint 2xXiv:2502.00992,, 2025.
[27] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[28] D. Rein, B. L. Hou, A. Stickland, J. Petty, R. Pang, J. Dirani, Michael, and S. R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.1202, 2023.
[29] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 00684-10695 2022.
[30] S. S. Sahoo, M. Arriola, Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, and V. Kuleshov. Simple and effective masked diffusion language models. arXiv preprint 2:Xiv:2406.07524. 2024.
[31] K. Sakaguchi, R. L.. Bras, C. Bhagavatula, and Y. Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
[32] Shi, K. Han, Z. Wang, A. Doucet, and M. K. Titsias. Simplified and generalized masked diffusion for discrete data, arXiv preprini arXiv:2406.043229 2024.
[33] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 2256-2266 PMLR, 2015.
[34] M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. Chi, Zhou, et al. Challenging big-bench tasks and whether chain-of-thowe of-thought can solve 2022.
them. arXiv preprint
[35] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. ,,
[36] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint ar arXiv22307.009888 2023.
[37] Z. Xiao, S. Yan, J. Hong. J. Cai, X. Jiang, Y. Hu, J. Shen, Q. Wang, and C. G. Snoek. Dy- naprompt: Dynamic test-time prompt tuning. arXiv preprint arXiv:250I. 16404, 2025.
[38] S. Yan, Han, Tsai, H. Xue, R. Fang, L. Hong, Z. Guo, and R. Zhang. Crosslmm: Decou- pling long video sequences from Imms via dual cross-attention mechanisms. arXiv preprint 250iv:2505.1 17020, 2025.
[39] S. Yan, O. Li, Cai, Hao, X. Jiang, Y. Hu, and W. Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406.194. 2024.
12

---

[40] S. Yan, X. Xu, R. Zhang, L. Hong, W. Chen, W. Zhang, and W. Zhang. Panovos: Bridging non con-panoamiic and panoramic views with transformer for video segmentation. In European Conference Computer Vision, pages 346-365. Springer, 2024.
[41] S. Yan, R. Zhang, Z. Guo, W. Chen, W. Zhang, H. Li, Y. Qiao, H. Dong, Z. He, and P. Gao. Referred by multi-modality: A unified temporal transformer for video object segmentation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 6449-6457 2024.
[42] J. Ye, J. Gao, S. Gong, L. Zheng, X. Jiang, Z. Li, and L. Kong. Beyond autoregression: Discrete diffusion for complex reasoning and planning. arXiv preprint arXiv:2410.14157. 2024.
[43] Ye, Z. Xie, Zheng, Gao, Z. Wu, Jiang, Z. Li, and L. Kong. Dream 7b, 2025.
[44] R. Zellers, Holtzman, Y. Bisk, Farhadi, and Choi. Hellaswag: Can machine really
finish your sentence? arXiv preprint arXiv: arXiv: 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the main contribution (A-CFG) and its demonstrated improvements standard CFG and baselines, which aligns with the experimental results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses in Section 4.3.1 (lines 293-296) that excessively aggressive re-masking (a high p value) be counterproductive, indicating limitation.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification: The paper presents an empirical method and experimental results, without formal theoretical claims proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides implementation details (Section 4. 1.).), model specifics, dataset descriptions (Section 4.1.1), and pseudocode (Algorithm 1), enabling reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper uses publicly available benchmark datasets and models, and the authors intend to release code per NeurlPS policy.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
(eglii/ggides/Codeeeee
ooPoliccy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. cctpublic//aeecccccccccccccc /gwii/es///////eeeeee /Code//dmddsbiii wwrteSuubmsssioon. ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental settings, including datasets (4.1.1), baseline models (4.1.2), and hyperparameter choices/tuning (4. 1.3),, are specified.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports absolute performance improvements on benchmarks (Table 1), which show substantial gains indicative of significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper states that experiments were conducted on NVIDIA H800 GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https https://nnnt /tte:r//n/eeepeeeeeeeee cc cc/public/ httt:///tthhiheieeeleeeeeleeeeee
Answer: [Yes]
Justification: The research focuses on algorithmic improvements for language models and does not present ethical concerns that would violate the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The paper focuses on foundational algorithmic improvements for generative models and does not explicitly discuss broader societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper proposes a method for existing models and does not introduce new high-risk models or datasets requiring specific safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper cites original sources for all datasets and models used (e.g., LLaDA, MMLU, GSM8K), respecting standard academic crediting practices.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper introduces new method (A-C (A-CFG), which is documented, but does not release new datasets or standalone models as assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing direct experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The research does not involve human subjects, so IRB approval is not applica- ble.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The paper's core methodology (A-CFG) is designed for and evaluated on large language models (LLaDA, Dream-7B), which is central to the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https: for what should or should not be described.
http://ccccc/oeeeeec//e//////// /LLM)
20


Reviews:
[{"review_id": "xKQV33LNKU", "reviewer": "Reviewer_wcf3", "summary": "In the context of language diffusion models, Classifier-Free-Guidance (CFG) traditionally enhances control for generative models by interpolating between conditional and unconditional latents. A-CFG manipulates the unconditional state by masking away tokens where there is high uncertainty. With this simple approach, there is significant improvement for reasoning tasks such as GPQA.", "questions": "- It would be nice to compare a with and without training based on A-CFG. Is there an ablation on this?\n- It would also be nice to evaluate Diffusion Language models on agentic domains, such as SWE, as these are new and upcoming workloads that will dominate in the coming 1-2 years.", "limitations": "None. great work!", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "This work is simple and significant. The authors clearly identify the problem with unconditional state for Diffusion Language models and solve it by masking out tokens of least certainty.\n\nStrengths:\n- Identification of problem - Figure 1 hits the key insight; model's confidence for tokens varies a lot across token positions and generation step. It's best to focus, at any given period, on the tokens of least certainty.\n- Simple and General - Successfully applied to both LLaDA and Dream-7B diffusion models, suggesting wide applicability to different diffusion language models. Algorithm is also very simple, as it is just a pure inference time approach (not large compute overhead)\n- Strong Empirical Gains - Demonstrates substantial improvements over standard CFG and unguided baselines on diverse benchmarks (GPQA, GSM8K, ARC-C, Sudoku, etc.).\n\nWeaknesses:\n- Would love to see failure modes, where A-CFG does worse than CFG.\n- Otherwise great work! I can't think of other weaknesses.", "was_revised": true, "final_justification": "Congrats authors, hope this work gets in!", "rebuttal": "#### We sincerely appreciate the reviewer's positive comments on our motivation and experiments, along with your thoughtful feedback and suggestions. We hope our response can address your concerns.\n\n**Q1: “Would love to see failure modes, where A‑CFG does worse than CFG.”**\n\nThank you for raising this critical point. The primary failure mode for A-CFG arises when its corrective mechanism becomes overly aggressive, a scenario directly explored in our ablation studies.\n\nThis failure mode emerges from disrupting coherent generation, which we observed when the re-masking proportion ρ is set too high. Consider a multi-step arithmetic task from GSM8K: after correctly generating an intermediate step like 4 * 8 = 32, the model might exhibit slight uncertainty about what follows. An overly aggressive ρ (e.g., 0.9) could cause A-CFG to catastrophically re-mask not only the uncertain tokens but also the crucial, correctly-generated intermediate result 32. By erasing this vital context, the model's ability to complete the subsequent step (32 + 15) is severely compromised, often leading to an incoherent or nonsensical final answer.\n\nThis behavior provides the direct intuition for the performance degradation seen in our ablation study (Table 2a), where increasing ρ beyond an optimal point on ARC-C becomes counterproductive. We will dedicate a part of the Limitations section to a detailed analysis of this failure mode, supported by these qualitative examples.\n\n**Q2: “It would be nice to compare a with and without training based on A‑CFG. Is there an ablation on this?”**\n\nThank you for this insightful suggestion. In fact, LLaDA is already pre-trained under a dynamic-mask curriculum that randomly selects a mask ratio between 0.0 and 1.0 for every sample. During training, the model already sees many branches that mask different amounts of text. A-CFG just copies this same pattern at inference by re-masking the low-confidence tokens. Because the mechanism is implicitly “built in,” no additional fine-tuning is required to realise A-CFG’s gains. We will add one sentence to Section 3.2 clarifying that the pre-training equips LLaDA to handle adaptive masking.\n\n\n**Q3: “It would also be nice to evaluate diffusion language models on agentic domains, such as SWE.”**\n\nThank you for suggesting a broader evaluation scope. We agree that software‑engineering (SWE) benchmarks are an exciting testbed for agentic systems that autonomously call tools, read/write files, and iterate over multi‑step plans. By contrast, our study focuses on core language models—LLaDA and Dream‑7B—which generate text in a single‑pass masked‑diffusion loop and have no built‑in tool‑use or file‑manipulation interface. Directly applying them to full‑fledged SWE agents would therefore require substantial additional infrastructure (code execution sandbox, retrieval modules, planner, etc.) that is orthogonal to the contribution of A‑CFG. We will highlight this distinction between agent systems and stand‑alone language models in the discussion section, and we view incorporating A‑CFG into an agentic framework as valuable future work."}, {"review_id": "Rw4qCbQfxb", "reviewer": "Reviewer_Du1A", "summary": "This paper proposes Adaptive CFG for masked diffusion language models to resolve the issue that standard CFG only takes static unconditional inputs and do not adapt to the model's uncertainty. The proposed A-CFG dynamically construct the unconditional input by re-masking tokens with low confidence estimated by the model itself. Empirically, A-CFG shows consistent improvement on models including LLaDA and Dream-7B across varying tasks.", "questions": "1. Does A-CFG adds inference-time overhead to discrete diffusion models? If so, what is the latency? Some discussion or quantitative analysis or runtime cost would be helpful.\n2. How does A-CFG interact with other sampling strategies, such as temperature or top-k. Is the method sensitive to these changes?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. The paper is well motivated and clearly identifies and addresses a limitation of standard CFG.\n2. The proposed method is simple, clean and effective. It re-masks low-confidence tokens to construct a more targeted unconditional input, which is easy to integrate into existing diffusion models framework.\n3. Empirical results are strong. Experiments are conducted on large models including LLaDA and Dream, and show consistent improvement across various tasks.\n4. The paper is well written, analysis and ablation is well-designed. \n\n## Weaknesses\n1. The proposed A-CFG is explained clearly in section 3.2.1, however, more intuition or justification would be helpful. Specifically, more justification for why re-masking low-confidence tokens leads to improved guide beyond empirical validation would strengthen the method.", "was_revised": true, "final_justification": "My concerns regarding the lack of justification and intuition have been satisfactorily addressed in the rebuttal. The authors provided helpful clarifications that improved my understanding of the method’s motivation. However, incorporating them would require substantial revisions to the current version of the paper. For this reason, I am maintaining a rating of weak accept rather than upgrading to clear accept.", "rebuttal": "#### Thanks for your insightful feedback and support. We sincerely appreciate your valuable comments and your recognition of the motivation of our method and the soundness of experiments.\n\n**W1: “The proposed A-CFG is explained clearly in section 3.2.1, however, more intuition or justification would be helpful. Specifically, more justification for why re-masking low-confidence tokens leads to improved guide beyond empirical validation would strengthen the method.”**\n\nThank you for this insightful question and for motivating a deeper explanation of the intuition behind A-CFG. Instead of feeding a static null prompt, we build the unconditional input by re‑masking the $\\tau\\%$ least‑confident tokens. For confident positions ($c_{i,t}\\approx1$) the conditional and A‑CFG unconditional logits are *identical*, so the guidance term vanishes—these tokens are left untouched.\n\nFor uncertain positions the unconditional branch now sees $[\\text{MASK}]$, making its logits close to a **uniform prior**. This maximizes the contrast\n\n$$\n\\Delta L_{t,i}\n= L_{t,i}^{\\mathrm{cond}} - L_{t,i}^{\\mathrm{uncond}}\n\\tag{1}\n$$\n\nexactly where it is most useful.\n\nBecause the gradient of the KL divergence between the two distributions equals $\\Delta p = p_{\\mathrm{cond}} - p_{\\mathrm{uncond}}$, Equation (1) effectively amplifies the per‑token gradient without increasing the global scale $w$. One can view A‑CFG as realizing a token‑wise scale:\n\n$$\nw_i = \n\\\\begin{cases}\nw, & \\\\mathrm{if} \\\\ c_{i,t} \\\\le \\\\mathrm{\\\\tau\\\\text{-}percentile} \\\\\\\\\n0, & \\\\mathrm{otherwise}\n\\\\end{cases}\n\\tag{2}\n$$\n\nLet\n\n$$\nD_t = \\\\sum_i \\\\mathrm{KL}\\\\bigl(p_{\\\\mathrm{cond}}(y_i\\\\mid z_t) \\\\big\\| p_{\\\\mathrm{uncond}}(y_i\\\\mid z_t)\\\\bigr) \\tag{3}\n$$\n\nA‑CFG increases $D_t$ selectively on uncertain tokens, raising the signal‑to‑noise ratio of the guidance term while leaving confident tokens (low local KL) unchanged. Empirically this accelerates convergence (fewer resampling steps) and improves final accuracy (Table 1).\n\n**Q1: “Does A-CFG adds inference-time overhead to discrete diffusion models? If so, what is the latency? Some discussion or quantitative analysis or runtime cost would be helpful.”**\n\nRelative to standard CFG, A‑CFG incurs no extra forward passes—we already compute both conditional and unconditional logits.  The cost of overwriting ≤ 70 % of tokens in the input tensor is negligible (<0.2 ms on H800 GPUs).  End‑to‑end wall‑time increases by 2.4 % over standard CFG and 11.8 % over the no‑guidance baseline (128‑sample GSM8K run).  We will report these numbers in Sec. 4.1.3.\n\n\n**Q2: “How does A‑CFG interact with other sampling strategies, such as temperature or top‑k?”**\n\nThanks for your great question. We swept temperature ∈ {0.3, 0.5, 0.7, 1.0} and top‑k ∈ {20,50,100} on ARC-E.  The relative gain of A‑CFG over standard CFG varied ≤ 0.4 pts, confirming robustness. A detailed plot will be added to Appendix C."}, {"review_id": "twuugVhA2y", "reviewer": "Reviewer_cK1A", "summary": "The paper proposes Adaptive Classifier-Free Guidance (A-CFG), a method to enhance controllability in iterative masked diffusion language models (e.g., LLaDA) by dynamically adjusting the unconditional input for CFG based on the model’s token-level predictive confidence. At each generation step, A-CFG identifies low-confidence tokens, re-masks them, and uses this adaptive input to compute the unconditional logits for CFG. The authors evaluate A-CFG on reasoning (GPQA, GSM8K), planning (Sudoku), and general language tasks (MMLU, TruthfulQA), and shows clear improvements over standard CFG.", "questions": "Please address the questions above.", "limitations": "N/A", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The proposed A-CFG seems to be a clean extension to CFG by adaptively re-masking low-confidence tokens, which intuitively focuses guidance where the model is uncertain.\n\n- Experiments demonstrate meaningful improvements across a range of reasoning and planning benchmarks, compared to both standard CFG and no-guidance baselines.\n\n- Qualitative examples showing how A-CFG helps correct errors over iterations, supporting the motivation behind the method.\n\n\n### Weaknesses\n\n- The core contribution seems to be a confidence-based heuristic for selective re-masking, which, while intuitive, appears incremental. Can the authors clarify why this design is considered novel in the field compared to related works?\n\n- The evaluation focuses mainly on LLaDA and Dream-7B diffusion models. Are these models representative of state-of-the-art diffusion models? What are the alternative strong baselines for comparison? Does the proposed method generalize to other model structures?\n\n\n-  As presented in Section 4.3, the method requires careful tuning of remasking ratio and guidance scale for each task. It is unclear whether the model can generalize to unseen tasks without thorough hyperparameter search.\n\n- Although experiments show A-CFG improves over diffusion-only variants, it appears to still underperform top-tier autoregressive models (e.g., Qwen/Llama) on many benchmarks (e.g., GSM8K, GPQA). What are the benefits of diffusion models compared to autoregressive models for the evaluation tasks used in this work? It would be valuable to evaluate on tasks where diffusion models have a unique advantage over autoregressive models and demonstrate the effectiveness of A-CFG.", "was_revised": true, "final_justification": "Thanks for the detailed clarification, which clearly answered my questions and helped me better understand this line of research. To reflect this, I've updated my evaluation score. Good luck!", "rebuttal": "#### Thank you for your thoughtful and comprehensive review. We value your perceptive comments and your support for the paper's contributions. We have worked to address your suggestions in our response below.\n\n**W1: “The core contribution seems to be a confidence-based heuristic for selective re-masking, which, while intuitive, appears incremental. Can the authors clarify why this design is considered novel in the field compared to related works?”**\n\nWhile the idea of using confidence is indeed intuitive, its novelty lies in fundamentally shifting the paradigm of unconditioning from a static, predetermined process to a dynamic, instance-specific one.\n\nA-CFG introduces a fundamentally different, dynamic mechanism. It is the first method, to our knowledge, that inspects the model's internal confidence state at every generation step and selectively adapts the unconditional input at the token level. This allows the guidance to be precisely targeted, creating a strong corrective signal (by forcing a reconsideration via [MASK]) only at points of high ambiguity, while leaving confident predictions untouched.\n\n\n**W2: “The evaluation focuses mainly on LLaDA and Dream-7B diffusion models. Are these models representative of state-of-the-art diffusion models? What are the alternative strong baselines for comparison? Does the proposed method generalize to other model structures?”**\n\nThank you for this question about the scope of our evaluation and the generalizability of our method.\n\n**1. About Model Representativeness:** We chose LLaDA and Dream-7B precisely because they are prominent, state-of-the-art exemplars of the Masked Diffusion Model (MDM) paradigm for language, which is the specific architectural class our work aims to enhance. These iterative, non-monotonic models represent a significant and promising direction in generative modeling, and our goal was to provide a robust evaluation within this context.\n\n**2. About Baselines:** Regarding baselines, our evaluation in Table 1 is designed to be comprehensive. We compare A-CFG not only against the necessary internal baselines (the base model with no guidance and with standard CFG) but also against a suite of powerful, contemporary autoregressive (AR) models like LLaMA3 and Qwen2-7B. This cross-paradigm comparison is crucial to demonstrate that our enhancements make diffusion models more competitive with, and in some complex reasoning tasks even superior to, leading AR models of a similar scale.\n\n**3. About Other Structures:** A-CFG is intrinsically designed for iterative, non-monotonic generation frameworks that operate by progressively refining a sequence, typically involving [MASK] tokens. This mechanism is fundamental to MDMs. Therefore, the method does not directly generalize to fundamentally different structures like autoregressive models. AR models generate text token-by-token in a strict left-to-right sequence and lack the iterative refinement loop where A-CFG's confidence-based re-masking could be applied. The novelty of A-CFG is in leveraging the unique architectural properties of diffusion models. While its application is specific to this class of models, our results demonstrate it is a powerful enhancement for this important and growing family of generative architectures.\n\n**W3: “As presented in Section 4.3, the method requires careful tuning of remasking ratio and guidance scale for each task. It is unclear whether the model can generalize to unseen tasks without thorough hyperparameter search.”**\n\nThank you for this question. In fact, as noted in our implementation details, a single, fixed hyperparameter setting (ρ = 0.7, w = 0.5) was used for other benchmarks presented in Table 1. This single setting was chosen based on performance on only one validation task (ARC-C) and then applied universally to nearly all other tasks, from mathematical reasoning (GSM8K) to planning (Sudoku), without any further search.\n\nThe fact that these settings achieve consistent improvements across such a diverse task set indicates that the method is not brittle and generalizes well to unseen tasks. Furthermore, our ablation study in Table 2a and the table below shows that while performance is sensitive to ρ, there is a relatively broad \"plateau\" of strong performance around the 0.6-0.8 range for ARC-C, GSM9K and Sudoku. This suggests that finding a good, general-purpose setting is straightforward and does not require exhaustive per-task optimization.\n\n| Task      |  ρ = 0.5 |  ρ = 0.6 |  ρ = 0.7 |  ρ = 0.8 |  ρ = 0.9 \n| --------- | :-------: | :-------: | :------: | :-------: | :-------: |\n| ARC‑C     |     46.8 |       47.3  | **47.8** |    47.1  |     46.0 |\n| GSM8K     |   72.1   |  72.9  | **73.5** |   72.6   |   71.0  |\n| Sudoku    |   36.0   |   **42.0**   | **42.0** |   40.0   |   34.0   |\n\n\n**W4: “Although experiments show A-CFG improves over diffusion-only variants, it appears to still underperform top-tier autoregressive models (e.g., Qwen/Llama) on many benchmarks (e.g., GSM8K, GPQA). What are the benefits of diffusion models compared to autoregressive models for the evaluation tasks used in this work? It would be valuable to evaluate on tasks where diffusion models have a unique advantage over autoregressive models and demonstrate the effectiveness of A-CFG.”**\n\nThank you for highlighting this important comparison. We fully agree that today’s best AR LLMs still enjoy a maturity advantage—larger parameter budgets, longer training runs, and a decade of optimisation tricks—so on per‑token perplexity they often outperform the current generation of diffusion language models (DLMs). Our goal in this paper is therefore not to claim overall supremacy, but to demonstrate that A‑CFG unlocks the distinctive strengths of the diffusion paradigm that are orthogonal to raw model scale.\n\n**Order‑free generation and parallel infilling**\nDiffusion models update all masked positions simultaneously rather than marching left‑to‑right. This is ideal for tasks whose solution does not follow a strict sequential dependency. For example, on Sudoku the board’s cells are conditionally independent given the constraints; LLaDA‑8B + A‑CFG solves 80 % of puzzles, whereas a comparable AR model (Qwen2‑7B) solves virtually none.\n\n**Non‑monotonic reasoning**\nMulti‑hop QA benchmarks such as GPQA benefit from the ability to revise earlier tokens as new evidence emerges. A‑CFG narrows the gap to AR SOTA: 36.8 % vs 33 % for baseline LLaDA and 39 % for GPT‑class AR systems.\n\n**Flexible editing and tool‑use potential**\nBecause DLMs treat text as an editable canvas, they integrate naturally with external tools (retrievers, code executors) that may overwrite specific spans. Although we do not build a full agent in this work, our findings suggest that A‑CFG’s confidence‑aware masking is a principled way to decide which spans an agent should defer to a tool.\n\nWe follow the task mix introduced by the original LLaDA and Dream‑7B papers—mathematical (GSM8K), factual (MMLU), reasoning (GPQA), and planning (Sudoku)—to provide apples‑to‑apples comparisons.\n\nIn short, while core language modelling benchmarks still favour large AR models, diffusion language models offer architectural advantages for orderless or editable text scenarios. A‑CFG magnifies those advantages without retraining, and we see bridging the remaining AR gap as an exciting avenue for future, larger‑scale DLMs."}, {"review_id": "EVbrjMceNY", "reviewer": "Reviewer_5qdm", "summary": "This paper introduces a simple technique called Adaptive Classifier-Free Guidance to enhance CFG for masked diffusion language modeling. A-CFG makes a simple modification to the original CFG by re-masking the tokens with low predictive confidence when computing the unconditional logits, which would help the model to adaptively focus on tokens that need more “attention”. A-CFG is tested on LLaDa framework and improves performance across benchmarks such as MMLU and Sudoku.", "questions": "- Did the authors analyze impact of re-masking proportion or scale beyond ARC-C? Do the hyperparameter values transfer across different tasks?\n\n- What is the exact size of the logit $L$ at each step? Do you compute the logits of non-masked positions as well in practice? As a newcomer to diffusion model I wish this kind of technical detail is more explicitly presented in the preliminaries section.", "limitations": "I think the limitations could be more detailed by including discussions on e.g. computational overhead. I would suggest to separate limitations section in the appendix, and add more than what is currently described in Section 4.3.1.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Overall, this is a good paper with solid technical detail and comprehensive evaluation.\n\n- S1: The idea is pretty simple - leveraging model confidence to moderate unconditional logits at each diffusion step. Yet it turns out to be surprisingly effective, e.g. improving GPQA by 3.9 points.\n\n- S2: Experiments are quite thorough, covering benchmarks across general knowledge domain, quantitative reasoning and symbolic planning / ablation studies on hyperparameters.\n\n- W1: Re-masking low-confidence tokens essentially trades off the computational efficiency (by delaying the generation of complete output) with potentially the quality. While the authors do discuss impact of re-masking proportion, the computational overhead introduced by this process needs analysis.\n\n- W2: Relatedly, introduction of additional hyperparameters poses challenges in adopting A-CFG. As shown in 4.3.1 the right value should be somewhere in the middle of 0.5 ~ 0.9, and the performance varies quite largely depending on this value. \n\n- W3: While the paper provides a case study, it does not discuss potential failure modes of A-CFG, such as scenarios where confidence-based re-masking might disrupt coherent generation or introduce biases. I think this is particularly important given that the confidence statistics is entirely based on the model itself.", "was_revised": true, "final_justification": "Overall the work has solid contribution with simple approach, and the authors have clarified most of the questions raised in the original review. I will keep my positive evaluation of the paper.", "rebuttal": "#### We sincerely appreciate your recognition of the meaning of our work. We especially appreciate your encouraging assessment of our work's direction and the insightful questions you raised. We hope our following responses clarify these points to your satisfaction.\n\n**W1: “Re-masking low-confidence tokens essentially trades off the computational efficiency (by delaying the generation of complete output) with potentially the quality. While the authors do discuss impact of re-masking proportion, the computational overhead introduced by this process needs analysis.”**\n\nThanks for your great question. Relative to standard CFG, A‑CFG incurs no extra forward passes—we already compute both conditional and unconditional logits.  The cost of overwriting ≤ 70 % of tokens in the input tensor is negligible (<0.2 ms on an H800 GPUs).  End‑to‑end wall‑time increases by 2.4 % over standard CFG and 11.8 % over the no‑guidance baseline (128‑sample GSM8K run).  We will report these numbers in Sec. 4.1.3.\n\n**W2: “Relatedly, introduction of additional hyperparameters poses challenges in adopting A-CFG. As shown in 4.3.1 the right value should be somewhere in the middle of 0.5 ~ 0.9, and the performance varies quite largely depending on this value.” & Q1: “Did the authors analyse impact of re‑masking proportion or scale beyond ARC‑C?”**\n\nWe appreciate the reviewer’s concern about the extra hyper‑parameters ρ and w; however, the updated sweep below—obtained with a fixed guidance scale w = 0.5—shows that A‑CFG is highly tolerant to the choice of ρ: scores remain virtually flat across the 0.6 – 0.8 range on ARC‑C, GSM8K and Sudoku, and our single default setting ρ = 0.7, w = 0.5, chosen once on ARC‑C, is reused unchanged for every other benchmark while consistently lying inside this plateau, so no per‑task tuning is required.\n\n| Task      |  ρ = 0.5 |  ρ = 0.6 |  ρ = 0.7 |  ρ = 0.8 |  ρ = 0.9 \n| --------- | :-------: | :-------: | :------: | :-------: | :-------: |\n| ARC‑C     |     46.8 |       47.3  | **47.8** |    47.1  |     46.0 |\n| GSM8K     |   72.1   |  72.9  | **73.5** |   72.6   |   71.0  |\n| Sudoku    |   36.0   |   **42.0**   | **42.0** |   40.0   |   34.0   |\n\n\n**W3: “While the paper provides a case study, it does not discuss potential failure modes of A-CFG, such as scenarios where confidence-based re-masking might disrupt coherent generation or introduce biases. I think this is particularly important given that the confidence statistics is entirely based on the model itself.”**\n\nThank you for raising this critical point. The primary failure mode for A-CFG arises when its corrective mechanism becomes overly aggressive, a scenario directly explored in our ablation studies.\n\nThis failure mode emerges from disrupting coherent generation, which we observed when the re-masking proportion ρ is set too high. Consider a multi-step arithmetic task from GSM8K: after correctly generating an intermediate step like 4 * 8 = 32, the model might exhibit slight uncertainty about what follows. An overly aggressive ρ (e.g., 0.9) could cause A-CFG to catastrophically re-mask not only the uncertain tokens but also the crucial, correctly-generated intermediate result 32. By erasing this vital context, the model's ability to complete the subsequent step (32 + 15) is severely compromised, often leading to an incoherent or nonsensical final answer.\n\nThis behavior provides the direct intuition for the performance degradation seen in our ablation study (Table 2a), where increasing ρ beyond an optimal point on ARC-C becomes counterproductive. We will dedicate a part of the Limitations section to a detailed analysis of this failure mode, supported by these qualitative examples.\n\n**Q2: “What is the exact size of the logit at each step? Do you compute the logits of non‑masked positions as well in practice?”**\n\nThe model always outputs a |V|‑dimensional logit for every position, masked or not, as per Eq. 2 (page 5).  In LLaDA‑8B |V| = 126464.  During sampling we only draw from positions whose current token is [MASK]; immutable positions are ignored at the decoding layer, so the extra logits do not incur memory movement."}]

Author rebuttal:

Meta-review: This paper proposes a novel Adaptive Classifier-Free Guidance (A-CFG) method, which adaptively constructs unconditional inputs to amplify the divergence between conditional and unconditional logits, thereby enhancing guidance efficacy. The method achieves consistent performance improvements under fixed hyperparameters across multiple datasets, demonstrating effectiveness and stability. However, the authors insufficiently justify the motivation or theoretical foundation for this approach. ​​Notably, the authors have addressed relevant reviewer inquiries with clarity and precision, and these responses have been accepted by the reviewers.​
Decision: Accept (poster)