Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impact
Authors: Kevin Rojas
Ye He
Chieh-Hsin Lai
Yuhta Takida
Yuki Mitsufuji
Molei Tao
Venue: ICLR 2026 Poster (2026)
Keywords: discrete diffusion
conditional generation
guidance
Primary area: generative models
TL;DR: We introduce an improved mechanism for applying classifier-free guidance in discrete diffusion

Abstract: Classifier-Free Guidance (CFG) is a widely used technique for conditional generation and improving sample quality in continuous diffusion models, and its extensions to discrete diffusion has recently started to be investigated. In order to improve the algorithms in a principled way, this paper starts by analyzing the exact effect of CFG in the context of a low-dimensional masked diffusion model, with a special emphasis on the guidance schedule. Our analysis shows that high guidance early in sampling (when inputs are heavily masked) harms generation quality, while late-stage guidance has a larger effect. These findings provide a theoretical explanation for empirical observations in recent studies on guidance schedules. The analysis also reveals an imperfection of the current CFG implementations. These implementations can unintentionally cause imbalanced transitions, such as unmasking too rapidly during the early stages of generation, which degrades the quality of the resulting samples. To address this, we draw insight from the analysis and propose a novel classifier-free guidance mechanism. Intuitively, our method smoothens the transport between the data distribution and the initial (masked) distribution, which results in improved sample quality. Remarkably, our method is achievable via a simple one-line code change. Experiments on conditional image and text generation empirically confirm the efficacy of our method.

Published as a conference paper at ICLR 2026
IMPROVING CLASSIFIER-FREE GUIDANCE IN MASKED DIFFUSION: LOw-DIM THEORETICAL INSIGHTS WITH HIGH-DIM IMPACT
Kevin Rojas
Ye He
Georgia Institute of Technology revin.rojas@gall edu
Georgia Institute of Technology #/e3678ggtecc. eduu
Chich-Hsin Lai
Yuhta Takida
Sony AI
Sony Al
Chieh-Hsin Lai paaissony.com
yuta.
Yuki Mitsufuji
Molei Tao
Sony AI
Georgia Institute of Technology mtao@gatech.
yuhki. wolki..mitsufujjeeson. com
ABSTRACT
Classifier-Free Guidance (CFG) is widely used technique for conditional gen- eration and improving sample quality continuous diffusion models, and its ex- tensions to discrete diffusion has recently started to be investigated. In order to improve algorithms in a principled way, this paper starts by analyzing the ex- act effect of CFG in the context of a low-dimensional masked diffusion model, with a special emphasis on the guidance schedule. Our analysis shows that high guidance early sampling (when inputs are heavily masked) harms generation quality, while late-stage guidance improves it. These findings provide a theoreti- cal explanation for empirical observations in recent studies on guidance schedules. The analysis also reveals an imperfection of the current CFG implementations. These implementations can unintentionally cause imbalanced transitions, such unmasking too rapidly during the early stages of generation, which degrades the quality of the resulting samples. To address this, we draw insight from the analysis and propose novel classifier-free guidance mechanism. Intuitively, our method smooths the transport between the data distribution and the initial (masked) distri- bution, resulting in improved sample quality. Remarkably, our method is achiev- able via a simple one-line code change. Experiments on conditional image and text generation empirically confirm the efficacy of our method.
Unlocking Guidance
Our Guidance
Key Benefits
Sharper Details
Improved Stability
Simple code change
Figure 1: We proposed improved guidance mechanism through column normalization. Our method produces sharper images while being more stable to the guidance strength. Notably, requires only minor code modification.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Continuous-ste diffusion models (Ho et al., 2020; Song et al., 2021) have proven effective in both unconditional and conditional generation tasks, such as generating data from natural language prompts. Prominent examples include text-to-image and text-to-video models like Stable Diffusion, Sora, and others (Rombach et al., 2022; Esser et al., 2024; Brooks al., 2024). More recently, progress in discrete diffusion modeling (Campbell et al., 2022: Lou et al., 2023; Huang et al., 2023; Gruver al., 2023; Ou al., 2024; Shi et al., 2024; Sahoo et al., 2024) has extended the applicability of diffusion-based generation to new domains, including molecular design, protein synthesis, and languages.
Despite their success, these models often produce outputs that lack fine detail or strong alignment with conditioning inputs (e.g., text prompts). A widely adopted technique to address this issue classifier-free guidance (CFG) (Ho & Salimans, 2021), I), which improves fidelity but typically at the cost of reduced sample diversity (Karras et al., 2024).
A growing body of work has sought to understand the theoretical foundations of CFG in diffusion models (Chidambaram et al., 2024; Pavasovic et al., 2025; Bradley & Nakkiran, 2024; Ye et al., 2025), while others have developed improved guidance algorithms (Karras et al., 2024; Li et al., 2024). Classifier-fres guidance has also been adapted to discrete diffusion models (Nisonoff et al., 2024: Schiff et al., 2024), yielding promising empirical gains.
Among these improvements. dynamic guidance schedules- where guidance strength varies over the generation trajectory- have shown especially effective. Strategies such guidance inter- vals (Kynkäänniemi et al., 2024) and gradually increasing schedules (Xi et al., 2024) can signifi- cantly enhance sample quality and are increasingly adopted in practice (Hoogeboom et al., 2024; Yu et al., 2024; Karras et al., 2024). However, such scheduling techniques remain exclusive to the continuous setting.
While recent adaptations of CFG to discrete diffusion have improved empirical performance, defin- ing and optimizing effective guidance strategies in discrete spaces remains fundamentally chal- lenging and open research problem.
In our work we aim to better understand the mechanisms by which guidance affects the sampling process in discrete diffusion. Specifically, we aim to answer the following questions:
How does the guidance schedule affect the distribution of the generated samples?
Is it possible characterize properties of good guidance schedules?
To do so, we start by deriving explicit formulas for the sampled distribution under varying guidance schedules in and 2 dimensions. Our analysis not only reveals flaws current CFG implemen- tations, but also leads to effective design principles for effective guidance schedules in masked diffusion. Our contributions can be summarized as:
We identify key flaw in existing discrete guidance mechanisms that complicates simula- tion, and provide theoretical explanation of its cause.
To address the flaw, we propose a novel classifier-free guidance mechanism based on simple yet principled column normalization of the rate matrix. This change is theoretically justified, easy to implement (pseudocode in Sec. I), and compares favorably to existing approaches in practice.
The first theoretical justifications to characterize guidance schedules and the mechanisms by which they improve sample generation
2

---

Published as a conference paper at ICLR 2026
def nermalized gww.dnce__eeeeree..e.oeeeeeeeeee transstioon
def sniter_guiddnee Sttiianne__uler__se. www.errrrenssttioo.
&, Er t, dt, w
X, C, t, dt, x
1:
): uncond model (x, cond-None) cond model conére) logits cend (1 s) unsond
uncoad = model (x, cond-None) cend model cond-c)
logits = x cond (1 w) uncend
p_theta Logits. softmmx (dim--1)
p_theta logits.
5, s_bar algma (t), aigma_bard (t)
6, a_bar afgma (t), atgma par (t). change dt return sample (x) change P_thetal
change dt (1 expí s_baz))
exp s_barl)
return sampletden (xdeltaxx change P_theta)
Our guidance in the special case of masked diffusion using Euler transitions. Our method is a simple one line change but clearly molivated by theory
Listing 1:
Listing 2: Unlocking/Simple guidance for the special case of masked diffusion using Euler transitions.
PRELIMINARIES
This paper considers vocabulary of size M and state space S = 11. MM,, with each element being a sequence tokens. The number of tokens d will also be referred to as the dimension. Each probability distribution on S' is represented as vector in RM whose entries sum to one.
2.1 INTRODUCTION TO DISCRETE DIFFUSION VIA CTMC
Given an initial distribution p E A discrete diffusion is defined by considering a rate matrix R2 E TR TRM xMd and defining a continuous time Markov chain (CTMC):
dp RePt. di
Po p.
(I)
pick RG such that when & - co, Pt converges to simple distribution. Additionally, Rd must satisfy that its non-diagonal entries are non non-negative and each column must add up to zero. The time reversal of this process corresponds different CTMC given by:
dpr-t -PT--P---
This process is considered as the time reversal since it has the same law as (1) for all values of and the reverse transition matrix be found through the following identities:
Re x) Re(x, v) (x)
Pe())
Re = - 2c().
water
The ratios Pr(y)
(.()) are called the concrete score and they enable sampling through Euler schemes, r-leaping (Lou et al., 2023) or higher order methods (Ren et al., 2025).
Masked Discrete Diffusion is special case of diffusion where clean sequence xo is gradually corrupted over time by randomly masking some of its entries. Typically, the forward process is chosen such that time 0, the data is completely unmasked, and at = T the data is completely masked. Formally, the distribution of each token can be written in a simple form:
[aj AM
with probability - with probability 1- 1
Where is an increasing function that defines the unmasking schedule. The forward dynamics are defined such that tokens transition only from a clean state to a masked state, remaining masked thereafter. Generation is achieved by starting from fully masked state and iteratively unmasking tokens until a clean sequence is recovered by following Equation (2).
Masked diffusion enjoys simple and structured design. which has enabled its successful scaling to large practical tasks (Nie et al., 2025; Xie et al., 2025; Ou et al., 2024; Sahoo et al., 2024; Shi et al., 2024; Campbell et al., 2022). For this reason, adopt it the primary setting for our analysis.
3

---

Published as a conference paper at ICLR 2026
2.2 CLASSIFIER- FREE GUIDANCE
Classifier-free guidance (CFG) (Ho & Salimans, 2021) was introduced to improve conditional dif- fusion models, like generating images from class labels text. Models often failed to capture fine details, which led to less accurate and misaligned samples (Karras et al., 2024).
CFG tackles this by comparing predictions with and without conditioning, and biasing generation toward the conditional signal. Formally, the method defines reweighted distribution:
( (arly) p (x:l))"~"(>>
Where w is called the guidance strength. Setting p(zly) while w = corresponds to unconditional sampling. The crucial insight is that by setting >> it is possible to emphasize the conditional part, effectively pulling the generation closer to satisfying the required condition. CFG is now a standard tool in conditional diffusion models, more controllable generations across tasks such text-to-image synthesis.
recovers the usual conditional distribution
While the original formulation contrasted the conditional model against its unconditional counter- part, later works recognized that this can be extended by replacing the unconditional distribution with other distributions. For example, Karras et al. (2024) used a weaker conditional model as the guiding distribution. This view has led to the understanding that the essence of guidance lies in balancing target distribution p with guiding distribution
(x) x p (x)4" "xx)
This view highlights that the unconditional model is simply one possible choice of q. By carefully selecting recent works (Karras et al., 2024; Li et al., 2024; Rojas et al., 2025) have proposed novel guidance strategies that further improve sample quality and control.
2.3 GUIDANCE FOR DISCRETE DIFFUSION MODELS
In parallel to advances in continuous domains, discrete diffusion models have emerged as powerful generative models, enabling diffusion-basee approaches on modalities that were previously out of reach-most notably, text. Improving the fidelity and controllability of these models is crucial, and guidance offers a natural path forward. Extending classifier free guidance to the discrete setting has therefore become active line of research with two main approaches having been proposed, which we describe below, followed by a discussion Section 3.3 comparing them to our method.
Unlocking Guidance (Nisonoff et al., 2024) introduced the first classifier-fres guidance mecha- nisms for discrete diffusion models. Inspired by the continuous case, they constructed a guided backwards transition by interpolating between two transition matrices in equation 2, yielding
T =
Pe(y)) S pr(x)
1-w (x(
(x,x)=-
(1, (5)
where Pt, qt follows the forward CTMC (I). Here Po = p is the distribution that we want to generate from and q serves the guiding distribution. !. Notice how the products mimic those present in equation 4. A useful way to interpret this is by introducing the notion of the tilted distribution:
(w) (x) 2000=;,;10 p (1)) 1- (x),
7
Σ p" (x) (1-
VES
The generation process follows the dynamics induced by the guided transition matrix substituted in equation 2. Nisonoff al. (2024) showed that guidance in the discrete setting serves role analogous to its continuous counterpart- providing an important step toward improving the quality of discrete diffusion generations.
the model toward more faithful conditional samples- thus
Simple Guidance. Concurrently, Schiff et al. (2024) proposed an alternative formulation of classifier-free guidance for discrete diffusion. Rather than interpolating the rate matrices as in
In existing literature, p is usually condiionniiio distribution, and is an unconditional distribution. We adopt the general setup since recent works have shown that can be chosen in different ways (Karras et al., 2024; Li et al., 2024; Rojas al., 2025).
4

---

Published as a conference paper at ICLR 2026
Nisonoff et al. (2024), they directly interpolate the transition probabilities themselves. Specifically, when transitioning from time to time t, the following transition was proposed:
P Psimple (z) p" (=sl=))"==" ((==(=e)
(w)
As hefore, increasing w biases towards the target distribution Although the construction appears different, in the limit & + & the transitions coincide with those of Nisonoff et al. (2024). In practice, however, finite number of steps is used, and the resulting methods are distinct. To implement these transitions, one can use equation (2) together with a suitable numerical integration scheme.
2.4 DYNAMIC GUIDANCE SCHEDULES
In our work will consider dynamic guidance schedules, i.c. making function of time. Such schedules have become more popular in practice. For instance, guidance interval (Kynkäänniemi al., 2024) only applies guidance a boost in the performance of diffusion models. However, existing work on dynamic guidance schedules (Kynkäänniemi al., 2024; Xi al., 2024) has been limited to continuous (state-space) diffusion models. It remains unclear whether such schedules are also effective in discrete state diffusion question that serves as the main focus of our investigation.
segment of the generation process. Doing so produces
Specifically, this work will consider [0.7] + R. i.. guidance strength as function of time, referred as the guidance schedule. The schedule induces generative process given by:
dpr-t =R
(wT-() PT-t
dt Understanding which schedules result in the best generation of crucial importance to further im- prove the sample accuracy of discrete diffusion models.
3 METHODOL.OGY
We begin by analyzing the guided process in the simplest case of a single token in Section 3.1, which already reveals key limitation of existing guidance. We then introduce proposed remedy in Section 3.2 via column normalization. Afterwards, we analyze the effect of guidance schedules on two tokens in Section 3.4. Finally, we present experimental results of methods in Section 4.
3.1 IDENTIFYING AN ISSUE IN THE GUIDANCE OF DISCRETE DIFFUSION
We start by studying guidance for d = 1 where an ex- act analysis is possible. The following result character- izes the distribution at time f under constant guidance:
2,-1
2.+2
2,+3
2,04
Theorem 3.1. (Informal) Along the dynamics of equa- tion (7), starting from a fully masked state, the distri- bution time is given by:
Z.-10
0.6
Fe
Pt 11
p(
(8)
- BT
We present a full proof, as well as more general result for varying guidance schedules in Theorem B.I. This shows that for d = 1 the guided process exactly recov- ers the tilted distribution, with the unmasking speed controlled by the factor in front of p!w). Although low-dimensional, this result already reveals important properties of the guided backwards process.
Time
We plot the unmasking rates as function of time under guidance. Faster unmasking (Z,. > 1) leads to worse nu- merical solvers, demonstrating an issue in the existing guidance mechanism.
Figure 2:
Crucially, the partition function Z., appears in the ex- ponent of the rate term, meaning that even small changes in can result in fast changes in the sampling rate. Figure shows the percentage of tokens that remain masked as function of time (M(M) for different values of Zw. Applying guidance can significantly accelerate unmasking rates. While this can lead to faster generation, it may also introduce stiffness (Rathinam et al., 2003) and inefficiencies if not properly controlled.
5

---

Published as a conference paper at ICLR 2026
3.2 IMPROVED GUIDANCE MECHANISMS FOR DISCRETE DIFFUSION VIA COLUMN NORMALIZATION
In order alleviate the unintentional fast unmasking rates, we propose simple yet effective change to the guidance mechanism. We begin by isolating the source of the issue by rewriting by decom- posing the rate matrix (2) into a jump rate and a jump distribution:
Py( (x(x,))
Rs (e x) = (x)
(e,(())
Σ Re UFL
(x) (x)
pa(s(()
Under this decomposition, (2) governs how often jumps occur from state while pr (v 2) de- termines where the process jumps. Using this decomposition the guided rate matrix in (5) becomes:
T (v. = rT (2)r (x)
3,0 Pp (u(z)a w (y/ax)
(10)
distribution
Where z., = =E Eyzz P (11l))"""((/lll/) is the normalizing constant of the guided jump measure. Crucially, (10) reveals that the zw does not merely affect the jump distribution, but instead rescales
the overall jump jump rate. This is unintentional and leads to disproportionately fast transitions.
To make this effect explicit, we consider the masked diffusion setting and explicitly write the tran- sition rates between masked state M non-masked state:
Lemma 3.1. The transition rates between masked state and an unnormalized state are given by:
ct I-e-a Zmpllu (v)
R M) (2, v)
The appearance of z as a multiplicative factor in the transition rate confirms the source of the observed pathology: guidance amplifies how often unmasking occurs, rather than only affecting which token is selected. Notably, when w = 1 (the purely conditional setting), 2,, = and the issue disappears, explaining why this effect is absent in standard conditional diffusion.
Normalized Guidance. The above analysis indicates that the normalizing constant z. should not influence the jump rate. To correct this, we explicitly decouple rate and distribution by normalizing the guided rate matrix column-wise:
R
(v. (x)) (x))
T. (yl=)4 (v)xx)
(11)
rall
distribution
In the case of masked diffusion this normalization admits a very simple form:
xx 1- 5,
Roor. x)
Softmax(w log (X*) + (1 w) log
(12)
The normalization introduced in (11) and (12) has the effect of smoothing the transport between the starting distribution and the data distribution. This simple change stabilizes the sampling process and allows for a cleaner theory. Notably, this can be done with a simple one line change to the code presented in the pseudocode in We further elaborate the experimental benefits on Section 4.
3.3 COMPARISON GUIDANCE MECHANISMS
We now clarify the distinctions between the various classifier-free guidance mechanisms. While some differences between our method and that of Nisonoff et al. (2024) were already discussed, we further highlight how our formulation also differs from the approach of Schiff et al. (2024). To better understand these differences, we begin by comparing the unlocking guidance mechanism of Nisonoff et al. (2024) with the simple guidance proposed by Schiff et al. (2024). For this analysis,
we keep the guidance strength fixed throughout. Notice that: p(x,(==)) = exp (J. (T") dr Pt- Therefore, if Pa denotes the law of 24, we can write the transition probabilities for each method:
Punlocking (I.( = exp (f. R. (-(o) (()r)

---

Published as a conference paper at ICLR 2026
Paimplo (12)221) = Ssimpio (exp (L.
( (1)))))))) (expp (((r(())))
where Zsimple a normalizing constant. Now we look at the w-dependence inside the exponential. For log Punlocking. the w-dependence is exponential as it appears the exponent of the rate matrices, while for log Psimple. the w-dependence is linear. Therefore, the transitions induced by the unlocking guidance method get much more aggressive when w increases. On the other hand, our normalization (depending on wi) normalizes the columns so that it maintains the smoothness of the transition when increases. Therefore, our method approximates the convergence rates of the original process.
3.4 ANALYSIS OF GUIDANCE SCHEDULES IN 2D
Having addressed the existing issue we switch our focus to the analysis of guidance schedules in the case of two tokens. Although the analysis be extended to higher-dimensions, the complexity of the problem grows exponentially with the dimension, leading to increasingly intricate expressions and reduced interpretability. This lin-dimenioonal analysis already reveals the underlying mecha- nisms that define a good guidance schedule, and its impacts in high-dimensions are remarkable.
We start by stating our main theorem, in simple to understand case that is used in practice. This simplification doesn't 't result in loss of generality, but significantly increases the interpretability of the results. We present more general version in Theorem C.I.
Corollary 3.1. Consider a time partition = to < l < l2 < 1a = T with guidance w; in the interval (Li, 1). With log( 81) and pT (M, M) 1. Then the sampled distribution follows the following formula:
+ عااي 2
2 w.) (i,5) + (1-te) ((
Pto (i.5) = (1-62 ((w)) p(,z) (i,j)
p
(i.j)
(1a (la-taa(tt- ti] (1 (w)) 122) (i, j) (ts t2]((1 tol (2))
(i,j) + 11--1)661 to) p (ono. (i,
where (i. = ( (i. j|_11 = i)g()) (X1 = i) + ( [i.j = (), = j), notice that this is not exactly probability distribution as is not normalized, but we will refer to it as one.
This theorem states that guidance schedules induce an interpolation of different distributions, which depend only on the guidance strengths and that the portion assigned to each one depends on the time parameters. We analyze the role of each component separately.
Coodded destern a h1
Chelknd Sartedie f -1
The role of guidance weights: We study a toy example in 2D with 4 clusters, two of which intersect in the middle (see Ap- pendix D for visualizations). Figure shows that increasing leads to concen- tration of mass in of the modes. Simi- larly, Figure 5 shows that ((w,7)) strongly resembles the tilted distribution of Practically, this means that the combined distribution will be more similar the guidance applied at the end of the gener- ation! Therefore, effective schedules have
055
86
Figure 5: Notice that when < y the combined dis- tribution doesn't bias the leftmost mode, making this setting less efficient for guidance. higher guidance at the final and middle phases of the generation while keeping early guidance small.
Figure Tilted distributions for varying values w. Large w concentrates mass on one mode.

---

Published as a conference paper at ICLR 2026
1d
-B-M -1h-17"
-1
-1
vally 11) 100 +1 (o
1ol 14 1015 14
Ceeloou Vilae
6il 10
0.2
110
Figure 4: Evolution of the coefficients in Corollary 3.1 for different values of fz. with t S f2. For
moderate f2, no single coefficient dominates, yielding balanced target distribution.
The role of the time parameters: As observed in corollary 3.1, the time parameters set the pro- portion of each distribution that will contribute towards the final output. As observed in Figures 3,5, biasing just one of the distributions usually results in oversampling from certain area. A good schedule is one that appropriately balances the contribution of each distribution.
We fix several values of l2 and plot the coefficients only have two intervals, and the curves change quickly; this implies that finding the right balance requires more careful tuning. On the other hand when f2 .75, many values t result in balanced combinations of all distributions, which ensures that we sample in a balanced way.
function of L in Figure 4. When L2
Which schedules perform best? Our theoretical analysis provides several insights into the design of effective guidance schedules. As discussed earlier, schedules that apply stronger guidance during the middle and later stages of the sampling process, while keeping early guidance small, tend to perform better. These selections seem to be the most critical, they govern which distributions are mixed. Moreover, our theory predicts that using three intervals (early, middle, and late) in the schedule facilitates easier tuning and yields more balanced output distributions. Based these principles, we evaluate (according to our theory) various guidance schedules for discrete diffusion in Table 1, and we validate these predictions empirically in Section 4.2.
Table I:
Comparison of several guidance schedules.
Low G. Beg
High G. Mid
High G. End
Params Tune
Difficulty to Tune
Constant Interval Increasing
1
High
3
Low
1
Low
Decreasing
X
1
Low
4 NUMERICAL RESULTS
In this section, we test whether low-dimensional theoretical insights extend to high-dimensional image and text domains. Section 4.1 studies the effect of normalization, while Section 4.2 examines different guidance schedules. Additional details and samples are provided in Appendix G.
4.1 EFFECT OF NORMALIZATION
Recall that our theory predicted that failing normalize complicates the simulation, so normaliza- tion should improve results in practice, which we confirm below.
Testing Imagenet: We assess MaskGIT the ImageNet dataset (Deng et al., 2009) and evalu- ate FID on ImageNet-256 using 50K samples, following standard practices. For our method and for the Unlocking Guidance baseline (Nisonoff al., 2024), we use the 7-leaping sampler. For Simple Guidance (Schiff et al., 2024), we interpolate Euler transitions. For all methods, we use 50 steps. Figure 6a shows FID function of guidance strength using a constant schedule. Our experiments demonstrate that failing to normalize can substantially degrade sample quality as suggested by our theory.
8

---

Published as a conference paper at ICLR 2026
lese Gresi Cal siece
20.0
7.5
2.5
Su duicee throgghh
Rayi Eripsiee
(ef E16r44
(a) Comparison of different guid- ance mechanisms,
(b) Right Interval Ramp-Up schedules.
(c) Left Interval vs Ramp- Ramp-Down schedules
Figure 6: Evaluation of different guidance mechanisms and schedules on Imagenet
The effect on diversity-quality tradeoff:: To assess tradeoff. Fig. 7 shows the precision-recall curves. Relative to the no-guidance baseline (w 1), both Unlocking Guid- ance and Simple Guidance exhibit reduced precision guidance increases, indicating a tradcoff between fidelity and diversity. In contrast, our method improves precision while maintaining comparable recall for moderate guid- ance strengths, before all methods degrade large guid- ance.
Reealll guidance strength
the effect of guidance on the
Setker lociky
Beller ldl
3.5
Predaloon (ndalty)
9.1
52-
Our Hetted
Unieding Gustance
Testing on text-to-image: We evaluate method the GenEval benchmark (Ghosh et al., 2023) using Meis- sonic (Bai al., 2024) as well as Show-O Xie al. (2024) which is a mixed model leveraging discrete diffusion for image generation and auto-regressive for text generation. This benchmark provides a comprehensive measure of both prompt alignment and perceptual image quality. Fig- ure 9 compares generations with and without normaliza- tion. Red regions indicate prompts where normalization improved the score. Overall, we observe consistent gains: normalization enhances prompt adher- ence and yields images that better match the target distribution.
Setier diersiy
0.6
0.88
0.70
0.14 Reca Idiversity)
0.72
0,76
0.16
0.50 0.52
Figure 7: Only our method simulta- neously improves fidelity and diversity. We plot the Precision/recall curves for different values of w. The dashed lines indicate the no-guidance baseline.
Testing on text generation: To assess the effectiveness of normalization in the text generation domain, we eval- uated using LLaDA-8B- Instrructtt (Nie et al., 2025) the MATH-500 dataset, generating up to 256 tokens. We sam-
Performance on MATH-500
H with Normaicator
- No Nomallzstio
ple autoregressively in blocks of 32 tokens using simple
Euler sampler with 32 denoising steps per block, resulting total of 256 steps for the full generation.
Performence
Figure 8 presents the results of such an experiment. The results clearly show that normalization consistently im- proves performance across all guidance strengths. We note that the results are not directly comparable to those reported in the LLaDA paper; we use a simple Fuler sam- pler without remasking to better isolate the effect of guid- ance and normalization in simple setting.
Guidance Weight
Figure 8: MATH-500 performance for -Laad-88B-Instruu under simple sam- pler without remasking to isolate the ef- fect of the guidance mechanism. Nor- malization always yields better results.
Empirical effect of normalization: All our empirical findings demonstrate that normalization is helpful step in improving the simulation of classifier-fi free guidance for discrete diffusion. This aligns with our -iwdiineniional theoretical analysis in Section 3.1, demonstrating that low-dimensional studies can have a signifi- cant impact in high dimensions.

---

Published as a conference paper at ICLR 2026
Performance gain by including normalization
Kod: remaliacionni wiu Blue: Ro rommlbution arr
Guidaance Wreightt
2
+1.5
+10.2
-35
+14.4
+17.0
+10.2
+1.9
+17.7
+15.9
43.2
+14.5
48.2
mm
+19.4
+11.9
+19.9
+19.7
+29
88
22.2
17.0
Single Object
Too Objects
Counting
Alributtin
Performance gain by Including normalization
Cuidance Weight
+0.0
+0.00
+0.0
+0.0
+0.0
40.0
40.0
W=
-2.3
+0..9
-10.6
-5.9
+0.3
-0.2
+2.2
W
+17.3
+11.9
+25.0
+21.9
+28.7
+13.2
+3.0
+27.2
+30.6
Overall
Single Object
Twu Sbjects
Counling
Colors
Color Altribution
Position
Figure 9: GenEval with and without normalization using Show-o (Top) and Meissonic (Bottom). Red denotes improved performance from normalization. Normalization improves prompt adherence and quality.
Table Description of guidance schedules.
4.2 STUDY OF GUIDANCE SCHEDULES
Schedule
Formula w(t)
Previously, our theory predicted that increasing schedules improve discrete diffusion while decreas- ing ones degrade generation. We test this theory on Imagenet-256 with 10K samples. For precise for- mulas for the schedules, see Table When testing increasing schedules (Ramp-Up and Right Interval) in 6b, we observe that both schedules can significantly improve the results. Furthermore, the Right Interval schedule exhibits convex trend with respect r, while the Ramp-Up schedule is mono- tone r, and reaches a lower FID value, indicating that a gradual, linear increase in guidance outperforms abrupt alternatives. When testing the decreasing schedules (Left interval and Ramp- Down), we observe that they consistently damage the generation as seen in Figure 6c. Overall, our experiments confirm our theory that increasing schedules are most effective for masked diffusion.
Left Interval Right Interval
1 1o.4 (1) w 1
(r.1)
Ramp-Up
min
w, w
+
Ramp-Down min
W. w
CONCLUSIONS
In this work, we introduced framework for analyzing guidance schedules in masked diffusion. Our analysis led novel approach for classifier- free guidance in the discrete setting. We validate effectiveness our method through experiments and show that guidance applied near T is harmful to the generation quality while near / 0 can improve the it. This insights enabled us to identify effective scheduling strategies. Our thcoretical insights align closely with empirical observations, bridging the gap between theory and practice.
Limitations and Future work. While our framework provides a principled and tractable approach theoretical analysis is currently limited to masked diffusion in
to CFG in discrete diffusion, low-dimensional settings. Although the method applicable to more complex real-world settings, current theoretical study does cover such regimes. Promising directions include extending the framework to other forms of discrete diffusion, such as uniform diffusion, scaling to higher dimensions, and analyzing the role of score estimation error the guidance dynamics.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jinbin Bai. Tian Ye. Wei Chow, Enxin Song, Xiangtai Li, Zhen Dong, Lei Zhu, and Shuicheng Revitalizing masked generative transformers for efficient high-resolution text- to-image synthesis. arXiv preprint arXiv:2410.08266 2024.
Yan. Meissonic:
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is predictor-correcto.. arXiv preprint ar arivv2448..00000 2024.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. https:/ 2024.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 35:28266-282799 2022.
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? fine-grained analysis in simple setting. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 255. leee, 2009.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focuser framework for evaluating text-to-image alignment. Advances Neural Information Processing Systems, 36: 2023.
Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances neural information processing systems, 36: 36:12489-12517, 2023.
Jonathan Ho and Tim Salimans. Classifier-free diffuusinn guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
Emiel Hoogeboom, Thomas Mensink, Jonathan Heek, Kay Lamerigts, Ruiqi Gao, and Tim Sali- mans. Simpler diffusion (sid2): 1.5 fid on imagene1512 with pixel-space diffusion. arXiv preprint arXiv:2410.1933 2024.
Han Huang. Leilei Sun, Bowen Du, and Weifeng Lv. Conditional diffusson based on discrete graph structures for molecular graph generation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 4302-4311, 2023.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with bad version of itself. Advances in Neural Information Processing Systems, 37:52996-53022 2024.
Tuomas Kynkäänniemi, Miik Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. In Conference Neural Information Processing Systems. 2024.
Tiancheng Li, Weijian Luo, Zhiyang Chen, Liyuan Ma, and Guo-Jun Qi. Self Self-guidance: Boosting flow and diffusion generation their own. CoRR, 2024.

---

Published as a conference paper at ICLR 2026
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint Xiv:2310.16834, 2023.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, JUN ZHOU, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy, 2025.
Hunter Nisonoff, Junhao Xiong. Stephan Allenspach, and Jennifer Listgarten. Unlocking guidance for discrete state-space diffusion and flow models. arXiv preprint arXiv:2406.01572, 2024.
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprini urXiv:2406.037336 2024.
Krunoslav Lehman Pavasovic, Jakob Verbeek, Giulio Biroli, and Mare Mezard. Understanding classifier-free guidance: High-dimensional theory and non-linear generalizations. arXiv preprint arXiv:2502.07849, 2025.
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1, 2014.
Muruhan Rathinam, Linda R Petzold, Yang Cao, and Daniel T Gillespie. Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. The Journal of Chemical Physics, 119(24):12784-122 12794, 2003.
Yinuo Ren, Haoxuan Chen, Yuchen Zhu, Wei Guo, Yongxin Chen, Grant M Rotskoff, Molei Tao, and Lexing Ying. Fast solvers for discrete diffusion models: Theory and applications of high- order algorithms. arXiv preprint arXiv:2502.0023 00234, 2025.
Kevin Rojas, Yuchen Zhu, Sichen Zhu, Felix XF Ye, and Molei Tao. Diffuse everything: Multimodal diffusion models arbitrary state spaces. In Forty-second International Conference Machine Learning. 2025.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling, 52(1 1):2864 2875, 2012.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 001113036-130880 2024.
Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P de Almeida, Alexander Rush, Thomas Pierrot, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models. arXiv preprint ar arXiv:2412.10193, 2024.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnud Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167 2024.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
WANG Xi, Nicolas Dufour, Nefeli Andreou, CANI Marie-Paule, Victoria Fernandez Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. Transactions on Machine Learning Research, 2024.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang. Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In The Thirteenth International Conference Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, et al. Dream-coder 7b: An open diffusion language model for code. arXiv preprint arXiv:2509.01144 2025.
He Ye, Rojas Kevin, and Tao Molei. What exactly does guidance do in masked discrete diffusion models. arXiv preprint arXiv:2506. 2025.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong. Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940, 2024.
13

---

Published as a conference paper at ICLR 2026
A NOTATION AND GENERAL RESULTS
A.1 SPECIAL PROPERTIES OF MASKED DIFFUSION
We will use the following notations specific to masked diffusion. Let X2 (x. ...,..,, xf) denote random variable on S, and M be the masked token. We will write x xUM for the set of elements such that x * M, meaning the entries that are not the masked token. Additionally, we will denote T, - a,ds.
Masked diffusion has several appealing properties, being the following shown by Ou et al. (2024):
Lemma A.1. Along the dynamics (1) given by the masked rate matrix, xf (x. x() and 8; (x, , x in such way that X + M and x M, have the following identity for the score
(x( (x(x))
- e- Po(x; )M
This result is of great importance, it tells that it is possible decompose the scores probability distribution independent of time multiplied by a time-dependent term.
B PROOFS IN ID
We first prove a small lemma:
Lemma B.1. Given a matrix of the form
81
A =
Un
Un + 0, then its matrix exponential is given by e
Fn
Proof. First notice that for h: > 0 it holds that A u AA then we can write:
A= + A +
+ ...
3!
I A +
Au +
Au
+
21
31
1
T A(1
+
2
+
...)
2!
3!
= 1 A(1 +
2
+ ...)
31 n
= I A(1 +
(-1 - + +1+ on +
21 2
31 1 + ....
Un
1
+I
A(1 +
(-1
En +cn))
Un
Pn
I +
Un
As we wanted.
We now state and prove the general version Theorem 3.1: Theorem B.1. Along the dynamics of equation (7). The distribution Pt is given by:
1
A T
Pt A1
AM-11
eA
A
A
14

---

Published as a conference paper at ICLR 2026
Where, for ......M-- 1:
e E a
M-1
T
A =
08 001 -- Lw, 1f
=
Σ
A
03
Lw ds.
e
3
1=0
Proof. Recall that the rate matrix the one mieedimenioona case is:
(1))(1)
0 0
(2 (2)
R₂ w+))
0t
Zuz
(13)
1 C
p (M -1)
-1
By direct integration we know that:
Pr exp ( nT Rr""'dr)pT
Therefore applying Lemma B.1 we get that (in vector notation):
1 000 c
Zw ppiius ds A
Dt - pr (M))
with
M-1
A, [
T
c σ
A
Σ
0x
3. Zw, e-
ds.
The result is proved.
We can the previous theorem to compute the distribution under constant guidance: Corollary B.1. If we start with distribution pt and keep guidance to be constant Then at time s the distribution is given by:
(t(1) = (i( + (MM 1 - e 0=
2n (w)
1- -0 -1)
for i + M and (M (1-0-%-- - (M)
Proof.
The proof follows by keeping w constant in the above theorem:
1 ed
vx Zw, So oo -
Ps Pe + Pc(M)
A
E
ed
c 2 ds.
plu)
Pr Pc(M
ed
(11 (1-e2)) ed
=p. + PM(M
Substituting A gives the desired result.
We can now chain the above argument to obtain a result for general piece-wise constant guidance schedules:
15

---

Published as a conference paper at ICLR 2026
Theorem B.2. Let = to < t1 on the interval (t,, t.+1l. Along the dynamics of equation (7), the sampled distribution ps given by:
<...<tt T be time partition and let we the guidance strength
P6 PT +
Σ Ph (M)
R-1
(1-
p(on).
(14)
Additionally,
probability 1 1
mass at M at
different
time
satisfies
Pr, (M)
Ziu for all i = 0, 1, 1.
Peue (M)
Lemma B.2. The transition rates between a masked state and an unnormalized state are given by:
c
(y. R2( 3) e Zup Zwplu) (y)
Proof. Using Lemma A. we can write:
P((x)) (t(
(((x))
1-w
T (y, M) Re(M,))
(MM)
T Po(u) "
E o
1-w
())
e- 40 2o(v))
= -R((2,8)
--ō, Pb (y)go ((y) (w()
0
= =R((z
- Z Zup (y)
The results for the normalized process are identical to the ones above, omit them for brevity.
C PROOFS IN 2D
We begin by writing a simple lemma that will come in handy later. Lemma C.1. Given matrix of the form
b
0
1
0
A
0
-1
0
0
-2)
Then for any a E R, it's matrix exponential is given by:
(ac+bd)(ee -18
a(1 -e-a)
6(1 6(1-e-a)
2
exp(aA)
0
e a
0
C (Eea
1) c 2a
0
a
(e
1) 20x
0
0
2a
Proof. The proof of the above statement is easy by noticing that A PDP-1 with:
dc
2
+
bd 2
a
0
0
P
0
1
0
1
0
0
2
0
0
0
0
-1
0
0
0
1
0
0
0
0
Then exp(aA)= Pexp(aD))P-1 and the result follows.
16

---

Published as a conference paper at ICLR 2026
Now for the main proof we start by explicitly writing down the rate matrix in the case of two tokens. In this case the rate matrix will have the following structure:
D1
0
Ci
R
OFE 81
0
D2
C2
0+
ae 0-
E
R
Rnor
where each block is an M x M matrix given by the following formulas:
plu) (X2 = X1 =1) plu) (X2 2 X1 =)
D =
( (X2 = M - 1 x1 =)
-
p(w) (X1 = i X2 =1 0
0
C
p ( (X1 i X2 = M- 1)
0
pl (X1 1)
1 0 -1
p (X2 = 1) ( (X2 = 2)
L=
p (w)) (X2 (X2 = M 1)
-2
We can now state the main theorem: Theorem C.1. Given a starting distribution Pt following the dynamics given by (7) the distribution at time is given by:
1 1 1-0-0)
p((i.)) +
p ((t) (c.,))((M,M)
1-
1 3
+
|0 (w) (,/00(X2 (X2 = i 1|x,
X1 = =.)(1, M)
1 0
+ pl (X, =
X2 = j) (M
if + M
1 - 3.
- (M
p=(i,))=
+ - c e- )
(1
(1-0-0 -1) plu)(Xs (M,
11 11-e-00
Fit M, j-M
1 の0
pc(M.j)
1 の o,
c c- 2
)
11-e-0
+ - -
(1-0-00 -1 p
( = -))
ifi= M, M
1
1 - 2 (1--00)) M)
fi=j=M M
Proof. By direct integration we know that:
=exp (f, o+e R(w)
exp (In (1-e e (w))
1-e-a, dr R(nor)
1 ._
17

---

Published as a conference paper at ICLR 2026
Due to the block structure of F it is enough to be able to compute the exponential of:
pl - i) X1 =
( (X1 = X2 1)
0
p(w) (X2
0
D: C2 0 L
plw) (X1
w) (X2 1)
pl ww) (X2
0
2
where once again can exploit the structured form of the matrix simplify the calculation. It is clear that when computing products of this matrix, coordinates will only get affected by the smaller sublocks:
p (ar)
(X2
X2 j |1 = i) p(w) plw)(X, = = = 5)
0
0
p w (X1 = i) (X₂ = j) -2
0
1
0
0
0
This not only clear from the structure, but it also reveals a true intuitive understanding. The probability mass at given position can only be affected by those states that are reachable from the current state by masking or unmasking the entries. We can now use Lemma C.I to find the exponential:
1
((w) (X2
11 = i) - ==)11e
e (X1 = = 5) a
p (i, 1)2e-
100
a
0
p fl (Xi 1) (e - 1) 22
0
0 0
e
p (X2 = 1) (e - 1) E 20
0
where a = In (1 e -0
c - and we used that (1.5) = (X) = - = 1)p (X1 =
+ p(u) (X1 = i X2 j)p(") (X2 = j). Putting this together, get that exponentiation each
block we get:
(X2 = Xi ()(1 - -)- "Xi = X2 1)(1 - -
p ( (1, 1)(e) - -1)26
(w)
(X2 X1 i)(1 -9
( (i, 1)(c -1)² 22a
(w)) (X1 = i)(1 e e 22
IX 1)(1 e en)e 20
( 2) e
18

---

Published as a conference paper at ICLR 2026
With this, we have full characterization of the matrix exponential. Therefore, can simply write down the probability distribution by multiplying by Pe:
pe(i,)) (1 - - (1)) (I,) (5) ( (X2 =j X1 ())
(1 -1-- --
if M (X1 = X2 5) (c
(p(, M)
pi(j,))
e (c - 1) = 1 (MM, M)
i M, j- = M
+e- (c - 1) (X2 = -))
if = M, j + M
2d (rMM,
if = = M
We can now replace a = In (1 1 (1-e- 0=
11 (1-c- into the formula above to obtain the result.
Corollary C.1. Given a starting distribution Pt following the dynamics given by (7) with T.
-log(1 ō1) the distribution at time is given by:
118 p (M
pe(i,))+ +
t 8-8 (p(m))
()(w)(sg (X2 = j X, P((,,)
(()) (X1 = il X2 = j) ((M
ifi,j + M
(1(.)) =
Pe(ii M) + (7) 2 (t s-
p [X1 = ()
if M, j= M
(M( + (7) 2 1-8 (1-8) p (x2= = 3)p(M,
(x2
ff= M, 1
(i) (M M)
fi=j-M
Proof. Notice that under this schedule we have that:
1
8s
I
t
Substituting this in gives the corollary above.
Proof of Corollary 3.1. We track the changes in the distribution in every time interval. This can be found by plugging in the result of the corollary above. Firstly, on the interval T - t2 we obtain:
t2
(M, M) =
T
19

---

Published as a conference paper at ICLR 2026
(M,
12
T - l2
p ( =j)
T
12
Pto (1, M) =
12
T 1/
p = i)
T
t2
Pta (1,
T
12
2
P
(wm))
(i.j)
T
Then on the interval from l2 + 1 we get:
f1
2
Pt, (M, M)
Pt, (M,
t1
Pes () +
11
82 t
p p (X2 -))p, (M, M)
12
1/
1
Pt, (t, M) =
11
Pia (1, M) +
(1]
2
1/ 11
p (x)))(1 = (M, M)
1/2
t/2
11
1/ t1
2
Pt, (1.5) = Ptz (2. +
p
(w1)) ple (i.j))1, (M)
t2
12
(1
+
[p (wi
(X2 = j|X1 = (1, M) + p (w)) (X1 i|X² =
t2
Replacing the values for pt, into this equation we get:
Pt. (M,
T
Pt. (),
t1 (T - t2)
p(aa) = +)+ +
11
t2 t
p p =))
T2
T
h
(t,
t1(T' -1))
p(aal(x, = i) +
[1
2
122 -1 1
p p(o))(X, = d)
T2
T
1
(i..
T - 12
2
p
(w2)
(i,
12
t1
2
1/2
2
p
(102) (i.5)
T
t2
T
t2 t1
t2
T -
to
(1o))
(X2
j|Xı = i)p(w>)(X) =i)
1/
T
l2
+
1/ 4
l/
(T
l/
p
piin) (X1 = x|X2 = j)p(ns)(Xx =})
1/
T
t2
T - t2
2
p
(w2))
(i,
t1
2
(in))
(12 - (T tx2
p p(w).wa)
T
T
T
Finally, we can proceed with the final step from 4 - l0- In this case, we have:
Pio (M, M)
Pro (M,j) =0 0
Pto (1, M) =0
(1 11-60 10 )² p( (i.j)Pr, (1 (M, M)
Pio (i. = =pt, (i,j) +
t
+ (1 - ln
( (x = -|X1 = i)pt, (1, M) + +p (X, = =|/22
-De,, (M,5))
Then substituting in the previous results:
Pro (i.j) = Pc, (1,5) +
f1 - to t
11
p (w)) (1o))
T
20

---

Published as a conference paper at ICLR 2026
11 -to to
+
p (w (x =
11(T - -22)
(
11
2
12 (2-4 (X)))(X)=)) (x)=)) =0)
72 D (ux)) (Xi 1) +
T
t
+ +p (wo)) (Xi = i|X2 = j]
t1 t1 (T -2) (20)) (X2 = j] +
2
( (b2-1)
1
p pim = (xi= )
-
7
11
Grouping by coefficient we get:
2
11 2
Pto (1 = Pt1 (i: +
11 - 60
p (wo) (i,
1
T
11 10
+
t1
11
(T 12)
(x
(X2 = x|X1 = i)pp(ms) (X1 = 1) + +p(mo)) (X1 = =|X²== = j)p(i=)) = -]]]
T2
t1
2
t2 t1
+
[p (xleo)(X2 = -||X1 = ()p(le) = 8) + p(ww)) = i|Xx2 = =))p(uu) = ز)]
T
1
Pis (i.j) +
11 to
2
tt
2
(ww (i.
T
t1
to
(T 12)
t1
2
*(o
t2 t1
p (m0)
t
T2
T
t
Simplifying and substituting the term of pt, this becomes:
Pto (i,5)
13
t2
2
(m (i. +
t2 12-41 t
2
(mx) +
{1 (1-t0) to pino (i.5)
13
13
17
(133 - (2) - 122 pl (m)) (i, +
(1s - (2) - [0)
pl (m) (i,
13
1/3
(122 - 11)(11 - (o) (o),
tz
D DETAILS ON TOY EXAMPLE
We now present the details of the toy example that we used to demonstrate our theoretical results. In figure 10 we present plots of each class and the full data distribution. Each cluster is defined via the following matrix
0.1
0.2
0.3
0.4
0.5
0.4
0.3
0.2
0.17
0.2
0.4
0.6
0.7
0.8
0.7
0.6
0.4
0.2
0.3
0.6
0.8
0.9
1.0
0.9
0.8
0.6
0.3
0.4
0.7
0.9
1.0
1.0
1.0
0.9
0.7
0.4
0.5
0.8
1.0
1.0
1.0
1.0
1.0
0.8
0.5
0.4
0.7
0.9
1.0
1.0
1.0
0.9
0.7
0.4
0.3
0.6
0.8
0.9
1.0
0.9
0.8
0.6
0.3
0.2
0.4
0.6
0.7
0.8
0.7
0.6
0.4
0.2
0.1
0.2
0.3
0.4
0.5
0.4
0.3
0.2
0.1
Each class is equally weighted. A pseudocode for generating the above dataset is:
21

---

Published as a conference paper at ICLR 2026
Figure 10: Definitions of the class and unconditional distributions for the toy problem.
height, Midth 30, 30
matrix1 torch. zeros ( theight, width)
[1:10, 1:10]
torch.
(cluster)
matrixl (9:18, 9:18] torch. Lenaor (cluster)
matrix2 torch. zeros ( theight, width)
matrix2 [11:20, 11:201
borch.tensen (eluster)
matrixa 19:28]
torch,tensou tensor (oluater)
Listing 3: Code to generate our toy dataset
22

---

Published as a conference paper at ICLR 2026
3:11
100
Deight
Suidene Reogle
(a) Percentage of valid (b) Percentage of unique (c) Percentage novel generated molecules
(d) Ring Count Mean as function of guidance strength
generated molecules
generated molecules
Figure 11: We display the percentage of valid, unique, and novel molecules. We find that our method the most robust an increase in guidance strength.
E NORMALIZATION FOR GENERAL DIFFUSION PROCESSES
In this section we present sample pseudocode for the general case in Listing E as well as some experiments using uniform diffusion section E.
def normal Lized www.idane_euuer__iite tralerr_raanittion dt,
1:
lcg_ssare_c get_score (x, t, condrc)
loga_nore_j gat_acore cond-None)
score_c log_score_p.exp
score_u log_score_u.exp
Set diagonal terms to zero
score_c. poore_c.sestter__11 soore_..catter__11 x(...r None], zeros_ like (score_s))
None], torch, zeroa_ kike (acore_c))
Maltiply matrix by edge to get the rates sate_c rate_u = edge score_u
Obtain the diagonal torm tetel_rate_c score_c sun keepdim-Truel total_rate_u score_u. (ddimm- keepdim True)
Get jump distribution jump_= rate_s total_rate_c jump_u rate_u / total_rate_u
Get quided jump distribution jump_y torch. serchhseftmax.t jump_e (1-w) jump_u)
jump__ Lorch.exp (M rate__ (1 rotei_r
Convert to the rate matzix rate_M = jump_w ttotll_jump
sate_w.soatler_111 xl..., Nonel, jumpa__jump_)
return a eample (delta
di sigma (L) zate_w)
Listing Our guidance in the general case using Euler transitions
E RESULTS ON QM9
We present results using our guidance mechanism in the context of uniform diffusion, applied to the QM9 small molecule dataset (Ruddigkeit et al., 2012; Ramakrishnan et al., 2014). QM9 is dataset containing small organic molecules containing up to heavy atoms. We train a conditional model on QM9 using uniform diffusion, based on the official implementation of Schiff et al. (2024), without modifying the architecture or hyperparameters. The model is conditioned on the number of rings in each molecule (ring count). Unlike ImageNet, evaluation on QM9 is more nuanced: generate 1,024 samples and assess several metrics. First, generated molecules must satisfy chemical constraints to be considered valid. Second, key goal of generative models is produce novel molecules not found the training data. Accordingly, we report both validity and novelty, along with the mean ring count (i.e., the conditioning signal), in Figure 11.
23

---

Published as a conference paper at ICLR 2026
1101
3:11
100
Deight
Suidene
(a) Percentage of valid (b) Percentage of unique (c) Percentage novel generated molecules
(d) Ring Count Mean as function of guidance strength
generated molecules
generated molecules
Figure 12:
We display the percentage of valid, unique, and novel molecules. We find that our method the most robust increase in guidance strength.
We find that our method is the most robust to increases in guidance strength. However, in general, all methods perform comparably well across the full range of strengths. This suggests that normal- ization may have less pronounced effect in the uniform diffusion setting. Due to the complexity of evaluating on QM9, further experiments on additional discrete datasets are needed to more conclu- sively determine the optimal guidance mechanism. We leave this for future work.
F EXTRA EXPERIMENTS ON QM9 FOR MASKED DIFFUSION
We present similar results using guidance mechanism but in the context of masked diffusion, applied to the QM9 small molecule dataset (Ruddigkeit et al., 2012; Ramakrishnan et al., 2014). We train conditional model QM9 using uniform diffusion, based the official implementation of Schiff et al. (2024), without modifying the architecture or hyperparametens. The model is condi- tioned on the number of rings in each molecule (ring count). We report statistics under the same setting as in Appendix E.
We plot the results in Figure 12, find that our method is the most robust to increases in guidance strength and generally achieves better results across various guidance strengths. Suggesting that normalization is also helpful technique on this domain.
24

---

Published as a conference paper at ICLR 2026
G GENERATED SAMPLES TEXT TO IMAGE
To generate these samples we made use of single node with 8 NVIDIA A100 GPUs. We present samples to compare our method against other guidance methods well as the detailed results of the GenEval benchmark in Table and 4. The results demonstrate that normalization key in order to improve the sample quality.
Table 3: Performance comparison across different guidance weights using Meissonic as base model
Ours
Unlocking
Metric
w
W
w
W
3
w
Overall
8.2
40.8
45,9
44.7
8.2
43.1
28.5
19.9
Objects Single
23.8
89.4
91.9
91.2
23.8
88.4
80.0
64.1
Objects Two
3.0
36.9
48.2
48.7
3.0
47.5
23.2
18.2
Counting
0.6
27.2
33.8
28.4
0.6
33.1
11.9
3.1
Position
20.7
72.3
77.4
77.7
20.7
72.3
48.7
29.0
Color Attribution
0.2
8.5
7.8
7.8
0.2
6.2
7.8
3.8
Colors
1.0
10.5
16.5
14.5
1.0
10.8
2.8
1.0
Table 4: Performance comparison across different guidance weights using Show-o as a base model
Ours
Unlocking
Metric
2
8
2
Overall
56.42
62.46
63.13
63.39
53.73
52.84
52.89
43.96
Objects Single
96.88
98.75
99.06
98.75
95.94
98.44
97.19
86.88
Objects Two
65.66
75.76
78.28
80.05
64.14
61.36
60.61
60.10
Counting
41.56
50.00
50.94
50.94
39.69
35.00
35.00
31.25
Position
78.19
81.38
79.79
81.12
73.14
75.00
76.60
51.33
Color Attribution
22.75
28.5
30.25
27.25
26.25
22.75
22.00
20.25
Colors
33.5
41.5
40.5
42.25
23.25
24.50
26.00
14.00
25

---

Published as a conference paper at ICLR 2026
G.1 GENERATED SAMPLES FROM MEISSONIC
We now present some samples from our method:
Normalized vs Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
Figure 13: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo bench (top). A photo of cow (bottom).
26

---

Published as a conference paper at ICLR 2026
Normalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
W
Figure 14: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of bike (top), A photo of clock (bottom).
27

---

Published as a conference paper at ICLR 2026
Normalized Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
Figure 15: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of carrot (top), A photo of suitcase (bottom).
28

---

Published as a conference paper at ICLR 2026
Normalized Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
chekcetee
W
Figure 16: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of fork (top), A photo of surfboard (bottom).
29

---

Published as a conference paper at ICLR 2026
Normalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
W
Figure 17: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompt: A photo of refrigerator (top), A photo of cup (bottom)
30

---

Published as a conference paper at ICLR 2026
G.2 GENERATED SAMPLES FROM SHOW-O
We now present some samples from our method using Show-o Xie et al. (2024):
Normalized vs Unnormalized
Normalized
Unnormalized
2.0
4.0
6.0
Normalized Unnormalized
Normalized
Unnormalized
4.0
6.0
Figure 18: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo dog (top). A photo of a tie (bottom).
31

---

Published as a conference paper at ICLR 2026
Normalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
2.0
4,0
= 6.0
8.0
Figure 19: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of laptop (top). A photo of computer mouse (bottom).
32

---

Published as a conference paper at ICLR 2026
Normalized Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
2.0
4.0
6.0
Figure 20: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of sandwich (top), A photo of baseball bat (bottom).
33

---

Published as a conference paper at ICLR 2026
Normalized Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
2.0
4.0
= 6.0
Figure 21: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompts: A photo of train (top). photo of cell phone (bottom).
34

---

Published as a conference paper at ICLR 2026
Normalized Unnormalized
Normalized
Unnormalized
Normalized Unnormalized
Normalized
Unnormalized
2.0
4,0
6.0
Figure 22: Comparison of samples generated by different guidance methods across various seeds and configurations. Using the prompt: A photo of chair (top), A photo of tv (bottom)
35

---

Published as a conference paper at ICLR 2026
H
GENERATED SAMPLES IMAGENET
H.1 GUIDANCE STRENGTH w = 2
Metiood
Unlocking Guidance
Guidance
Our Metboo
Unlocking Guidance
Simple Guidence
Our Method
Unlocking Guidance
Simple Guidance
Method
Guidance
Figure 23: Comparison of samples generated by different guidance methods across various seeds configurations.
36

---

Published as a conference paper at ICLR 2026
H.2
GUIDANCE STRENGTH 3
Mathed
Unlocking Guidance
Simple Guidence
Our Methoo
Unlocking Guidance
Simple Guidance
Methed
Unlocking Guidunce
Our Method
Unlocking Guidance
Simple Guidance
Figure 24: Comparison of samples generated by different guidance methods across various seeds or configurations.
37

---

Published as a conference paper at ICLR 2026
H.3
GUIDANCE STRENGTH 4
Mathed
Unlocking Guidance
Simple Guidence
Our Method
Unlocking Guidance
Simple Guidance
Methed
Guidance
Simple
Our Methood
Unlocking Guidance
Simple Guidance
Figure 25: Comparison of samples generated by different guidance methods across various seeds or configurations.
38

---

Published as a conference paper at ICLR 2026
H.4
GUIDANCE STRENGTH 5
Mathed
Unlocking Guidance
Simple Guidence
Our Mothod
Unlocking Guidance
Simple Guidance
Methed
Unlocking Guidance
Guidance
Our Methood
Unlocking Guidance
Simple Guidance
Figure 26: Comparison of samples generated by different guidance methods across various seeds or configurations.
39

---

Published as a conference paper at ICLR 2026
STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
This work made use of large language models to assist with proofreading and improving the clarity of the writing. All research ideas, theoretical development, and experiments were carried out solely by the authors, When used for coding, was solely used for plotting purposes.
40


Reviews:
[{"review_id": "s3jF7mwO4E", "reviewer": "Reviewer_sPcA", "summary": "The paper analyzes classifier-free guidance (CFG) in masked discrete diffusion and shows—theoretically (in low dimensions)—that strong early guidance harms quality, whereas late guidance has a larger positive impact. It identifies a flaw in common CFG implementations, where imbalanced transitions unmask tokens too quickly. To remedy this, the paper proposes a simple one-line column/softmax normalization that smooths transport. Empirically, this normalization improves ImageNet-256 FID, ImageReward text-to-image alignment, and MATH-500 text generation accuracy.", "questions": "1. Since the theory is developed in 1–2D masked diffusion, could you provide bounds or a proof sketch that extends to realistic high-d CTMCs?\n\n2. With results limited to ImageNet-256 FID and MATH-500 on a single LLM backbone, is there evidence that the approach generalizes to other discrete diffusion tasks?\n\n3. Although normalization is cheap, schedule changes can alter unmasking rates and step counts. Therefore, reporting the resulting overhead (wall-clock, GPU-hours, memory) is important to demonstrate the strength of the proposed method.\n\n4. The proposed framework mainly targets masked diffusion, what changes are needed for uniform or other discrete diffusions?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper presents a principled low-dimensional analysis of CFG for masked discrete diffusion, showing that strong early guidance is harmful while late guidance is beneficial.\nBuilding on this insight, it introduces an elegant, theory-grounded tweak—a one-line column/softmax normalization—that corrects imbalanced early unmasking.\nImportantly, by linking this tractable normalization to improved robustness and better FID/ImageReward/MATH-500 results, the paper offers a practical change likely to be widely adopted in discrete diffusion implementations.", "weaknesses": "1. The paper develops analysis and closed-form expressions only in 1–2 dimensions for masked discrete diffusion. As a result, guarantees for realistic high-dimensional CTMCs remain implicit, leaving the theoretical treatment somewhat loose.\n\n2. Some results isolate the mechanism using a simple sampler without remasking and with fixed step counts (e.g., 50 steps on ImageNet-256), which may limit generality. Moreover, sampling schedules and samplers are crucial to implementing the guidance mechanism, yet the paper provides neither theoretical analysis nor empirical evaluation of their effects.\n\n3. Although the evidence spans ImageNet-256 FID, ImageReward, and MATH-500 with a single LLM backbone, broader discrete domains (e.g., ASR, text, protein) and larger vocabularies are not explored. Because the approach is closely related to Unlocking Guidance and Simple Guidance, running additional experiments on the datasets used in those works is crucial.\n\n4. Quality gains are reported, but diversity metrics are missing. Because guidance mechanisms can reduce sampling diversity, it is important to report diversity measures and quantify the proposed method’s impact.\n\n5. It is recommended to ensure the citation style is applied uniformly across the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Gfq25PkieP", "reviewer": "Reviewer_xrKo", "summary": "The paper analyzes CFG for masked diffusion, showing (in a low-dimensional setting) that strong early guidance hurts quality while late guidance helps. It also identifies an implementation flaw that causes over-eager early unmasking and proposes a one-line fix to smooth the transition from the masked prior to the data distribution.", "questions": "1. Prior work reports that higher guidance weights can distort images and reduce fidelity, yet in your Figure 8 larger guidance weights yield better performance. Can you justify this discrepancy?\n2. The appendix includes many generated images—could you also include the text prompts used to produce them?\n3. Unified reward models are surging. Could you evaluate your method with Show-o [1] and Dual Diffusion [2] to assess its effectiveness in that setting?\n\n[1] Xie, J., Mao, W., Bai, Z., Zhang, D. J., Wang, W., Lin, K. Q., ... & Shou, M. Z. (2024). Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528.\n[2] Li, Z., Li, H., Shi, Y., Farimani, A. B., Kluger, Y., Yang, L., & Wang, P. (2025). Dual diffusion for unified image generation and understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 2779-2790).", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. CFG is a well researched topic in continuous diffusion. But in masked/discrete diffusion is under active exploration. Clarifying scheduling effects is valuable for both image inpainting/masked modeling and text infilling models\n2. Section 3.4 provides exhaustive analysis across factors such as time parameters and guidance strength.\n3. The experiment in Section 2.3 is intuitive, improving explainability.\n4. The method can be performed in inferencing time.", "weaknesses": "1. **Novelty:** The importance of guidance scheduling and rescaling by conditional/unconditional norms has been reported by Kynkäänniemi et al. (2024). Can you clarify how your approach differs?\n2. **Metrics/Benchmarks:** For text-to-image evaluation, you only report ImageReward. Could you also evaluate with **HPSv2** to check for aesthetic trade-offs? Additionally, please test on T2I benchmarks like **GenEval** and **T2I-CompBench**.\n\n[1] Kynkäänniemi, T., Aittala, M., Karras, T., Laine, S., Aila, T., & Lehtinen, J. (2024). *Applying guidance in a limited interval improves sample and distribution quality in diffusion models.* NeurIPS 37, 122458–122483.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZoqNPGoeak", "reviewer": "Reviewer_DWkQ", "summary": "This paper analyzes the effect of using guidance schedules in discrete diffusion models and how such schedules can improve performance compared to constant guidance. Drawing inspiration from continuous diffusion models, the authors note that high guidance scales are necessary to enhance sample quality and condition alignment; however, using a constant high guidance scale results in suboptimal performance due to large changes during the early sampling stages. They first demonstrate that transition matrices should be normalized (via SoftMax) in the presence of guidance. Then, using a low-dimensional setup, they show that a constant guidance scale leads to poor performance, reducing diversity and increasing bias in the final generation. To address this, the authors propose an increasing weight schedule to balance the effect of guidance across sampling steps, leading to improved performance across various diffusion models for both image and text generation.", "questions": "1. Do normalization and time-dependent schedules also improve the performance of Simple Guidance?\n\n2. Could you provide additional metrics, such as Precision and Recall, to separately evaluate diversity and quality, rather than relying solely on FID?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Since guidance in discrete diffusion models is underexplored, this paper bridges an important gap between CFG in continuous diffusion models and discrete diffusion models. The results have potential implications for all systems that rely on discrete diffusion.\n\n- The proposed method is simple and can be easily integrated into existing sampling pipelines.\n\n- The theoretical results provide intuition and reasoning behind the method, although their presentation could be significantly improved.\n\n- Experiments are conducted on both image and text generation benchmarks.", "weaknesses": "- In my opinion, the main weakness of the paper is its presentation. Several parameters are either misused in notation or not defined prior to their introduction in the text. This makes the paper difficult to follow and obscures the intuition and analysis behind the proposed method.\n\n- Section 3.4 appears to contradict the main message of the paper. It states that “effective schedules have higher guidance in the beginning and middle phases of generation,” whereas the best performance is reported for “schedules that apply stronger guidance during the middle and later stages of the sampling process, while keeping early guidance small.”\n\n**Minor**:\n- There might be a typo in Equation 6.\n- Table 1 indicates that the increasing schedule has one parameter, while Table 2 shows it has two parameters ($w$ and $r$).\n- The theoretical analysis could also consider the effect of guidance on condition alignment, in addition to its impact on diversity and stability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XZOCSqQWgY", "reviewer": "Reviewer_Yvo9", "summary": "This paper investigates Classifier-Free Guidance (CFG) in the context of masked discrete diffusion models. The authors claim to have identified a key flaw in existing CFG implementations, where high guidance strength can lead to imbalanced transitions and unmasking too rapidly, which degrades sample quality.\n\nTo address this issue, the authors propose a novel CFG mechanism based on a column normalization of the rate matrix, achievable via a one-line code change. The authors provide a low-dimensional (1D and 2D) theoretical analysis to motivate this change and to characterize the properties of effective guidance schedules.  Experimental results are also provided to validate the technique.", "questions": "1. Can the authors resolve the central contradiction (Weakness #1)? Which finding is correct: the 2D theory suggesting high early guidance, or the empirical results suggesting high late guidance? Why does the theory not seem to match the practice here?\n\n2. In the abstract, what is meant by \"late-stage guidance has a larger effect\"? Is this a positive or negative effect on generation quality? (This also relates to Weakness/Question #1.)", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper's primary contribution is a fix for a problem in discrete CFG that can be implemented with a one-line code change. I always like seeing examples of this kind of simple and effective solution, especially when they are theoretically motivated.\n2. The methodology of using low-dimensional (1D and 2D) theoretical analysis to derive insights that are relevant to the high-dimensional setting is a strong and interesting approach.\n3. Based on the experimental results, the proposed normalization fix appears to be effective.", "weaknesses": "My current impression of the paper is that the authors have indeed likely identified a flaw in CFG for masked discrete diffusion models and found a simple fix for it. However, the more serious weaknesses I outline below prevented me from more deeply appreciating the authors' contributions, which has me currently sitting on the negative side of the fence. That said, I admit to being a good deal less familiar with the discrete diffusion literature than I am with the continuous diffusion literature, hence the low confidence in my score. I will keep an open mind during the discussion period, and I look forward to seeing the authors' responses and the feedback from the other reviewers.\n\nI'll list my concerns in decreasing order of seriousness:\n1. There seems to be a major contradiction in the paper's claims regarding effective guidance schedules. In Section 3.4 (lines 368-370), the 2D theoretical analysis concludes: \"Therefore, effective schedules have higher guidance in the beginning and middle phases of the generation, and their effect towards the end is negligible.\" This directly contradicts empirical results from numerous other works as well as statements in this paper's abstract and the discussion in Section 4.2: \"[S]chedules that apply stronger guidance during the middle and later stages of the sampling process, while keeping early guidance small, tend to perform better.\" Unless there is a serious misreading on my part, the results of Section 3.4 are never reconciled with these statements and the empirical record.\n2. Considering the weight given to the paper's theoretical results, I found them very hard to follow in their current form. Notation is occasionally not defined, and it was sometimes challenging to match the theorems or lemmas with their counterparts and proofs in the appendix.\n3. On the topic of notation, in most of the CFG literature, in both continuous and discrete diffusion, $x$ is a state and $y$ is a condition. But in this paper $y$ (never explicitly defined) is apparently a member of the state space (judging by the formula for the partition function) as well as a condition. I found this quite confusing.\n4. There are some minor typos and other errors in the paper (e.g. *diffusion* is misspelled in the header for Section 2.1, the regionalism *smoothen* is used in the abstract instead of the standard *smooth*).\n5. In the literature, the probability ratio the authors refer to as the *score* is referred to as the *concrete score*. I recommend sticking to this terminology to avoid confusion with the score as it's understood in the continuous diffusion literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)