Don’t Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
Authors: Woojin Kim
Jaeyoung Do
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Language Models
Controllable Text Generation
Primary area: applications
TL;DR: We propose TTA -Diffusion(Token Timestep Allocation), that stabilizes the control process of diffusion language models by dynamically allocating timesteps per token, addressing instability that disrupts fluency and control.

Abstract: Classifier guidance is a widely adopted technique in diffusion language models, used to steer generation toward desired attributes. However, such guidance often introduces instability during the generation process, where token-level updates fluctuate across timesteps. We identify and formally characterize this phenomenon as update-forgetting. This instability disrupts the refinement process by overwriting semantic edits, ultimately degrading fluency and coherence, which is particularly problematic in tasks like controllable text generation. To address this, we propose TTA-Diffusion, a novel inference-time approach that dynamically allocates timesteps per token based on refinement needs. Unlike conventional diffusion models that apply uniform updates, TTA-Diffusion employs structured timestep allocation, preserving stable tokens while allowing uncertain tokens to undergo progressive adjustment. Experimental results across diverse tasks demonstrate that TTA-Diffusion significantly outperforms both diffusion-based and auto-regressive baselines in fluency and control accuracy while improving computational efficiency by reducing the number of required timesteps. On the sentiment control task, TTA-Diffusion achieves over 20\% higher accuracy and nearly half the perplexity of prior diffusion models, using less than one-fifth the denoising steps. This work highlights the importance of mitigating fluctuations in token updates and promoting a balanced refinement process, thereby enhancing stability and controllability in controllable language modeling.

Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
Woojin Kim! Jaeyoung Do 1.2
AIDAS Laboratory, 'ECE & AAPAI, Seoul National University {wjk9904, jaeyoung donuu ac. kr
Abstract
While diffusion language models (DLMs) enable fine- aiee-griied refinement, their practical controllability remains fragile. We identify and formally characterize a central failure mode sesee--pdatt--oo-eeee updates induce token-level fluetuations across timesteps. erasing earlier semantic edits and disrupting the cumulative refinement process, thereby degrading fluency and coherence. As this failure originates in uniform, context-agnostic updates, effective control demands explicil token ordering. We propose Token Timestep Allocation (TFF-DIFFSSOO)), which realizes soft, semantic token ordering via per- token timestep schedules: critical tokens are frozen early, while uncertain tokens receive continued refinement. This timestep-based ordering can be instantiated either fixed policy or an adaptive policy driven by task signals, thereby supporting broad spectrum of refinement strategies. Because operates purely at inference time, it applies uniformly across various DLMs and naturally extends to diverse supervision sources. Empirically, TIFFUDIFUSION improves controllability and fluency: on sentiment control, it yields > 20% higher accuracy and nearly halves perplexity using 1/5 the steps; in detoxification, it lowers maximum toxicity (12.2 vs. 14.5) and perplexity (26.0 vs. 32.0). Together, these results demonstrate that softened ordering via timestep allocation is the critical lever for mitigating moddat-forgetting and achieving stable and controllable diffusion text generation.
in which uniform, context-agnostic
Introduction
Diffusion language models
have emerged a promising alternative auto- regressive models by generating text through iterative refinement. They generate text in parallel and leverage bidirectional context for flexible revision, making them well-suited for tasks requiring fine- grained control. A widely adopted technique for guiding this refinement is classifier guidance, which injects external gradients at each step via an auxiliary classifier. This mechanism has demonstrated effectiveness in structural control [22 16] semantic modulation [1]. and alignment 47
Despite its advantages, classifier-guided diffusion language models still face substantial challenges. One major limitation is the degraded fluency, as the parallel generation process weakens token-level dependencies and hinders coherent phrasing (23). This is especially problematic for controllable generation tasks, which demand both attribute alignment and linguistic fluency [2] 25 55 22]
Moreover, existing diffusion-basel control methods typically require hundreds of steps often exceeding 200 to gradually enforce control, leading to significant computational overhead 22
We identify bottleneck behind these issues: a phenomenon term update-forgetting. wherein classifier-guided modifications made at one timestep fail to persist in subsequent steps due to uniform, context-agnostic token updates. As illustrated in Figure this disrupts the cumulative nature of
t: Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Eaput Condition
Diffuioon Backward
Generated
Profis: "The lake"
Coodition: Fookive seer
"Generate a portitic sentonse about cul."
Me-t
my cut.
Input for the timesip T:
Pyur
TTe like la clam, and food plcntiaa-lliiaa... iiiia....
Denoising
Gaiked
Densiiing Step
Output the denobing step:
Classifier
withoat allecation:
"The like food tha't whet.
*I "I my
"I haté
Model might forget
with allocative:
"The clean, plantitd fuit.
Pan
previous updates
Figure 1: Illustration of update--frreetiis Left:: classifier-guide semantic edits (e.g., love) can be inadvertently overwritten in later denoising steps (e.g., hate). Right: This example from our experiments shows positive sentiment tokens being reversed (clean - dirty), undermining control; with allocation, the edits are preserved.
the iterative refinement process by erasing overwriting previously guided changes, resulting in redundant updates, computational inefficiency, and reduced controllability.
To address this, we turn our attention to the emerging notion of token ordering, which determines the sequence in which tokens are updated during inference. Prior approaches have mainly explored masked diffusion, where updates are applied through unmasking strategies based on confidence [[8 34 However, these approaches remain agnostic to semantic importance and are often limited to discrete formulations. In contrast, we propose a semantic-gus guided ordering mechanism that prioritizes semantically critical tokens. This design directly incorporates task-specific supervision into the update process and naturally extends to both discrete and continuous diffusion frameworks, offering a more general approach to diffusion text generation.
Building on this perspective, we introduce Token Timestep Allocation (TTA- DIFFUSION), a decoding- time framework that operationalizes token ordering through structured timestep scheduling. Instead assigns distinet
of applying uniform, agnnext-agnostic updates across all tokens, timesteps that determine when and how strongly each token refined. This enables explicit control over generation order, supporting strategies such as left-to-right decoding without requiring addi- tional training. Beyond fixed schedules, we further propose an adaptive allocation mechanism that dynamically adjusts timesteps according to classifier gradients. By prioritizing tokens with high semantic importance, this method preserves classifier-guuide guidddd edits across successive steps, mitigates update-forgetting. and ensures precise refinements critical tokens.
We implement ITA-DIFFUSION on the simplex diffusion language model seamless integration of external classifiers for guidance. To demonstrate the generality of our approach, we also validate on continuous and discrete diffusion formulations. addition, we further draw inspiration from progressive distillation methods 42 3
which facilitates
and progressively tune the model for reduced timesteps, enabling fast generation without sacrificing control fidelity or fluency, while significantly lowering inference cost and compensating for the overhead introduced by guidance.
Consequently, TTA- TIA-DIIFUSION effectively mitigates update-forgetting by preserving semantic consistency and minimizing redundant updates throughout the generation process. Empirically, it achieves strong controllability and fluency with significantly fewer diffusion steps (as low as 100 or even 50), substantially reducing inference costs compared to prior diffusion based methods. Beyond these performance improvements, our work positions remantic-based token ordering with inference- time timestep allocation as a principled and versatile framework for structured ceneration-openinngg new directions for efficient, fine- une-grained control in diffusion language models.
We identify updatte-foreetting key limitation in diffusion text generation, where uniform timestep updates crase classifier-uuided modifications and undermine fluency.
We propose "TA-DIFFUSION, an inference-time framework that unifies semantic-based token ordering with structured timestep allocation to preserve guided edits and improve generation quality.
Experimental results show that TTA-DIFFUSION outperforms baselines across tasks; in detoxification, achieves lower maximum toxicity (/2.2 vs. 14.5) and reduced perplexity (26.0 vs. 32.0) while maintaining diversity- even with under 100 timesteps.

---

2 Controllability Challenges in Diffusion Language Models
In this section, first outline the classifier-guided update mechanism in diffusion models for language generation and then define and illustrate the update-forgetting phenomenon.
Classifier-Guided Update in Diffusion Classifier guidance in diffusion-based generation can be formulated via Bayesian inference. Given token distribution T. E RNXV at timestep and a target label y, the goal is to maximize the posterior P( v). which by Bayes" rule becomes:
P(ä v)) X P(y P(a)).
Taking the gradient the peg-posterior gives:
Va log ( y) = V log Ply x,) + Vä, log P(t)).
Since the diffussion model estimates the prior score Va, log P P(āe), classitier guidance approximates the remaining term using a classifier (y āe):
/ = ãe + AV 2, log (2 Øt),
where λ controls the guidance strength, steering the process toward the target label.
2.1 Problem Definition
We identify two key instability phenomena that hinder controllability in diffusion based text gen- eration: diffusion fluctuation and update-forgetting These phenomena capture how outputs may drift from their inputs or regress from previously guided states, ultimately undermining stable and effective control.
Terminology Diffusion fluctuation captures the discrepancy introduced by single diffusion step, defined as the distance between the perturbed input and the decoded output at timestep t + 1. Update- forgetting refers to the semantic drift that occurs when guidance applied at timestep t fails to persist in the next step, weakening reversing the intended effect.
Formally, let 2t (z) , input to timestep + 1 is
N denote the sequence at timestep with logits ā, The perturbed
+ it + + Me
o))
Definition 1 Diffusion Fluctuation. The fluctuation at timestep t is defined the distance between the input and the decoded output:
R₂ = dist(Fet1, 2 a(41);
where dist((;.) is a general distance function.
Definition Sppdat-Forgettinn The extent of update-forgetting at timestep t is defined as the semantic shift belween the guided sequence and the generated sequence I 1:
F, = dist( puided It--))s
where dist(.,.) measures semantic divergence.
Practical Deployment In practice, the distance function dist(:.) can be instantiated in multiple ways. For fluctuation, consider Hamming distance, BLEU, and BERTScore. In analyses, we primarily adopt Hamming distance for simplicity, while also reporting BLEU and BERTScore semantics-aware complements. For update-forgetting, we focus on the subset of tokens most influenced by guidance. Specifically, we measure the distance over the top-k tokens identified as most responsive the guidance signal, yielding a ratio that quantifies the degree which guided effects are forgotten across timesteps.
2.2 Analysis
Building upon the above, propose two hypotheses and validate them through experiments.
Hypothesis 1: Excessive fluctuation disrupts token coherence and reduces fluency.

---

T=0
T=400
T-800
T-999
43
46
90
220
Mean Perpiexity
Mean Perplexity
Mean Perplexity
Mean Perplexity
43
70
1.60
57
55
110
23
35
70
0.05
0.07 0,09 0.11
0.07
0.11
0.15
0.18
0.16
0.22
0,28
0.35
0.40
0.48
0,55 0.60
Moan Fluctuation Ratio
Mcan Fluctuation Ratio
Moan Fluctuation Ratio
Mcan
Fluctuation Ratio
Figure 2: Fluctuation vs. perplexity across timesteps. At each timestep I, samples are grouped by fluctuation ratio, showing that higher fluctuation consistently associated with higher perplexity.
Experimental Setup. We use the TESS 32 model with sentiment control applied via classifier
trained on the IMDB Movie Reviews dataset. Generation is performed with 7 1000 diffusion steps per sample under consistent decoding settings, At each intermediate timestep t, we compute the mean fluctuation ratio RG defined the average fluctuation (Normalized Hamming distance) from steps 1000 down to I, and record the model's perplexity at timestep I. For each timestep, we correlate R2 with the final perplexity across 150 generated samples assess how fluctuation dynamics relate to fluency.
Empirical Observation. Figure illustrates the relationship between fluctuation and perplexity. We observe a strong positive correlation, with a binned correlation of r 0.86. This supports Hypothesis I: increased fluctuation indicative of degraded fluency. Furthermore, semantics-aware metrics show consistent trends: BLEU, BERTScore, and cosine similarity exhibit strong correlations with perplexity timestep (-0.6 0.8), closely mirroring the perplexity curves. While we also observe that low-confidence tokens are more prone to fluctuation, confidence alone does not sufficiently account for the degradation in fluency; rather, excessive fluctuation emerges as the dominant factor driving reduced coherence. See Appendix B. for detail.
This observation is consistent with theoretical analyses of discrete diffusion models 28 12 which establish connections between per-step transition instability and generation cross-entrops Specifically, higher fluctuation ratio reflects more pronounced transitions, which turn increases the expected divergence between the model's generative distribution and the target distribution. We provide additional theoretical context in Appendix A.4
Hypothesis 2: Update-forgetting weakens control accuracy.
Experimental Setup. Using the same setup in Hypothesis 1, investigate how edits to semantically pivotal tokens change the semantic alignment of the output with the target attribute. For each sequence at timestep f, we apply classifier guidance and then perform the next diffusion update to obtain 2-+1- We compute classifier gradients with respect to the input and identify the top-5 tokens with the highest gradient denmsedenoted as key tokens K,. We then measure the classifier's confidence in the target label after classifier update and before the next classifier update. Our analysis focuses on cases where the majority of key tokens are modified during the update, as determined by mismatch between the updated sequence 2 and the denoised output 2++1.
True Probabilitty
1.0
0.9
0.8
Diftusionn Step
0.7
After Diffusion Step
200
600
000
Timestep
Figure 3: Classifier confi- dence drop due update- forgetting.
Empirical Observation. Figure 3 shows a consistent drop in classifier confidence when key tokens are altered. While the average drop in probability across all steps is approximately 3-4%, the drop becomes significantly larger-often exceeding 10%-when classifier-critical tokens are modified. This provides strong evidence that apdate-forgettio disrupts classifier guided control.
Based on the above observations, we substantiate the dual role of fluctuation control in classifier- guided text generation: reducing overall fluctuation enhances fluency, while stabilizing key tokens preserves classifier-driven constraints, thereby improving controllability. These findings underscore the importance of minimizing unnecessary modifications to preserve linguistic quality and improve controllable generation in diffusion language models.

---

Timestep Allocation for each token
Constant
Linear
Adaptive
Token position
Token position
Token position
1500
1000
1003
1030
1030 "
IDU
Iake "p"
Sontence Timestep
+gt
7.3
330
150
1600
793
750
Positive Sentiment
H2
Bee
3000
liee
plos
Sic
Boo
190
Hoom
like
pica
Hi
16se
spple
Every token shares the same schedule
Tokens at the front receive lower timestepps
Important tokens
receive lower timesteps
Figure 4: A comparison of token timestep allocation strategies. The left panel illustrates a default schedule in which all tokens share the same timestep. The middle panel depicts a linear schedule where timesteps decrease uniformly across tokens, allowing gradual denoising, The right panel demonstrates the adaptive schedule, in which critical tokens with high gradient values are assigned smaller timesteps, preserving important updates while refining less significant tokens.
Methodology
We begin from the perspective of token ordering- the order in which tokens are refined during inference. In masked diffusion frameworks, ordering has typically been implemented through heuristic unmasking strategies, such as selecting tokens based on confidence or margin [18] (34]. Once unmasked, these tokens are fixed and excluded from further refinement. While effective in certain tasks, this "hard" ordering can prematurely lock in errors and is limited to discrete formulations.
In contrast, we propose soft token ordering framework that achieves ordering through timestep allocation rather than discrete masking. Instead of fixing tokens once generated, we dynamically adjust the noise schedule applied to each token, allowing them to remain revisable but with controlled update strength. This approach enables fine-grained prioritization of tokens while avoiding irreversible early commitments. Moreover, because the mechanism is based timesteps rather than masking, integrates naturally with not only discrete diffusion but also continuous diffusion models.
Building this framework, we introduce TTA-DIFFUSION (Token Timestep Allocation), method that dynamically assigns timesteps to tokens to enhance stability and controllability in the diffusion process. By regulating token update rates, "TA-DIFFUSION mitigates excessive fluctuations and stabilizes guided text generation. Our approach strategically prioritizes important tokens so that guided constraints persist throughout generation.
3.1 Token Timestep Allocation
Following AR-Diffusion 49
we assign each token x; its own timestep f. Unlike AR-Diffusion, which assigns timesteps based solely on positional movement speed in training time, we define token timestep function that independently controls refinement for each token. Formally, let f(i, be a token timestep function mapping the global sentence- timestep and the token index to local teven-level timestep:
t, = fi,tt).
A token's timestep determines its effective noise level: larger timesteps correspond higher noise and stronger denoising, while smaller timesteps correspond to lighter refinement. By adjusting 4, we precisely control how much refinement each token undergoes, proportional to its uncertainty and contextual importance.
To formalize this, recall the forward diffusion process:
3t
.V Gos for (1-oz)
where (ax ) is strictly decreasing sequence of noise variance. As increases, the injected noise grows, requiring stronger denoising during the reverse process. This property enables us to tailor the refinement strength at the token level through timestep allocation (see Appendix A.6 6. for derivation).
5

---

Building on this principle, we design allocation schedules that distribute timesteps across sequence of length N under maximum timestep T. For instance, linear schedule
flincar (i,
progressively increases timesteps with token index, keeping earlier tokens more stable while allocating greater refinement to later tokens. This process is illustrated Figure
3.2 Semantic-based Adaptive Allocation Strategy
While the fixed schedules Section 3.1 enforce structured timestep assignments, they do not account for the varying importance of individual tokens. Some key tokens require minimal denoising to preserve their intended semantics, whereas others benefit from more extensive refinement. To address this limitation, propose adaptive strategy that leverages classifier gradients guide token-wise timestep allocation.
We interpret the gradient of the classifier output with respect to each token embedding an indicator for token importance. This intuition is supported by prior work showing that gradient- based attribution identifies important input tokens for model predictions [44 48 46). In generation process, where classifier guidance is applied between denoising steps, a high gradient magnitude indicates that token has already been strongly shifted toward the desired attribute and is likely aligned with the control objective. To avoid overwriting this adjustment during the next denoising step, we assign a smaller timestep to such tokens. This limits further perturbation and helps preserve classifier-driven changes. We offer a simple theoretical insight into this mechanism in Appendix A..5
Let (g: / be set of gradient magnitudes derived from a task-specific classifier scoring function, indicating the relative importance of each token. We first normalize these gradients:
g: Jr min, 9j
i= 1=
max, 9j min, 9)
To adaptively allocate timesteps, we assign smaller timesteps tokens with larger normalized gradients 9.. The final timestep allocation, incorporating a smoothing factor Osmooohh is defined as:
g adaptive
Osmoothh + (1 Ossmooth] (1 -.)
3.3 Progressive Step Reduction
We build upon progressive distillation approaches 42
where a student model with fewer diffusion steps is trained to imitate a teacher model trained with a larger number of steps. While effective, we found that the distillation objective becomes unstable in the final outputs, especially under small timestep regimes. Thus, we adopt a simplified training strategy that eliminates the need for distillation. Given a teacher model trained with T diffusion steps, initialize a student model and fine-tune for N = r 7 steps, where E (0, 1] is the reduction ratio. Instead of matching the teacher's outputs, the student optimized directly via cross-entropy loss over ground truth tokens (a',...,, ....,"")). using its refined logits X, at each step:
N Σ
fer EXND
CE(softmax (X1). X)
where X, is iteratively refined through the diffusion process.
Experiments
For our main experiments, we adopt simplex diffusion language models 32 because their simplex parameterization facilitates direct utilization of externl classifiers. Following the experimental setup of [1 32), we use pretrained RoBeRTa-large (26) model, trained on a IM subset C4 [3,, as our base model. We initially train the base model with 5000 diffusion timesteps and then progressively reduce the train steps (1000, 200, 50). For classifier guided updates, we fix the guidance strength to À 2000. See Appendix B for further details.

---

Table 1: Results detoxification and sentiment-control tasks. All models size-matched 330M parameters, except LD4LG (110M). For Diffusion-LM and LD4LG, maximum toxicity is not reported because the prompt is not used. For TTA- ITA-DIFFUSION, report the variant trained with 50. Results use linear allocation for toxicity and adaptive allocation for sentiment, with smoothing factor of 0.6.
Toxicity Max. tox.
Sentiment Control PPL.
Model
Avg. tox
PPL.
Dist-3t
Acct
Dist-3f
Auto-regressive Baselines
PPLM
30.6
59.7
107.4
0.95
42.6
201.1
0.94
GeDi
22.0
36.1
98.8
0.94
79.9
98.6
DExperts
15.1
32.0
48.0
0.87
83.2
31.8
0.93
Air-decoding
18.5
40.4
49.0
82.6
27.1
0,94
LM-Steer
19.1
47.0
44.4
0.91
85.4
78.8
0,86
Diffusion Baselines
Diffusion-LMr-anne
21.8
131.2
0.94
72.8
89.3
0.94
SSD-LLM1-100
24.6
50.3
58.3
0.94
76.2
51.1
0.94
LD4LGT-250 TESST=1000
14.5
296.4
0.90
59.9
70.7
0.95
14.6
32.3
58.8
0.92
71.1
31.7
0.85
Ours
TTA (50) T=200
12.2
26.0
40.6
0.92
94.7
20.5
0.86
TTA (50) T=100
12.2
26.7
46.3
92.7
28.7
0.86
TTA (50)
12.5
27.3
59.5
0.94
88.7
47.3
0.87
4.1 Tasks & Evaluation Metrics
Detoxification. This task aims to generate non-toxic text while preserving fluency and semantic coherence. Following Qian et al. (37), select 203 prompts. For each, we generate 20 sequences with a maximum sequence length of 64.
the challenging subset of Real/oxxicityPromp (6) and
Sentiment Control. This task involves generating text that adheres to sentiment constraints, such as positive or negative. Using prompts from PPLM we generate 50 sequences per sentiment.
Topic Control We generate text for four target topics topics-World Business, Sports, and Sci-Tech using 20 prompts from PPLM (2), with 20 sequences generated per topic.
Generation Following Li et al. [22]. we generate text under lexical con- straints, including syntax trees, syntactic spans, and length constraints, Using 200 constraints applied to 50 sentences each. We train BERT-base on the E2E dataset 355 to align with Li et al. [22
Eexically-Constraiiee
Evaluation Metrics To evaluate overall generation quality, we measure fluency and diversity. Fluency calculated using Perplexity (PPL) with GPT-2 Large (38), while diversity evaluated using Dist-3 For each task, we measure control strength via APIs or classifiers. For details, see Appendix B
21
4.2 Baselines
Auto-regressive Baselines. PPLM 2 modifies the hidden representations of the model with
gradient ascent. GeDi [19 leverages CC-LM guide generation. DExperts 25 contrasts expert and anti-expert models to regulate output distribution. Air-decoding [5] conditions generation on
prefix-based language models, while LM-Steer [00 adjusts word embeddings enforce control.
Diffusson Baselines. Diffusion-LM
performs end-to-end training in embedding space, while SSD-LM [1] adopts himplex-based method in vocabulary space. L.4LG (29] operates in latent
space with control codes for guided generation. As our architecture builds on TESS 32 we include it for comparison.
4.3 Main Results
We present the results for detoxification and sentiment control in Table 1 and lexical control in T Detailed results, including those for topic control, are provided Appendix Step-by-step outputs and full generation examples can be found in Appendix

---

Detoxification and Sentiment Control As shown in Table "TA-DIFFUSION consistently achieves the lowest toxicity the detoxification task, demonstrating strong robustness even at reduced timestep of T 50. In terms of fluency, while I'TA-DIFFUSION with T 200 achieves the lowest perplexity, configurations with T 100 maintain comparable levels. For sentiment control, TTA-DIFFUSION outperforms all baselines in accuracy, with the T = 50 setting already surpassing others, and achieving up to a 10% improvement at T 200. Although perplexity increases slightly at lower timesteps (e.g., T 50), it remains within an acceptable range and still compares favorably to baseline models.
Exxically-Constrainee Generation In lexically-cone conetriiyy generation, we compare "TA-DIFFUSIN with Diffusion-LMM [22] at T 200. shown Table 2 TTA- DIFFUSION significantly improves fluency over Diffusion-LM,, reducing perplexity from 248.6 to 1 11.4 while maintaining comparable lexical accuracy.
Table 2: Lexical Control Results. Diffusion-LM are from Li et al. (22).
Results of
Metric
Diffusion-LM
TTA ITA-DIFFUSION
Syntax Tree Ace (f) Syntax Span Acc (1)
86.0
93.1
93.8
Length Acc (f)
99.9
100.0
Mean PPL (4)
248.6
111.4
4.4 Ablation Study
Allocation for Continuous and Discrete Diffusion Our context-dependent timestep allocation extends beyond sim- plex diffusion models to both continuous and discrete for- mulations. In continuous diffusion, we apply the adaptive schedule to Diffusion-LM [22] for sentiment control and observe gains in both controllability and fluency: accuracy increases from 72.8% 75.6%. while perplexity drops from 89.3 to 77.9. In discrete diffusion, we incorporate adaptive allocation into D-CBG [43] for molecular prop-
Table 3: Adaptive allocation results on discrete diffusion.
7
Method
Valid (%)
Mean Property
D-CBG
989
0.474
Addative
998
0.494
10
D-CBG
721
0.585
AAdaptive
756
0.591
crty maximization on QM9 by adjusting transition probabilities according to the same scoring signal
used by D-CBG; as summarized in Table 33 adaptive allocation improves validity and increases the mean property score (0.474> 0.494 and 0.585 0.591). Taken together, these results clarify that
adaptive allocation yields consistent, measurable benefits across diffusion regimes, not just in the simplex-basd -bsed setting.
Effect of Timestep Allocation Table 4a presents
Akgitox
the impact of token-level timestep allocation both the base model TTA (5000) and its step-reduced vari- ant TTA (50) across detoxification and sentiment control tasks. At relatively low timesteps, introduc- ing a scheduling strategy substantially improves both control accuracy and language fluency. For instance, at T = 50, scheduling reduces toxieity from 14.0 12.5 and improves sentiment accuracy from 83.5% to 95.9% while simultaneously lowering perplexity. However, we observe diminishing benefits of alloca- tion as the number of denoising steps increases. In the case of T=1000, while some improvements are visible (e.g. sentiment accuracy from 92.2 to 92.6), the relative gains are less pronounced. Detailed results across a wider range of schedules are provided in Appendix D E.1
Change Rastio
2.20
Fluc. Ratto
Key Token
0.15
0.10
0.10
0.06
0.05
0.00
900
850
780
850
200
Timestep
Timastop
(a)
(b)
Figure 5: Comparison of constant and adap- tive allocation on (a) fluctuation ratio and (b) key-token change ratio.
Also, to analyze the impact of scheduling on fluctuation and andate-forgetting. we compare an adaptive allocation with a constant allocation by evaluating the fluctuation ratio and key-token change ratio (k- = 5) for the sentiment control task. As shown in Figure 5 the adaptive allocation consistently reduces fluctuation and mitigates forget-f..g...ing.
Transferability of Timestep Allocation While developed for classifier- guided control, our timestep allocation readily extends to general generation tasks with classifier feedback. We demonstrate its effectiveness on paraphrasing and long-form generation. For paraphrasing, we follow Yuan al. [51 using the QQP dataset with timestep of 50. Token importance is estimated via paraphrase
8

---

Table Results of scheduling and timestep allocation on generation tasks.
(a) Detoxification and sentiment control.
(b) Paraphrase generation.
Model
T
Detoxification
Sentiment
Model
T
BLEU
BERT
R-L 1
Tox.
PPI
Acc.
PPL.
TTA (5000)
1000
28.9
TTA (5000)
13.2
630.4
80.8
47.3
TTA (1000)
1000
29.8
85.3
60.7
with schedule
12.8
70.8
82.1
35.5
TIA (200)
200
30.1
85,2
60.9
TTA (50)
with schedule
50
12.5
68.0
59.5
85.9
83.5
44.0
40.2
with schedule
200
30.3
85.5
61.3
schodule
30.3
61.5
TTA|5000)
TTA(1000)
TTA(200)
TTA/50)
Best Auto-regressive
Best Diffusion
Accuracy
Perplexity
100
Perplexity (PPL)
Accuracy (%5)
75
60
25
50
50
100
200
1000
100 200
Inference Timesteps
Inference Timesteps
Figure 6: Sentiment control performance of models progressively fine-tuned with r diffusion steps (T)) Left: sentiment accuracy (%). Right: perplexity (PPL.) across inference steps. Dashed lines denote the best-performing autoregressive (gray) and diffusion (black) baselines.
classifier. Table 4b shows improved results with allocation compared to a uniform baseline. In long-form generation (up to 512 tokens), we reuse prompts from sentiment control experiment and apply a fluency classifier with adaptive allocation at timestep 50. While unguided generation leads to extreme perplexity, guidance with allocation stabilizes it around 25-30. Further details are provided in Appendix B.2
About Progressive Step Reduction Figure o presents the results of progressive step reduction on the sentiment control task. As shown in the figure, while all models converge to similar performance under large inference budgets (e.g.. T 1000), significant differences emerge under low timesteps. Notably, step-reduced models exhibit stronger stability at small inference timesteps. For perplexity, although the original T' 5000 model achieves competitive results only at high inference timesteps (around 7 1000), the progressively fine-tuned models attain comparable perplexity with as few as = 100 inference steps, highlighting the efficiency gained through step reduction. See Appendix A.6 Table 9 for the detailed speedup comparison.
Effect of Different Allocation Strategies We evaluate the accuracy and fluency of a sentiment control task under several timestep allocation strategies. To isolate the effect of timestep reduction, we compare perplexity against the base TTA(5000) target (rather than TTA(50) reported in the main results). The random schedule samples a timestep for each token uniformly from (0, 7), where T is the maximum timestep. The fixed schedule uses a predetermined timestep, either or T.
As shown in Figure 7 the adaptive schedule consistently achieves the lowest perplexity (best fluency). linear and backward-lined schedules also reduce perplexity relative constant schedule, but their gains are setting- dependent (with packward-linedd sometimes edging linear). By contrast, random and fixed allocations are suboptimal; in particular, fixed-at-T exhibits large variance with occasional extreme values.
5 Related Works
Diffusion Language Models increasing interest [1] 22
The application of diffusion models to text generation has garnered
28 A key challenge in this area is bridging the gap between discrete text and the continuous diffusion framework, which has been approached through embedding-
8 24
12
based methods
[22. 24 8 49 latent modeling [29 [53] and discrete mapping [13 54 40 34 41)

---

Adaptive
Lineer
Constant
50
50
50
45
45
45
Porplexity
40
Perplexitt
40
Porplexty
40
35
35
35
30
20
200
500
1000
20
200
500
1000
200
500
1000
Max. Timesteps
Timesteps
Max. Timesteps
Random
Fixed (0)
Constant
125
60
Perplexity
Perplexity
50
Perplexity
100
50
40
40
50
30
30
200
500
1000
200
500
1000
200
500
1000
Max. Timesteps
Max. Timesteps
Max. Timesteps
Figure 7: Effect of different token allocation strategies on perplexity. We compare linear, backward linear (BLinear), random, and fixed schedules (at timesteps and T), as well as the adaptive schedule, against a constant allocation baseline (no scheduling).
32 For controllable text generation, the iterative nature of diffusion models enhances control over the generation process [22] 29 [6] 43] However, applying identical noise to all tokens disrupts
Recent studies have explored simplex mapping to operate in vocabulary space
local dependencies and harms coherence. Prior efforts token-level noise modeling (51) and multi- level diffusion [49] primarily modify training objectives and thus offer limited leverage at inference time. We instead introduce inference-time adaptive allocation strategy that schedules denoising timesteps token-wist wise to improve controllability. This perspective connects naturally to the emerging notion of token ordering-the sequence in which tokens are (un)masked and revised-studice in discrete diffusion [18] 34]; our approach realizes a soft ordering via timestep scheduling, making it applicable beyond discrete unmasking schemes.
Controllable Text Generation As language models scale, effective mechanisms for controlling their outputs become essential. Training- based approaches [17] [6 [6) demand extensive com- putational resources and large-scale annotated datasets, constraining their scalability. In contrast, inference-stagee methods offer more flexible solution by enforcing constraints through contrastive techniques [19 25), prompt tuning (20, [5), or modulation via latent space [0). However, these methods offer only indirect control and remain constrained by the model's 's inherent generative capa- bilities. Plug-and sig-andd-lay methods 2 33), which employ external classifiers to steer generation, are closely related to our work. Despite their effectiveness, these methods struggle in regreseeee models due to sequential dependencies that hinder the modification of previously generated tokens.
6 Conclusion
In this work, identify updatt--orgelling central limitation in controllable text generation with diffusion language models, wherein uniform token updates overwrite previously guided modifications and disrupt the generation trajectory, compromising both fluency and controllability. To address this, we propose ITA-DIFFUSION, an inference-time. framework that implements soft token ordering by dynamically allocating denoising timesteps across tokens according to their semantic importance. By prioritizing critical tokens and structuring the update schedule accordingly, "TA--IFFUSION improves generation stability and enhances controllability. Our findings demonstrate that token-wise timestep allocation provides a principled mechanism for structured generation. UTA-DIFFUSION not only improves quality and efficiency through progressive reduction but also introduces a flexible, inference-ting approach to operationalize token ordering without additional training. While this work adopts classifier-informes allocation, future research may explore alternative heuristics, semantic- aware scheduling, extensions to multi-attribute control. More broadly, we view TTA-DIFFUSION as a step toward general-purpone frameworks that integrate soft ordering strategies into diffusion models, opening promising directions for efficient, fine- fine-grained, and controllable text generation.
10

---

Acknowledgements
This work was supported in part by National Research Foundation of Korea (NRF) grant (RS-2025- 00560762, 005-202-00044999 Evaluation (ITTP) grant and Institute of Information & communications Technology Planning RS-2021-1121 1333333 J. Do is with ASRI, Seoul National University.
RS-2025-25442338, 2---2222--R--222-0037772
References
[1] J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. V. d. Berg. Structured Denoising Diffusion Models in Discrete State-Spaces. In Proc. Neural Information Processing Systems (NeurIPS), volume 34, 2021.
[2] S. Dathathri, A. Madotto, J. Lan, J. Hung, E. Frank, P. Molino, J. Yosinski, and R. Liu. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. In Proc. of Int'l Conf. on Learning Representations (ICLR), 2019.
[3] J. Deschenaux and C. Gulcehre. Beyond autoregression: Fast LLMs via self-distillation through time. In Proc. of Int'l Conf. on Learning Representations (ICLR), 2025.
[4] Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2019.
[5] Z. Feng, H. Zhou, K. Mao, and Z. Zhu. FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation. In Proc. of Annual Meeting of the Association for Computational Linguisties (ACL), 2024.
[6] S. Gehman, S. Gururangan, M. Sap, Choi, and N. A. Smith. RealToxicityPrompts Eval- uating Neural Toxic Degeneration in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020, 2020.
[7] A. Gokaslan and V. Cohen. Openwebtext corpus. 2019.
[8] S. Gong, M. Li. J. Feng, Z. Wu, and L. Kong. DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models. In Proc. of Int'l Conf. on Learning Representations (ICLR), 2022.
[9] I. Gulrajani and T. Hashimoto. Likelihood- -ased Diffusion Language Models. In Proc. of Neural Information Processing Systems (NeurIPS), volume 36, 2023.
[10] C. Han, Xu, M. Li, Y. Fung, C. Sun, N. Jiang, T. Abdelzaher, and Ji. Word Embeddings Are Steers for Language Models. In Proc. of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.
[11] X. Han, Kumar, and Y. Tsvetkov. SSD-LM: Semi-autoregressive Simplex-base -baee Diffusion Language Model for Text Generation and Modular Control. In Proc. of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
[12] E. Haxholli, Y. Gurbuz, O. Can, and F. Waxman. Efficient perplexity bound and ratio match- ing in discrete diffusion language models. In Proc. of Int'l Conf. on Learning Representations (ICLR). 2025.
[13] Z. He, T. Sun, Q. Tang, K. Wang, X. Huang, and X. Qiu. DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models. In Proc. of Anmual Meeting of the Association for Computational Linguisties (ACI.), 2023.
[14] Ho, A. Jain, and P. Abbeel. Denoising Diffusion Probabilistic Models. In Proc. of Neural Information Processing Systems (NeurIPS), volume 33, 2020.
[15] A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi. The curious case of neural text degeneration. In Proc. of Int'l Conf. on Learning Representations (ICLR) 2020.
[16] 7. Hu, C. Liu, Y. Feng, A. T. Luu, and B. Hooi. Poetrydiffusion: towards joint semantic and metrical manipulation in poetry generation. 2024.

---

[17] N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher. CTRL: A Conditional Transformer Language Model for Controllable Generation. arXiv preprint N.5.v.999.0555. 2019.
[18] Kim, K. Shah, V. Kontonis, Kakade, and S. Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. Proc. of Int'l Conf. on Machine Learning (ICMI.), 2025.
[19] B. Krause, A. D. Gotmare, B. McCann, N. S. Keskar, S. Joty, R. Socher, and N. F. Rajani. GeDi: Generative Discriminator Guided Sequence Generation, In Findings of the Association for Computational Linguistics: EMNLP, 2021.
[20] B. Lester, R. AI-Rfou, and N. Constant. The Power of Scale for Parameter-Ffffcien Prompt Tun- ing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2021.
[21] J. Li, M. Galley, C. Brookett, J. Gao, and B. Dolan. A Diversit Objective Function for Neural Conversation Models. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2016.
[22] X. L. Li, Thickstun, I. Gulrajani, Liang, and Hashimoto. Diffusion-LM Improves Controllable Text Generation. In Proc. of Neural Information Processing Systems (NeurIPS), volume 35, 2022.
[23] Y. Li, K. Zhou, W. X. Zhao, and J.RR. Wen. Diffusion models for non-autoregressive text generation: a survey. In Proceedings of the International Joint Conference on Artificial Intelligence, 2023.
[24] Z. Lin, Gong, Y. Shen, T. Wu, Z. Fan, C. Lin, N. Duan, and W. Chen. Text generation with diffusion language models: pre-training approach with continuous paragraph denoise. In Proc. of Int'l Conf. on Learning Representations (ICLR). 2023.
[25] A. Liu, M. Sap. X. Lu, S. Swayamdipta, C. Bhagavatula, N. A. Smith, and Y. Choi. DExperts: Decoding-Tinee Time Controlled Text Generation with Experts and Anti-Experts. In Proc. of Annual Meeting of the Association for Computational Linguistics (ACL), 2021.
[26] Y. Liu, M. Ott, N. Goyal, Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv. 09..//9779.999 2019.
[27] Y. Liu, S. Feng, X. Han, V. Balachandran, C. Y. Park, S. Kumar, and Y. Tsvetkov. P'Sum: Preserving author's 's perspective in news summarization with diffusion language models. In Pro- ceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2024.
[28] A. Lou, C. Meng, and S. Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In Proc. of Int'l Conf. on Machine Learning (ICML), 2024.
[29] J. Lovelace, V. Kishore, C. Wan, E. S. Shekhtman, and K. Q Weinberger. Latent Diffusion for Language Generation. Proc. of Neural Information Processing Systems (NeurIPS), volume 36, 2023.
[30] Lovelace, Kishore, Chen, and K. Weinberger. Diffusion guided language modeling. 2024.
[31] A. L.. Maas, R. E. Daly, T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning Word Vectors for Sentiment Analysis. In Proc. of Annual Meeting of the Association for Computational Linguistics (ACL), 2011.
[32] R. Mahabadi, H. Ivison, J. Tae, J. Henderson, I. Beltagy, M. Peters, and A. Cohan. TESS: Text-to-Tex celf-Conditioned Simplex Diffusion. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024.
12

---

[33] F. Mireshghallah, K. Goyal, and T. Berg- Krng-Kirkpatrick. Mix and Match: Learning-free free Con- trollable Text Generation using Energy Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.
[34] S. Nie, F. Zhu, Z. You, X. Zhang, J. Ou, J. Hu, J. Zhou, Y. Lin, J.-R. Wen, and C. Li. Large language diffusion models. arXiv preprint arXiv:2502. 09999, 2025.
[35] Novikova, O. Dušek, and V. Rieser. The E2E Dataset: New Challenges For End-to- Generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, 2017.
[36] I.. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welin- der, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. In Proc. of Neural Information Processing Systems (NeurIPS), volume 35, 2022.
[37] J. Qian, L.. Dong, Y. Shen, F. Wei, and W. Chen. Controllable Natural Language Generation with Contrastive Prefixes. In Findings of the Association for Computational Linguisties, 2022.
[38] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language Models are Unsupervised Multitask Learners. 2019.
[39] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 2020.
[40] S. S. Sahoo, M. Arriola, A. Gokaslan, E. M. Marroquin, A. M. Rush, Y. Schiff, J. T. Chiu, and V. Kuleshov. Simple and effective masked diffusion language models. In Proc. of Neural Information Processing Systems (NeurIPS), 2024.
[41] S. S. Sahoo, J. Deschenaux, A. Gokaslan, G. Wang, J. T. Chiu, and V. Kuleshov. The diffusion duality. In Proc. of Ini'l Conf. on Machine Leurning (ICML), 2025.
[42] T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models. In Proc. of Int't I Conf. on Learning Representations (ICLR), 2022.
[43] Y. Schiff, S. S. Sahoo, H. Phung. G. Wang. S. Boshar, H. Dalla-torre, B. P. de Almeida, A. M. Rush, T. PIERROT, and V. Kuleshov. Simple guidance mechanisms for discrete diffusion models. Proc. of Int'l Conf. on Learning Representations (ICLR), 2025.
[44] K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint wrXiv.1312.6034. 2014.
[45] Song, C. Meng, and S. Ermon. Denoising Diffusson Implicit Models. In Proc. of Int'l Conf. on Learning Representations (ICLR), 2020.
[46] M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In Proc. of Int'l Conf. on Machine Learning (ICMI), 2017.
[47] J. Tae, H. Ivison, S. Kumar, and A. Cohan. Tess 2: A large-scale generalist diffusion language model. arXiv preprint 2502. 2025.
[48] E. Wallace, J. Tuyls, J. Wang, S. Subramanian, M. Gardner, and S. Singh. AllenNLP interpret:
A framework for explaining predictions of NLP models. In Proceedings of the Conference Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, 2019.
[49] T. Wu, Z. Fan, X. Liu, H.-T. Zheng, Y. Gong, Y. Shen, J. Jiao, J. Li, Z. Wei, J. Guo, N. Duan, and W. Chen. AR-Diffusion: Aego-Regressive Diffusion Model for Text Generation. In Proc. of Neural Information Processing Systems (NeurIPS), volume 36, 2023.
[50] K. Yang, D. Liu, W. Lei, B. Yang. M. Xue, B. Chen, and J. Xie. Tailor: A Peitt-rromp-ssse Approach to Attribute-Base Controlled Text Generation. Proc. of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
13

---

[51] H. Yuan, Z. Yuan, C. Tan, F. Huang, and S. Huang. Text Diffusion Model with Encoder- Transformers for Sequence-to-Sequees Generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics, 2024.
[52] X. Zhang, J. Zhao, and Y. LeCun. Character-level convolutional networks for text classification. In Proc. of Neural Information Processing Systems (NeurIPS), volume 29, 2015.
[53] Y. Zhang, J. Gu, Z. Wu, S. Zhai, J. Susskind, and N. Jaitly. Planner: Generating diversified paragraph via latent language diffusion model. In Proc. of Neural Information Processing Systems (NeurIPS), volume 37, 2023.
[54] L. Zheng, J. Yuan, L. Yu, and L. Kong, A Reparameterized Discrete Diffusion Model for Text Generation. arXiv preprint ar arXiv.330..05737 2024.
[55] Zhong, Q Wang, J. Han, Y. Zhang, and Mao. Air-Decoding: Attribute Distribution Recon- struction for Decoding- Time Controllable Text Generation. In Proceedings of the Conference on Empirical Methods Natural Language Processing, 2023.
[56] W. Zhou, E. Jiang, E. Wilcox, R. Cotterell, and M. Sachan. Controlled Text Generation with Natural Language Instructions. 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have included paper's contributions and scope in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have included the limitations the conclusion part.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

We provide the theoretical proofs on appendix and referenced it in correspond- ing part of the writing.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided the experimental details appendix and plan to release the code for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
We plan to publicly release our code either during the review period or shortly thereafter.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
oolicy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccpublle pwwppppguuuiiieeeep hetpc///uuudess/eeeuubuiiiis
(CodeSubiisss wwtplubmmmisioiio ssiiiiolccyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide experimental details in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For all the main generation tasks, we have experimented at least 1500 samples.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the resources information in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have adhered to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential societal impacts in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We provide the usage guidelines in Appendix F.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We have followed the license and usage guidelines provided by creators the assets.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the license, guidelines and training details in Appendix B-F.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the datasettcooeemmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy for what should or should not be described.
21

---

Supplementary Material
Contents
Diffusion Formulation
23
Diffusion Model Formulation
23
A.2
Simplex Diffusion Language Model Formulation
23
A.3
Connection of Simplex Diffusion to Uniform- State Discrete Diffusion
24
A.4
Relation Between Diffusion Fluctuation and Perplexity
25
A.5
Theoretical rationale for token-timesten timestep allocation
26
A.6
Relation of Denoising Ratio and Diffusion Timestep
27
Ex
Experimental Details
28
B.
Experiment Configuration for Fluctuation and
28
B.2
ITA-DIFFUSION Configuration
28
B.3
Baseline Implementation & Generation
29
4
Dataset Details
30
About Progressive Step Reduction
30
Hyperparameter Sensitivity
31
D.I Effect of Control Hyperparameter X
31
D.2 Smoothing Factor
31
Detailed Results
32
Sentiment Control
32
Topic Control
32
Generated Samples
33
Ethics Statement
33
Licenses
34
22

---

A Diffusion Formulation
A.1 Diffusion Model Formulation
Diffusion probabilistic models [14] are generative models that learn a probabilistic distribution by iteratively denoising latent variable, following Markov chain. The diffusion forward pass progressively corrupts the original data representation into pure noise, and a learned reverse process (backward pass) recovers the original representation by iterative denoising.
Diffusion Forward Pass data point žo into sequence of latent variables { by iteratively injecting Gaussian noise. This process is defined as:
The forward process follows a Markov process that transforms a given
g( (,-1) ( Vor (1-a))),
where ae is the noise schedule that controls the scale of noise. For sufficiently large T and appropriate noise schedule, the final distribution of ir converges to pure Gaussian noise N I).
Diffusion Backward Pass To recover the original data from pure noise, approximate the reverse Markov process, which progressively removes injected noise at each timestep. ultimately reconstructing Zo. Given the sampling from prior äT NO, 1), the reverse process is modeled as:
Po ( k,) R (i po( t), Ee ())
where μe (ă,, t) and [e t) are parameterized by a neural network trained to predict the denoised representation timestep f
To incorporate classifter control within the generation procedure, the diffusion backward process can be modified using an external classifier p(y (..).
Guided Update Procedure
Controlling ,o. corresponds to sampling from the posterior:
p(āo Sr v) II p(T.-1 (r,y).
By applying Bayes' rule, the conditional transition each step is expressed as:
p 32, v) p(8.-1 [b)) p(y (2-1,
Following prior work controlled diffusion models [22] simplify the dependency structure
using a conditional independence assumption.
p(y 21 T2) = p(y) ã;-1).
Under this assumption, the gradient- based update at each step of the reverse process is given by:
V logp(ã.-1 Tt, y)
Va. pogp(a.11 ie) + V p(y Æz-1).
where 10,-( iz) is modeled by the diffusion process and log p(y ăz-1) is modeled by an
external classifier.
A.2 Simplex Diffusion Language Model Formulation
This section provides a detailed formulation of the simplex diffusion process applied to diffusion language modeling.
Simplex Mapping Since text is inherently discrete, we map tokens into a continuous simplex space following [11. Let V be the vocabulary set, and x y be token. The token-to-simplex transformation is defined as:
z (= [ if in)ex(
[- 1-
otherwise
where predefined simplex constant. This mapping produces logit representation X, over the vocabulary, forming the input for the diffusion process.
23

---

Forward Diffusion Process now becomes:
The forward process for standard diffusion process described above
(X, X,-1) (X,s Va, - aa))), The latent state at any step t can be expressed directly as:
X = vā.Xo + v1 a.Z, ~~
~~~ NO, 1
Backward Diffusion Process To reconstruct the original representation, approximate the reverse diffusion process as:
Pa( X,) R 15 Eo(
where μe (Xr,t) representation.
() and Eo(X,, 1) predicted by neural network trained to estimate the denoised
Sampling and Decoding Sampling begins from a Gaussian prior Xy ~ No,,, K2J) and iteratively refines through reverse diffusion:
X,-1 va- v1 11-a-12,
where Z N(0, K21). At each step, logits are predicted from the current noisy simplex representa- tion:
X l (Xx,, t).
The corresponding token distribution is obtained via softmax:
P(X,) = softmax Xtagis.t)-
To project back into the vocabulary space, employ top-p sampling 15 in M. ensuring
diversity while preserving fluency. The selected token is mapped back to the simplex representation as:
[K, if = top-p(((()))
-K, otherwise.
11 noted that this transformation maintains the sampled representation near the original simplex structure while ensuring the model retains its non-autoregressive properties
A.3
Connection of Simplex Diffusion to Uniform-State Discrete Diffusion
Let V = = and let E {0, 1) be the one-hot vector for token y. Our simplex embedding uses the constant K > 0 and maps y
(yy)
K (2ey 1) ER,
so the correct coordinate is +K and all others are -K. The forward (continuous) corruption used in A is
X, Vas (5)) + 1-52 Z~
Define the discrete projection arg max fei....,
Normalization to the Gaussian dual form. There exist scalars at > 0 and by E R such that the affine reparameterization
Wi a, Xt b, 1
obeys
Wt - N arey (1- ā,) Iv
(2)
with the (Gaussian) correlation parameter
4
Vo
at
b
(3)
1+ ++3a;
K KVT+30
VT+30
Moreover, arg max(w)) arg max (x,x because arg max is invariant to adding b, 1 and to positive rescaling de.

---

Discrete marginals via Diffusion Duality. Applying the Diffusion Duality result of Sahoo et al.
41 to 22, the pushforward of wi (equivalently. of X2) under arg max yields the marginals of Uniform State Discrete Diffusion Model (USDM):
P22 = 1 20 = Ccy = Cat at ey + (1 (1-ap disc + 41).
at T(ā:),
(4)
where 7: [0. 1| - [0 1] is the diffusion transformation operator of Sahoo al. 41 (their Eq. (10)),
defined for a -class simplex. Combining (3 and gives the induced discrete schedule directly from the simplex process:
disc at
T
(5)
1+
3
Remarks, The map āe + or in (3 is monotone and independent of the embedding scale K due to the matched noise variance Z~~N (0, K0)
I..
A.4 Relation Between Diffusion Fluctuation and Perplexity
High empirical diffusion fluctuation, when measured as deviation from an optimal baseline, is theoretically grounded indicator high upper bound model perplexity. This follows because excess fluctuation tower-bounds the per-step denoising error that appears in the perplexity certificate.
Proposition.
We adopt the discrete diffusion setting of Haxholli et al. [12.. By definition, PPL = exp (H pe)), and Theorem 4 of Haxholli et al. [12] provides upper bound the cross-entropy of the form
Formalism.
H(Pp. Po) I UB(H) = C++++ + L. E'Eon.. ['Ean.n.PPia(a xx) pl mx))] dt,
(6)
where q( 24) is the Bayes-optimal reverse kernel and po(- xx)) is the learned reverse kernel.
Let T. be the sequence length, d the Hamming distance, and define the (sampled) fluctuation rates at time by
R(2) (zt) Ex Ex-ro (-(aa)
del dx((X, X. 22)
1
R! (xt) Eyng(tl (x2)
(8)
L
The excess fluctuation is AR((x.) = (x R! (Iz).
Proof of relation.
Step 1: Excess fluctuation total variation. For the bounded function f(=) = dn E [0, 1].
(x.) = EIIl < drv(g(: ax), pol: xz)).
(9)
which is the standard variational bound for TV when f E [0, 1].
Step 2: TV KL (Pinsker). Pinsker's inequality yields
drv(g(: 1x), Pe(- (x)) s / DKu(a(: x1) pol- xx)).
(10)
Combining and (10) gives the pointwise lower bound
DRu(g(= (x)) pa (2)))))))))))))) 2 (AAR()))))
(11)
Step Impact on the cross-entropy upper bound. Taking expectations over (æo, (x) (the forward process) and integrating over t in (II).
S0 Ero.z. [DLL(( I (x) p xx))] dt 2 2 2 Bro.z. [(A dt.
(12)
25

---

Substituting 12 into 66 yields
UB(H) C' + 222
2 Exow
(A))))))))) dt.
(13)
Therefore, the large fluctuation forces the certified upper bound to be large. Equivalently,
UB(PPL) = exp exp(UB((1) 2 exp ( + Exo.ze (AR:( dt
Conclusion. The excess fluctuation A RR, principled indicator of generation quality: large deviations from the Bayes-optimal flip rate imply large per-step denoising KI., which enlarges the certified upper bound on cross-entropy and thus on perplexity. Empirically observed instability therefore direct, theoretically grounded symptom of underlying denoising error.
A.5 Theoretical rationale for token-timestep allocation
Setup and notation. Let (ot (x',...,=+) be sequence of length N and [0, T) the global
reverse time. A schedule a mapping [1. N) x [0, + [0. 7) with local times 4 = [ 1). We assume is monotone in time: 0t:/0t [Pmin: Prnax] with 0 Pmin S Pmax 00. Let
(,(t) E (0, 1] be the signal coefficient, and define the per-token forward variance (t) = (t).
At reverse time t, we impose budget constrains Σ;= (t C(t) with box constraints oiin (t) s (1) < a (t) induced by the integrator and training schedule. Classifier guidance augments the score by XV = log p(y it); write gi(t) == V| a (Zx) and let (t) E [0, be a normalized importance weight.
(A1) (Score-error bound form) The cross-entropy upper bound can be written (or upper bounded)
Assumptions.
J2 S / Σ (oi(()) dt,
where each di : [0, 1] R2 is nondecreasing and conver. (A2) (Local sensitivity) For small perturbations of the noise around baseline, Φ a, + b of with b, w; and weights w; := (t) nondecreasing in (t). (A3) (Margin smoothness) The classifier margin m(ï) is L-smooth: /Vm(u) Vm(v)ll S Ll[u v/l. Under one-step Buler-Maruyama update with independent Gaussian perturbations s¹ ~ af I). the expected one-step margin drop satisfies
E A|Am]
n of with T co + Ci
for constants co, Cj 2 0 determined by the diffusion and guidance drifts.
Perplexity bound improvement. By (A1) and (A2), at each t minimizing the instantaneous contribution to Jz under the budget and box constraints reduces to
N of = C(I), omi 5 o m
min
b, o
S.t.
Because the objective is linear and the feasible set a product of intervals intersected with simplex, the optimum assigns smaller of larger b; until box constraints are met. When C(t) lies strictly inside the box (no saturation), the interior KKT solution affine the weights:
(1(
( [ ( (c(t) x(t) b,(t))
for Lagrange multipliers e(t) 0, where II denotes clipping the box. Since b, is nondecreas- ing in gt. practical normalized rule
a3(1) x ((()
a realizable instance of the KKT solution up to scaling and clipping. Integrating over & yields Jz( J2(Suuuttive) < J2( (, with strict inequality whenever the by are not all equal and no box constraint forces equality. By Haxholli al. [2]. the adaptive schedule yields tighter upper bound, UBpPL (fadapive) < UBPPL (f constant) whenever the above conditions hold.
26

---

Impact on control accuracy.
By (A3), the expected one-step margin drop obeys
E[Am]
T of with 7 non non-decreasing in Ji-
Minimizing this upper bound under the same budget and box constraints the same program (*) with b; <- 1/i, hence has the same KKT form and the same monotone allocation: assign less noise
larger Pt. Therefore the adaptive rule of (1 9s) minimizes the margin-drop bound among
feasible schedules, yielding
ACC (adapiiee z
ACC([conntaat)
with strict improvement under the same "no-saturation and non-identical weights" condition.
Conclusion. Under (A1) and a fixed per-time noise budget with box constraints, the allocation of (1-9:) is the affine KKT solution of the per-time convex programs that minimize the cross-entropy upper bound integrand and (ii) minimize the one-step margin-drop bound. Hence it is Pareto improvement for both perplexity (via the bound) and control accuracy. Strict improvements obtain whenever token importances are not identical and box constraints do not force equality.
A.6 Relation of Denoising Ratio and Diffusion Timestep
We analyze the relationship between the magnitude of denoising updates and the diffusion timestep. Specifically, we show that the noise injected at timestep + 1 greater than the noise injected at timestep 1, implying that later timesteps require proportionally larger denoising updates.
We examine the forward diffusion process given by
St Vaa - a4-4 4 ~0,,T,,
where {ar} C (0, 1] strictly decreasing sequence I. We show below that the variance of the newly injected noise at timestep (t + 1) strictly larger than that timestep t. First, observe that the newly injected noise step & can be written
Org Exs Noise; V1
where E, (O( I). Its variance then
Var Noise:]
(1 ar) I.
Next, because } is strictly decreasing,
0++1
az
1- --0+11 > 1- ap.
Thus, for the noise term at step (1 1).
Var /I a+41 cet1) (1 04+1)
> (1 av.) I,
which is exactly Var [Nose,]] Consequently, the noise injected at each successive timestep (t+ 1) is strictly larger in variance than the noise injected at timestep t. In summary,
Var ( (0+1 > Var (V1
04
Since the forward diffusion process injects increasing noise at later timesteps, the corresponding re- verse (denoising) steps must compensate by removing larger noise magnitudes. Hence, the magnitude of the denoising update greater for higher timesteps in the reverse diffusion chain.
27

---

B Experimental Details
B.1 Experiment Configuration for Fluctuation and Update-Forgettng
We conduct our analysis using
diffusion language model [32 trained with subset of C4, equivalent to a fully non-autoregressive extension of SSD-LM. For controllability, we apply classifier-guided sentiment control using a classifier trained on the IMDB Movie Reviews dataset, with a guidance strength of À = 2000. Each sample is generated using T 1000 diffusion steps under fixed decoding parameters.
Bluctuation-Perpless Analysis. To quantify how token-level fluctuations affect fluency, compute the mean fluctuation ratio RG from steps ' down to each intermediate timestep / (i.e., over [ ,,).. For each timestep I, correlate RG with the final sequence perplexity across 150 samples. We repeat this procedure using BLEU and BERTScore (F1) computed between each step's output and the previous step's sequence. This allows us to track how instability during the denoising process relates to semantic drift and final output quality. To evaluate lcxical subtlety, we also detect synonym replacements using WordNet. Across all timesteps. only 0.117 tokens per 64-token sequence (on average) are replaced by synonyms, suggesting that most fluctuations involve non-trivial changes.
Podate-Forgettingg Analysis. For each sequence at timestep we compute classifier gradients and select the top-5 tokens with highest gradient norm as key tokens KG, We record classifier confidence before and after single diffusion step and compare it for cases where at least of the 5 key tokens are modified. The "before" state refers to the output immediately after classifier guidance, while the "after" state the output following noise injection and denoising. The classifier's s confidence is measured as the probability assigned to the target label.
Correlation Computation. We compute Pearson correlation coefficients between fluctuation ratio and semantic similarity metrics (BLEU, BERTScore), as well as between those metries and final perplexity. All correlations are computed across (sample, timestep) pairs from 150 generations. In addition, we compute correlation trends within timestep bins isolate convergence effects later steps. Even in the final 100 steps, we observe non non-trivial correlations (e.g., BLEU- -a-UUueeplx:tty -0.23), indicating that sample-level fluctuation trajectories meaningfully affect final outcomes.
Synonym Analysis by Timestep. To rule out the possibility that observed fluctuations are predom- inantly benign, we compute the proportion of changed tokens that qualify as WordNet synonyms within 100-step bins. The proportion remains consistently low (1-5%) even in later timesteps, re- inforcing the interpretation that update-forgen forgetting and fluctuation often correspond to semantically meaningful deviations rather than minor paraphrasing.
B.2 TTA-DIFFUSION Configuration
RoBERTa-large RoBERTa-large 266 serves the base model for sentiment/topic-conroee text generation, and detoxification tasks. For training, we extract a 1M subset from the C4 [39]. The model was fine-tuned for 300K steps using a batch size of 64 and learning rate of 10 During training, we randomly select between 2 and 10 tokens as prefix, with the model learning to generate the remaining tokens. The training objective follows the standard cross-entropy loss, as in [1 32). A cosine noise schedule [45] with a simplex parameter of k 5 was employed. The training process used 5,000 diffusion timesteps, For progressive step reduction, we begin with a base model trained using 5000 diffusion steps and progressively reduce the number of steps to 1000, 200, and 50. The base model is trained for maximum of 300K timesteps, while each fine-tuned model trained until convergence (300K, 100K, 40K). Training was performed on single NVIDIA H100 GPU, requiring approximately 50 hours to complete.
BERT-base BERT-base is utilized to maintain consistency in model size with 22 which employs BERT-base 4 as the base architecture. The model is trained on the E2E dataset [35 using a batch size of and a learning rate of x 10-4 The maximum sequence length set to 64, with 2,000 diffusion steps during training and 200 steps during inference. A cosine noise schedule is
28

---

applied, with simplex parameter of l = 5. The training process was conducted on single NVIDIA A100 GPU, requiring approximately 10 hours complete.
Generation Configuration For the detoxification task, we set À to 2000 and use the control classifier s-hlpp/robertattoxeeeee toxicity_classifier For sentiment control, we set to 2000 and employ For topic control, we set À to 20,000 and train classifier on the AGNews dataset, independent of the evaluation classifier. For the syntax tree and syntax span tasks, we set À to 2 and 20,000, respectively, using a control classifier trained on the E2E dataset. For length control, no classifier is required. Instead, constraint enforcement achieved by fixing the <cos> token at the desired position. We set À to 20,000 for this task. Also, We adopt DDPM [14] over DDIM [45], as the latter resulted in reduced diversity.
For the text paraphrasing task, we use the QQP dataset curated by [51] and fine-tune our progressively step-reduced model on training set, following the setup of [2] with 50K training steps and a batch size of 6. Classifier-based control is applied using the publicly available paraphrase classifier with a control strength (A = 2000). For the long-form generation task, we retrain each step-reduced model with maximum generation length of 512 tokens. We adopt the same prompts used in the sentiment classification task from [2), generating 50 outputs per prompt. For classifier guidance, we employ fluency classifier, but apply control only during the first half of the denoising process to preserve generation diversity in later steps.
Evaluation Configuration For detoxification, we report the average toxicity, maximum toxicity (averaged per prompt,, and toxicity probability (the proportion of sentences with a toxicity score greater than 0.5) using the Perplexity API For sentiment control, we compute the average results from three external classifiers: one from [55], and two from and -haarttannn/snttment---ee provided in [55]. Lastly, for lexical constraints, we follow the evaluation setup used in Diffusion-L.M
wooberaa-lagg--eeee large-english For topic control, we utilize the classifier
large-english-
(2
B.3 Baseline Implementation & Generation
The toxicity and sentiment classifiers are trained using the same datasets and hyperparame- ters specified in the original PPLM paper. However, the classifier sizes are scaled following (25), as this configuration has been shown to enhance performance. The generation process is conducted using the same settings described in the original work.
PPLM
GeDi For all tasks, we utilize the class-conditional language models provided by the original authors and employ GPT-2 Medium (38] as the generation model. The generation process follows the original setup, using a discrimination weight of 30 and a filter/target probability of 0.8.
DExperts Since the expert and anti-expert models provided by the original authors are based on GPT-2 Large, we retrain both models using GPT-2 Medium while adhering to the original configuration described in the DExperts paper. For the topic-guided text generation task, we use the AGNews dataset [52) and maintain the same hyperparameter settings, training only the expert model with the same architecture. Generation is performed using an alpha value of 2.0 for detoxification. 3.2 for sentiment control, and 2.0 for expert-only topic generation.
Air-Decoding We use the models released by the original authors and adhere the recommended generation settings for each task. Control strength parameters are set to 120 for detoxification, 140 for sentiment control, and 60 for topic control.
LM-Steer
The sentiment and detoxification models are trained following the original implemen- tation provided by the authors. For topic modeling, we train four separate models on the AGNews dataset, adopting the same training configuration as the sentiment model. Generation is conducted using a steering value of for all tasks, consistent with the original implementation.
Diffusion-LM To ensure consistency with other baseline models, we initialize the model using BERT-large [4. The training process utilizes the same subset of the C4 dataset as our model, with a learning rate of 1 10-5, 2,000 diffusion steps, a batch size of 64, and an embedding dimension of
29

---

128. The model is trained for I million steps until convergence. Due to instability during training, we select the checkpoint that produces coherent outputs as the final model.
Given the substantial computational resources required for training, we use the pretrained model provided by the original authors, which was trained on the Wext dataset [ For controlled generation, we employ the same guidance classifiers used in our model, with control hyperparameter 2000. Other hyperparameters, including a maximum timestep of 1000 and a block size of 25, remain consistent with the original paper.
SSD-LM
LD4LG Initially, we attempted to train the model using BART-lrrge 4;
however, due to instability, instead utilized the pretrained BART- model while maintaining the hyperparameters specified by the original authors. The model is trained in a class- conditional format using the AGNews dataset [52] for topic control, the Jigsaw Unintended Bias in Toxicity classification dataset (100K subset) for detoxification, and the IMDB Movie Reviews dataset [31] for sentiment control. While LD4LG allows the use of either control codes prompts, we solely rely control codes for generation. Sampling is conducted using 250 timesteps with DDPM sampling.
B.4 Dataset Details
Details of the C4 subset used for our model and Diffusion-LM are shown in Table Dataset details of AGNews [52], IMDB Movie Reviews [3 and Jigsaw Unintended Bias in Toxicity for training baseline models are illustrated in Table 6 Table and Table g
Table Dataset configuration for C4 subset.
Train
Valid
Num samples 9,500,000 500,000
Table 6: Dataset configuration for AGNews.
World
Sports
Business
Sci-tech
Num samples 15,000 15,000 15,000
15,000
Table 7: Dataset configuration for IMDB Movie reviews dataset.
Positive Negative
Num samples
25,000
25,000
Table 8: Dataset configuration for Jigsaw Unintended Bias in Toxicity.
Toxic
Non-Toxic
Num samples 50,000
50,000
C About Progressive Step Reduction
Result of Progressive Step Reduction preure the results of progressive step reduction on the sentiment control task. As shown in the figure, while all models converge similar performance under large inference budgets (e.g.. T = 1000). significant differences emerge under low timesteps. Notably, step-reduced models exhibit stronger stability at small inference timesteps. For perplexity, although the original T 5000 model achieves competitive results only at high inference timesteps (around T 1000), the progressively fine-tuned models attain comparable perplexity with as few as T = 100 inference steps, highlighting the efficiency gained through step reduction.
Speedup Comparison Table summarizes relative decoding speed ratios (GPT-2 base= 1.0); for diffusion-based methods, T is the number of inference steps.
30

---

Table 9: Relative decoding speed ratios measured in setup (GPT-2 base 1.0). For diffusion-style methods, Timestep denotes the number of inference steps T.
Method
Timestep
Speed ratio (x)
GPT-2 (base)
1.0
DExperts
2,6
PPLM
270.1
LM-steer
1.2
Diffusion-LM
200
14.7
Diffusion-LMM
1000
72.7
SSD-LM
1000
109.8
LD4LG
250
2.2
DGLM
50
4.4
TTA (w/o control)
1.4
TTA (w/o control)
1000
27.4
TTA (with control)
50
2.0
TTA (with control)
100
4.0
TTA (with control)
200
7.9
TTA (with control)
1000
39.2
D Hyperparameter Sensitivity
D.1 Effect of Control Hyperparameter
Table 10: Effect of control À on sentiment control task. We report the result with TTA (5000) with inference step of 1000. Sentiment accuracy, Perplexity and Dist-3 is reported.
Accuracy (% )
PPL
Dist-3
0
35.8
17.0
0.86
2
42.6
15.8
0.86
20
56.2
16.3
0.85
200
74.1
19.4
0.83
2000
90.1
26.8
0.85
20000
94.2
42.4
0.90
The generation quality of controllable diffusion highly sensitive to the choice of the con- trol hyperparameter X. Throughout experiments, select the optimal À from the range {2, 20, 200, 2000, 20000} based on its effectiveness in achieving a balance between accuracy and fluency.
Table TO presents the impact of different values on the sentiment control task. A higher À generally leads to increased accuracy but at the cost of reduced fluency, reflecting the well-known trade-off between control strength and naturalness. Among the tested values, = 2000 provides the best balance between accuracy and fluency, making it our choice for sentiment control.
The effect of À follows a similar trend across other tasks well. Consequently, we determine the optimal À for each task by selecting the value that best balances these two competing objectives.
D.2 Smoothing Factor
To regulate the balance between the global timestep and the adaptive token timestep, introduce smoothing factor, defined as follows:
final Osmooth + (1 Osmooth)
2 adaptive
In our experiments, we set ammooh 0.6, as it provides an optimal trade-off between accuracy and fluency. To examine the impact this factor, we conduct a sentiment control task, generating 600 samples for each value of Osmooth in the range of 0.0 1.0. Here, asmooth 0.0 corresponds to purely adaptive gradient schedule, while Qsmooth = 1.0 represents a constant schedule.
Figure 8 illustrates the effect of Otsmonth on both accuracy and fluency. As shown in Figure 8(a). accuracy tends to decrease as Ossmooth increases from to i, supporting our hypothesis that adaptive scheduling enhances key token preservation. For fluency, rather than exhibiting monotonous
31

---

T=1000
T=500
T=200
40
88
Perplexity
35
Accuracy
80
0.0
0.2 0.4
0.6
0.8 1.0
0.0
0.2
0.4
0.6 0.8 1.0
Smoothing Factor
Smoothing Factor
(a)
(b)
Figure 8: Effect of smoothing factor on (a) Sentiment accuracy and (b) Perplexity.
decrease in perplexity, U-shaped trend emerges, with the lowest perplexity observed within the range of 0.4 to 0.8, as depicted in Figure 8 b).
E Detailed Results
E.1 Sentiment Control
We present sentiment-contrrolle generation results using the fully progressively reduced models in Table II All results are reported under fixed control strength of À 2000, with adaptive timestep allocation (smoothing factor 0.6) and control iterations. Under the same configuration, Table 13 compares adaptive and constant allocation strategies for TTA(200) and TTA(50), focusing on inference steps fewer than 100.
Table 11: Accuracy (Acc) and Perplexity (PPL) across different inference steps and training time
TTA schedules for sentiment control task. (Iter 2)
T
TTA (5000)
TTA (1000)
TTA (200)
TTA (50)
TTA (20)
(%)
Acc (%)
PPL
Ace (%)
PPL
Acc %)
Acc (%)
PPL
20
59.0
119.9
60.8
109.3
71.6
96.8
69.8
90.4
66.5
106.5
71.4
60.2
76.8
58.6
85.9
40.2
85.6
42.0
46.6
100
79.6
84.7
38.3
90.0
30.1
91.0
25.4
89.1
29.4
200
84.2
29.2
89.5
27.8
92.1
23.2
91.6
19.4
91.2
23.4
500
87.5
23.9
91.9
22.5
92.3
18.7
92.8
16.1
92.5
18.3
1000
87.6
21.5
93.3
21.0
92.6
17.1
93.4
15.7
93.5
17.5
Table 12: Accuracy (Ace) and Perplexity (PPL. across different inference steps and training-time TTA schedules for the sentiment control task. (Iter 3; values rounded to 1 decimal)
T
TTA (1000)
TTA (200)
TTA (50)
TTA (20)
Ace (%)
PPL
Ace (%)
PPL
Acc (%)
PPL
Ace (%)
PPL
20
62.9
137.0
74.7
120.0
71.3
109.7
66.8
107.8
50
79.2
75.3
89.0
51.1
88.7
47.3
85.3
56.7
100
86.4
48.7
93.8
31.9
92.7
28.7
91.4
36.4
200
90.8
32.9
95.1
24.3
94.7
20.5
93.4
23.6
500
93.6
25.0
95.3
19.5
95.6
95.0
18.9
1000
94.6
22.6
95.1
18.3
94.9
16.4
95.5
17.0
E.2 Topic Control
For topic control task, we generate text conditioned four topics: World, Business, Sports, and Sci-Tech. Using 20 prompts from PPLM (2) we generate 20 sequences per topic. Adaptive allocation with smoothing factor of 0.6 with control lambda of 20000 is reported.
32

---

Table 13: Accuracy and perplexity comparisons for different TTA values with and without token timestep allocation.
TTA
200
TTA 50
TTA
acc
ppl
acc
ppl
20
68.9
98.5
69.2
94.0
+ with allocation
71.6
96.8
69.8
90.4
50
84.5
44.0
85.0
43.7
with allocation
85.9
40.2
85.6
42.0
100
89.1
30.2
90.4
27.4
+ with allocation
90.0
30.1
91.0
25.4
Table 14: Results of topic control task. We report the result with TTA (5000) for T=1000 abd TTA (200) for T=200.
Model
Acct PPL. Dist-3f
S-BLI
Auto-regressive Baselines
GeDi
96.4
55.6
0.94
0.11
DExperts
94.5
59.9
0.94
0.13
Air-dccoding
92.9
32.4
0.94
0,21
Mith
63.4
64.6
0.94
0.06
LM-Steer
52.2
44.3
0.88
0.24
Diffusion Baselines
Diffusion-LM LM1-000000
63.1
135.6
0.93
0.10
SSD-LMr=1000
46.1
61.8
0.95
0.11
LD4LG1=250
88.3
129.5
0.92
0.10
Ours
TTA- DIFFUSION
94.6
41.2 1.22
0.89
0.12
TTA-DIFFUSIONN..00
85.8
58.8
0.90
0.10
Result for topic control is provided in Table 14 While topic control at 7 = 1000 achieves strong performance in both fluency and accuracy, reducing the number of timesteps below 200 still yields superior results compared to existing diffusion- based models, yet remains inferior to auto-regressive baselines. We hypothesize that this performance gap may stem from the increased complexity of topic classification, which often involves multiple overlapping labels and nuanced semantics that are more difficult to capture with single-step classifier during the diffusion process.
F Generated Samples
We provide the generation results for our model as in Table [6 and Table per step generation results in Table 13
17
We also provide the
G Ethics Statement
Our model's enhanced control mechanisms could be misused to generate misleading, biased, or harmful content, including fabricated news, impersonation, and propaganda. While it is designed for detoxification, adversarial modifications could enable the evasion of toxicity detection systems. Additionally, our model may unintentionally amplify biases present training data, leading to discriminatory outputs if classifier guidance is miscalibrated. We do not intend for our model to generate biased or toxic content and strongly advocate for its ethical use, restricting its application to responsible and constructive domains.
33

---

H
Licenses
many We plan release our code under the Apache 2.0 license. Our implementation is based on Diffusion- LM and TESS-Diffusion both of which are also distributed under the Apache 2.0 license.
Table 15: We present generated samples at each timestep, underlining the Top-5 tokens with the highest classifier gradient magnitudes.
Sentiment Control (Negative)
Prompt
Timestep
Generated Sample
200
The pizza is is the the the the the the the the the............................
160
The pizza and the pizza are not the hest. have been to.........
120
The pizza at this place is not the best pizza I have ever had and was not the good. The. They have the quality, too. I would not recommend going to this place.
The pizza
80
The pizza this place is not the best quality have ever had and would recommend for it. The food is terrible, too. I would recommend going to this place.
40
The pizza from this place is not the best quality I have ever had and would not recommend paying for it, The food is terrible, too, I would not recommend going to this.
0
The pizza from this place is not the best quality I have ever had and would not recommend paying for it, The food is Iérrible, too, I would not recommend going to this.
Sentiment Control (Positive)
Prompt
Timestep
Generated Sample
200
The lake is the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the
The lake
160
The lake is beautiful and beautiful place to live, and and.mmmee....................... the lake, a...............................
120
the lake is beautiful and beautiful place to live, and it? to to..... from...... lake, and.,, lake, and many,.--.... the property...
80
The lake beautiful and exciting place to live in but it's also great place to relax. There are numerous recreational areas around the lake, the the areas of the lake where you can enjoy boating, kayaking, and many other recreational activities on the lake. Take advantage of these areas and enjoy your
40
The lake not only makes exciting place to live, but it is also a beautiful place to live. There numerous open areas between the north and south west ends other activities offered by the lake. Take ad advantage of these opportunities and enjoy summer
lake where you can enjoy boating, kayaking,
0
The lake not only makes great place to live, but it is also a bcautiful place to explore. There are numerous open areas along the north and south west ends of the lake where you can enjoy boating, kayaking, and many other activities offered by the city. Take advantage of these opportunities and enjoy summer
34

---

Table 16: Generated text samples under detoxification and sentiment control settings.
Detoxification
Prompt
Generated Sample
Not only is he total babe.
but he's also amazing photographer. Check out some of his beautiful work below, and you'll get sneak peek at what he's up to!
In the Year of the Horse, stop being an
introvert and live the life of a horse. Put aside all your fears and develop the confidence to do big things, no matter what you're afraid do.
Three Lions support- ers have reportedly been heard chanting: "F*
during England's friendly game against Slovakia at Rickenham Stadium. This is as international governing bodics express deep regret that England is unlikely to be involved in 2020. Having already failed at the 2016 World Cup.
Mitchell wasted no time in diving in to say what most fans probably were already thinking by beginning, "George, you heartless
man who just stolc my heart." I wholcheartedly agree with Mitchell's point: if George hadn't 't been there, I wouldn't 't have been"
Sentiment Control (Positive)
Prompt
Generated Sample
The city
of San Francisco presented exciting opportunity for me. With its diverse cultures, world-class schools, and excellent public transportation. it's a great place live and work. had the pleasure of working with several companies, but the one in San Francisco was the most rewarding for me.
The chicken
love this dish because it's so easy and delicious! First, cut up all dry ingredients: water, olive oil, salt, and baking soda; put them in a food processor, and you 're done! In another bowl, whisk together onion, garlic, and curry powder.
The book
provides excellent introduction to many aspects of computer science and program- ming, it covers important topics like: Systems and Programming, Graphs/Objects, Networking, ctc..
The year is 1910.
We are going through a very exciting time, with lot of really interesting events. One those really interesting events had to with the 100th anniversary Boxing Day! Wow, that's 's big onc!
Sentiment Control (Negative)
Prompt
Generated Sample
The city
of Radnica is in need of whole bunch of cajax service and auto maintenance, they all established around 2005 and are by local cajax company, their prices are pretty high for this industry, but do not like using this company or their service.
The chicken
isn't good and I'm not happy with my order, I'll have to order again! I like the sauce, but the quality is not that great!
The book
they are advertising just had a big success and then they lost a lot of people, it was so hard. And now they have another one coming out and think that they are having hard time advertising that.
The year is 1910.
We've ve had a little snow, and now we're in the middle of a terrible industrial recession and they' re trying to lock us out. They won't 't give us an opportunity to talk about what's really going on
35

---

Table 17: Generated text samples under topic control settings.
Topic Control (World)
Prompt
Generated Sample
The connection
between India and Afghanistan goes back long way, before any political situation and before any war between India and Pakistan. For that reason, there has always been an agent of India in London, England; and an agent of Afghanistan in Washing- ton, D.C.
summary
the Report provides an independent assessment the current security situation India and its major political parties and its main armed groups, and its consequences for the upcoming national polls in the country.
This essay discusses
major social, political, and economic changes that affected teaching in the Arab World and subsets, offers contemporary views on workers' rights, and also considers alternative views on teachers' unions and professional development, before conclud- ing the chapter,
Topic Control (Sports)
Prompt
Generated Sample
The connection
between John and his team great, it doesn't it matter where we came from and knows what players we had, it doesn't 't matter where we are now and he knows where we are going. The challenge of being a head coach is one he faces and he gives all he has to support his players.
summary
yesterday was a good day for us, as we played the best team-not our highest-ranked opponent-bus but played a fantastic match and advanced to the District Final!
This essay discusses
the positive influence that sports can now have on someone that just got started.
Topic Control (Business)
Prompt
Generated Sample
The connection
between a trader and an investor goes back many years, there are three phases the relationship. Trader adds Stock to his portfolio, an Investor places a bid on a Stock and
summary
this is an introduction to the fundamentals all business activities - including advertising and marketing, the sale and distribution of consumer products and services; building supplies and insurance; financial services; government, internal and foreign exchange; the public and private sectors.
This essay discusses
the relationship between international trade and consumer prices, focusing on the impact of medicines and other imports from different regions and their effect on consumer prices. In international trade, some advanced economies, like France, maintain an Import deficit:
Topic Control (Sci-tech)
Prompt
Generated Sample
The connection
of two Devices via USB is quite simple in itself, and consists of two separate components. one being Power Supply and the other being Lightning Cable. First, the power supply must be positioned between the two USB Devices, and the Cable must connected the USB Device using standard Lightning connector
In summary
this research evaluated various transmission systems for transmitting high-fre frequency radiation, shed some light on the environmental characteristics of these systems, and analyzed them a more detailed way.
This essay discusses
the link between mental and physical illnesses and how it might impact people. The main aim of understanding the link is to understand how little control a person has how long condition lasts, and hetter understand the underlying mechanisms of mental and physical illnesses.
36


Reviews:
[{"review_id": "EuPh8ENCuq", "reviewer": "Reviewer_sFKt", "summary": "This paper proposes TTA-Diffusion, which is an inference time method to adaptively assign a differing number of diffusion time steps to every token in the sequence. The authors argue that this mechanism helps perform better classifier-guided controllable generation. The number of steps for a token is set by using signal from the classifier. The authors also show that their method is effective when diffusion models are progressively distilled to reduce the number of diffusion steps.", "questions": "Here are some questions I have about the paper:\n- Does forgetting, as illustrated in figure 1, empirically occur? Doesn't the classifier guidance applied at every denoising step prevent \"love\" to change to \"hate\" when stearing for positive sentiment?\n- The experimental setting for hypothesis 2 is unclear. Is classifier guidance being applied? If so, why does the probability drop?\n- What is the input to the classifiers? Is it tokens, is it vectors? Appendix B2 mentions off-the-shelf classifiers are used for the different tasks, but these classifier operate on clean text. Are you using these same classifiers on noisy intermediate sequences during diffusion?\n- Section 4.4 mentions for T=1000 the sentiment accuracy improves from 92.2 to 92.6, but these numbers are different from the ones reported in the table in the main paper and the appendix? Which table are these numbers from?\n- Why are the results from TTA (5000) bad? Table 3a shows a very high perplexity for  TTA (5000). Table 9 also shows that TTA (5000) is worse than TTA (1000).\n- In figure 2, how many samples correspond to each mean fluctuation ratio? Is the distribution close to uniform?", "limitations": "Yes, the authors address limitations.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- TTA-Diffusion works with just 50-100 time steps. Compared to baselines, this should make TTA-Diffusion much faster.\n- The analysis done for hypothesis 1 is a convincing motivating story for TTA-Diffusion.\n- TTA-Diffusion is an inference-time method that can potentially be easily applied to many different diffusion methods.\n\nWeaknesses: \n- There are a few missing baselines. DGLM ([1]) is a very related method and there are no comparisons to DGLM. AR-Diffusion [2] proposes using something similar to linear schedule for assigning time steps, but there are also no comparisons to AR-Diffusion. \n- The main contribution of the paper is to propose an adaptive strategy to assign time steps to each token. However, for their detoxification experiments they use a linear allocation schedule as mentioned in caption of table 1. Does this mean the adaptive strategy is only useful sometimes?\n- The writing clarity can be improved. Some details are unclear; for e.g. is the time step allocation done once at the start of denoising or is the allocation changed at every step. Are the time steps for all tokens set according to the equation in 199 or only a subset of tokens?\n\n[1] Lovelace, J., Kishore, V., Chen, Y., & Weinberger, K. (2024, August). Diffusion Guided Language Modeling. In Findings of the Association for Computational Linguistics ACL 2024 (pp. 14936-14952).\n[2] Wu, T., Fan, Z., Liu, X., Zheng, H. T., Gong, Y., Jiao, J., ... & Chen, W. (2023). Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36, 39957-39974.", "was_revised": true, "final_justification": "It is nice to see that the proposed method also woks in the discrete diffusion setting. However, my one remaining concern is that the linear schedule outperforms the adaptive schedule in many of the toxicity experiments. This makes me question the effectiveness of the proposed method.", "rebuttal": "**Dear Reviewer sFKt,**\n\nThank you for your thoughtful and constructive review. We address your comments below:\n\n## **W1. Comparison with More Baselines**\n\nWe appreciate your emphasis on broader baseline comparisons. Since the official implementation of DGLM [1] has not been released, we closely replicated their experimental setup and evaluated our results against the values reported in their paper.\n\nFor sentiment and toxicity control tasks, we adopted the same evaluation datasets as DGLM: 5,000 OpenWebText prompts for sentiment and 5,000 neutral prompts from RealToxicityPrompts for toxicity. We report perplexity (measured by Olmo-1B) alongside control accuracy for a fair comparison. These results suggest that **TTA performs competitively with DGLM**, achieving lower perplexity and strong control performance.\n\n**Toxicity Control Results**\n\n| Method | Perplexity ↓ | Max Toxicity ↓ | Toxicity Prob. ↓ |\n| --- | --- | --- | --- |\n| DGLM (s=5) | 30.7 | 0.182 | 0.025 |\n| DGLM (s=20) | 38.2 | **0.101** | 0.005 |\n| TTA (50) | **29.2** | 0.119 | **0.004** |\n\n**Sentiment Control Results**\n\n| Method | Perplexity ↓ | Accuracy ↑ |\n| --- | --- | --- |\n| DGLM (s=100) | 28.6 | 96.6 |\n| DGLM (s=500) | **26.0** | 96.8 |\n| TTA (50) | 26.7 | **97.9** |\n\nScaling AR-Diffusion [2] to 330M parameters on the C4 dataset was quite unstable. Instead, we adopted the AR-Diffusion objective (Multi-Level Diffusion) and fine-tuned our TTA(50) model accordingly. As shown below, this AR-style objective yields slightly better fluency, but at the cost of noticeably reduced classifier accuracy.\n\n**Sentiment Control Results**\n\n| Method | Perplexity ↓ | Accuracy ↑ |\n| --- | --- | --- |\n| TTA (50): AR-Diffusion | 38.7 | 77.4 |\n| TTA (50): adaptive | 42.0 | 85.6 |\n\n## **W2. Adaptive allocation strategy in detoxification experiment**\n\nWe appreciate your observation. Indeed, the detoxification experiments in Table 1 used a linear allocation schedule intentionally to demonstrate that various allocation strategies can be effectively applied within the same framework without requiring retraining.\n\nAs the table below shows, both adaptive and linear schedules consistently outperform the default strategy across multiple timesteps. We chose to highlight results using the linear schedule to emphasize the flexibility and practicality of our method during inference.\n\n**Toxicity Control Results with different allocation methods**\n\n| **Timestep** | **Token Allocation** | **Avg. Toxicity** ↓ |\n| --- | --- | --- |\n| **50** | Adaptive (0.6) | 12.7 |\n|  | Linear | **12.5** |\n|  | Default | 14.0 |\n| **100** | Adaptive (0.6) | 12.3 |\n|  | Linear | **12.2** |\n|  | Default | 13.8 |\n| **200** | Adaptive (0.6) | **12.2** |\n|  | Linear | 12.2 |\n|  | Default | 13.6 |\n\n## **W3. Experimental details on token update mechanism**\n\nThank you for your suggestion. As mentioned in Line 161, timesteps are assigned to **all** tokens in the sequence, using the formula provided in Line 199. This allocation is dynamically updated at **each** denoising step based on the gradient magnitude with respect to the classifier. We have clarified this further in Section 3.2 of the revised manuscript.\n\n## **Q1. Forgetting phenomena in Figure 1.**\n\nThis is a great point. You're absolutely right that the classifier guidance step would eventually steer the token \"hate\" back toward something more positive. However, what we intended to illustrate in Figure 1 is that the *denoising step* of the diffusion mode (*before* applying classifier guidance) can sometimes undo or weaken the effect of the previous classifier update. This can lead to redundant or oscillating updates across timesteps, potentially causing instability in the generation process.\n\nThe token transitions in Figure 1 (e.g., “clean → dirty”) are drawn from actual empirical traces. While drastic reversals (like “love → hate”) are rare, we frequently observe classifier-steered tokens reverting to a neutral or unintended form. Our proposed method is designed to mitigate this instability.\n\n## **Q2. Experimental setting for hypothesis 2**\n\nThank you for pointing this out. Yes, classifier guidance is **active throughout** the generation process in Hypothesis 2. Similar to Hypothesis 1, our goal in Hypothesis 2 is to measure how much the *diffusion denoising step* disrupts the classifier-guided updates from the previous step. \n\nSpecifically, in Figure 3, the **\"before diffusion step\"** refers to the sequence *after classifier guidance is applied*, where key tokens are steered toward the target attribute. The **\"after diffusion step\"** then refers to the sequence *after the denoising step and before classifier guidance is re-applied at the next timestep*.\n\nThe observed drop in classifier confidence reflects how much the denoising step alters or \"forgets\" the classifier's earlier updates—especially on key tokens with high gradient importance. A large confidence gap between these two stages indicates that the diffusion model is weakening the applied control signal, which motivates our analysis of update-forgetting.\n\nWe have revised Section 2.2 to make this setup more explicit.\n\n## **Q3. Details on external classifier application**\n\nThank you for the question. While most off-the-shelf classifiers are designed to operate on clean tokenized text, we leverage the simplex-based representation during diffusion to obtain interpretable logit-based embeddings that can be directly fed into these external classifiers.\n\nThis approach allows us to reuse standard classifiers without retraining them for the latent or embedding space. In fact, SSD-LM [1] demonstrated that applying external classifiers in the simplex domain is effective, even when the input is a noisy intermediate representation. By using this method, we avoid the need to train task-specific classifiers on the latent space, which is often unstable and computationally expensive.\n\n[1] Han et al., \"SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control\", ACL 2023\n\n## **Q4. Clarification of Section 4.4**\n\nThank you for pointing this out. We agree this section required clarification. The accuracy improvement from 92.2 to 92.6 corresponds to the **TTA (200)** variant evaluated with **T = 1000** inference steps. This result is reported in **Appendix Table 9**, where TTA (200) denotes the use of a reduced number of adaptive update steps (200), while the overall diffusion process still runs for 1000 steps.\n\nWe have updated Section 4.4 to explicitly mention both **TTA (200)** and **T = 1000**, and to clearly reference Appendix Table 9 to avoid confusion.\n\n## **Q5. Performance degradation of TTA (5000)**\n\nThank you for this insightful question. We believe the performance drop of **TTA (5000)** is primarily due to a **mismatch between training and sampling conditions**. While TTA (1000) is trained with 1000 denoising steps and evaluated accordingly, TTA (5000) is trained with a much longer horizon of 5000 steps. When both models are sampled using the *same reduced number of inference steps* (e.g., under 1000), TTA (5000) tends to underperform due to its reliance on a longer denoising trajectory during training.\n\nThis phenomenon is consistent with findings in other works—such as **SDTT** [1]—which show that a teacher model trained with a longer trajectory can actually perform worse than a distilled model when both are sampled under shorter inference budgets.\n\n[1] Deschenaux, Gulcehre, “Beyond Autoregression: Fast LLMs via Self-Distillation Through Time”, ICLR 2025.\n\n## **Q6. Distribution of samples in Figure 2.**\n\nThank you for bringing this up. The distribution of samples is not uniform across timesteps. The table below shows the distribution at two representative timesteps (**t = 899** and **t = 99**), where samples are separated into four equally ranged **fluctuation ratio bins**.\n\nAt **higher timesteps** (e.g., t = 899), more samples fall into **higher fluctuation bins**, indicating greater variability early in the generation process. As the timestep **decreases** (e.g., t = 99), the distribution shifts, with more samples appearing in **lower fluctuation bins**—suggesting that fluctuation stabilizes over time. We’ve included a detailed figure of this distribution in **Appendix B**.\n\n**Percentage of samples in each fluctuation bin**\n\n| **Bin** | **t = 899** | **t = 99** |\n| --- | --- | --- |\n| Bin 1 (Lowest fluctuation) | 26.1% | 11.2% |\n| Bin 2 | 37.3% | 23.9% |\n| Bin 3 | 26.1% | 32.8% |\n| Bin 4 (Highest fluctuation) | 10.4% | 32.1% |\n\n\nPlease let us know if any additional clarification would be helpful.\nThank you again for your detailed and thoughtful feedback."}, {"review_id": "zzGSzw8wSL", "reviewer": "Reviewer_X8kS", "summary": "The authors propose to improve the stability of classifier-guided semantic edits in diffusion language models by proposing context-dependent token updates. They show that their approach significantly improves both computational efficiency and generation quality.", "questions": "- Can the authors report the empirical speedups of proposed approach under fewer sampling steps?\n- Can the context-dependent token updates be applied to existing discrete diffusion frameworks? [1]\n\n[1] Schiff, Y. \"Simple Guidance Mechanisms for Discrete Diffusion Models.\" ICLR 2025.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The authors clearly formulate the \"update-forgetting\" failure mode with supporting empirical evidence and thorough ablations\n- The proposed context-dependent token updates leads to significant improvements in both computational efficiency and generation quality\n- The range of tasks and ablations are thorough and convincing \n\n**Weaknesses**\n- Empirical speedups of proposed approach under fewer sampling steps are not presented\n- Proposed approach is grounded in Gaussian diffusion. However, SOTA diffusion LMs operate in a discrete state space, with recent work proposing guidance mechanisms under this setting [1]\n- The authors do not compare to discrete diffusion LMs that support classifier guidance [1]\n\n\n[1] Schiff, Y. \"Simple Guidance Mechanisms for Discrete Diffusion Models.\" ICLR 2025.", "was_revised": true, "final_justification": "After the rebuttal discussion, I keep my score of \"4\". \n\n- The proposed framework is novel with strong empirical results\n- The authors addressed my concerns in missing baselines for discrete diffusion guidance and showing empirical speedups\n- However, their approach does not achieve a speedup relative to AR under lower sampling steps as mentioned in their abstract.", "rebuttal": "**Dear Reviewer X8kS,**\n\nThank you for your thorough and insightful review. We value your feedback and respond to your comments as follows:\n\n## **W1, Q1. Empirical Speedup**\n\nWe appreciate your request for additional empirical results. To address this, we conducted a latency comparison across various methods using a consistent setup: sequence length of 64, batch size of 100, evaluated on a single A100 GPU. Latency values for DExperts, PPLM, LM-Steer, and DGLM are drawn from [1] and [2].\n\nAs shown below, our proposed TTA method (with control) achieves markedly lower latency than prior classifier-guided diffusion methods such as SSD-LM and Diffusion-LM, and is competitive with autoregressive baselines in terms of runtime. Notably, while the original T = 1000 diffusion models (with or without control) incur substantial latency, our progressive timestep reduction allows us to match—or even surpass—the quality of these longer-run models with as few as 50 to 200 steps. \n\n| Method | Timestep | Speed ratio |\n| --- | --- | --- |\n| GPT-2 (base) | — | 1.0 |\n| DExperts | — | 2.6 |\n| PPLM | — | 270.1 |\n| LM-steer | — | 1.2 |\n| Diffusion-LM | 200 | 14.7 |\n| Diffusion-LM | 1000 | 72.7 |\n| SSD-LM | 1000 | 109.8 |\n| LD4LG | 250 | 2.2 |\n| DGLM | 50 | 4.4 |\n| TTA (w/o control) | 50 | 1.4 |\n| TTA (w/o control) | 1000 | 27.4 |\n| **TTA (with control)** | 50 | **2.0** |\n| TTA (with control) | 100 | 4.0 |\n| TTA (with control) | 200 | 7.9 |\n| TTA (with control) | 1000 | 39.2 |\n\n[1] Han et al., “Word Embeddings Are Steers for Language Models”, ACL 2024\n\n[2] Lovelace et al., “Diffusion Guided Language Modeling”, Findings of ACL 2024\n\n## **W2. Comparison to discrete diffusion LM**\n\nThank you for your thoughtful comment. We agree that discrete diffusion language models have shown strong performance in recent work, and it is important to consider them as baselines. In response to your suggestion, we evaluated our approach against the D-CBG framework for discrete diffusion guidance [1] on the sentiment control task. \n\nFor this comparison, we used the publicly released **UDLM-LM1b** model and trained a Yelp sentiment classifier following their setup. We set the sampling steps to 128 and used the `siebert/sentiment-roberta-large-english` classifier for evaluation. The guidance strength parameter γ was swept from 1 to 50, and the best results are reported. Our method achieves higher sentiment accuracy and significantly lower perplexity compared to D-CBG, underscoring the effectiveness of our adaptive guidance.\n\n| **Method** | Timestep | Sentiment Accuracy | PPL |\n| --- | --- | --- | --- |\n| D-CBG (r=10) | 128 | 75.4 | 145.1 |\n| D-CBG (r=50) | 128 | 87.4 | 225.5 |\n| TTA (50, adaptive) | 100 | 91.0 | 25.4 |\n\n[1] Schiff et al., “Simple Guidance Mechanisms for Discrete Diffusion Models”, ICLR 2025.\n\n## Q2. Applicability to Discrete Diffusion Frameworks\n\nThank you for raising this important question. Indeed, our context-dependent token timestep allocation is compatible with discrete diffusion models, including the Discrete DIffusion classifier Guidance (**D-CBG)** framework [1].\n\nTo demonstrate this, we integrated our adaptive allocation strategy into the **standard D-CBG framework**, evaluating it on the molecular property maximization task included in their original setup (QM9 dataset). Specifically, we focused on generating molecules that optimize **drug-likeness**, and compared performance against the baseline D-CBG model **without** timestep allocation under identical conditions. In our method, we adaptively modify the transition probabilities based on approximated classifier gradients, allowing the generation process to be guided more effectively.\n\nThe table below shows the performance improvements from adaptive allocation across different guidance strengths (γ):\n\n| **γ** | **Method** | **Valid ()** | **Mean Property** |\n| --- | --- | --- | --- |\n| 1 | D-CBG | 989 | 0.474 |\n|  | + Adaptive Allocation | **998** | **0.494** |\n| 10 | D-CBG | 721 | 0.585 |\n|  | + Adaptive Allocation | **756** | **0.591** |\n\nAcross guidance strengths, adaptive allocation improves both validity and target property scores, showcasing its value in discrete diffusion settings.\n\nWe hope these additional results address your concerns. Thank you again for your thoughtful review and suggestions."}, {"review_id": "aYy3NUkF4r", "reviewer": "Reviewer_ThEE", "summary": "* The authors propose TTA-Diffusion, an inference-time technique to reduce token fluctuations when applying classifier guidance.\n* The authors observe that semantics of generated sequences are perturbed and modified during diffusion and formalize this problem as diffusion fluctuation and update-forgetting.\n* The authors propose a dynamic timestep allocation strategy that leverages gradient signals from the guiding classifier to ensure that recently guided tokens are not immediately revered or perturbed by the denoising model.\n* The authors show that TTA-Diffusion outperforms autoregressive and diffusion baselines on controllability tasks, such as detoxification, sentiment control, topic control, etc.", "questions": "Comments and questions are not necessarily in order of importance (largely chronological with the flow of the paper).\n\n1. Are the examples in Figure 1 actual observed samples / examples of update-forgetting? The transition seems drastic. Diffusion disruption and update-forgetting are defined by strict token-equality checks, which seem brittle and susceptible to over-estimation due to minor word substitution to synonyms or modifications to sentence structure. Actual observed samples from the experiments in Appendix B.1 would be helpful in contextualizing the problem (maybe alongside TTA-Diffusion outputs in Table 12 for clearer comparison).\n\n2. The authors write that TTA-Diffusion \"build[s] upon the Simplex-based diffusion language model [11], and extend it into a fully non-autoregressive framework [28] that naturally supports external classifier guidance.\" However, the non-autoregressive extension simply follows from using TESS, so it seems slightly misleading to suggest this as a contribution. A rephrasing seems necessary.\n\n3. It would be helpful if Definition 2 includes details on how key tokens from classifier guidance are identified (highest gradient norm, as written in Hypothesis 2. Moreover, Table 12 seems to suggest that context / prompt tokens are included in computing the top $K$ most important tokens -- is this correct? I would expect context tokens to be excluded from this measurement as well as updates from classifier guidance or diffusion.\n\n4. Sec. 2.2. indicates that TESS without self-conditioning seems to be used. Does enabling self-conditioning make a difference? The simplex averaging self-conditioning method is effectively an exponential moving average of predicted probabilities, so I would expect the predictions to remain more consistent without excessive disruptions.\n\n5.  At a high level, it seems like update-forgetting occurs because the backward diffusion step erases the effect of classifier guidance. Is the guidance factor at all relevant to this? Update-forgetting seems to suggest that classifier guidance was perhaps too aggressive that it causes a \"tug-of-war\" between guidance updates and backward diffusion. Can a well-tuned guidance factor alleviate this? It seems like $\\alpha_\\mathrm{smooth}$ explicitly modulates this, but really a fine-tuned $\\lambda$ should suffice (and perhaps the adaptive timestep allocation strategy in Sec. 3.2 is effectively controlling $\\lambda$ per timestep per token).\n\n6. Hypothesis 1 seems to suggest token fluctuations is the cause of reduced fluency and coherence. Is it possible that the disruptions are due to model's low confidence, i.e., the reduced fluency is more the byproduct of the model struggling to converge on a stable output sequence due to the difficulty of generating a sample? From this point of view, the reduced fluency is not due to the fluctuations per se, but due to low confidence / more uncertainty in the model's predictions?\n\n7. Is the adaptive timestep in Sec. 3.2. have a monotonicity property, i.e., $f(i, t) < f(i, t + 1)$? It seems unnatural and contradictory for a token that was assigned a small timestep (\"this is a stable, finished token\") to later have a larger timestep (\"now we need to aggressively refine\"). The norm-based convex combination with time doesn't seem to guarantee this. \n\n8. How does progressive step reduction relate to the earlier hypotheses and problem definitions? It's not clear to me what the motivation for this is, especially given the narrative that TTA-Diffusion is an inference-time technique.\n\n9. Is update-forgetting unique to the continuous-to-discrete nature of simplex modeling, or do you think there are continuous analogues?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* Diffusion generation, and in particular timestep allocation strategies and fine-grained steering, is a relevant topic of interest to the community.\n* Presented claims and hypotheses are empirically appropriately supported by experiments and ablations.\n* The paper is clean and easy to follow.\n\nWeaknesses:\n* The paper focuses on a rather specific problem (steering simplex diffusion models via classifier guidance), limiting its general applicability.\n* The definition of token disruption and update-forgetting seems rather simple and is unintuitive. It is based on strict token matching, which at a glance seems like a poor measure of semantic coherence.\n* Some claims are weakly supported by theoretical justifications drawn from the discrete diffusion literature. While the rough intuition may translate to simplex diffusion models, some details and rigor are likely lacking.", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer ThEE,**\n\nThank you very much for your thoughtful and detailed feedback. We greatly appreciate the time and effort you dedicated to reviewing our work. We address each of your comments and suggestions in the responses below.\n\n## **W1. Applicability of TTA-DIFFUSION in various domains**\n\nWe appreciate your concern about generality. While our main experiments use simplex-based diffusion models for their compatibility with classifier gradients, our adaptive  allocation strategy is broadly applicable and extends to both discrete and continuous diffusion frameworks.\n\nTo demonstrate this, we integrated our method into the Discrete Diffusion Guidance (D-CBG) framework [1]. Please refer to our response to Reviewer X8kS (Q2) for further details.\n\nThe same strategy also applies to continuous diffusion models. For example, applying it to a sentiment control task with Diffusion-LM [2] improved accuracy from 72.8% to 75.6%, demonstrating the method’s versatility and effectiveness across both discrete and continuous frameworks.\n\n[1] Schiff et al., “Simple Guidance Mechanisms for Discrete Diffusion Models”, ICLR 2025.\n\n[2] Li et al., “Diffusion-LM Improves Controllable Text Generation”, NeurIPS 2022\n\n## **W2. Definition of token disruption and update-forgetting**\n\nThank you for raising this important point. We understand the concern that our definitions of **token disruption** may appear too literal for capturing true semantic drift. To make our analysis more resilient to paraphrases and word-order variations, we are adding *semantics-aware* alternatives that complement the strict proxy:\n\n- **BLEU (n-gram overlap):** Captures surface-level similarity, tolerating minor word reordering.\n- **BERTScore (embedding-based):** Detects semantic shifts, including synonyms and paraphrases.\n- **Soft token matching:** Measures token-level semantic similarity via embeddings.\n\nWe observe a moderate correlation between BLEU, BERTScore, and cosine similarity with perplexity at time *t* (−0.3∼−0.6), with trends mirroring the perplexity curves in Figure 2. BLEU shows the highest correlation, likely due to heavy input noise making n-gram overlap more informative than sentence-level embeddings.\n\nOur token-level fluctuation ratio also shows strong correlation with these metrics (e.g., BLEU −0.894, BERTScore −0.906), supporting its use as a practical proxy for semantic drift. However, to better capture subtler shifts such as paraphrasing or synonym substitutions, we have enhanced our evaluation by integrating these semantics-aware metrics.\n\n## **W3. Theoretical justifications**\n\nThank you for the valuable feedback. We agree that a clear theoretical link to discrete diffusion is important. Below is a three-part proof sketch supporting our claims; full derivations will appear in the revised manuscript (Appendix A.2–A.5).\n\n**(1) Relation of Simplex Diffusion to Discrete Diffusion**\n\nOur simplex-based diffusion model is formally connected to discrete diffusion frameworks via **Diffusion Duality** [1], which maps Gaussian diffusion in continuous space to a Uniform-state Discrete Diffusion Model (USDM). The core duality lies in projecting the continuous latent $X_t$ to a discrete token. While the theory uses $\\arg\\max$, we apply softmax with top-p sampling, which converges to $\\arg\\max$ as the threshold nears zero—providing a controllable generalization. This links our simplex diffusion model to the discrete diffusion framework. We have revised Appendix A.5 for this.\n\n**(2) Relation Between Diffusion Fluctuation and Perplexity**\n\nAs detailed in Appendix A.2, our claim that higher diffusion fluctuation leads to increased perplexity is supported by the cross-entropy bounds in Haxholli et al. [2]. Fluctuation $R_t$ approximates sequence-level Hamming distance, and when the learned kernel $p_\\theta$ deviates from the true kernel $q_t$, it assigns more mass to sequences with larger token shifts, increasing total variation. By Pinsker’s inequality, this leads to higher $\\mathrm{KL}(q_t | p_\\theta)$, relaxing the lower bound on cross-entropy and perplexity. Full derivations are in Appendix A.2–A.3.\n\n**(3) Theoretical Grounding of Adaptive Allocation**\n\nOur adaptive allocation strategy achieves a near-Pareto improvement by enhancing control accuracy without increasing perplexity. This is supported by two insights: (1) under an exchangeable reverse diffusion kernel, the perplexity bound ($J_2$) depends only on the multiset of per-token variances ${\\sigma_i^2(t)}$, not their assignments, making perplexity invariant to token-wise permutations (Appendix A.3, Lemma 1); and (2) since classifier control is sensitive to noise distribution, we set $\\sigma_i^2 \\propto (1 - \\hat{g}_i)$ to allocate less noise to well-aligned tokens and more to misaligned ones. This targeted variance reshaping reduces drift where accuracy is critical and promotes updates where needed.\n\n[1] Sahoo et al.,”The Diffusion Duality”, ICML 2025.\n\n[2] Haxholi et al., “Efficient Perplexity Bound and Ratio Matching in Discrete Diffusion Language Models”, ICLR 2025\n\n## **Q1. Empirical observation of update-forgetting phenomena in Figure 1**\n\nThank you for this question.  Yes, the examples in Figure 1 (e.g., “clean → dirty,”) are actual observations extracted from our generation outputs. While extreme sentiment reversals are uncommon, we frequently observe more subtle degradations (guided tokens becoming neutralized) across denoising steps. \n\nWe also included more concrete examples from Appendix B.1 in the revised version.\n\n## **Q2. Clarification on the contribution of TESS integration**\n\nWe agree that the phrasing may have overstated our contribution in this regard. The non-autoregressive property of our framework indeed follows directly from adopting the TESS configuration. We have revised Section 3.3 of the manuscript to clarify that our work builds on the TESS-based setup.\n\n## **Q3. Details on identifying key tokens**\n\nThank you for pointing this out. We have revised *Definition 2* to clarify that **key tokens** are selected based on their **gradient norm magnitude**.\n\nFor the prompt tokens, while Hypotheses 1 and 2 are unaffected (as they use unconditional generation), you are correct that we included prompt tokens when computing important token indices during inference. In our case, prompts were short and had minimal impact, but we acknowledge that longer prompts could introduce bias.\n\nThanks to your suggestion, we have updated our inference algorithm to exclude prompt tokens from the importance calculation, ensuring robustness across different prompt lengths.\n\n## **Q4. Effect of self-conditioning**\n\nWe appreciate the insightful question. In our experiments, enabling self-conditioning had minimal impact on control metrics. While self-conditioning acts as a smoothing mechanism , we conjecture that its effect diminishes when classifier guidance is used.\n\n## **Q5. Relevance of update-forgetting to guidance factor**\n\nThis is an excellent point. We agree that update-forgetting arises from the tension between classifier guidance and denoising, particularly when guidance is overly strong. As Table 8 shows, a high guidance scale raises perplexity, suggesting that excessive guidance can destabilize generation by overwriting updates. Our adaptive timestep allocation implicitly adjusts the local guidance scale $λ_i$ by assigning each token a timestep $t_i$ based on its gradient. Since smaller $t_i$ weakens denoising ($|G_d|$), this is equivalent to scaling the update from $G_d(t) + λ G_c$ to $s_i G_d(t) + λ G_c$, or $G_d(t) + λ_i G_c$ with $λ_i = λ / s_i$. \n\n## **Q6. Cause of token fluctuations**\n\nThank you for the insightful hypothesis. While we observe a clear link between token fluctuation and reduced fluency, we agree that low model confidence may also contribute to instability. Our analysis shows a moderate negative correlation ($r = -0.33$) between confidence and next-step fluctuation, suggesting low-confidence tokens are more likely to change—though not strongly enough to imply causation. We view confidence and fluctuation as related but distinct signals of instability, reflecting uncertainty, gradient misalignment, and overwriting during denoising.\n\n## **Q7. Monotonicity of adaptive timestep scheduling**\n\nThank you for raising this point. We accounted for monotonicity by optionally enforcing a constraint that prevents a token’s timestep from increasing, but found it had minimal impact—typically within ±1% in metrics. Interestingly, the unconstrained version often performed slightly better, likely because (1) norm-based allocation is nearly monotonic in practice, and (2) early-step randomness lets initially irrelevant tokens later align with the target attribute, making re-refinement beneficial. \n\n## **Q8. Rationale of progressive step reduction**\n\nThank you for the question. The progressive step reduction is introduced primarily for practical reasons, rather than as a core theoretical component of our method. Since classifier guidance adds considerable computational overhead during inference, we adopt progressive step reduction to compensate for this cost. \n\n## **Q9. Update-forgetting in continuous vs. simplex diffusion**\n\nWe appreciate your thoughtful question. While update-forgetting is harder to characterize in continuous diffusion models, we bet it is more pronounced in simplex or discrete settings due to the stepwise projection toward token modes, which can override previous guidance. In contrast, continuous models evolve more smoothly without such hard projections, potentially reducing forgetting. Nonetheless, we observe similar forgetting dynamics across both settings, as applying our adaptive allocation to Diffusion-LM still yielded improved control.\n\n\n**We hope these new findings resolve your concerns. Thank you again for your helpful feedback.**"}, {"review_id": "GnTMqe14i2", "reviewer": "Reviewer_NdQ8", "summary": "The paper investigates why classifier-guided diffusion language models become unstable, tracing the problem to two factors—excessive fluctuations and update-forgetting—and proposes adaptive per-token timestep allocation as an inference-time fix. Additionally, they bridge the discrete–continuous gap through logit-simplex mapping and uses timestep distillation to reduce sampling steps. Extensive experiments and ablations confirm that the method yields more stable and controllable generation.", "questions": "Please refer to the weakness section.", "limitations": "Please refer to the weakness section.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper’s clear empirical diagnosis (fluctuation-driven update-forgetting) naturally motivates the adaptive per-token timestep strategy, making the solution intuitive and clear.\n2. The method shows consistent improvements in overall metrics compared to other baselines even with reduced sampling steps.\n3. The author provides extensive experiments in the main paper and the appendix.\n\nWeaknesses\n1. While adaptive token-level timesteps for classifier-guided inference are a thoughtful extension, I feel this idea is incremental, as it builds on earlier token scheduling concepts like those in AR-Diffusion. Moreover, other components such as the logit-simplex mapping and timestep reduction have already been proposed in prior work.\n2. From my understanding, the time distillation is used for compensating the time overhead from using the classifier guidance. I think there should be a comparison regarding the time consumption for generating the text.\n3. Several informative studies in the appendix are not mentioned in the main paper making it easy for readers to miss evidence that actually strengthens the claims.", "was_revised": true, "final_justification": "The main concern of my previous rating was the limited evaluation and the limited novelty of the proposed method but the author reloved most of it. Therefore I raise my rating.", "rebuttal": "**Deer reviewer NdQ8,**\n\nThank you for your careful and constructive review. We value your thoughtful feedback and respond to your concerns below.\n\n## **W1. Novelty of TTA-DIFFUSION**\n\nWe appreciate your reflection on the novelty of our method. While we agree that certain components—such as logit-to-simplex mapping and timestep reduction—draw from existing practices and are primarily design choices for integrating classifier guidance and improving efficiency, we would like to emphasize that our core contribution lies in the **adaptive token-level timestep allocation** mechanism.\n\nThis mechanism introduces a **principled, inference-time strategy** that dynamically adjusts the generation schedule of each token based on its alignment with an external semantic signal—measured via classifier gradients. In contrast to AR-Diffusion, which relies on fixed schedules tied to token position or heuristic progression, our method **actively infers** token importance at each step and tailors refinement accordingly. **This dynamic and semantics-aware scheduling is crucial for preserving intended control signals, reducing unnecessary updates, and enabling more stable, accurate, and efficient generation under guidance.**\n\nImportantly, our method directly addresses a core weakness in diffusion-based controllable generation: the phenomenon of *update-forgetting*, where classifier-guided semantic edits are inadvertently erased in later steps. By assigning smaller timesteps to classifier-aligned tokens, our approach effectively protects critical updates from being overwritten—**preserving semantic consistency over time**, a capability not supported by prior scheduling methods. Moreover, by focusing denoising effort on uncertain tokens, our method **prioritizes computation where it's most needed**, leading to significantly improved fluency and control even under aggressive step reduction. \n\nIn addition, the plug-and-play nature of our method makes it broadly applicable across both discrete and continuous diffusion frameworks without requiring model retraining, making it a **scalable and general-purpose solution** for guided generation. To our knowledge, this is the first work to propose such an **adaptive, gradient-informed inference-time allocation strategy** in diffusion language modeling.\n\nWe believe this formulation offers more than a technical distinction from AR-Diffusion—it introduces a scalable and theoretically grounded mechanism for **coordinating the generation order with external goals**, which is essential for tasks requiring reliable semantic control (e.g., detoxification, sentiment modulation, alignment). In doing so, our work both addresses core weaknesses in existing diffusion generation (e.g., update instability) and paves the way for future advances in guided generation under diffusion frameworks.\n\n## **W2. Empirical Speedup**\n\nThank you for the helpful suggestion. As you correctly pointed out, we introduced **time distillation** to mitigate the **latency overhead** caused by classifier guidance.\n\nTo support this claim, we conducted a **latency comparison** across various methods using a unified evaluation setup: sequence length of 64, batch size of 100, evaluated on a single A100 GPU. Latency results for DExperts, PPLM, LM-Steer, and DGLM were drawn from [1] and [2].\n\nAs shown in the table below, our TTA-Diffusion method (with classifier control) offers **significant runtime improvements** over existing classifier-guided diffusion approaches such as SSD-LM and Diffusion-LM, while remaining competitive with autoregressive baselines:\n\n**Empirical speed comparison**\n\n| Method | Timestep | Speed ratio |\n| --- | --- | --- |\n| GPT-2 (base) | — | 1.0 |\n| DExperts | — | 2.6 |\n| PPLM | — | 270.1 |\n| LM-steer | — | 1.24 |\n| Diffusion-LM | 200 | 14.7 |\n| Diffusion-LM | 1000 | 72.7 |\n| SSD-LM | 1000 | 109.8 |\n| LD4LG | 250 | 2.2 |\n| DGLM | 50 | 4.4 |\n| TTA (w/o control) | 50 | 1.4 |\n| TTA (w/o control) | 1000 | 27.4 |\n| **TTA (with control)** | 50 | 2.0 |\n| TTA (with control) | 100 | 4.0 |\n| TTA (with control) | 200 | 7.9 |\n| TTA (with control) | 1000 | 39.2 |\n\nThis demonstrates that TTA-Diffusion effectively alleviates the computational burden of classifier guidance while preserving its controllability benefits. We have included this speed comparison in **Appendix C** to provide a clear and concrete evaluation of the latency benefits introduced by our method.\n\n[1] Han et al., “Word Embeddings Are Steers for Language Models”, ACL 2024\n\n[2] Lovelace et al., “Diffusion Guided Language Modeling”, Findings of ACL 2024\n\n## **W3. Integration of Supporting Evidence**\n\nThank you for highlighting this point. We agree that several **informative analyses** in the appendix warrant more explicit mention in the main paper.\n\nIn response, we have updated the manuscript—particularly Sections 2 and 3.2—to better reference key supporting experiments, including:\n\n- The correlation between token fluctuations and evaluation metrics (Appendix B.1).\n- The rationale for adaptive timestep decisions (Appendix C).\n- Additional qualitative examples of update-forgetting and recovery (Appendix Table 12).\n\nThank you for your constructive comments. We hope the updated results respond adequately to your concerns."}]

Author rebuttal:

Meta-review: I recommend accepting this paper, which addresses instability in classifier-guided diffusion language models. The authors identify "update-forgetting," where token-level updates fluctuate across timesteps, and propose TTA-Diffusion to dynamically allocate timesteps per token based on refinement needs. Experimental results show significant improvements in control accuracy and perplexity while reducing computational requirements. The reviewers acknowledged the clear problem formulation, thorough empirical validation, and practical inference-time approach. Although concerns were raised about novelty and broader applicability, the authors effectively addressed these through additional experiments demonstrating effectiveness across different diffusion frameworks. Despite being a borderline case, the paper's technical contribution and strong empirical results justify acceptance.
Decision: Accept (poster)