Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
Authors: Yeongbin Seo
Dongha Lee
Jaehyung Kim
Jinyoung Yeo
Venue: NeurIPS 2025 spotlight ()
Keywords: diffusion LLM
large language models
decoding
semi-autoregressive
non-autoregressive decoding
Primary area: deep_learning
TL;DR: new decoding method for diffusion LLM, alternative of semi-AR

Abstract: Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the \textbf{long decoding-window problem}, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks (sacrificing bidirectionality), but we find that this also leads to \textbf{time-interval expansion problem}, sacrificing the speed. Therefore, semi-AR eliminates the main advantages of diffusion models. To overcome this, we propose Convolutional decoding (\textit{Conv}), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements. The code is available online (\url{https://github.com/ybseo-ac/Conv}).

Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
Yeongbin Seo
Dongha Lee
Jaehyung Kim
Jinyoung Yeo
Department of Artificial Intelligence Yonsei University
(suhcrates, donalee, jaehyungk, jinyeo) @yonsei. ac.kr
Abstract
Autoregressive (AR) language models generate text one token at time, which limits their inference speed. Diffusion-based language models offer a promising alternative, they decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the long decoding-windoel where tokens generated far from the input context often become irrelevant or repet- itive. Previous solutions like semi-ause autraarrrrerrssvvee address this issue by splitting windows into blocks (sacrificing bidirectionality). but find that this also leads to time-interval expansion problem, sacrificing the speed. Therefore, semi-AR eliminates the main advantages of diffusion models. To overcome this, propose Convolutional decoding (Conv), a normalization-basee method that narrows the decoding window without hard segmentation, leading to better fluency and flex- ibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-arr results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines. with significantly lower step size than previous works, demonstrating both speed and quality improvements. The code is available online (https: https:///gtttuut github. com/ yysee--ae aasee--cccae ac/ ac/Conv).
problem,
Introduction
Autoregressive (AR) language models (LMs) have shown remarkable performance in recent years [31, 32, 6, 1]. Diffusion- based LMs are gaining attention [3, 36, 29, 2] as they offer potential solutions to two fundamental limitations of ARLM: (1) ARLMs must generate text one token at time, which imposes an inherent limit on speedup. In contrast, diffusion models can be further accelerated by reducing decoding steps while maintaining the same output length. (2) ARLMs inherently unidirectional, relying only on past context, whereas diffusion LMs can leverage bidirectional context [49]. Since human language understanding involve both past and future context, bidirectionality offers more flexible and human-like approach.
However, find critical inherent drawback of diffusion LLMs, called long decoding-window (LDW for simplicity) problem, which is a key bottleneck fluent text generation. While AR models focus predicting single token that is directly attached to the previous context, diffusion LMs treat all positions as potential targets for decoding. Here, the problem occurs: tokens suggested at positions far from the previous context tend to be less aligned and more random. As result, naive categorical sampling during decoding often yields outputs that are poorly grounded in the given context.
Several approaches have been proposed to address this issue, including semi-AR (e.g., LLADA [29]. Block-diffusion [2]). Semi-AR divides the generation window into multiple blocks and decodes them sequentially, therefore shortening the decoding area. This suppresses decoding of positions far from
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

previous context, directly mitigating the LDW problem. It also reported to improve performance in downstream tasks [29].
However, we analyze that semi-AR still has significant inherent limitations. (1) Most notably,
sacrifices decoding speed. Reducing the step size in semi-AR is challenging. as dividing the total
steps across multiple blocks leads to a rapid degradation in overall text generation quality, due to
an issue we term the time-interval expansion problem. (2) Also, semi-AR inevitably sacrifices
bidirectionality, As semi-Ar AR lacks both the speed advantage and bidirectionality, it removes most of the motivation to choose diffusion LMs over AR models.
Therefore, we propose two approaches that both address the LDW prob- lem while retaining speed and flexibility. (1) Convolutional decoding (Conv): This narrows the decoding window similarly to semi-AR, but uses normalization instead of strietly splitting it into blocks, bypassing quality degradation. (2) Rejecting Rule-based negative Fine-Tuning (R2FT): post-hoc training scheme that aligns distant tokens with previous context, removing the need for narrowing the window. We empirically demon- strate that combining these methods leads to more fluent and coherent open-ended responses.
(Alpace Eval)
Competia seoding
per step
Figure 1: G-eval score and sampling speed on AlpacaEval. Ours
For the experiments, focus testing instruction- guriionn open- ended answer generation ability. Previous studies primarily evaluated diffusion LMS using natural language understanding (NIU) benchmarks that are measured with multiple-choice metrics or string matching metrics [29]. However, this makes it difficult to assess the biggest challenge of diffusion LMs: fluent open-ended text generation. We observe that previous diffusion LMs [29] which showed remarkable scores on NLU tasks (e.g., MMLU [16)) perform poorly on benchmarks to evaluate open-ended answer generation such as AlpacaEval [44], often generating broken sentences. Therefore, we use benchmarks like AlpacaEval and the metric of G-Eval [26], which is known as most aligning with human evaluation, showing that our method achieves
achieves SOTA.
performance among diffusion- LM baselines, with even around one-third the step size of previous work Figure 1. For bidirectional generation, we provide a proof of concept through sampling patterns (Figure 7). Since most LM tasks assume unidirectional scenario, benchmark experiments are deferred to future work.
Our novel contributions are summarized as follows:
We define the long decoding-window problem, one of the core challenges in diffusion LMs.
We identify and analyze the inherent speed limitation of semi-AR (i.e., block diffusion), termed the time-interval expansion problem, which was unrecognized in prior studies.
We propose two alternative methods to address the long decoding- problem without sacrificing speed or bidirectionality, Conv and R2FT.
We demonstrate that models equipped with methods achieve state-of-the-arr performance among diffusion LM basclines, on open-ended answer generation tasks.
2 Preliminaries and motivation
2.1 Diffusion LM
In the image generation domain, diffusion model refers model trained through a forward process that adds gaussian noise for T' steps to the original data (xo), and reverse process that learns to remove the noise, gradually denoising from pure noise (Xy) through intermediate steps (X,, .... X1 xn)
[17, 40, 42, 10, 41]. The forward process is formulated q(x, (xo) N(x v Vaxxa, (1 [
where ,. II 04 ar E [1, The reverse process is training minimize Negative ELBO (NELBO):
ENELBO = Bag
pa pxoo
DKa. 1 (o) x()]
+ DKL. (xo)ll (1)
Lrecons
Cprior
Latltssonn
As the forward process designed for continuous data (e.g., image), applying this framework to discrete data (e.g., text) has been major bottleneck in the study of diffusion LMs. However, the

---

Notation of π and p In this work, two kinds of conditional probability are defined. The conditional probability defined with the ı term, л (atg -i); is the probability of token at position i, conditioning on the tokens provided at positions -i. On the other hand, the conditional probability defined with remarkable work of D3PM by Austin et al. [3] opened the door by defining the forward process as follows:
g(x;
Xo) Cat( Q,xa), Q: Qu...QQ44
Where Cat(x: is a categorical distribution over the one-hot row vector x (dimension of vocab size) with probabilities given by the row vector p, and Q; is transition matrix at timestep t. This formulation laid the foundation for many subsequent studies to follow [19, 15, 7, 27, 34, 43, 22, 48, 38). Furthermore, Sahoo et al. [36] proves that the NELBO objective of D3PM can be extremely simplified to Equation by following certain assumptions:
t=
NELBO
Egg
0x+
dt--t
log t), xo)dt, E 1], ae [1,0]
(3)
1=
a2
This objective function is the same as masked language model (MLM) such as BERT [9], while the only difference is the forward process with more complex masking pattern. The assumptions are as follows: (1) The time space is continuous, which indicates T - oc, df - 0. (2) The fully noised state 2T means all tokens are masked, and the reverse process unmasking them. Therefore, diffusion LM is trained to predict tokens at masked positions, and during decoding, it unmask each position of a fixed-size decoding window over S steps (S T).
This formulation demonstrates the most stable convergence of the validation loss compared to earlier approaches (e.g., SEDD and D3PM) [36]. Several concurrent works (LLADA, Block-diffusion, MD4 [38]) reach the same formulation by intuition and empirical study, establishing it as the current consensus in the field. In this work, following Sahoo al. [36], we refer diffusion LMs trained with this objective as MDLM (masked diffusion language models). (See $F for more detailed review of related works.)
the p or q term, p( (1), refers to the probability of token given position (e.g., usually not stated) and the timestep t, conditioned on the token of the same position at timestep 1.
2.2 Bottleneck of MDLM: long decoding-windov problem
We analyze that the main bottleneck of MDLMs in open-ended text generation lies in the overly broad decoding candidate space, called long decoding-window (LDW) problem. Unlike AR models, which decode one token at a time, MDLMs predefine a decoding window with fixed window size (denoted L, fixed to 1024 in our work) and treat all L positions candidates for decoding at every step. However, if the position is farther from the previous context (equal to the prompt if the first step), the model tends to predict more context-irrelevat tokens. In our analysis, the pattern of irrelevance can be summarized as a combination of "high-prior tokens" and "repetition of the previous context".
Question: Who is Abraham Lincoln?
Answer:
surri AR
ConV
Abrattemn
Lingoin
LiROOn Abraham
Repetition,
Lincal
Lincain
Linçon
Lircoin
Lincaln
Lincoin
High prior
Abbaham
Lincoln
Abraham
.the
No
.AGr aham
Frezdenn
5
Abraham
Abraham
Abraham
Abraham
Abraham
of Who
Meaning
Who
States
Fruiido
Viho
Vho
Who
Wio
Qieetton 'you
Arwwer
Questioo and Preskenn
Quiecios
John
Wos Fooin
Question
and
he
knlow
he
Arswer
queztion
and
Distance Context
from
Close to context
Far from context
Figure 2: Candidate zone of the first inference step given the previous context "Q: Who is Abraham Lincoln? A:". The left box shows positions close to the context (0 to 4), and the right box shows positions relatively far from the context (25 to 29). Darker red indicates higher confidence of the model. Informative token for given question (e.g., "president", "United States") outlined.
3

---

An example of candidate zone is presented as table in Figure 2 (with the x-axis indicating the unmasked position which is distance from the previous context, and the y-axis showing the top-k ranks for each position). MDLM infers this table at every step, and samples tokens from it.
We observe that the candidate zone is typically composed of four types of tokens clustered like layer (repetition, high-prior, meaning, noise), often appearing sequentially from the top rank, as illustrated in Figure 2 and Figure 11. (1) Repetition: zone where the model repeats tokens from the previous context. In Figure 2, we observe excessive repetition of tokens from the previous context (e.g., "Questiun"". "Answer", ":"), which is not helpful in the current step. This phenomenon likely stems from the inherent tendency of LLMs to repeat previous context, which has also been reported in AR models when generating longer responses [18, 46]. (2) High-prior: A zone composed of tokens with high term- -enn-freuency, which are often functional words (c.g., ".", "the", "is"). This zone tends to appear thicker when farther from the previous context. We analyze that, from the Bayes-form decomposition T (x4((2)))) X x (I ( (x;)) if the likelihood term (indicating the relevance with context) weakens, the prior (proxy for term-frequency) dominates. (3) Meaning: Tokens containing meaningful information aligned with the previous context (e.g.. "president", "United States"). This zone becomes more prominent when it is closer to the previous context. (4) Noise: Tokens that are unrelated to the previous context, filling up the rest of the lower rank.
The key property is that, as the distance from the previ- ous context gets farther, the meaning zone tends to fall below the repetition and high-prior zones. This trend is also consistent with statistical observation (Figure 3). We measure the sum of probabilities of high-prior (top-100) and repetition (tokens overlapping the prompt) tokens at varying distances from the instruction prompt. For both, the probabilities are lowest at distance 0, indicating weak dominance. However, they peak after around 5-10 posi- tions. The high-prior continues to occupy around 50% throughout the window.
45
High prior
Repetition
54
0.31
0.33
X-axis is distance from instruc- tion prompt, Y-axis is summed probabil- ity of high-prior and repetition.
Figure 3:
This property leads to the following problems. If sampling is performed over the entire decoding window, deterministic decoding becomes difficult. In other words, if tokens are sampled from top-ranked candidates, there is high chance of selecting only repetition or high-prior tokens in positions far from the previous context, leading to generating repetitive and dull text (Figure 11).
The tendency to generate repetitive and dull text under highly deterministic decoding (e.g., greedy decoding) is also observed in ARLMs [18]. However, AR has the advantage of sampling from very short decoding window (size of 1) that is closest to the previous context, where meaning tokens gain more preference, As a result, this issue can often be effectively addressed simply by using slightly more probabilistic yet deterministic decoding strategies, such as nucleus or top-k sampling [46].
Formulation We provide a theoretical explanation of the LDW problem. Consider scenario in which the model decodes I; at the first sampling step, where F; denotes a mask token that is in the i-th position from the given context. AR decoding employs fixed value 1, whereas in diffusion LM with LDW, we may assume = 20 or higher values. The probability of candidate c for 22 is x (cl7) Ex, | [T(T)))((C) where T is all possible combination of {x1, 2:2, x220.. The cardinality of T is V 22 (V is vocab size), infinity. Therefore, posterior π [ I converges to (c).
T(c) represents the probability of a candidate that be easily predicted without any contextual information. Typical examples include high-prior tokens (c.g., function words) or simple repetition of given context. This applics regardless of whether = 20, 21, or 22. Consequently, as illustrated in the upper part of Figure 11, top candidates often include patterns such as "Question: Question: Question:" (repetition of given context) "the the the" (high-prior tokens).
3 Existing solutions for LDW problem and their limitations
3.1 Categorical sampling
Sahoo al. [36], who introduced MDLM, addressed this problem using categorical sampling. This method randomly samples from the full vocabulary instead of focusing on the top-ranked tokens. is expressed as Equation 4, where a; is a noise schedule, and s = f - dt. Both m and x x-(() are V + dimensional probability distributions, where m assigns probability I to the mask index (V + 1), and

---

xx(-) is softmax distribution inferred by the model based on the state X2.
Cat(xs;; Xt):
Xt 7 m. xx(xx))
po(xx|[x:) = g(x,|xx, x xxxxxx, = Cat m m+
(4)
de
Xx m,
1-a:
This setup allows both high- and low-ranked tokens to remain as candidates, reducing the chance of sampling from repetition or high-prior tokens that dominate the top ranks. In turn, it also increases the likelihood of bypassing the meaning zone. jumping directly to the noise zone. For this reason, text generated by categorical sampling often appears grammatically fluent but less aligned (Figure 15).
3.2 Semi-AR: time-interval expansion problem
Recent studies (LLADA [29] and Block diffusion (2]) systemized semi-AR decoding to successfully address the LDW problem. Although it was not explicitly mentioned, we analyze that the performance improvements of those methods were the result of addressing this problem.
This approach divides the decoding window into smaller blocks and decodes them sequentially, which restricts sampling from positions far from the context. As a result, it directly bypasses the LDW problem. Nic et al. [29] reports that applying semi-AR decoding raises downstream task performance to level comparable with AR models, while the performance is poor without it.
However, semi-AR has two critical drawbacks that undermine the motivation for using diffusion LMs. sacrifices decoding speed and directionality.
Semi-AR sacrifices decoding speed We found that though the total step size is maintained, splitting the decoding window and step size into multiple blocks makes text quality drop. Therefore, when using semi-AR, the total step size must be kept high to maintain the quality of the overall output, which removes the speed advantage.
This observation contradicts intuition, as it is more natural to expect that if the sample size per step re- mains, the text quality should also be maintained. For example, decoding a window of size 1024 in 128 steps, or splitting it into two 512-token blocks and decoding each in 64 steps, both result in an average of 8 tokens sampled per step. Thus, might expect the sampling quality be similar in both cases. However, in practice, the quality drastically degrades (Figure 4). We term this phenomenon time- interval expansion problem.
0.90
Total
The rationale behind is follows. Let the decoding win- dow size L, the total step size S, and the number of blocks b. Then, the window size and step size per block are de- noted as L,- = L/b and S₁- = S/b, respectively. If standard decoding of diffusion LMs is a single reverse process with time span / E [1, 0j. then semi-AR performs multiple re- verse processes, each also with time span /. [1, 0). When the infinitesimal time interval is dt = 1/S, the time in- terval within each block becomes dt, = L/S,. which is b times larger than dt. As b approaches S. dt, approaches 1, which is no longer infinitesimal.
120
250
Block length (eride)
Figure 4: Perplexity (y-axis) of the text samples from pretrained MDLM, apply- ing semi-AR decoding with different block sizes (x-axis) on fixed I. = 1024. Each line corresponds to a fixed S.
This leads to a violation of the continuous tirim assumption in the objective of MDLM ($2). The training objective of MDLM is simplified thanks to assuming a continuously
divisible time span (dl- + 0). However, d/, is b times larger than di, leading to a mismatch between training and inference. For this reason, text quality depends more on d/, than on L/S, as observed in Figure 4. To state more intuitively, when La and Si, get smaller, the model has less freedom to choose an unmasked position, which seems to cause the drop in performance. (We provide another version of theoretical explanation based on the hazard function in $G.1)
Then, how did the previous works address this? LLADA [29] and Block diffusion [2] also share the same training objective and assumption MDLM and are likely face the same problem. However, they avoid facing the problem by just setting very high S as default (S as 512-1024 and L as 1024).
It may seem natural to assume that decoding with a window size 1024 with total step size 1024 results in "1 token per step" manner, giving a speed comparable to AR. However, this a misconception. In practice, the actual content in the decoding output typically consists of only around 200-300
5

---

tokens on average, with the rest being padded by EOS (i.e., end of segment) tokens. Therefore, in reality, diffusion LMs with 1024 and L 1024 actually generate 300 tokens with 1024 steps. which is over three times slower than AR. This implies that, to claim a speed advantage, diffusion LMs must reduce the total number of steps a substantially smaller fraction (e.g., one-third) of the decoding window size L. However, such reduction causes severe degradation in generation quality for semi-AR as discussed. Consequently, is inherently difficult for semi-AR approaches to claim speed advantage.
Semi-AR sacrifices bidirectionality Since semi-AR assigns blocks sequentially from the left, regardless of whether the input context is on the left or right, it loses its bidirectional decoding ability, We provide detailed explanation for this in $H.1, and note that bidirectional generation tasks are included in this paper.
In 11, highlighted the key advantages of diffusion LMS as speed and bidirectionality. However, semi-AR sacrifices both, making a poor alternative for AR in practice.
Proposed methods
In this section, we propose two approaches that address the LDW problem without suffering from the time-interval expansion, therefore preserving the sampling speed and also bidirectionality.
Figure Convolution normalizer (s:) for each position across decoding win- dow, given bidirectional context.
4.1 Convolutional decoding
Teat(sc)
Number 14
Conndiatir
22 unraadee
Figure 6: Pipeline of convolutional decoding, from (1) to (5). Gray boxes are mask tokens.
We introduce convolutional decoding (Conr), method that narrows the decoding window in semi-AR, but achieves this via a normalization mechanism instead of hard segmentation. This helps mitigate the time-interval expansion problem. Conv is an alternative semi-AR with the following advantages: (1) It allows more flexible adjustment of the decoding window than semi-AR, leading improved performance. (2) simpler to implement. (3) preserves bidirectionality. We provide more clarification about bidirectionality in $H.1
(a) Com (uni-direction)
(b) Conv (bi-direction)
(c) Semi-AR
(d) Categorical
Figure 7: Unmask positions (Y-axis) over decoding steps (X-axis) for different decoding strategies.
L = 1024, S = 128, with kernel size and block size of 128. All strategies begin with BOS (begin- of-ei--of-sentencee token the first position. In (b), BOS token is also placed at the last position.
The proposed method is as follows. Let p(ı;y) denote the probability of a token at top-j rank of i-th position within the decoding window inferred by the model. The probability after applying the convolutional transformation is pCone (x.z) p(ıaz)
8norm, where the transformation function for each position 8; = g(u)). As illustrated in Figure 6, u; is the number of unmasked positions within a fixed distance (i.c., kernel size) around position computed using ID convolution

---

filter. g(u,) is a function of u4 and we find that tanh performs best. The normalizing constant is EFEY p(ans) which ensures that E, (C 1 holds for any function g(-).
Snorm E1 p(ag))s,
As illustrated in Figure 5, the convolutional transformation restricts the unmask probability at positions far from the previous context, functioning similarly to semi-AR. However, unlike semi-AR which rigidly decodes in blocks as shown in Figure 7(c), the Conv allows the decoding area to move more gradually and smoothly across positions. can handle any directionality under the same setup (Figure 7(b)). Additionally, the implementation is very simple, as it only requires adding a normalization step.
Total stea size
While semi-AR decoding divides the decoding window and step size by 5. resulting in increase of time interval drg, Conv retains the original step size and instead narrows the context window using a normalization. This seems to help bypass the time-interval expansion problem ($3.2), avoiding drastic degradation when narrowing the window. The Figure 8 shows that Conv maintains generation quality even at small kernel sizes (e.g.. 32), unlike semi-AR decod- ing, which rapidly degrades with only slight reductions in block size (Figure 4). We provide a theoretical guarantee for enhanced structural coherence of Conv, in $G.1
120
M12
V10
312
Figure 8: Perplexity (y-axis) of the text samples from pretrained MDLM, apply- ing Conv decoding with different kernel size (x-axis) fixed L = 1024. Each line corresponds to a fixed S.
4.2 Rejecting rule-based negative fine-tuning
An additional method to address the LDW problem is to directly mitigate the model's preference for repetition and high-prior tokens, thereby removing the need to narrow the decoding window to only nearby positions. To achieve this, we consider a short additional training stage after standard fine-tuning (SFT), aimed rejecting such patterns. We refer to this as rejecting rule-based negative fine-tuning (R2FT), which is training to reject rule-based synthesized negative behaviors. Following DPO [33] and SimPO [28]. we formulate the training objective:
(RIFT(TO)
7 ywrl
log To (y (r) EEE,D Jog a
log To (Vwl+)
3 [ynl
log To (wulx))
(5)
CNELBO
Toly z)) denotes the probability of predicting the remaining sequence y (Ne > y1) conditioned on the previous context (II,,.., ...yy.)... In other words, corresponds to the model outputs
(i) (x:) at all positions i. 7, 3 are weighting hyperparameters (we set = 0.1, 3 1). and is a sigmoid function. In general setups of DPO and SimPO, the model-generated texts are divided into negative (y1) and positive samples (yw) based human feedback. However, this approach has drawbacks to fit our goal: it is overly costly for just correcting specific unpreferred patterns, and it is diffieult to directly target the patterns aim to mitigate. In contrast, R2FT allows a more targeted approach. We use a standard fine-tuning dataset for Yuu, and we generate a corrupted version y by applying rule-based repeating patterns to portion of the same data of yw- More details on the corruption rule in SA.
Xg
We formulate the motivation of R2FT follows. As formulated in $2.2, the LDW problem occurs when (") c) is marginalized out in [(c)7())) for large n, causing т (c) to dominate. -(") is the positions between the initial prompt and the target position, containing mask tokens. R2FT directly
reduces (c). thereby amplifying π (7()) c) and restoring the model's consideration of context.
Why rase- corruption is safer and more effective than other options A more general option would be to use model-generated answers rejective samples. However, we argue that utilizing moner-generated negative samples has drawbacks, because of the possibility of "false negative". While the model tends to show preference for repeating or nigh-prior tokens, it also assigns meaningful tokens with high preference, depending on the context. Therefore, false negatives can easily occur.
In contrast, the main risk of our rule-based repetitive patterns is the presence of "unlikable negatives" (errors the model is unlikely to make). Importantly, in R2FT, unlikable negatives tend to be ignored
7

---

during training, while false negatives can lead to serious artifacts, making them far more harmful. This can be supported by analyzing the gradients of Reject of Equation 5: VoLRejec(
5g
Vo log To (Voll)) Nwl
Tmal Vo log No (u) (u)zz)
Penalty term sy follows:
88 o (Ivll log (o(l)))
B Buul
log To To
(6)
Here, the case of "unlikable negatives" result in (o( 2) O. which leads to sø - 0, therefore the model remains unaffected. However, for self-generated negative samples, (3(t z)) is relatively high it is the most probable sample of the model, resulting in high so. Therefore, if y is "false negative", the model is forced to strongly unlearn a positive sample, leading to degradation.
R2FT mitigates preference for both repetition and high-prior tokens By closely analyzing Equation 6, we can also see that R2FT mitigates not only preference for repetition but also for high-prior tokens. The mechanism for mitigating each is as follows:
(I) repetition: In the Bayes-form decomposition (a(yi z) x no( 31) (Br). the likelihood term To(x) w) captures the relationship between the bad sample y and the given context æ, reflecting the repetition pattern. Therefore, when za (1) xx) > To 2t), 8o increases, and the model is updated to lower (o x)), which leads to mitigating the repetition pattern. (2) high-prior: Since y is randomly generated from x using rule- based repetition, it may include repetition patterns of high-prior tokens by chance. In such cases, To (b()) represents such high prior, resulting in (T())) > TT (yw As a result, the model is updated to lower To (3r), leading to mitigating preference for high-prior tokens.
We observe that, after R2FT, both the high-prior and repetition in the candidate mitigated, which results in the meaning tokens shifting to higher ranks. Models already converged after 34 epochs of SFT showed improvements through 300 steps of R2FT ($A.2).
significantly
Top-k sampling Once R2FT aligns the model's candidate zone more closely with the context, decoding can be performed using only the top-k candidates, enabling more deterministic generation. As shown in the upper and lower boxes of Figure 11, top-k sampling before R2FT results in noisy outputs filled with repetition and high-prior tokens, whereas after R2FT, the model generates well- structured and coherent text. We explain the normalization method in $A.3.
Inference time intervention A simple option is to directly mitigate the probability of repetition and nigh-prior in the model's 's output during inference. This approach would behave similarly to what model learn from log To (Bu (z) part in Equation 5. However, this simplicity may come at the cost of limited optimization capability. Empirically, R2FT achieves higher performance.
EOF-fill We additionally introduce a EOS-fill, which substantially improves decoding speed. When an EOS token is sampled in the middle of the decoding window, all positions to the right are filled with EOS. Using with caching [36], EOS-fill significantly reduces required steps. More details in $B
Experiments
Since diffusion LMs trained with the MDLM objective represent the current state- of-the-art, we use the MDLM backbone as the basis for comparing different methods. All baselines are trained using standard finetuning (SFT) on the Alpaca instruction dataset [44]. Subsequently, we apply R2FT to a subset of models, as proposed in $4.2. Then, various decoding strategies are applied: In Table 1, categorical refers to categorical sampling, the default decoding strategy implemented in [36]. LLADA represents the ideal decoding setup including semi-AR with stride 512 as described in the paper of [29]. Top-k Glob corresponds to top-k decoding with global normalization (k = 20 in our paper) proposed in AA.,, and Cony refers to convolution decoding 1). Since speed is one of the key advantages of diffusion LLMs, all models follow the default setting proposed in the [36]: decoding window size L = 1024 and total steps 5 = 128, a highly compressed generation process. We mainly test on the checkpoint (180M) from [36], and assess consistency in larger models, also report experiments on the LLaDA- -B---B-------- checkpoint from Nie et al. [29]. For fine-tuning LLaDA, we LoRA adapter with the optimal hyperparameters provided in the original paper [20].
Baseline setup
Benchmark setup Since our focus is on instruction-based open-ended text generation, one of the most challenging and vailed abilities for diffusion LMs, we center our evaluation around AlpacaEval
8

---

[44] with G-eval, which is known as most aligning with human evaluation. Under G-eval, we also evaluate the methods on other benchmarks. MT-Bench [51] and Wiki. The Wiki dataset consists of 500 entities randomly sampled from Wikipedia (in the test set of Mintaka [37]) each prompted with an instruction of the form: "Please give me an explanation on [Entity] "). For larger models, we additionally test on the Arena-Hard-Auto [23] benchmark. All evaluations are conducted using baselines fine-tuned on the Alpaca instruction dataset, and we assess the response with the same metric as AlpacaEval. Details on evaluation setup are in $D.1.
Additional experiments with CoT
Table I: AlpacaEval score of small baselines across different decoding strategies. LC is length- [44]. Block and kernel size 256. We highlight first and second best.
setting In SE, we conduct addi- tional experiments under a setting de- signed to better reflect our scenario: LMs generate long-form text without
L.Winrer
1:04
47)
relying on random sampling. To re-
MBLM
1054
138
13:38 (3.22)
11.9911400
flect this setting, we instruct models to generate responses in Chain-of- Thought (CoT) format to provide de- tailed reasoning, and we apply top-5 for more deterministic decoding. As average response length increased, we set S=128, which still guarantees reasonably fast (around 3x) sample rate. As longer response is more challenging for models, this setting highlights the advantages of our method more clearly. Moreover, we report experiments and analysis on decoding methods with revocability (e.g., ReMDM [45]). Alongside Win rate, which measures alignment, we also report metrics for structural coherence (c.g., inlier rate).
MBLM
1054
138
(45413.0)
18:6 (1.0
MBLM
conprical
1054
136
3131
1350
11:18:1233
MILM
12p- Clas
4..7
4058
up-l SQRAAR
MILK
40.32
43.1911.01
MILM
lup-l
Cocy
128
48.32 13.54)
45.92.4449)
Results From the main results (Table 1), we observe the following key result: (1) Combination of our methods (R2FT, Comv) achieves the highest performance, even com- parable to AR models of the same scale. It significantly outperforms categorical decoding and LLADA decoding. (2) Even with just R2FT or Cony, the model achieves significant performance gap over other baselines. (3) The performance of R2FT drops with categorical decoding. This suggests the two components (R2FT and top-k) are synergetic, as top-k decoding helps focus on the top-ranked candidates from the R2FT model. (4)
Table 2: Comparison between small MDLMS across MT-Bench and Wiki. I=1024, S=128.
31 26
(111)
28 (3:4--144
(100)
31 (3511
(08)
1455-196)
ELADA
41.81 (360
56:2
23.56
40:27-1.311
48.87
35.55 113
40.29 (130)
(430)
44.42 (m
44.9-1.9
Eos-fill contributes to performance gain, though it is designed to improve speed. This is likely because it removes distracting content beyond the EOS token. (4) These trends were consistent across other datasets (Table 2).
Table 3: Comparison between large (8B) MDLMs. L-1024, S=128, block and kernel size 512.
Alpaco eval
Besch
Will
Arena
Buckbone
Decoding
LC Win
Win
LC
Winn
Win
LLADA- Bic 2e So peee PT (BB) LoRA SFT (BMM)
LLADA
25.63 (1:21)
35.52 (1.59)
24.31 (0.12)
25.35 (0.35)
15:26 (1:09)
25.51 (1.59)
25.73 (0.25)
25:41:11.55 (1.75)
curcociiall
51.71 (L.1)
31.92 (1 54)
40.05 (1.44)
36.93 (5.11)
16.06 (0.12)
19:22 (1.50
19:37 (0.16)
19.17 (60)
LLADA
16.07 (0 19:
(1
27.48:11442 (1:42)
31.80
23.05 (0.05)
21.17 255
LLADA
17.04
16,02 (1.15)
(1.30)
17.25 0..1)
(1.12)
(0.15)
12:45
(1.65)
45900121
48:0 (5.31)
25.52
(0.17)
Topsk Glob- Com
43.70 (3.000.00)
43.64 (1.55)
45.04:1443 (1-43
44.30 (5.30)
22.35 (0.14)
26.55
(1.5(11.6)) 667
37.81
(0.22)
22.55 (1855
Top + Gikb
54.39 (1.21)
54.05 (1 (44
51.14 (0.9))
51,02 (4.35)
30.35 (1
27.41 00.16)
36.98 02)22
35.43 (2,19)
Topsk + Glib- Core
55.96
(1.5)
46 (1.00)
50.0 343
38.55
28.17 (9086)
54
(0.15)
53.30 953
(5) LLADA decoding shows poor performance, mostly generating broken sentences (SI). We analyze this as mainly due to the small S (128) compared to the L (1024), which aligns with the report of [29], where performance degrades seriously under small S. However, when the model is trained with R2FT, even LLADA decoding yields better performance. (6) Experiments using a large pre- trained model as the backbone show a similar trend (Ta- ble 3). Notably, the combination of R2FT and Conv with small LoRA outperforms the fully fine-tuned model (LLaDA-88) 8B- Instruct) by [29].
Table 4: GSM8K test accuracy for large MDLMs. L=512, S=64, block and ker- nel size 128.
Backhone
RIFT
Decading
Accuracy
PT LORA SFT (ROM)
cuseporical
7.49
LLADA
17.33
LLADA
23,50
CLADI- BB--ttrutt
LLADA
29.89
Comr
39.34
9

---

Reasoning tasks We also conducted experiments on one of the reasoning benchmarks that requires long text generation. GSM8K [8]. We evaluated only the large (LLADA-8B) model, following the optimal settings in Nie et al. [29]: decoding window size I, 512, kernel size of 128, and deterministic (top-1) decoding. However, to accelerate the speed, we set 64. In Table 4, Conv exhibits significant performance gains.
without any additional fine-tuning, compare only the decoding strategies of Conv and the original LLADA. Even in this setting, Cony achieves substantially higher performance.
Additionally, we use
Ablation on Conv and semi-AR We conduct an ablation study to assess whether the Conv can serve for semi-AR. Keeping = 1024 and S = 128 fixed, we varied the stride (block size) for semi-AR and the kernel size for Conv, and evaluated on AlpacaEval. The comparable lower bound is the best- performing among full-window decoding methods (R2FT + top-k, red dashed line in Figure 9).
effective replacement
80
Defout
As shown in the Figure 9, semi-AR rarely outperforms the lower bound and exhibits sharp performance degradation as the stride decreases. In contrast, the Conv achieves significantly better peak performance than the lower bound and remains stable as the kernel size is reduced, with only minor degradation. This is consistent with analysis in $3.2
Stride Nermel size
Figure 9: AlpacaEval win rate (Y-axis) of semi-AR and Conv with different block size (stride) and kernel size. The red line is the best of top-k.
Sampling speed To evaluate sampling speed, we fix the decoding window size at L = 1024 and evaluate across various total step sizes S on AlpacaEval (Figure 10). We compare the baseline Semi-AR with our method, combination of Conv and R2FT. As result, method with S 128 achieves comparable performance than Semi-AR with S = 1024, significantly more efficient in terms of decoding speed. Additionally, our method can further improve generation quality as the total step size increases.
slightly better
52.5
50.0
42.5
Conv R2FT
SemiAA
Furthermore, even with the same total step size, our method achieves an additional 1.5x-3x speedup through EOS-fill (Figure 1), resulting in about 3x faster decoding speed (tokens per step, excluding EOS) compared to AR. We show this evaluation $B.
Conv RIFT (5-120)
40.0
128
256
512
1024
Total size
Figure 10: AlpacaEval win rate of semi-AR and combina- tion of Conv and R2FT. Red line is score from Conv + R2FT with S 128.
6 Conclusion and limitation
In this work, we defined the long decoding window problem as key bottleneck of diffusion LMs, and analyzed the limitations of the semi-AR as sacrificing speed and bidirectionality. Furthermore, we demonstrated that our proposed methods Conv and R2FT achieve enhanced performance generation tasks even at very small steps.
srrooding-window
However, while Conv enables bidirectional text generation unlike semi-AR, we do not evaluate its effectiveness on bidirectional downstream tasks detail. This is primarily because most generation tasks are tailored AR LMs. However, bidirectional generation holds significant potential, as it enables goal-aware behavior that is not achievable with AR. An example is goal-oriented dialogue [50, 5], which benefits from goal-aware (i.e., bidirectional) generation. We leave this investigation for future work. We provide more clarification about bidirectionality in 1..1
10

---

7 Acknowledgements
This research was supported by the MinirMinistry of Science, ICT), Korea, under the Global Research Support Program in the Digital Field Nogram(RS-20244-004333 (IIsttnt for Information Communications Technology Planning & Evaluation). This project is supported by Microsoft Research Asia. This research was supported by grant of Korean ARPA-H Project through the Korea Health Industry Development Institute (KHIDI, funded by the Ministry of Health & Welfare, Republic of Korea (grant number: RS ASS-022--055 (2044-00512374) 12374). This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant Artificial Intelligence Graduate
supervised by the
funded by the Korean government (MSIT)(N..RS) (0S-2020-I120111 School Program (Yonsei University)). Jinyoung Yeo is the corresponding author.
References
[1] Chatgpt: Optimizing language models for dialogue. 2022. URL https::/openai...eo chatgpt/.
[2] Arriola, A. Gokaslan, т. Chiu, Z. Yang. Qi. Han, S. Sahoo, and Kuleshov, Block diffusion: Interpolating between autoregressive and diffusion language models, 2025. URL https: /ab///////////////// / //arxiv.orgaall abs/ 2503. 09573.
[3] J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg. Structured denoising diffusion models in discrete state-spaces, 2023. URL https: / /arxiv. org/ abs aaa//bb//1111 2200 03006.
Y. Boget and A. Kalousis. Critical iterative denoising: A discrete generative model applied graphs. arXiv e-prints, pages Xrvv-2503,, 2025.
[5] A. Bordes, Y.-.. Boureau, and Weston. Learning end-to-end goal-oriented dialog, 2017. URL. https: //arxiv.org/ www.ii.orr//aooo /atg//a////// 605. 07683.
[6] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry,
A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877- 2020.
[7] A. Campbell, Benton, V. De Bortoli, T. Rainforth, Deligiannidis, A. Doucet. continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 35: 28266-28279, 2022.
[8] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. Training verifiers solve math word problems, 2021. URL https:/ petts::/arxtie /awwvv.xvvvv..////////.//.//.. .rrg abs 22100.11..... 111111111111 14168.
[9] Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL htwpt:tt////ee. //aaa:///xiiiii wxv.ioooo www.vooor///sss..o 810 0. 04805.
[10] P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis, 2021. URL https:
/ //arxiv.org/ abs/ 221..2205... 05233.
[11] J. Eisenstein. Introduction natural language processing. MIT press, 2019.
[12] N. Fathi, T. Scholak, and Noël. Unifying autoregressive and diffusion-based sequence generation. arXiv preprint arXiv:2504.06416. 2025.
[13] A. Gokaslan and V. Cohen. OpenNebToxtCorpu 2019.
Openwebtext corpus.
http://8kk/ion000.ggg hht..//.li...........u.......... 1o/
[14] S. Gong, S. Agarwal, Y. Zhang, J. Ye, I.. Zheng, M. Li, C. An, P. Zhao, W. Bi, J. Han, et al. Scaling diffusion language models via adaptation from autoregressive models. arXiv preprint 170.v.240..1891. 2024.
[15] Z. He, T. Sun, K. Wang, X. Huang, and X. Qiu. Diffusionbert: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.1500 2022.
[16] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and Steinhardt. Measuring massive multitask language understanding, 2021. URL https::/ /ttp:///////ia/////a///a org/ abs/2009 03300.
11

---

[17] J. Ho. A. Jain, and P. Abbeel. Denoising diffusion probabilistic models, 2020. URL https: /arxiv. org/ aaaaab/// /ataaaaa///0.... 11239.
[18] A. Holtzman, J. Buys. L. Du, M. Forbes, and Y. Choi. The curious case of neural text degeneration. arXiv preprint www.:ix:110400999.0 2019.
[19] E. Hoogeboom, D. Nielsen, P. Jaini, P. Forré and M. Welling. Argmax flows and multinomial diffusion: Learning categorical distributions, Advances neural information processing systems, 34:12454-12465 2021.
[20] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang. L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models. 2021. URL httpt::/t//t/ //rr:v////w/v/.///aa/ wwwpa/wwwwwwwaa..r.oaooere//eaaa /216 09685.
[21] N. S. Keskar, B. McCann, I. R. Varshney, C. Xiong, and R. Socher. Ctrl: conditional transformer language model for controllable generation, 2019. URL https httppps///aiii..t/tt//pptppptttt http://xxvvvv./o.o abs/ 05858.
[22] O. Kitouni, N. Nolte, Hensman, and B. Mitra. Disk: A diffusion model for structured knowledge. arXiv preprint arXiv:2312.0525. 2023.
[23] T. Li, W.-L.. Chiang, F. Frick, I., Dunlap, T. Wu, B. Zhu, Gonzalez, and I. Stoica. From crowdsourced data to high-quality henchmarks: Arena-hard and benchbuilder pipeline, 2024. URL httph::/t//aatte //ctps://ariv.. org/ / aaaabb/44 22os/206.... 11939.
[24] A. Liu, O. Broadrick, M. Niepert, and d. Broeck. Discrete copula diffusion. aXXv:2410.01949, 2024.
arXiv preprint
[25] S. Liu, J. Nam, A. Campbell, H. Stärk, Y. Xu, T. Jaakkola, and R. Komez-Bombarelee Think while you generate: Diserete diffusion with planned denoising, arXiv preprint arXiv:2410.0626,, 2024.
[26] Y. Liu, D. Iter, Xu, Wang, R. Xu, and C. Zhu. G-eval: Nig evaluation using gpt-4 with hetter human alignment, 2023. URLhttps: : /axxi aaraiivvvoo weiivvorrggoo. org/ hatgg//b//2/// 2303. 16644.
[27] C. Meng. K. Choi, J. Song, and S. Ermon. Concrete score matching: Generalized score matching for discrete data, Advances in Neural Information Processing Systems, 35:34532-34533 2022.
[28] Y. Meng. M. Xia, and D. Chen. Simpo: Simple preference optimization with a reference-free reward, 2024. URL https::t///........ //wpxivvvvv//////////////aaaa/wa www.v.org/aabb...o.o..oo org/abs// /2abas/220 2.405.. 14734.
[29] S. Nic, F. Zhu, Z. You, X. Zhang. J. Ou, J. Hu, Zhou, Y. Lin, J.RR. Wen, and C. Li. Large language diffusion models, 2025. URL https: / /arxiv.v.vv..////ii/..aaaaai.vv. org/abs 250./0099.......2.22222222292992
[30] L. Ouyang. J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal. K. Slama,
A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback, 2022. URL https: / arxiv. org/ abs /2203. 02155.
[31] A. Radford, K. Narasimhan, T. Salimans, L. Sutskever, al. Improving language understanding by generative pre-training. 2018
[32] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models unsupervised multitask learners. OpenAl blog, 1(8)99 2019.
[33] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and Finn. Direct ppefereeeccceciiiiiiiiiiiii optimization: Your language model secretly reward model, 2024. URL https::/arxiv..orgg//////////// 182900
[34] Reid, V. Hellendoom, and G. Neubig. Diffuser: Discrete diffusion via edit- bdstd reconstruction, 2022. URL: http://arxvv. org/abs/2210.1 16886.
[35] R. Rombach, Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19684-10699 2022.
[36] S. S. Sahoo, M. Arriola. Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, and V. Kuleshov. Simple and effective masked diffusion language models, 2024. URL 2436. 07524.
abs/
[37] P. Sen, A. F. Aji. and A. Saffari. Mintaka: A complex, natural, and multilingual dataset for end-to-ena question answering. arXiv preprint arXiv:2210.01613,022 2022.
12

---

[38] J. Shi. K. Han, Z. Wang, A. Doucet, and M. Titsias. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37:103131-103161 2024.
[39] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. arXiv preprint 20.000000000000000000 2020.
[40] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models, 2022. URL org/ / abs /2100. 02502.
[41] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations, 2021. URL https: http:///aavvvvv/.e wttp://www..ga//////////// 13456.
[42] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models, 2023. URL httss//::// org/ / aaaab/b30000 01469.
[43] H. Sun, L. Yu, B. Dai, D. Schuurmans, and H. Dai. Score-based continuous-time discrete diffusion models. arXiv preprint arXiv:2211. 2022.
[44] R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. Hashimoto. Alpaca: strong. replicable
model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/113aapucau html, 3(6):7, 2023.
[45] G. Wang, Schiff. Sahoo, and Kuleshov. Remasking discrete diffusion models with inference-time scaling, arXiv preprint arXiv:2503.00300 003077 2025.
[46] S. Welleck, I. Kulikov, S. Roller, Dinan, K. Cho, and Weston. Neural text generation with unlikelihood training, 2019. URL https: //tpx//x..vvv///////////../aaavl org/ abs/ /1908. 00.9888043111 04319.
[47] S. Welleck, I. Kulikov, S. Roller, Dinan, K. Cho, and J. Weston. Neural text generation with unlikelihond training, 2019. URL https::tt///.......t http:///wwwwxw.....o..a / 908 01431. 9.
[48] T. Wu, Fan, Liu, H.-T. Zheng. Y. Gong. J. Jiao, Li, J. Guo, Duan, W. Chen, et al. Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36:39957-39974, 2023.
[49] J. Yc, J. Gao, S. Gong, L.. Zheng, X. Jiang, Z. Li, and L. Kong. Beyond autoregression: Discrete diffusion for complex reasoning and planning, 2025. URL https::/// ///t://///xiii// orgo//////aooo abs 2410. 14157.
[50] T. Zhao and M. Eskenazi. Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning, 2016. URL. https: http:///wwwwww.aaa///e/so abbs/////////// 606. 025600
[51] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. Judging llm-as-a-judge with mi-bench and chatbot arena, 2023. URL ht https::/ //tppiv///xvvv../// org/ abs 1.306.00655555
[52] I.. Zheng, J. Yuan, I., Yu, and I.. Kong. A reparameterized discrete diffusion model for text generation. arXiv preprint arXiv:2302.05737, 2023.
13

---

A Details on R2FT
In this section, we discuss two points: (1) training details of R2FT including the corruption pattern design, and (2) whether R2FT effectively mitigates the preference of the model for repetition and high-prior tokens.
A.1
Training details
A.1.1 Corruption pattern design
To construct undesired samples yt. we design corruption patterns based on the following observation: Unmasked tokens typically form clusters (e.g., instruction text), and the boundary regions where these clusters meet masked areas most often the source of repetition.
We provide a corruption algorithm (Algorithm 1) tailored for an instruction- baased text generation dataset (c.g., Alpaca instruction [44]). This algorithm can be easily adapted for bidirectional scenarios by extending its principles.
R2FT is of HERL This form of corruption can be seen a different way of incorporating human preference compared to standard HFRL (human feedhack reinforcement learning). Traditional HFRL injects human preference into the model by using datasets labeled with desired outputs based on human annotation. In contrast, R2FT incorporates human preference by generating datasets composed of undesired samples. Importantly, unlike desired patterns, undesired patterns can often be easily reproduced through simple algorithms. This allows R2FT to inject preference without relying on expensive human annotation, using only the existing fine-tuning dataset. In implementation, the undesired samples y are not pre-generated: instead, they are created randomly at each training step.
Algorithm 1: Corruption for Instruction Tuning Data
Require: zo: Original data with question + answer with padding to the total length L
Require: Lo: Length of the question
Require: La: Length of the answer
Require: 9max: Hyperparameter for maximum length of repetition unit
Require: Zmin, Zmax: Hyperparameter for minimum and maximum lengths for repetition
Ensure: Tb: Corrupted version of zp
U(LQ LQ + LA). ), where e N 9 ~~~ U1,, Imax). where g E N
> Sample a length of unmasked content of z Sample a length of repetition unit
~~ U1 Eminn Zmaxx,, ), where E N
e + --c
> End position of repetition Repetition unit .hb is xo prefix with padding length L Repeat to fill length > Model evaluates no (y (22))
+ zolc g c]
Initialize N4 Rb =m [0:c] [
2b(c: e| + [r F, ... return Tb. where F6b {y1, (11
A.1.2 Training setup
For SFT, we use a global batch size of 512, the learning rate of 3e-5, 2500 warm-up steps, and AdamW optimizer across 8 GPUs ([NVIDIA RTX A5000). For the small model, the loss value converged around 33 epochs. The large model converged faster and was trained for 3 epochs, following the optimal setting of [29]. We use the same hyperparameters for R2FT SFT, but is trained only for 200-300 steps, resulting in a peak learning rate of around 5e-6. The small model (182M) was fully fine-tuned from the pretrained checkpoint of [36], while the large model is trained on LLADA- -B-base with a 80M LoRA adapter for both SFT and R2FT. Throughout our work, decoding window size L is set 1024, and step size S to 128, following [36].
Training the model to predict the EOS token. In response generation, the model must be trained to predict the end of an answer using the EOS token. Following the setup in [29], we build the SFT dataset where all positions after the question and answer are filled with EOS tokens. For SFT-only models, we allocate an attention mask to these FOS-filled positions (also following [29]). This setup encourages the model to predict an EOS token at position where the answer is likely to end.
14

---

In contrast, for SFT before R2FT, we assign attention only up to the EOS token that immediately follows the answer during SFT. During R2FT, for Yus the attention mask is constructed in the same way as in SFT. For yt, we insert EOS token at a random position within the answer. This setup intended to teach the model to place the EOS token the end of the response, rather than in the middle. As result, the length of generated responses remained similar to that of SFT-only models.
A.2
Effect of R2FT
SFT 35 epoch
Decoded from Top-5
Abraham Lincoln.
Abraham Lincoin, Abreham Lincole's
Question: What
atiiwer to, Arswwer? Answer Answer the cuestion. on. Question: What the answer to, the Answert Y Answer the question. the next question...
Pl0M P1000
12
40
-00
200
140
30
200
20
repetitive dull
Dolance Porn Certect
SFT 34 epoch
R2FT 300 step
FERN
-120
100
1000
He was the President to lsad the Union in 1865, He traveled to illincis for campaign tour in order diécuss the upcoming elections and discuse economy, those
JBL
5000
11000
14
11
10
10
100
140
the economy improved and Union highest successful state..
200
-000
30.0
Auent Informative
Detence Porm Contect
Figure 1I: Candidate zone of first 25 positions from instruction "Q: Who is Abraham Lincoln? A:". X-axis is distance from instruction, Y-axis is top-k rank. Green stands for informative tokens for instruction (e.g., "president" "United States"), while blue stands for those with high-prior (".", "the"" or repetition of instruction. (":", "Question").
Mitigating preference for repetitive pattern As the R2FT objective (Equation 5) consists two losses, L. log a ( (yw (x)) and Li - log a (=o(yw((z) where yo > V1- Li represents the negative log-probability of the repetitive pattern, while L. corresponds to the standard SFT loss. We can track Li during validation steps to observe how R2FT affects the preference of the model.
As shown in the Figure 12a, the validation E.p remains nearly constant throughout training, since the model was already converged from SFT. In contrast, L; steadily increases, indicating that the model is assigning lower probabilities to repetitive samples, demonstrating the intended unlearning effect of R2FT. Taken together, this suggests that R2FT successfully mitigates the preference of the model for repetitive patterns without harming its original language understanding or generation capabilities.
Mitigating preference for high-prior tokens To examine whether R2FT mitigates the preference of model for high-prior tokens, we measure the mean log prior E! log Tprior (xp (x:))/=1 of each token in text samples generated during validation using a top-k decoding strategy. The prior of each token is computed based on its term-frequency [11] the MDLM pretraining corpus C, which is Open WebText [13] in [36].
According to the Figure 12b, the mean log prior starts relatively high level but quickly converges to a much lower value within just 300 steps. This indicates that R2FT effectively mitigates the model's preference for high-prior tokens, especially during generation using top-k decoding.
To assess whether R2FT too much suppresses the preference for high-prior tokens, we compare the mean log prior of generated tokens against the median value of the reference corpus C (-8.01, red dash
15

---

(a) Validation loss
(h) Mean log prior
45
jog_rrior__g mean median reference corpus
-7.4
35
30
val/loss_
-7.6
25
val/loss_
20
10
500
750
1500 1750
Global Step
Global Step
Figure 12: Validation values during R2FT training, plotted against global steps (with a global batch size of 512). In (a), the Y-axis represents the loss value of L. and L1. In (b), the Y-axis shows the mean log prior of tokens sampled text data (length 512) from MDLM. The red line indicates the median log prior mean of texts from the Open WebTeet corpus.
line in Figure 12b). shown, the log prior mean of samples from the model converge a degree close to this reference within 300 to 1000 global steps. This suggests that the token distribution in the generated text is similar to that of the reference data, indicating a high likelihood that the model produces well-structured sentences similar to the references.
A.2.1 Why does R2FT help?
We provide an explanation of why preference for high prior and repetition can be problematic, and what R2FT aims to achieve by mitigating this preference.
High prior
Rapetition
The preference pattern is partially learned from the training data. We observe that the model's preference for high-prior tokens and repetition is learned from the training data. To analyze this, we examine the frequency of high-prior tokens (top 100) and repetitive tokens in the training data, following the same procedure as Fig- ure 3. The frequency pattern in the training data (Figure 13) closely resembles the probability distribution assigned by the model (Figure 3), except that the peak for repetition occurs earlier.
65
04
03
02
0.5
0.1
0.2
0.0
Figure 13: X-axis is distance from
instruction prompt, Y-axis is summed probability of high-prior and repetition.
Why is this problematic? If the model is simply mimicking the data distribution and therefore fa- voring high-prior or repeating tokens, why is this problematic? We suggest the following explanation.
LLM is probabilistic model that, when faced with high complexity. often learns shortcuts to minimize loss. From the training data, the model learns that simply choosing easy tokens always guarantee least worth loss, even without contextual understanding.
We can formulate this problem as follows. A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). can be exemplified as decoding T21, T22, I25 at the first step, where I; denotes a mask token that is i-th position from the given context. where 7
The probability of candidate e for F21 E, (T(C)) of {x1, x2, .... r20). The cardinality of T is 2 (V is vocab size), the infinity. Therefore, the posterior क (c 7) converges to (c), where easy tokens dominate. This holds for Ø22; 223 as well, which explains why patterns in candidate zone, such as "lincoln lincoln lincoln" emerge in Figure 11.
all possible combination
similar malfunction case can be seen in candidates such as "Question" and "Answer" shown the Figure 11. The model selects these candidates only because they are repetitions, even though they
16

---

have no contextual meaning. The model arbitrarily applies the pattern seen from the training data (e.g,, repetition, without contextual understanding.
Why R2FT is introduced? The motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is reducing (cc) in E| (r(e))))(()) By doing so, the posterior shifts toward tokens with higher average likelihood within T. In other words, we choose the candidate patterns like "president president president", which at least contain more contextual information".
A.3 Top-k decoding with global normalization
Once R2FT aligns the model's candidate zone more closely with the context, as illustrated in Figure 11, decoding can be performed using only the top-k candidates, enabling more deterministic generation. However, two issues remain, which call for normalization strategy to address them. (1) As shown in Equation 4, the unmask probability for each position at timestep t is Ita where I-aa
dt Σ x00
(V is vocab size, 2y is top-j'h element in x6" (xx)). However, when sample from only the top-k candidates, the unmask probability ( '1ta/ d' = I-oy Σj 2j, k < V) is reduced, leading to skipping of sampling during decoding steps. (2) In Equation 4, the unmask probability is solely a function of t, without consideration of position i. As result, unmasking positions selected from random positions.
To address this, we apply a global normalization term to the predicted probability at each position. When p(z,g) ) the probability predicted by the model for the top-j candidate token at position within a decoding window of length L, global normalized probability is DGN (s:i)) = p(r.;) x Σ Σy p(as) Σ Ef
The normalization term of G. (Iiz) enhance properties: (1) the unmask probability of step
remains. (2) Compared to position-wis wise normalization p(x,) > p(as)
Σ, p(z,)* global normalization reflect
confidence in top-ranked candidates assigned by the model.
A.4 Justification Against the Bitter Lesson
R2FT's rule-basee annotation could be criticized as going against the "Sutton's Bitter Lesson," which argues that approaches relying on manually designed rules tend to be short-lived. We can provide the following justification for this point.
(1) In R2FT, the rule-based component only used for data construction, while the learning lies with the model itself. So R2FT is closer to SFT modeling than rule-based system. Also, we provide the theoretical justification for why this rule-based annotation is safer than other, more natural-looking options in $4.2.
(2) R2FT can be viewed as low-cost approach to human data annotation. Creating datasets for desired model behavior has been a guaranteed approach in LLM development. Research fields are further mining data to prevent unwanted behaviors. However, as "deconstruction is always much simpler than construction", unwanted behaviors often be easily reproduced through simple rules, which is R2FT.
datasets to broaden model coverage. Likewise, can annotate "undesired"
B Details on the sampling speed
B.1
The Illusion of sampling speed
The sampling speed of diffusion LMs reported in previous work is often overestimated, due to overseeing the following point: the gap between default decoding window size (L) and the number of content tokens without EOS tokens (we denote this as L*).
For example, previous work [29, 2] set the decoding window size 1024 and the total steps also to 1024, generating one token per step on average. This is regarded as the same speed of AR in those
2To avoid those three tokens decoded simultaneously (resulting in structural breakdown), possible strategies include 1) reducing T (e.g.. semi-AR, Conv), 2) lowering the sampling rate, or 3) introducing revocability.
17

---

works. However, in practice, this assumption does not hold. The token for actual content (L*) is about 200-400 tokens on average, and the rest are filled with EOS tokens. While AR would need only 200 steps from generating 200 tokens, diffusion LM needs 1024 steps to generate 200 tokens, making it about 5 times slower than AR. Considering the difficulty of implementing KV-cache in diffusion LMs, the actual computation speed is likely even slower than this.
Therefore, fairly claim a speed advantage over AR models, the default inference step count should be set much lower than L. So we adopt 128 as the default step size in our work, even though most diffusion LMs have reported significant performance degradation this small step size [29].
B.2 Solution for further acceleration
We have shown that, with R2FT and top-k decoding, diffusion LMs can achieve text generation quality comparable to AR models even with a very small step size of S = 128. However, the following methods offer additional speed advantages beyond this baseline.
Caching We adopt the caching method proposed by [36]. As shown in the Equation 4, decoding in diffusion LMs is inherently a random sampling process, with the multiplier of unmask probability di increasing over time (f). Therefore, when the total number of time steps (S) is large, by chance 1-04 some steps may result in unmasking. In such cases, caching reuses the model output from the previous step instead of performing a new inference, effectively reducing computation. Importantly, this strategy does not degrade generation quality, as the model is not trained to perceive time variable t in the continuous time-step assumption. However, when small, caching rarely occurs.
EOS-fill We introduce EOS-fill, method designed for unidirectional generation scenarios that is synergetic with caching. During response generation, when the model outputs an end-of-sequence (FOS) token, EOS-fill fills all subsequent positions to the right with EOS tokens. This significantly reduces the number of masked tokens, increasing the number of cacheable steps and therehy reducing the total count of model inference. This method not only further reduces the number of inference steps even with small S (e.g.. 128), but also leads to improvements in generation quality, by removing the distracting tokens on the right side of EOS token ($5). However, this method is not applicable to bidirectional generation settings.
B.3 Detail of experiment on the sampling speed
We evaluate the speed advantage of ap- plying cache and EOS-fill in $5. The comparison includes four baselines: MDLM with cache, MDLM with EOS. fill, MDLM with both, and standard AR decoding. All MDLM variants use top-k decoding; however, similar results are observed with other decod- ing methods such as categorical and LLADA decoding.
Table 5: Sampling speed of small and large baselines during generating responses for the AlpacaEval benchmark.
(a) Small
fill
1 (ypi8)
Lourns
Li
X
108.4
128.0
MDLM
127.9
MELM
1024
128
119.1
431
114
45.4
2.51
(b) Large
In Table 4, L denotes the decoding window size, and S represents the total number of diffusion steps. L refers to the content length, numbers of sampled tokens excluding EOS tokens, while S indicates the actual number of model inference steps without caching. u shows the content size measured by GPT-4, to reflect the actual answer length.
Backhone
FOS-6Il
(pp14)
Fours
127.9
MDLM
1169
To prevent from the variance in L overly affecting the measurement of sampling speed, we use normalized content length Li which is defined as the average of the three L" values from MDLM baselines. Finally, the sampling speed is defined as tokens per step Liamm SS* the normalized content size divided by the actual number of inference steps. This normalization is intended to avoid the statistical illusion discussed in $B.1. To maintain the assumption that AR generates one token per step, we set S of AR equal to Lorm
18

---

Results We observe the following trends in the experiment of Table 5: (1) If EOS-fill is not applied, there little difference in speed between MDLM without caching and with caching, mainly because the total step size S is too small for caching to take effect. (2) In contrast, applying EOS-fill significantly improves the sampling speed. Especially for the small model, the actual number of inference steps 5* drops under half of the S. This is likely because small models tend to generate shorter answers than larger models. (3) Large models produce longer content on average (about 341 tokens), which naturally results in higher sampling speed. Nevertheless, EOS-fill still brought additional speed improvements.
C Details on baselines
C.1 LLADA decoding
In this section, we describe LLADA decoding as proposed by [29], and explain how it differs from categorical decoding and top-k decoding. According to [29], LLADA decoding can be summarized as the following steps. Given total decoding- -widdoww size and the step size S, (1) first computes the number of tokens to be sampled at each step = 1 s E N. (2) At each step, unmask positions, selecting them in descending order of confidence scores. Additionally, the semi-AR strategy is applied by default.
The key difference from categorical (also top-k) decoding from [36] lies in how the number of tokens unmasked per step (i.e., s) is determined. In categorical sampling, the number of unmasked tokens per step is flexibly determined based on the model's inferred probabilities. In contrast, LLADA decoding fixes this number (s) uniformly for every step, making the unmasking process more rigid and less adaptive.
LLADA decoding performs well when s = 1 or s = 2, where its rigidity is not a significant issue ([29] sets S 512, L 1024, resulting in 2). However, when becomes larger (i.e., when S is small), its performance degrades substantially, likely as a consequence of this rigidity. Therefore, when speed-up is required, top-k may be more suitable choice, which flexibly determines for each step.
D Details on experiment setup
D.1 Evaluation setup
We primarily use AlpacaEval [44] for benchmark evaluation, which measures the win rate of model's response compared to that of a reference model. The reference model is an ARLM of similar size, fine-tuned with the same epoch on Alpaca instruction dataset and the same setting of LoRA adapter. For small size (180M) models, we use the 180M AR checkpoint provided by Sahoo al. [36], further fine-tuned, For experiments based on LLaDA 8 Ba se, the reference model 1LaMA-3-88R BBsee, further fine-tuned with LoRA. All AR models were decoded using nucleus sampling with p = 0.95. In addition, we report G-eval results using gpt - 4--44ttuu -turbo as the evaluator.
D.2 Compute resource detail
For small models we mainly conducted experiments on, we used NVIDIA A5000 (24GB) GPUs. The per-GPU batch size was 4. SFT required approximately 190 x GPU-minutes for 35 epochs, with an average GPU memory allocation of 60R2FT required 20 x 8 GPU-minutes for 300 steps, also with 60% memory usage.
For large models, we used NVIDIA A6000 (48GB) GPUs. The per-GPU batch size was 1. SFT took approximately 220 x 8 GPU-minutes for 800 steps, with 80% average memory allocation. R2FT required 90 x 8 GPU-minutes for 200 steps, with 100% memory allocation.
For SFT only version and SFT R2FT version, total training steps were set to be equal.
19

---

E Experiment with CoT setting
We conduct additional experiments under setting designed better reflect our scenario: LMs generate fong-form text without relying on random sampling.
Experiment settings We instruct models to generate responses Chain-of-Thoow (CoT) format to provide detailed reasoning, and we apply top-5 for more deterministic decoding. As average response length increased, we set $=128, which still guarantees reasonably fast (around 3x) sample rate. Since the small model did not understand the CoT instruction well, conducted experiments only on the large model. We do not report the cength-controlled win rate (LC win rate), since it penalizes long responses, whereas possible responses. We report experiments and analysis on decoding methods with revocability (e.g., ReMDM [45]). The block size and kernel size (Lt) are fixed 128, unless otherwise specified. Although AlpacaEval showed particularly high scores at L = 256, overall benchmark performance and stability in answer length were best with L,- 128.
setup explicitly encourages generating the longest
Metrics Alongside AlpacaEval, which measures alignment, we also report metrics for structural coherence (e.g., inlier rate). Structural coherence is typically assessed using generative perplexity (gen PPL). However, gen PPL known to overestimate texts containing repeated patterns [18], which makes it easy be gamed. Accordingly, we computed the mean (µ)) and stds (a) of gen PPL from the training dataset, and then measured the proportion of mnnel-ggnerrtted texts whose PPL falls within the interval μ 2o. Additionally, we report the portion of outputs that have non-zero length before the EOS token ("len > 0" in tables), since the absence of such content indicates another incoherence pattern. Responses with zero length are automatically regarded as outliers.
Table 6: Comparison between large (8B) MDLMs on AlpacaEval. LLADA, semi-AR, and Conv, if without comments.
512, 128. L, = 128 for
RFT
Decoding
Win rate
Avg length
len > 0
inlier (95%)
semi-AR
21.50
265.5
0.97
0.35
categorical
37.98
360.1
1.00
X
ReMDM + top-k
41.64
358.1
0.97
0.95
X
RcMDM + Conv top-k
42.12
470.5
1.00
1.00
X
LLADA topk-k: + repetition
39.18
405.1
1.00
0.75
X
36.14
340.5
0.96
0.95
top-k
42.48
354.5
0.97
0.95
Conv + top-k
46.99
354.8
0.99
0.99
semi-AR
26.97
452.5
1.00
LLADA
47.37
394.8
1.00
0.82
categorical
50.22
403.5
1.00
1.00
ReMDM top-k
51.68
472.3
1.00
1.00
ReMDM + Conv top-k
50.41
439.9
1.00
1.00
top-k Conv top-k
55.38
459,8
1.00
1.00
$7.38
454.2
1.00
Conv (Le = 256)**)) + top-k
67.73
320.1
1.00
1.00
Table 7: Comparison between large (8B) MDLMs on AlpacaEvval. I-=512, S=128 The inlier criterion is set to the central 95%
MT Bench
Wiki
Arena hard auto
R2FT
Decoding
Win rate
Inlier
Win rate
Inlier
Win rate
Inlier
categorical
26.19 (4.42)
1.00
44.66
(1.75)
0.97
27.41 (1.79)
0.98
LLADA
17.16 (3.80)
0.76
39.03 (1.71)
0.62
28.75 (1.85)
0.76
LLADA
36.91 (5.07)
0.91
45.66 (1.73)
0.73
35.34 (1.93)
0.87
categorical
24.64 (4.35)
1.00
56,44 (1.77)
1.00
34.77 (1.91)
1.00
X
Top-k Glob
26.13 (4.64)
0.97
47.42 (1.80)
0.97
30.11 (0.95)
0.95
Top-k + Glob + Conv
32.80 (4.87)
1.00
50.27 (1.79)
33.32 (1.91)
0.99
Top-k + Glob
34.11 (4.93)
1.00
63.77 (1.76)
0.99
40,25 (2.02)
0,99
Top-k Glob Com
36.59 (5.18)
1.00
64.74 (1.72)
0.99
42,47 (1.99)
0.99
20

---

Table Comparison between LLADA- Ins.-r-nntttutttt on MMLU. 512, S 128.
Decoding
La - 32
L 64
La
128
L.
256
La 512
LLADA
40.89
42.54
34.61
20.24
17.85
LLADA Conv
45.94
46.31
41.13
27.94
22.33
top-k Conv
54.13
54.16
54.65
54.02
53.21
Table Comparison between LLADA
on GSM8k, 512, = 128.
Decoding
la = 32
14, =
1,, = 128
1,, = 256
L. = 512
LLADA
48.82
50.72
47.38
26.16
21.30
LLADA Conv
51.18
51.48
50.72
44.88
26.76
top-k + Conv
51.71
48.75
44.73
43.37
40.03
E.1 Results
The experiments under the CoT setting are consistent with our previous results in $5. And it further support our hypothesis: The categorical baseline achieves very high structural coherence (as measured by inlier rate) but suffers in alignment (as measured by win rate). Top-k sampling improves alignment but degrades coherence. contrast, both R2FT and Conv improve both alignment and coherence.
Effect of repetition To clarify the advantage of R2FT. which mitigates the sampling of repetitive tokens, we deliberately inserted a meaningless repetitive phrase (c.g.. "Response:") at the right end of the window. This setup was designed to simulate worst-case scenario in which meaningless repetitive phrase sampled at an early step. As a result, we observed a notable degradation in alignment and structural coherence across the generated outputs ("top-k repetition" in Table 6).
Reasoning tasks The consistent superiority of Conv was observed for reasoning tasks as well. When Conv was applied, achieved relatively strong and stable performance regardless of the kernel size. At 5 = 64, Conv maintained its performance, whereas LLADA performance dropped sharply. This demonstrates Conv's robustness with respect segment size.
E.1.1 Analysis other baselines
Table 10: The effective sample rate (r*) with respect to step size (S). Here, r" L"/S, where L denotes the to- tal number of unmasked tokens.
(1) ReMDM: ReMDM [45] introduces revocability into MDLM, and in theory. it is expected to yield better perfor- mance. However, the results show that ReMDM does not lead to additional performance gains and, in some cases, results in degraded performance.
L
S
r* of ReMDM ys of MDLM
512
65
12.9
8.0
As discussed in $F2, we attribute this to the fact that revocation incurs additional cost (i.e., larger sample rate).
512
128
8.6
4.0
512
256
6.6
2.0
In general, a larger sample rate implies a higher risk of harming structural coherence ($G.1). 1). And revocability has similar effect of raising the sample rate, since it re-assigns tokens be unmasked again. To investigate this, we compared the average unmasking per step (denote r*) of both revocable MDLM (i.e., ReMDM) and irrevocable MDLM. Table 10 shows of the revocable model is 2 3 times larger than that of MDLM, equivalent to operating with a step size 2-3 times smaller in MDLM. As sample quality is decreasing function of r*, this the direct cause of the performance degradation of ReMDM over MDLM in Table 6.
(2) sliding annealing: As the sliding annealing approach of [12] appears similar to Cony, also experimented on this method, implemented in an irrevocable MDLM manner. However, we found that its performance severely degraded under the L 512, S 128 setting. Structural coherence was significantly broken, so AlpacaEval winrate is near zero.
We suggest the following rationale for this observation: The work [12] appears to mainly assume the case of r = (where r = L/S), under which slide annealing can behave like AR. However, when r gets larger, the behavior shifts to resemble autoregressive decoding with stride of r, particularly during early steps when the decoding window empty. more intensive settings like ours (r=4 or 8), this leads to very rigid sampling, forcing the unmasking of 4-8 continual tokens at once. This
21

---

is prone to cause structural incoherence (due to independency). and once such case occurs in early steps, propagates through later steps, resulting the breakdown of the entire output.
To state it more in detail, the implementation of sliding annealing divides the "active" window into two distinct regions (the size of window is denoted in Figure 3(b) of [12], but we refer it as Lt). This property arises from the fact that the active window must shift by exactly r = L/S positions to the right at each decoding step.
(a) The former to r positions: No masks are allowed to remain in this region after one decoding step. Otherwise, those masked tokens may never be resolved in later steps.
(b) The latter r + to La positions: They may contain masked tokens after one decoding step.
In our theoretical analysis on performance guarantees ($G.I), we assumed that the property of structural coherence degrades the density d = 5 increases. From this perspective, part (b) satisfies W the guarantee because it operates at a sufficiently low d, similar to Conv. contrast, (a) operates at a much higher which violates the theoretical guarantee, especially when gets larger. particular, during early decoding steps (when few tokens have been unmasked), d reaches its maximum value of 1. This is where the key difference from Conv arises, and where the breakdown of structural coherence is most likely to occur.
F Related works
We first provide an overview of the general domain related to our work. We then discuss in detail the two lines of diffusion LMs: the absorbing type and the uniform type.
F1 Overview
Diffusion language modeling The foundational work of D3PM by Austin al. [3] formulated diffusion processes for discrete data by defining the forward noising process as sequence of transition matrices over categorical distributions (Equation 2). This formulation enabled the adaptation of denoising diffusion probabilistic models (DDPMs) [17, 41, 10] from continuous domains (c.g.. images) to discrete domains such as text. This breakthrough inspired wide range of follow-up studies [19, 15, 27, 34, 43, 22, 48, 52, 14, 25, 12, 45, 4], which proposed various designs for the denoising model and decoding strategies.
More recently, Sahoo et al. [36] provided a theoretical simplification of the D3PM objective under continuous-tine and full-masking assumptions, arriving at loss function (Equation 3) equivalent to the masked language modeling (MLM) objective used in BERT [9]. but with diffusion forward process. This formulation has become the de facto standard for diffusion LMs, adopted by several concurrent works including LLADA [29].
[2], and MD4 [38]. Throughout this paper, we refer to this family of models as MDLM (Masked Diffusion Language Models).
Decoding in diffusion LMs Diffusion LMs generate texts by iteratively sampling from model- inferred probability distributions of vocabularies, following the reverse diffusion process. In MDLMs, this process is simplified via SUBBSparameteriization [36], which interprets the fully noised state masked decoding window and denoising procedure as unmasking it. This formulation, now standard in recent works [2, 38), enables efficient training and stable convergence. LLADA [29] adopts similar but slightly different procedure, where the number of tokens decoded per step is fixed to a predefined constant.
Recent works [29, 2, 36] suggest an additional decoding method, semi cemi-autoregressivee (semi-AR) decoding. This involves partitioning the decoding window into blocks and sequentially sampling each block.
Repetition in language models Repetition has also been an issue in AR LMs. Holtzman et al. [18] identified that AR LMs tend to generate repetitive or generic outputs in deterministic decoding. To mitigate this, various strategies have been proposed, including more stochastic sampling methods (e.g.. nucleus sampling [18]), repetition penalties [21], and unlikelihood training [47]. While unlikelihood training is similar mitigating repetition, the objective lacks penalty terms 89 as Equation 6, which enable automatic handling of unlikable negatives and suppression of high-prio prior tokens. Reinforcement
22

---

learning with human feedback (RLHF) [30] is also another approach that aims to mitigate such unwanted generation patterns by aligning model outputs with human preferences.
F.2 Absorbing vs Uniform
As framed in previous works [3, 25], discrete diffusion LMs can be broadly divided into two types according to their unmasking strategy: absorbing and uniform. (1) Absorbing: The denoising process is defined as unmasking masks into tokens. Once mask is unmasked to tokens, it is not revised to other tokens. This family referred to as MDLM, or mask diffusion. (2) Uniform: A token that has already been unmasked can still be redefined noise, and thus remains revocable. Revocability also leads to more contextual dependency between decoded tokens.
In this work, we concentrate solving the problem of unrevocable MDLM first. This is because we anticipate that the absorbing family will remain the dominant paradigm within the ongoing line of diffusion LM research, and that revocability and dependency may be incorporated in way compatible with the absorbing paradigm. The rationale for this expectation is as follows.
Rationale for why absorbing faailly seems to remain dominant
(1) Performance is already proven. First, the absorbing baarbing-bbsedd approach has already proved its downstream task performance comparable AR LMs [29]. Regardless of its issues, method that already works this well is unlikely to be entirely abandoned. In contrast, uniform methods have been mainly evaluated auxiliary metries such gen PPL [24, 25, 12, 45J. We believe this stems from the following theoretical limitations.
(2) Revocation introduces an additional cost. As the sample quality of diffusion LM is function of sample rate = L/S, revocability increases the number of masks to decode, similar to raising leading to degradation downstream task performance. We demonstrate this in $E.1.1.
(3) Irrevocability may not be critical property. As noted in [25], people often consider vocability as an inherent and necessary property of diffusion models (DM), thinking of the image generation, transforming existing image into entirely different one (e.g., person eating bread + a lion eating bread).
Output
Visual prompt
revocable
Severely noised at Oth step
Not much revocable during the denoising
Figure 14: the process of image editing with latent diffusion [35], the transformation from visual prompt to the output quite substantial, giving the impression that diffusion method generally has high degree of revocability. However, after severe noising the initial step, the subsequent denoising steps are largely anchored to the previous steps. Once points, lines, or silhouettes are established, they tend not to change.
However, closer look at the process reveals that this is not as flexible as the entire output seems (Figure 14). To replace person with lion, the DM first degrades the original image into almost
23

---

unrecognizable noisy state. Then, during the denoising phase, the model gradually reconstructs. Importantly, once strong edges or dots emerge certain location in the early steps, these elements tend to remain stable, functioning like anchors. Then, surrounding blurred regions progressively refine around these anchors, determining how to interpret the anchors (e.g.. as an eye or a nose). Thus, while the overall process is revocable after the initial destruction, subsequent steps exhibit limited revocability, This may be related to the fact that the reverse process, following DDIM [39], is more deterministic than explorative.
Likewise, existing successful generation models (e.g.. AR) are not revocable. It may be that well- calibrated sampling can provide an enough solution.
(4) If revocability is still needed, a compatible way is promising. [25] provide theoretical justification for this compatibilityy In [25]. the diffusion posterior p(z]22) can be decomposed as p(=]),)) p(N la p( N), where N is noise. The "uniform" approach models the entire, which theoretically complete but practically beyond the model's capacity. In contrast, the "absorbing" focuses only on modeling p(r/2,, N), which is theoretically incomplete but highly practical.
As alternative, [25] proposes operating p( p()) independently, allowing better control the denoising trajectory while preserving the practicality of the absorbing paradigm. Remasking (ReMDM [45]) also lies in this context. As long as absorbing models can operate independently, Conv and R2FT remain fully compatible, as also empirically shown in Table 6.
G Theoretical guarantee
We provide theoretical explanations for why our methods (Conv, R2FT) gain performance.
G.1
Conv
We provide two theoretical explanations for why Conv may outperform semi-AR. The following discussion assumes a mask (i.c., absorbing) diffusion type [25, 3].
G.1.1 Violation of the training assumption
This point is discussed in detail in $4.1. Semi-AR splits given timestep S into b segments, explicitly raising éd to b times larger, moving far from the infinitesimal assumption of the training object (Equation 3). In contrast, Conv preserves the entire S while narrowing the window through standardization. Of course, Conv does not perfectly match the condition from training either; as a result, performance degrades when the kernel size is excessively small (Figure 8). Nevertheless, it is far more robust compared to Semi-AR
G.1.2 Explanation with the hazard function
A more intuitive explanation is that Cony is more flexible than semi-AR, granting the model greater freedom. This can be theoretically justified using the hazard function, which measures the probability of corruption.
Let us follow the notation $3.2: L as window size. S step size, b as the number of blocks, La as the block size or kernel size, and S₁ as the number of steps assigned for each block in semi-AR.
We newly define the sample rate T.SS, and W as the number of mask tokens that the model have to sample from at current step. For example for default diffusion LM, W L. holds at the first decoding step, and W ~ L previous step, on average. For semi-AR, it is La to La -r.
the second step, since the model might have unmasked masks at
We further define the hazard function P, representing the probability (or degree) that the final decoded text becomes structurally (grammatically, syntactically) harmed. Conversely, 1 P representes the probability of being unharmed (i.e., structurally coherent). We define Q = log - P). And let small Pt denote the probability that the final output becomes harmed due to the decoding at intermediate
timestep f. Here, the hazard function is commonly defined as = 1 - (1 - -p:), which in turn Q ΣS Di= log(1 1 Pt). We assume that p. depends on both and W. thus denote p: pr
24

---

Assumptions about the factor in the quality of a text
We assume that the quality of generated text is related to density of decoding (denoted as d = r in two opposing ways (though there can be more other factors). Raising d can be benefit and harm the quality at the same time.
(I) Alignment: When the positions to be decoded are far from the given context, the model tends to suggest less aligned candidates-22 phenomenon referred as the LDW problem. This situation often associated with low (though not always), which indicates lots of masked positions are between the given context and the targeted position (e.g., often in early timestep). A common strategy to mitigate this issue is to increase d by reducing W (e.g., semi-AR, Conv).
(2) Structural coherence: Conversely, when d w becomes too high, it can reduce the dependency among decoded tokens and lead to structural degradation. This effect stems from a key property of mask diffusion: multiple tokens generated simultaneously cannot attend to one another at the output layer, resulting in low inter-token dependency. Consequently, when many tokens are generated within narrow span, the likelihood of syntactic conflicts increases.
However, when the distance between two generated tokens z; and æj large "enough", even if their immediate dependency is weak, the number of possible connecting sequences between them grows exponentially as [V]G-() (V: vocab size). This implies that later decoding steps have the opportunity restore structural connectivity. This is why pt depends and W. If large and small W mean too packed generation, thus low Pt.
We formalize this property as follows:
r1 = r2 and W1 > W2, then Pr, (Wi) S Pro (W2)
Equality approximately holds when r sufficiently small or when both W1, W2 are sufficiently large over some threshold. The precise threshold for "sufficiently" and the shape of the curve (,WW) as function of W may vary across models.
Comparison across decoding types
From the two factors discussed earlier, using the hazard function across different decoding strategies, leaving (1) aside for simplicity.
focus (2) structural robustness and model it
(a) Default mask modeling: For a standard mask diffusion model without narrowed decoding window, the hazard function Qo as follows.
P 1 (1-p (t (t- = 1- 1111--11111 (p (t.r))
(8)
Qo (t-r)) q= 1og(1-p)) -
=trr
(9)
9- (-(W) represents the log probability that decoding at a step does **not** introduce corruption. W decreases gradually by per step.
(b) semi-AR: For semi-AR, when is divided into 4 blocks (i.e., Lu = 128):
Qua = b Σ (t.r) =t
(10)
To compare, since S > Sb, the value of (W;) for semi-AR at any given step is strictly smaller than that of the default setting (because W. < La L for semi-AR).
Therefore:
Qu. - Qo
Equality occurs when the block size La approaches L, or when S is large enough so that the argument of p-W) exceeds threshold for most steps. This theoretical behavior aligns with the empirical observation in Figure 4, where the evaluated structural quality increases as Ls or S are raised.
25

---

(c) Conv: For Conv, assume the kernel size L3 128. In this case, at every step we have W 2 128. Although the actual variation of W more complex, for simplicity, we assume W 128.
128/r
Q. = ( 128) q (128)
Σ q(trr)
(11)
Here, the second term is shared with semi-AR and the default. but the first term is always larger than that of semi-AR, except at the start point of each block.
Therefore, 2a 2 Qs
As before, equality approximately holds when S is sufficiently large.
Compared to the default mask diffusion, the first term is always smaller than the default, so we have:
Qc S Qu-
Equality holds when the fixed kernel size is "sufficiently" large, so that the effective W over the threshold. This behavior is consistent with the empirical observation shown in Figure 7, where kernel size over some threshold yields structural stability.
To wrap up, In terms of structural robustness, generally observe the ordering:
semi-AR conv < default
(12)
However, from the perspective of alignment, the relationship reverses:
semi-AR, conv > default
(13)
Consequently, adopting Conv offers a favorable trade-off, as it provides substantial gains in alignment while maintaining significantly better structural robustness compared to semi-AR.
G.2 R2FT
We divide the explanation into two steps: the decoding step and the training step.
G.2.1
Decoding step
(This part is variation of $A.2.1 with minor modifications.) First, let us describe the scenario where the issue arises. A key characteristic of diffusion LMs decoding tokens that are far from the immediate context (LDW problem). To formalize this, consider a scenario in which the model decodes #21, Z222 Z22 the first sampling step, where x; denote a mask token that is i-th position from given context. The probability of candidate for 221 is (c|r) = Fr (x([(r) where E T all possible combination of {x1, x:2, ..., x:203. The cardinality of T is 20 (V vocab size), the infinity. Therefore, the posterior TTc r) converges TTc).
p(c) represents the probability of candidate that can be easily predicted without any contextual information. Typical examples include high-prior tokens (e.g.. function words) or simple repetition of given context. The same applies to 222 and Ø23. Consequently, illustrated in the upper part of Figure 11, top candidates often include patterns such as "Question Question Question" (repetition) or "the the the" (high-prior tokens).
The motivation of R2FT is: if guessing is inevitable, let be at least more aligned. Formally, this is training the model to reduce TC)) in (c)] By doing so, the probability mass shifts toward tokens with higher average likelihood within T. In other words, we choose the candidate patterns like "president president president", which least contains more contextual information than "Question Question Question". This is consistent with our empirical observation on AlpacaEval that R2FT achieves better alignment with the query.
G.2.2 Training step
How the model reduces (C) during the training step is described $4.2.
H Additional explanations
H.1 Bidirectionality
We provide more explanation the bidirectionality of diffusion LM.
26

---

(1) Diffusion LM has bidirectional attention. To provide clearer explanation, we distinguish between "bidirectional attention" and "bidirectional generation", AR models inherently employ unidirectional attention in their architecture, whereas diffusion-based LM architectures leverage bidirectional attention, which applies both Conv and semi-AR, Therefore, even when diffusion LMS perform unidirectional generation, they continue to maintain locally bidirectional attention. This applies to both semi-AR and Conv variants.
For example, consider the prompt:
"Who is the president:
If, in the next step, a token happens to be sampled in the middle, resulting in:
"Who is the president:
is
then the masked positions between "president:" and "is" will attend to context on both sides (e.g.. "president:". "is"). Thus, bidirectional attention occurs in local level.
This behavior can be observed Figure 7(a), where sampling does not proceed strictly sequentially as in AR but occurs in scattered manner. The mask tokens between scattered tokens are applied with bidirectional attention. This bidirectionality is a difference from AR models.
However, this situation (activating bidirectional attention) is largely byproduct of the model sampling multiple positions in a scattered manner, which might provide little benefit in most of the existing LLM tasks. This is because, in most LLM tasks (e.g.. QA), useful information is mainly provided on the left side the window. In fact, this property can even introduce the LDW problem, where tokens sampled far from the given context become misaligned, forcing harmful bidirectional attention.
(2) Conv is capable of bidirectional generation. "Bidirectional generation" refers to the overall direction in which generation is globally grounded. Semi-AR enforces strict rule to proceed from left to right, whereas Conv, when context is provided on both sides, automatically grounds generation multi-directonnally. This distinction is illustrated in Figure 7.
Tasks that provide context on both sides are goal-oriented, as explained $6. For example, negotiation requires aligning with both previous utterances and desired responses. And even in QA tasks, desired responses or scores can be framed as explicit goals. This naturally offers way reinforce L.MSs toward alignment with annotated preferences. This is also the scenario where the bidirectional attention of diffusion LMS provides a clear benefit. We helieve that solving such tasks represents one of the key motivations behind the growing interest in diffusion LMs.
However, as noted in $6, such bi-context tasks are currently rare- likely because most LLM tasks have historically been designed for AR-based models. If diffusion LMs gain broader adoption, we anticipate that more of these tasks will emerge.
27

---

I
Samples
[Context]] Who Abrahamm Lincole? At"
MDLM
"The American poop
upon
Answen: Lincoln. question: Is Abreham Linco
He wes President the United States of America for 30 years until his death 1895. In those years Lincoin oversaw pettd the American Givil War and prosided ouer the American Prograssive Fra...
otherwise unlnowwn game of history, Lincoln once
Answer: Yas. Question: What questions can you
said monograph to cart, agol8. (Loft to right]
answer to Obema' Anewer: You are Aoreham ( Lincoin? Answer: Unooin; quastion: Barack Obama...
Lincoln walts, above, to isten to speech by his..
Figure 15: Response samples for the same query from different decoding strategies.
L11 Samples of ours
L.11 MDLM (small), SFT R2FT, L=1024, S=128, Top-k (k=20), Conv (kernel size= 256), EOS-fill
Benchmark Alpaca Eval [Instruction] Who is Larry Page?
[Response] Larry Page is an American-born long-time technolegy en trepreneur who has been on the boards of several prominent high- profile technology companies. Page is known for improving the discoverability of websites by providing formation. Page also known for his expertise in optimization technalogies, such as search engine optimization and optimization, He is also known for his technical expertise in Linux operating systems and AI AI-powered appl applications.
greater breadth of in-
Benchmark Alpaca Eval [Instruction] List the concepts that should be learned before approaching the given complex concept.
Deep Learning
[Response] Deep learning should first learn various concepts, such as model regularization, linear overfitting, rule-based minimiza- tion, detailed analysis of its outputs, and even the potential values of the parametors. Then, we should first train the model, identify the features of the data, and analyze the data. Finally, we should then test nodel by examining additional inputs and outputs, detecting new features, and perform further exploration.
Benchmark Wiki
[Instruction] Please give me an explanation about "Alice Coachman in 1948"
[Response] Alice lee Coachman in 1948 was middle-aged daughter of prosperous Jewish family. She was loving grandmother who worked to raise children and was strong advocate for civil rights. She entered the White House in 1948 for the first time history. She had once been separated from her family and was looking for à job but was making enough to fund her education. She seen learned far from perfect, and needed steady job to support her progress. She also worked hard through adclescence and developed positive relationship with her friends and family. Eventually, she managed to support herself financially, earning her fair share of success in the pusiness and government. She eventu- ally connectea with the Jewish community and gained recognition in the media. She remains an icon Jewish history and continues her work in politics.
that her education
28

---

Benchmark Wiki [Instruction] Please give me explanation about "Super Mario Sunshine."
[Response] super Mario Sunshine is a platform game that use plat- forming to travel around the world. Players must jump, climb and attack obstacles to explore the envircnment. Players must use plat- forms and platforms to change the direction the game. also collect coins or upgrades to help them progress through differ- ent levels. As they progress through certain levels, bosses will help them collect coins, and use other platforms to access them, or may collect bonuses to boost their speed. As they progress in each level, they must solve puzzles, as well break down enemies and clean up the environment. Lastly, players can unlock more con- tent, by collecting or trading coins and enemies to unlock ther. Players will also have to gain numerous other items, such their health, throughout the game. The experience can also be gained by the player furthering the different stages, increasing staring or speed, or increasing difficulty. Players can also gradually progress through new set of enemies, allowing them to defeat them using single
oostacles, to fight various kinds of player may
enemies,
L..2 MDLM (Large), SFT R2FT, L=1024, S=128, Top-k (k=20), Conv (kernel size= 256), EOS-fill
Benchmark Alpaca Eval
[Instruction] How do I wrap a present neatly?
[Response] To wrap a present neatly, is important to make sure you have a good paper roll and ribbon. Start by choosing the right type of paper for the present and wrapping, ensure that the finished size of the paper is slightly larger than the present.
Begin by folding the paper along the length of the present and fold- ing the paper to the inside of the box. when reaching the top of the present,
tape to secure the paper in place.
Roll the paper inwards, making sure to push the layers down to en- sure a secure wrap. Fold the paper twice, with each fold inside the previous one. This will prevent the paper from rolling unevenly, secure open.
and make the present
Now you can trim off any excess paper and adjust the fit to the box's s size. Trim the paper so that it is flush to the top surface of the present.
Staple the edges place to give the present a neatly presentable appearance. With a little practice, you will soon wrap present neatly and the gift will look as special as it should be a special box for special gift.
Benchmark Alpaca Eval
[Instruction] I like to host guests at my home from time to time, and am gathering recipes of different dishes and drinks to keep things interesting. I am interested in trying some Somali dishes. Can you give me a recipe for Canjeero?
29

---

[Response]
Sure! Here is recipe for Canjeero, popular Somali
dish.
Ingredients:
50g of flour
300g of chicken or lamb
200g of potatoes
tbsp of canjeero tomate paste
small onion, chopped
tbap of olive oil
Salt and pepper to taste
Instructions:
1. Preheat the oven to 185%0 (375 and add in the flour and set it aside.
2. Using non-stick pan, heat up the clive oil until het. Brown the chicken or lamb, cooking until browned and crispy. Add the chopped onion and garlic, and cook for an additional 5 minutes.
3. Remove the pan from heat and add in the tomato paste and pota- toos. Sprinkle the flour over the top and season it with salt and pepper.
4. let the dish cook for 40 minutes in the sionally
stirring occa- the dish from sticking together.
5. Remove the oven and let the dish cool for five minutes. Slice into small bite-sized pieces and enjoy with white rice.
Can also served with herbs and spices from the region or side dishes. Enjoy!
Benchmark Alpaca Eval [Instruction] Do you know why turkeys became the official food of thanksgiving?
[Response] Yes, it is believed that turkey become the symbol of Thanksgiving festival due to an important event in the history of the US and its surrounding region.
The first Thanksgiving festival, also known as First Thanksgiving, which took place in 1621 in Massachusetts consisted of Pilgrims inviting Native American tribes for a three-day meal to share the first harvest. The Indians brought turkey gratitude the pilgrims' help in preparing for the harvest. Since then, turkey has been the symbol of Thanksgiving in America.
expression of
Additionally, the origin of turkey Thanksgiving food can be traced back to Ottoman Empire, where the domesticturkeys originated. These types of turkeys were introduced to Europe in the early 16th century, and were brought to North America during the colonization of the continent
As the Thanksgiving festival spread all over the United States, turkey became symbol of the festival, and eventually, the es- sential food of Thanksgiving meal. Today, turkey is still widely celebrated as dish on the Thanksgiving holiday, which was created in 1774 during the American Revoluticnary War.
In conclusion, turkeys became a symbol of Thanksgiving after it was
30

---

part a three-day feast when the Pilgrins invited Native American tribes to northeastern United States, It then became associated with the Thanksgiving festival as that tradition spread throughout the United States.
Benchmark Arena-Hard-Aur
[Instruction] Tell me how to implement SCIM server in PHP using slim as router
[Response]
Below is example implementation of Flask
that provides ЕТTР basic authentication with user database for authentication:
from flask import Flask, request, flash, redirect, render_template from flask_babel import identity, auth
app Flask name__ __
@auth def ()entity if request request.authorizate authorization:
return identity () user ()
return None
def authenticate
if request.headezt has_kaas_keeee ( I'Authorization'))
username = request.. get_heederrruseeeeeee return username and request ')
('password
return None
route
def main(): if heauers. has ('Authorization") user authenticate if user: Dession['authen '] = True
return redirect (1/home')
else:
return render_ pender_template// Login. @app t'
def ()out():
return render_templed aabubt h)
_ame___ "__ain_"".
app. ()
How it works:
This server provides basic HTTF authentication, The user 's username and password are in the request's Authorization header. If the request includes the Authorization header, the user's identity is authenticated using the 'authenticata function provided. This function parses the request's 's "Authorization' header to retrieve the user's username and password. If the user is suthenticated, authentication is added to the 'user' session to
helper
31

---

prevent further urauthenticated requests and recirects to the home page. If authentication is not successful, the main function will render the 'login. html template. The main function has a route named '' that redirects to the home page, and the v/about route to render the 'about. template. This way, the home page is authenticated securely and the user can change it's password by editing the user's password.
L1.3 MDLM(Large), SFT, L=1024, S=128, LLADA decoding (stride=512)
Benchmark Alpaca Eval
[Instruction] How did us states get their names?"
[Response] The
is: The
their states
were
from either the and, and, and, and, and, and and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and,
and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and
1.2 Failure cases
We present the failure cases of other baselines. However, we also note that similar failures occasionally occur even when our method applied. The main cases include the repeated appearance of high-prior or repetitive tokens, or responses that terminate prematurely. Nonetheless, applying our method reduces the likelihood such failures ($E). It seems that diffusion LMs still struggle to fully control these failures due to their stochastic nature and the LDW problem.
1.2.1 MDLM(Large)) SFT, I=1024, S=128, LLADA decoding (stride=512)
Benchmark Alpaca Eval
[Instruction] How did US states get their names?"
[Response] The answer is: The names of their states states were from either the and, and, and, and, and, and and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and,
and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and
I.2.2 LLADA-9B-Instruce (stride=512), L=1044
L=1024, S=128
Benchmark Alpaca Eval
[Instruction] like host guests my home from time time, and I am gathering recipes of different dishes and drinks to keep things interesting. I am interested in trying some Georgian dishes. Can you give me a recipe for Kubdari?"
32

---

[Response] Certainly! Kubdari is a traditional Georgian dish made from beef, Kub, and, with with herbs and spices. Here's s a simple recipe for a delicious recipe:-- Kubi Kub Kubdari
Ingredients:
-1 lb
33

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The paper identifies the long recoding-windood (LDW) problem in diffusion- based LMs and proposes methods (Conv decoding and R2FT) solve it. These claims are clearly supported by empirical results and analysis throughout the paper.
Justification:
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation section contains a discussion of our method's limitations.
Guidelines:
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The paper provides details on decoding settings (e.g., L, S, kernel size, block size), evaluation metries (G-eval), and training protocols (SFT, LoRA). Anonymized code and implementation details are provided in the supplementary material.
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide URL for the anonymized code.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies decoding window size, step count, kernel/block sizes, model sizes, finetuning protocols, and dataset splits in both main text and appendix.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For AlpacaEvval, we report error bars using standard deviations provided by the official AlpacaEval evaluation interface. This enables appropriate estimation of variability in win rates across different decoding methods.
8. Experiments Compute Resources
34

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper reports model sizes, GPU types, batch sizes, and total training steps.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:ps/////usss..e.eeee http:///.......ooottttttoo.eeee.
Answer: [Yes]
Justification: No sensitive or private data is used. All datasets are publicly available, and no high-risk models released.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work focuses on improving decoding algorithms for diffusion based language models, and is primarily technical in nature. While broader impacts the main focus
Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper does not introduce new pre-trained models or datasets that pose a misuse risk.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper clearly references and builds upon public models and datasets (e.g.. AlpacaEval, LLADA), and proper citations and licenses are included.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The proposed decoding algorithms and R2FT implementation are included as anonymized code with documentation and README.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No human subjects or crowdsourcing were involved in this research.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
35

---

Answer: [NA]
Justification: No human subjects were involved in this research.
16. Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: the core method development this research does not involve LLMs as components.
36


Reviews:
[{"review_id": "jMHt7HPB6b", "reviewer": "Reviewer_7TXB", "summary": "This paper works on diffusion-based language models and the long decoding-window problem. The authors propose two new techniques: Convolutional Decoding and Rejective Fine-Tuning. These methods improve speed, and bidirectionality without block-wise compromises. The paper provides strong empirical results on AlpacaEval and other benchmarks.", "questions": "-- Can Conv methid generalize to structured generation tasks (e.g., code or math)?\n-- Could R2FT interfere with learning domain-specific high-prior tokens?\n-- How sensitive are the methods to kernel size or corruption rules in R2FT?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n-- Clear motivation and problem formulation.\n-- Proposes simple, effective solutions that do not need architectural changes.\n-- Strong experimental results, especially in open-ended generation tasks.\n\nWeaknesses:\n-- Paper is dense and somewhat hard to follow in parts. Lots can be done here to make it easier to follow and less dense.\n-- Weak investigation of bidirectional downstream tasks, despite bidirectionality being a central claim.\n-- Some response samples still show factual or stylistic oddities.\n-- Evaluation on tasks like reasoning or structured output is limited.\n-- Paper is too packed. Figures are too small. This limits readability and potential impact of this work.", "was_revised": true, "final_justification": "Thanks again for the hard work and well executed response.\nI maintain my score due not limited evaluation and benefits of bidirectionality.", "rebuttal": "Thank you for raising such valuable questions that make our work more robust. We will incorporate all related discussions into the camera-ready version.\n\n---\n\n**[Q1] The paper is dense and somewhat hard to follow in parts. Lots can be done here to make it easier to follow and less dense. Paper is too packed. Figures are too small. This limits readability and potential impact of this work.**\n\n= We appreciate your observation. The formatting likely became dense as we attempted to cover all the necessary details. We will improve readability in the camera-ready version.\n\n---\n\n**[Q2]  Some response samples still show factual or stylistic oddities.**\n\n= That is correct. While our method improves performance over previous approaches, diffusion LMs still face several inherent limitations—most notably, sensitivity to the step size ($S$) and the lack of revocability. The latter issue has been partially addressed by recent work such as reMDM [1], which introduces remasking strategies, that are also compatible with our approach. Exploring such limitation remains an important direction for future research.\n\n---\n\n**[Q3] Evaluation on tasks like reasoning or structured output is limited.  Can Conv methid generalize to structured generation tasks?**\n\nHow sensitive are the methods to kernel size?\n\n=  Conv generalize to structured generation tasks, significantly outperforming the competitive baseline LLADA on reasoning tasks (GSM8K, MMLU) (ETable 1-2). And Conv also exhibits minimal sensitivity to kernel size even in small step size ($S = 64$).\n\nETable 1. GSM8k with LLADA-8B-instruct  ,   L=512,  S=64,  CoT instruct.  Each column  is kernel size or block size\n\n|  | 32 | 64 | 128 | 256 | 512 |\n| --- | --- | --- | --- | --- | --- |\n| LLADA | 27.22 | 31.24 | 32.75 | 23.73 | 29.64 |\n| Conv | 47.61 | 44.28 | 40.33 | 39.27 | 39.73 |\n\nETable 2. MMLU with LLADA-8B-instruct  ,   L=512,  S=64,  CoT instruct.  Each column  is kernel size or block size\n\n|  | 32 | 64 | 128 | 256 | 512 |\n| --- | --- | --- | --- | --- | --- |\n| LLADA | 9.63 | 14.46 | 17.98 | 15.68 | 26.41 |\n| Conv | 44.21 | 46.91 | 50.13 | 51.78 | 54.90 |\n\n---\n\n**[Q4] How sensitive is the R2FT to the corruption rule?**\n\n= As shown in Algorithm 1, the corruption rule conducts random sampling based on a few predefined rules. Consequently, slight variations of hyperparameters in these rules appeared to have minimal impact on performance, and we were able to reach strong results without extensive tuning of these rules.\n\nHowever, the training objective shows more sensitivity to the hyperparameters $\\gamma$ and $\\beta$. Excessive unlearning of unpreferred cases can lead to the elimination of high prior tokens, thereby removing essential functioning words (e.g., *is*, *a*). This issue appears to stem from the nature of the unlearning process itself. Nevertheless, we addressed it by incorporating an additional objective term $\\mathcal{L}_{NELBO}$ and setting $\\gamma = 0.1$, which mitigated this problem effectively.\n\n---\n\n**[Q5] Weak investigation of bidirectional downstream tasks, despite bidirectionality being a central claim.**\n\n=As you point out, we did not evaluate on bi-directional tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.\n\nHowever, we believe that the bidirectionality introduced by *Conv* offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.\n\nIn our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.\n\nWe also expect Decision Transformers (DT) [2] to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.\n\nHowever, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints.  In this scenario, Conv can be a better option than a more rigid semi-AR.\n\n---\n\nReferences\n\n[1] Guanghan Wang et al., Remasking Discrete Diffusion Models with Inference-Time Scaling, 2025 arxiv\n\n[2] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling NeurIPS 2021"}, {"review_id": "asiMuAYsFh", "reviewer": "Reviewer_QCLw", "summary": "The paper introduces advancements in diffusion-based language models (LMs) to address the long decoding-window (LDW) problem, where tokens generated far from the input context become irrelevant or repetitive. Unlike autoregressive (AR) models, diffusion LMs can generate multiple tokens in parallel and leverage bidirectional context, but their performance suffers from the LDW issue. The submission proposes two key methods to mitigate this:\n\n- Convolutional Decoding (Conv): A normalization technique that narrows the decoding window without segmenting it into blocks, preserving both decoding speed and bidirectionality. This contrasts with semi-autoregressive (semi-AR) approaches, which sacrifice these advantages.\n- Rejecting Rule-based Fine-Tuning (R2FT): A post-hoc training method that reduces the model's tendency to produce repetitive or high-prior tokens by aligning distant tokens with the context, enhancing fluency and coherence.\n\nIn addition, the submission introduces EOS-fill, a technique that improves decoding speed by filling positions with end-of-sentence tokens when one is sampled. The proposed methods were evaluated on open-ended text generation benchmarks like AlpacaEval, MT-Bench, and Wiki, using the Masked Diffusion Language Model (MDLM) backbone. The results show state-of-the-art performance among diffusion LM baselines, achieving high-quality text generation with significantly fewer decoding steps compared to prior work.", "questions": "- The submission provides response samples but lacks a detailed discussion of scenarios where Convolutional Decoding (Conv) or Rejecting Rule-based Fine-Tuning (R2FT) fail or produce suboptimal outputs. Could you include a dedicated section or table analyzing specific failure cases (e.g., types of prompts or contexts where repetition persists or coherence degrades)?\n\n- The submission lacks theoretical analysis or guarantees for why Conv avoids the time-interval expansion problem or why R2FT effectively mitigates repetition. Could you provide a theoretical explanation (e.g., a simplified model or derivation) of how Conv’s normalization preserves decoding quality compared to semi-AR, or how R2FT’s loss function reduces high-prior token preference?\n\n- The submission specifies GPU types (NVIDIA RTX A5000) and training steps but omits details on execution time, memory usage, or total compute cost for training and inference. Could you provide a table or paragraph detailing these resources (e.g., hours for SFT/R2FT, VRAM per GPU, total GPU-hours) for both small (18M) and large (8B) models?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. Quality\n\n*** Strengths ***\n\n*Rigorous Experimental Design*: The submission conducts comprehensive experiments using established benchmarks (AlpacaEval, MT-Bench, Wiki, Arena-Hard-Auto, GSM8K) to evaluate the proposed methods (Convolutional Decoding, R2FT, EOS-fill). It compares against strong baselines, including MDLM with categorical sampling, LLADA, and top-k global normalization, ensuring a robust evaluation of performance improvements. For example, Table 1 highlights AlpacaEval results, with clear reporting of first and second-best performances.\n\n*Reproducibility*: The submission provides detailed experimental settings, including decoding window size ($ L = 1024 $), step size ($ S = 128 $), kernel/block sizes, model sizes (18M and 8B), and training protocols (SFT, R2FT, LoRA). The inclusion of anonymized code and a URL for supplementary material enhances reproducibility. Hyperparameters, such as learning rate ($ 3e-5 $), batch size (512), and optimizer (AdamW), are explicitly stated.\n\n*Statistical Rigor*: The paper reports error bars using standard deviations for AlpacaEval, adhering to the official evaluation interface. This allows for a reliable assessment of variability in win rates across decoding strategies, strengthening the statistical significance of the results.\n\n*Theoretical Grounding*: The submission builds on the simplified NELBO objective for diffusion LMs and provide a clear explanation of the LDW problem using statistical observations. The mathematical formulation of R2FT’s training objective is well-defined, drawing from established methods like DPO and SimPO.\n\n*Thorough Baselines*: The submission evaluates multiple decoding strategies (categorical, LLADA, top-k global, Conv) and includes both small (18M) and large (8B) models, ensuring a comprehensive comparison. The use of the Alpaca instruction dataset for fine-tuning and the detailed setup for SFT and R2FT further strengthens the experimental quality.\n\n*** Weaknesses ***\n\n*Compute Resource Details*: Although the submission specifies GPU types (NVIDIA RTX A5000) and training steps, it lacks precise information on execution time or memory requirements for training and inference.\n\n2. Clarity\n\n*** Strengths ***\n\n*Clear Problem Definition*: The submission clearly articulates the long decoding-window (LDW) problem, supported by intuitive explanations and visualizations. The breakdown of token types (repetition, high-prior, meaning, noise) is accessible and well-illustrated.\n\n*** Weaknesses ***\n\n*Overuse of Acronyms*: The submission uses multiple acronyms (MDLM, LDW, R2FT, SFT, Conv, LLADA) without always redefining them in later sections. This could confuse readers who skip sections or are less familiar with the domain.\n\n3. Significance\n\n*** Strengths ***\n\n*Addressing a Key Limitation*: The LDW problem is a significant barrier to the practical adoption of diffusion LMs, which promise faster and bidirectional decoding compared to AR models. By tackling this issue, the submission advances the field toward more viable non-autoregressive language models.\n\n*Practical Impact*: The proposed methods (Conv, R2FT, EOS-fill) achieve state-of-the-art performance among diffusion LMs with a reduced step size ($ S = 128 $). This improves decoding speed, making diffusion LMs more competitive with AR models for real-world applications like chatbots or content generation.\n\n*Broad Applicability*: The techniques are model-agnostic within the MDLM framework and could potentially be adapted to other diffusion-based architectures. The use of the Alpaca instruction dataset and benchmarks like MT-Bench and Wiki ensures relevance to practical NLP tasks.\n\n*Advancing Diffusion LMs*: The submission builds on foundational work (e.g., D3PM, LLADA) and addresses critical limitations of semi-AR decoding (speed and bidirectionality loss). This positions diffusion LMs as a stronger alternative to AR models, potentially influencing future research in non-autoregressive generation.\n\n4. Originality\n\n*** Strengths ***\n\n*Novel Methods*: The proposed Convolutional Decoding (Conv) and Rejecting Rule-based Fine-Tuning (R2FT) are original contributions. Conv introduces a normalization-based approach to narrow the decoding window without sacrificing bidirectionality, distinct from semi-AR’s block-based segmentation. R2FT is a novel post-hoc training strategy that mitigates repetition and high-prior tokens without expensive human annotations, unlike RLHF.\n\n*EOS-fill Innovation*: The EOS-fill technique is a creative optimization that leverages caching to accelerate decoding by filling positions with EOS tokens, a method not widely explored in prior diffusion LM work.\n\n*** Weaknesses ***\n\n*Incremental over LLADA*: While Conv and R2FT are novel, they build heavily on the LLADA framework and MDLM backbone. The core idea of narrowing the decoding window is inspired by semi-AR decoding, which reduces the perceived novelty of Conv. A clearer differentiation from LLADA’s semi-AR approach could strengthen the originality claim.", "was_revised": true, "final_justification": "Congratulations on the submission. Excellent paper. Hope it gets accepted.", "rebuttal": "We greatly appreciate the reviewer’s acknowledgment of our work’s strengths and the insightful questions that contribute to its robustness. These discussions will be incorporated into the camera-ready submission.\n\n---\n\n**[Question2-1] Theoretical guarantee for why Conv avoids the time-interval expansion problem.**\n\n= We provide two theoretical explanations for why Conv may outperform semi-AR. The following discussion assumes a mask diffusion type [1].\n\n**(1) Violation of the training assumption.**\n\nThis point is discussed in detail in Section 3.2 (line 174). \n\n**(2) Explanation with the hazard function.**\n\nA more intuitive explanation is that Conv is more flexible than semi-AR, granting the model greater freedom. This can be theoretically justified using the hazard function, which measures the probability of corruption.\n\nLet us follow the notation in the paper, line 174: L as window size, S as step size, b as the number of blocks, $L_b$ as the block size or kernel size, and $S_b$ as the number of steps assigned for each block in semi-AR. \n\nWe newly define the sample rate $r=L/S$, and $W$ as the number of mask tokens that the model have to sample from at current step. For example for default diffusion LM, $W=L$ holds at the first decoding step, and $W \\approx L-r$ at the second step, since the model might have unmasked $r$ masks at previous step, on average. For semi-AR, it is $L_b$ to $L_b - r$.\n\nWe further define the hazard function $P$, representing the probability (or degree) that the final decoded text becomes structurally (grammatically, syntactically) harmed. Conversely, $1-P$ representes the probability of being unharmed (i.e., structurally coherent). We define $Q = \\log(1-P)$. And let small $p_t$ denote the probability that the final output becomes harmed due to the decoding at intermediate timestep $t$. Here, the hazard function is commonly defined as $P= 1- \\prod_{t=1}^S (1-p_t)$, which in turn $Q= \\sum_{t=1}^S \\log(1-p_t)$. We assume that $p_t$ depends on both $r$ and $W$, thus we denote $p_t=p_r(W)$.\n\n**#. Assumptions about the factor in the quality of a text.**\n\nWe assume that the quality of generated text is related to **density of decoding** (denoted as ****$d=r/W$) in two opposing ways (though there can be more other factors). Raising $d$ can be benefit and harm the quality at the same time.\n\n(1) Alignment:  When the positions to be decoded are far from the given context, the model tends to suggest less aligned candidates—a phenomenon referred to as the LDW problem**.** This situation is often associated with low $d$ (though not always), which indicates lots of masked positions are between the given context and the targeted position  (refer to response of **Question2-2)**. A common strategy to mitigate this issue is to increase $d$  by reducing $W$ (e.g., semi-AR, Conv).\n\n(2) Structural coherence: Conversely, when $d = \\frac{r}{W}$ becomes too high, it can reduce the dependency among decoded tokens and lead to structural degradation. This effect stems from a key property of mask diffusion: multiple tokens generated simultaneously cannot attend to one another at the output layer, resulting in low inter-token dependency. Consequently, when many tokens are generated within a narrow span, the likelihood of syntactic conflicts increases.\n\nHowever, when the distance between two generated tokens $x_i$ and $x_j$ is large “enough”, even if their immediate dependency is weak, the number of possible connecting sequences between them grows exponentially as $|V|^{(j-i)}$ (V: vocab size). This implies that later decoding steps have the opportunity to restore structural connectivity. This is why $p_t$ depends on $r$ and $W$. If large r and small W mean too packed generation, thus low $p_t$.\n\nWe formalize this property as follows:\n\n$\\text{if } r_1 = r_2 \\text{ and } W_1 > W_2, \\text{ then } p_{r_1}(W_1) \\leq p_{r_2}(W_2)$\n\nEquality approximately holds when $r$ is sufficiently small or when both $W_1, W_2$ are sufficiently large over some threshold. The precise threshold for “sufficiently” and the shape of the curve $p_r(W)$ as a function of $W$ may vary across models.\n\n**#. Comparison Across Decoding Types**\n\nFrom the two factors discussed earlier, we now focus on **(2) structural robustness** and model it using the hazard function across different decoding strategies, leaving (1) aside for simplicity.\n\n1. Default mask modeling\n\nFor a standard mask diffusion model without a narrowed decoding window, the hazard function $Q_0$ is as follows. \n\n$P \\approx 1-\\Pi_{t=1}^{S}(1-p_r(L-(t-1) \\cdot r))=1-\\Pi_t^{S}(1-p_r(t \\cdot r))$\n\n$Q_0 = \\sum_t^S q_r(t \\cdot r)$     ,   $q = log(1-p)$  ,  $W_t = t\\cdot r \\leq L$\n\n$q_r(W)$ represents the log probability that decoding at a step does **not** introduce corruption. $W$ decreases gradually by $r$ per step.\n\n1. semi-AR\n\nFor semi-AR, when $L$ is divided into 4 blocks (i.e., $L_b = 128$):\n\n$Q_{sa}=b*\\sum_t^{S_b}q_r( t \\cdot r)$ *,     $W_t = t \\cdot r \\leq L_b$*\n\nTo compare, since $S > S_b$, the value of $q_r(W_t)$ for semi-AR at any given step is strictly smaller than that of the default setting (because $W_t \\leq L_b \\leq L$  for semi-AR). \n\nTherefore:  $Q_{sa} \\leq Q_0$\n\nEquality occurs when the block size $L_b$ approaches $L$, or when $S$ is large enough so that the argument of $p_r(W)$ exceeds its threshold for most steps. This theoretical behavior aligns with the empirical observation in Figure 4, where evaluated structural quality increase as $L_b$ or $S$ are raised.\n\n1. Conv\n\nFor Conv, assume the kernel size $L_b=128$. In this case, at every step we have $W\\geq 128$.  Although the actual variation of $W$ is more complex, for simplicity, we assume $W = 128$.\n\n$Q_c = (L-128) * q_r(128)  +  \\sum_t^{128/r}q(t \\cdot r)$\n\nHere, the second term is shared with semi-AR and the default. but the first term is always larger than that of semi-AR, except at the start point of each block. \n\nTherefore, $Q_{sa} \\geq Q_b$\n\nAs before, equality approximately holds when $S$ is sufficiently large.\n\nCompared to the default mask diffusion, the first term is always smaller than the default, so we have: $Q_c \\leq Q_0$. \n\n Equality holds when the fixed kernel size is “sufficiently” large, so that the effective $W$ go over threshold. This behavior is consistent with the empirical observation shown in Figure8, where kernel size over some threshold yields structural stableness. \n\nTo wrap up, In terms of structural robustness, we generally observe the ordering:\n\n$\\text{semi-AR} \\prec \\text{conv}\\prec \\text{default}.$\n\nHowever, from the perspective of alignment, the relationship reverses:\n\n$\\text{semi-AR}, \\text{conv} \\succ \\text{default}.$\n\nConsequently, adopting Conv offers a favorable trade-off, as it provides substantial gains in alignment while maintaining significantly better structural robustness compared to semi-AR. \n\nThis argument is consistent with our empirical findings. Categorical sampling has good structural coherence (inlier in ETable1), while alignment (winrate) degrades. Conv has better alignment and winrate, while semi-AR fails in both.\n\nETable 1. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.\n\n| RFT | decode | winrate | count | avg Lenght | inlier (95%) |\n| --- | --- | --- | --- | --- | --- |\n|  | semi-AR | 21.5 | 777 | 265.5 | 0.34 |\n|  | categorical | 37.98 | 803 | 360.1 | 1 |\n|  | topk | 42.48 | 777 | 354.5 | 0.99 |\n|  | conv | 45.27 | 799 | 371.3 | 0.98 |\n| o | topk | 55.38 | 805 | 459.8 | 1.00 |\n| o | conv | 67.73 | 805 | 320.1 | 1.00 |\n\n---\n\n**[Question2-2] Theoretical guarantee for why R2FT effectively mitigates.**\n\n=We divide the explanation into two steps: the decoding step and the training step.\n\n(1) Decoding step\n\nFirst, let us describe the scenario where the issue arises. A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). To formalize this, consider a scenario in which the model decodes  $x_{21}, x_{22}, x_{23}$  at the first sampling step,  where $x_i$ denote a mask token that is i-th position from given context. The probability of candidate $c$ for $x_{21}$ is  $p(c|\\tau)=E_\\tau [ p(\\tau|c)p(c)]$, where $\\tau \\in T$  is all possible combinations of $\\{x1,x2,...,x20\\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), infinity. Therefore, the posterior $p(c \\mid \\tau)$ converges to $p(c)$.\n\np(c) represents the probability of a candidate that can be easily predicted without any contextual information. Typical examples include high-prior tokens (e.g., function words) or simple repetition of given context. The same applies to $x_{22}$ and $x_{23}$. Consequently, as illustrated in the upper part of Figure 11, top candidates often include patterns such as “: : :” (repetition) or “the the the” (high-prior tokens).\n\nThe motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is training the model to reduce $p(c)$ in  $E[p(\\tau|c)p(c)]$. By doing so, the probability mass shifts toward tokens with higher average likelihood within $T$. In other words, we choose the candidate patterns like \"president president president\", which at least contains more contextual information than \": : :\".  This is consistent with our empirical observation on AlpacaEval that R2FT achieves better alignment with the query.\n\n(2) Training step\n\nRationale on reducing p(c) is described in Section 4.2\n\n---\n\n**[Question 1] Failure cases**\n\n(1) Conv: As discussed above, Conv exhibits reduced structural coherence when the kernel size or the step size becomes excessively small.\n\n(2) R2FT: Without incorporating the NELBO term during training, R2FT tends to eliminate high-prior tokens entirely. However, this issue is resolved by adding the NELBO term and setting $\\gamma = 0.1$.\n\nSuch cases will be included in the camera-ready version.\n\n---\n\n**[Question 3] GPU usage and time**\n\nWe will include in the camera-ready version or report in the discussion period."}, {"review_id": "7rVDJkWWHt", "reviewer": "Reviewer_krtD", "summary": "This paper addresses the long decoding-window problem in diffusion LMs, where distant tokens become irrelevant or repetitive. It proposes Convolutional decoding (Conv) to narrow the window without hard blocks and Rejecting Rule-based Fine-Tuning (R2FT) to align far-away tokens. Experiments on AlpacaEval show better quality and faster decoding with fewer steps.", "questions": "Line 135 states that sampling from top-ranked candidates “high chance of selecting only repetition or high-prior tokens positions far from context.” Why? Do high-probability tokens actually cluster near the prefix, as suggested by darker colors in Figure 2?", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n1. Block decoding sacrifices speed and bidirectionality, but R2FT with convolutional decoding overcomes this trade-off.\n2. The method better aligns tokens far from context, yielding faster decoding and improved results.\n3. Evaluating on AlpacaEval instead of only NLU tasks better reflects open-ended generation performance.\n4. Decoding-window analysis is clear and offers valuable insight.\n\n**Weaknesses**\n\n1. Claim that semi-AR sacrifices speed may not hold when KV-cache is used in diffusion models.\n2. All analyses in Section3 use relatively small MDLMs. It’s unclear if results hold for larger models (e.g., 7B LLaDA).\n3. Convolution direction needs to be clarified. Uni-directional convolution mimics causal decoding in AR models. Which convolution type is used in the end, uni- or bi-directional?\n\n**Writing suggestions**\n\n1. Intro detail: the motivation and goal of methods are introduced, but without design details. Adding a brief overview of how Conv and R2FT work could be better.\n2. Figure 4: Zoom into x=[256,512,1024], y=[0,1000] or use log-scale for PPL to improve readability.\n3. Citations: the following key related works on diffusion language models are not cited.\n- A Reparameterized Discrete Diffusion Model for Text Generation. https://openreview.net/pdf?id=PEQFHRUFca\n- Scaling Diffusion Language Models via Adaptation from Autoregressive Models. https://openreview.net/forum?id=j1tSLYKwg8", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your positive assessment of the strengths of our work. We will incorporate all of the discussions below into the camera-ready version, along with your suggestions on citations and writing.\n\n---\n\n**[Weakness 1] Claim that semi-AR sacrifices speed may not hold when KV-cache is used in diffusion models.**\n\n= We agree with this point.  However, the speed gains from KV-cache in semi-AR models are compensated by the increased cost from timespan expansion, implying that diffusion LMs cannot fully leverage their inherent speed advantage with semi-AR.\n\nIn contrast, Conv can also theoretically leverage KV-cache, as the boundary of the mask tokens shifts consistently, as illustrated in Figure 7. For this reason, Conv can be considered to have a speed advantage.\n\n---\n\n**[Question 1] Line 135 states that sampling from top-ranked candidates “high chance of selecting only repetition or high-prior tokens positions far from context.” Why? Do high-probability tokens actually cluster near the prefix, as suggested by darker colors in Figure 2?**\n\n= A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). To formalize this, consider a scenario in which the model decodes  $x_{21}, x_{22}, x_{23}$  at the first sampling step,  where $x_i$ denote a mask token that is i-th position from given context. The probability of candidate $c$ for $x_{21}$ is  $p(c|\\tau)=E_\\tau [ p(\\tau|c)p(c)]$, where $\\tau \\in T$  is all possible combination of $\\{x1,x2,...,x20\\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), the infinity. Therefore, the posterior $p(c \\mid \\tau)$ converges to $p(c)$. This is consistent with the empirical observation in Figure 3, where the probability of high-prior or repetitive tokens increases and then plateaus at a high level as the distance from the given context grows.\n\n$p(c)$ represents the probability of a candidate that can be easily predicted without any contextual information. Typical examples include high-prior tokens (e.g., function words) or simple repetition of the given context. The same applies to $x_{22}$ and $x_{23}$. Consequently, as illustrated in the upper part of Figure 11, top candidates often include patterns such as “:  :  :” (repetition) or “the the the” (high-prior tokens). \n\nThe motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is training the model to reduce $p(c)$ in  $E[p(\\tau|c)p(c)]$. By doing so, the probability mass shifts toward tokens with higher average likelihood within $T$. In other words, we choose the candidate patterns like \"president president president\", which at least contains more contextual information than \": : :\".  This is consistent with our empirical observation on AlpacaEval that R2FT achieves better alignment with the query.\n\n---\n\n**[Weakness 2]** All analyses in Section3 use relatively small MDLMs. It’s unclear if results hold for larger models (e.g., 7B LLaDA).\n\n= That is a valid point. We are preparing the experiment and will share the results during the discussion period.\n\n---\n\n**[Weakness 3]** Convolution direction needs to be clarified. Uni-directional convolution mimics causal decoding in AR models. Which convolution type is used in the end, uni- or bi-directional?\n\n= The bidirectional decoding we referred to applies to scenarios where context is provided on both sides (left and right), unlike simple QA settings, and we did not evaluate on such tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.\n\nHowever, we believe that the bidirectionality introduced by *Conv* offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.\n\nIn our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.\n\nWe also expect  **Decision Transformers (DT) [2]** to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.\n\nHowever, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints.  In this scenario, Conv can be a better option than a more rigid semi-AR.\n\n---\n\nReferences\n\n[1] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling NeurIPS 2021"}, {"review_id": "yG4h9SIRLy", "reviewer": "Reviewer_iU8F", "summary": "This paper identifies the long decoding-window (LDW) problem in diffusion language models, the authors propose Convolutional Decoding and Rejecting Rule-based Fine-Tuning (R2FT) to address this problem. Experiments on benchmarks like AlpacaEval and MT-Bench show their approach achieves state-of-the-art performance among diffusion LMs, enabling efficient generation with fewer decoding steps.", "questions": "Most questions/suggestions are reflected in the weakness part, for the last point mentioned before, I suggest the authors conduct a more fine-grained evaluation, analyzing the quality and contextual relevance of generated tokens as a function of their distance from the prompt/context. \n\nOne more thing, I was also curious about the performance on bi-directional substream tasks, as mentioned by the authors in the limitations, it would be good if they can provide some basic results.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. This paper systematically analyzes the LDW problem, a core challenge for diffusion LMs, which is a fundamental issue but not fully exploited.\n2. The proposed Convolutional Decoding is a simple yet effective, comparing with commonly adopted block/semi-AR methods\n3. The R2FT method introduces a targeted, rule-based negative fine-tuning stage, which is also shown to be effective.\n4. Results on multiple benchmarks exhibit the potential of the proposed strategies.\n\n### Weakness\n1. R2FT relies on simple rule-based negative sample generation, more sophisticated negative mining methods can be considered\n2. In most cases,  window size is set to1024(512 in some experiments) , it's hard to judge the robustness/generalization of the method to the window size.\n3. The results pure rely on win rate, lacking analyses of more diverse dimensions.\n4. It's not clear how exactly the performance improvement comes from, does it primarily stem from scenarios requiring long contextual information?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful question. We will incorporate these discussions into the camera-ready version.\n\n---\n\n**[Weakness 1] R2FT relies on simple rule-based negative sample generation; more sophisticated negative mining methods can be considered.**\n\n= The point you raised remains an avenue for future research. However, it can be justified as follows.\n\n(1) We already discussed the strengths of our approach compared to other seemingly more natural alternatives in section 4.2\n\n(2) R2FT can be viewed as a low-cost approach to human data annotation.\n\nCreating datasets for desired model behavior has been a guaranteed approach in LLM development.   Furthermore, the ongoing progress in LLMs relies heavily on continuously generating new datasets to broaden the model coverage. Likewise, we annotate new data for LLMs to prevent unwanted behaviors. However, as “deconstruction is always much simpler than construction**”**, unwanted behaviors can often be easily reproduced through simple rules, which is R2FT. \n\n---\n\n**[Weakness 4] It's not clear how exactly the performance improvement comes from, does it primarily stem from scenarios requiring long contextual information?**\n\n= If the intent of this question is to inquire about the theoretical guarantees of our method, we would kindly ask you to refer to our response to **QCLw** for further details. We apologize for not including the full details here, as we did not want to occupy excessive space if this was not your intended concern.\n\n---\n\n**[Weakness 3,4 ] The results are purely based on win rate, lacking analyses of more diverse dimensions. \\\\ It's not clear how exactly the performance improvement comes from; does it primarily stem from scenarios requiring long contextual information?**\n\n= Acknowledging your concern, we provide an additional metric. Text quality involves two key aspects: alignment and structural coherence. Among these, alignment is primarily assessed using win rate. Besides, s structural coherence is typically assessed using generative perplexity (gen PPL). However, gen PPL is known to overestimate texts containing repeated patterns [1], which makes it easy to be gamed.\n\nAccordingly, we computed the mean ($\\mu$) and stds ($\\sigma$) of gen PPL from the training dataset, and then measured the proportion of model-generated texts whose PPL falls within the interval $\\mu \\pm 2\\sigma$ (reported as *inlier* in ETable 1). Additionally, we report the *count* of outputs which has zero length before the EOS token, indicating another pattern of structural incoherence.\n\nFurthermore, as you mentioned, our method assumes scenarios where long-form text must be generated, without relying on random unaligned sampling, following recent expectations for LLM-based services.  Thus, we conducted additional experiments under a setting designed to better reflect this assumption. To reflect this setting, we instructed models to generate responses in a Chain-of-Thought format to provide detailed reasoning, and we applied top-5 decoding. As average length increased, we set S=128, which still guarantees around 3x sample rate.\n\nETable 1. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.\n\n| RFT | decode | winrate | count | avg Lenght | inlier (95%) |\n| --- | --- | --- | --- | --- | --- |\n|  | semi-AR | 21.5 | 777 | 265.5 | 0.34 |\n|  | categorical | 37.98 | 803 | 360.1 | 1 |\n|  | llada | 39.18 | 805 | 405.1 | 0.75 |\n|  | topk | 42.48 | 777 | 354.5 | 0.99 |\n|  | conv | 45.27 | 799 | 371.3 | 0.98 |\n| o | semi-AR | 26.97 | 805 | 452.56 | 0.80 |\n| o | llada | 47.37 | 805 | 394.8 | 0.82 |\n| o | categorical | 50.22 | 805 | 403.5 | 1.00 |\n| o | topk | 55.38 | 805 | 459.8 | 1.00 |\n| o | conv | 67.73 | 805 | 320.1 | 1.00 |\n\nResults in new setting are consistent with previous one. Furthermore, this multi-dimensional evaluation reveals stronger evidence for our hypothesis about the proposed method, which is explained in the next question.\n\n---\n\n**[Question 1] I suggest the authors conduct a more fine-grained evaluation, analyzing the quality and contextual relevance of generated tokens as a function of their distance from the prompt/context.**\n\n= Contextual relevance as a function of distance from the prompt is basically illustrated in Figure 3. The probability of candidate $c$ on the given context $x$ is  $p(c|x)=p(x|c) p(c)$, where $p(c)$ represents a frequency without any contextual information. This term corresponds to the high-prior tokens or naive repetition, that we identified as problematic (Section 2.2). Figure 3 visualizes $p(c)$, showing that p(c) peaks at the 5–10 steps away from the context, then remains at a high level thereafter (30–50%). This suggests that the influence of the contextual term $p(x∣c)$ relatively diminishes substantially with increased distance.\n\nR2FT explicitly reduces $p(c)$, which leads to the relative increase of $p(x|c)$, which leads to contextually more aligned sampling.  As shown in **ETable 1,** our multi-dimensional evaluation of alignment (win rate) and structural coherence (inlier) captures these trends well. First, categorical sampling with SFT alone (with high $p(c)$) achieves enough structural coherence (inlier=1.00) but low alignment. More deterministic Topk-k decoding has better alignment, but risks structural degradation. In contrast, R2FT results in a noticeable improvement in both alignment and structural coherence.\n\n---\n\n**[Weakness 2] In most cases, window size is set to1024(512 in some experiments) , it's hard to judge the robustness/generalization of the method to the window size.**\n\n= Thank you for pointing out this important aspect. We are preparing the experiment and will provide the results during the discussion period if possible.\n\n---\n\n**[Question 2] One more thing, I was also curious about the performance on bi-directional substream tasks, as mentioned by the authors in the limitations, it would be good if they can provide some basic results.**\n\n=  As you point out, we did not evaluate on bi-directional tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.\n\nHowever, we believe that the bidirectionality introduced by *Conv* offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.\n\nIn our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.\n\nWe also expect  **Decision Transformers (DT) [2]** to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.\n\nHowever, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints.  In this scenario, Conv can be a better option than a more rigid semi-AR.\n\n---\n\nReference\n\n[1] A. Holtzman et al., The curious case of neural text degeneration\n\n[2] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling"}, {"review_id": "tnGa0F9etr", "reviewer": "Reviewer_TLks", "summary": "This work seeks to improve mask-based diffusion language models in the \"highly compressed generation\" regime: each model call generates many tokens on average (e.g., 8). The work identifies failure modes in SOTA models when they're pushed to these extremes, conjecture on the causes/mechanisms underlying these failure modes, and propose concrete solutions in the form of Rejecting Rule-based negative Fine-Tuning (`R2FT`) and Convolutional decoding (`Conv`). `R2FT` is a reinforcement-learning-inspired method to penalize the model for generating samples that do not satisfy some hard rules. `Conv` is like block diffusion, but smoother. Experiments show that, together, these two mechanism improve on the baselines in this \"highly compressed\" regime.", "questions": "### (Not a question.) Clarification on \"*independently* and *irrevocably*\"\n\nSuppose that the next two tokens are either \"admirable aadvark\" or \"benevolent beaver\", with equal probability. However, \"admirable beaver\" and \"benevolent aadvark\" are **impossible**. When $S$ is large, an MDLM is likely to generate those two tokens on distinct steps: the first token can be sampled 50/50, and the second token will later be deterministic. If the two tokens are to be predicted on the same step (which is likely when $S$ is small), then there is no way for the MDLM to guarantee a valid generation: AB and BA will occur as often as AA and BB. In view of this situation, [1] breaks the independence assumption; [2,3,4,5] can fix their mistakes.\n\n\n### Question 1: Do you agree that the conjectured causes/mechanisms in Section 2.2 are incomplete, premature and/or overstated?\n\n*(This is my main issue with the manuscript.)*\n\nLine 113:\n> In Figure 2, we observe excessive repetition of tokens from the previous context (e.g., “Question”, “Answer”, “:”), which is not helpful in the current step. This phenomenon likely stems from the inherent tendency of LLMs to repeat previous context\n\nThe training set likely contains Q&A data. In those datasets, a question is followed by an answer, *then by another question and another answer etc.* The MDLM is trained to predict those structures at a global level, not just to answer questions. The model may not so much \"repeat previous context\" as \"reproduce a pattern seen in training\".\n\nLines 124:\n> The key property is that, as the distance from the previous context gets farther, the meaning zone tends to fall below the repetition and high-prior zones. This trend is also consistent with statistical observation (Figure 3). We measure the sum of probabilities of high-prior (top-100) and repetition (tokens overlapping the prompt) tokens at varying distances from the instruction prompt.\n\nMy prediction is that the same \"zones\" would be observed in the training data, particularly with Q&A datasets (i.e., get the statistics for the tokens that follow question contexts in the raw dataset, as you would find it in a github repo).\n\n(Line 131)\n> However, they peak after around 5-10 positions.\n\nWhat is the typical length of an answer in the training set? My bet is that it is \"around 5-10 positions\". If this were to be confirmed, it would further support my point about the model just doing its job of learning the training set. In this view, part of what `R2FT` does is to train the model to deviate from that training distribution, to satisfy our specific practical needs.\n\nTaking a step back, like in the \"(admirable|benevolent) (aardvard|beaver)\" case, it is fine that \"Lincoln\" has a high independent probability to come up in consecutive positions: the problem is that we independently sample multiple tokens at once, which may cause \"Lincoln Lincoln Lincoln Lincoln\" to be generated. To be clear, I'm not saying that my own perspective is \"better\": like the authors, I'm making conjectures of my own. **My point is that there are alternative explanations** compatible with the presented evidence: the proposed causes/mechanisms are incomplete, premature and/or overstated. I see two possible paths forward:\n\n1.1: **Tone down** these conjectures, presenting alternative interpretations and noting that some of these observations may be specific to Q&A (or similar) tasks.\n\n1.2: **Double down** by gathering more data, disproving the alternative interpretations that I bring up above, and testing the \"zones\" hypothesis for arbitrary contexts (or better qualify what kind of context this applies to).\n\nDo you agree with my viewpoint? Would you consent to revise an eventual camera-ready accordingly?\n\n\n### Question 2: Can you comment or rebut my point concerning the specificity of `R2FT` to Q&A-like tasks?\n\nI said that, in its current format, this fix appears rather specialized to Q&A and similar tasks. I acknowledge that, *a priori*, `R2FT` could be generalized to expanded rulesets, but having humans leverage their domain knowledge to craft such rulesets would be against the \"Bitter Lesson\" that general methods leveraging computation have historically shown to be ultimately the most effective by a large margin.\n\nWhat are your thoughts? You may also suggest how more general rulesets may be obtained automatically, and/or provide evidence that such more general rulesets are not needed. \n\n\n### Question 3: Can you say a little more about what `R2FT` \"does\" to the probability distribution?\n\nIn my view, instead of dropping the independence and/or irrevocability assumptions, `R2FT` alters the independent token distributions to improve the coherence of a given generation algorithm. Do you agree with this take? If yes, what form does this \"alteration\" take? From Figure 11, I can see that the temperature has increased for latter tokens, and the blue \"high prior\" options have lowered in rank. But this is only a snapshot before the inference process has started: how different would Figure 11 be after the token \"Lincoln\" has been selected at distance 1 from context? Have you measured the $D\\_{KL}$ between these two models at different steps during generation? I welcome any data/observations in this general direction of inquiry. \n\n### Question 4: Can you clarify when `Conv` refers to the uni/bi-directional version?\n\nMy current understanding/guess is that, except in Figure 7, every instance of `Conv` refers to the bidirectional version. Is this the case? In any case, this should be crystal clear in an eventual final version.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- The issues identified in Section 2.2 are real and important. I have myself experienced such failures modes, and I am not aware of existing work reporting them so explicitly. I believe that it is good for this information to be made more broadly known.\n- Section 2.2 alternates between *stating observations* about failure modes and *conjecturing causes/mechanisms*, and I judge these latter conjectures to be incomplete, premature and/or overstated.\n- In my personal opinion, the fact that multiple tokens are sampled *independently* and *irrevocably* is an important cause of the failure modes observed by the authors in that regime. I'm aware of at least one work [1] that attempts to fix the issue by lifting the independence assumption, and many recent papers [2,3,4,5] conversely lift the irrevocability assumption. I believe that this situation should be properly discussed.\n- My understanding is that `R2FT` lifts neither assumptions of independence and irrevocability, but instead uses a novel third way: alter the independent token distributions to improve the coherence of a given generation algorithm.\n- In its current format, this fix appears rather specialized to Q&A and similar tasks. *A priori*, `R2FT` could be generalized to expanded rulesets, though this would go against Rich Sutton's Bitter Lesson, so I would not bet on it too much on the long term. Nonetheless, realizing the existence of this novel third approach is what made me revisit the significance score from \"fair\" to \"good\". \n- Although the unidirectional version of `Conv` (Figure 7(a)) appears to be very similar to the \"slide annealing\" from [3], the bidirectional version (Figure 7(b)) appears novel to me.\n- The evaluations are mainly pitted against variants of MDLM. I believe that these results suffice to answer \"yes\" to the question \"can `R2FT+Conv` improve on a given MDLM?\", but does not give a clear picture of how the resulting model scores in the fast-growing ecosystem of diffusion models.\n\n[1] Liu et al. Discrete Copula Diffusion. https://arxiv.org/pdf/2410.01949  \n[2] Liu et al. Think while You Generate: Discrete Diffusion with Planned Denoising. https://arxiv.org/pdf/2410.06264  \n[3] Fathi et al. Unifying Autoregressive and Diffusion-Based Sequence Generation. https://arxiv.org/abs/2504.06416  \n[4] Wang et al. Remasking Discrete Diffusion Models with Inference-Time Scaling. https://arxiv.org/abs/2503.00307  \n[5] Boget et al. Critical Iterative Denoising: A Discrete Generative Model Applied to Graphs. https://arxiv.org/abs/2503.21592", "was_revised": true, "final_justification": "The authors have agreed to make changes to the wording of their observations, as well as clarifying the mechanism through which their RL operates. Some misunderstandings remain (see https://openreview.net/forum?id=HvIRFV0J90&noteId=4v6V1jYx1E ), but no further iteration is required on my end.\n\nI support the publication of this work.", "rebuttal": "We are deeply impressed by the reviewer’s profound knowledge and insightful comments. Addressing these questions was an enjoyable experience. All discussions will be incorporated into the camera-ready version.\n\n---\n\n**[Q-1] *Why is the preference for high priors and repetition problematic, and what does R2FT aim to achieve? (Regarding your Question1, 3)***\n\n= First, let me briefly respond to the statistical questions:\n\n$ Average length of response in training data:  \n\n57.47 (GPT-2 tokenizer)\n\n$ Does a single instance in the training data consist of multiple question–answer pairs?\n\nNo. The behavior (“Question” and “Answer” appearing on top candidates) is not directly learned.\n\n$ Do the probability patterns for repeating and high-prior tokens in the training data resemble those in Figure 3?\n\nIn the training dataset, the probability of high-prior and repetitive tokens (we will note them “**easy tokens**”) resembles the pattern of the model, declining after an initial peak. However, the peak occurs much earlier (within 1~2 positions). This suggests that the model’s distribution behaves slightly differently, likely due to more influence of conditional likelihood at the positions near the context. We will contain the figure in the camera-ready.\n\nThis observation might lead you to three questions.\n\n**[Q-1-1]  If the model is simply mimicking the data distribution and therefore favoring high-prior or repeating tokens, why is this problematic?** \n\n=  LLM is a probabilistic model that, when faced with high complexity, it often learns shortcuts to minimize loss. From the training data, the model learns that simply choosing easy tokens can always guarantee a least worth loss, even without contextual understanding.\n\nWe can formulate this problem as follows. A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). It can be exemplified as decoding $x_{21}, x_{22}, x_{23}$ at the first step,  where $x_i$ denote a mask token that is i-th position from given context (similar to the “Lincoln” example). The probability of candidate $c$ for $x_{21}$ is $E_\\tau [ p(\\tau|c)p(c)]$ , where $\\tau \\in T$  is all possible combination of $\\{x1,x2,...,x20\\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), the infinity. Therefore, the posterior $p(c \\mid \\tau)$ converges to $p(c)$, where easy tokens dominate. This holds for $x_{22}, x_{23}$ as well,  which explains why patterns in candidate zone, such as “lincoln lincoln lincoln” emerge.\n\nSuch a problem is not a matter of “independently decoding” property. This is because, regardless of how much a model accounts for dependency with surrounding context on the output layer, no model can feasibly consider $|V|^{20}$ possibilities in advance.\n\n**[Q-1-2] On the other side, why do tokens like *\"Question\"* and *\"Answer\"*, which were not even present in the training dataset, appear in the candidate zone?**\n\n= This is a good example of the stated problem in Q-1-1. The model chooses these candidates only because it is repetition, though it has no contextual meaning.\n\n**[Q-1-3] Then, what does R2FT do?**\n\nThe motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is reducing $p(c)$ in  $E[p(\\tau|c)p(c)]$. By doing so, the posterior shifts toward tokens with higher average likelihood within T. In other words, we choose the candidate patterns like \"president president president\", which at least contains more contextual information. To avoid those three tokens decoded simultaneously (resulting in structural breakdown), possible strategies include 1) reducing T (e.g., semi-AR, Conv), 2) lowering the sampling rate, or 3) introducing revocability as you mentioned.\n\n(We will show p(c) in each step for both SFT and R2FT in camera-ready, regarding your Question 3)\n\n**[Q-1-4] Isn’t it rather a problem of irrevocability and independence? Isn’t Conv and R2FT focused only on the Absorbing approach?  (Absorbing vs uniform)** \n\nThe issue you raised can also be framed as a debate between *absorbing (i.e., MDLM, mask diffusion)*  and *uniform* approaches, as discussed in [1]. We anticipate that the absorbing family will remain prevalent, and revocability and dependency may be implemented in a way compatible with absorbing. That is why we concentrated on solving the problem of unrevocable MDLM first (i.e., “third way”). We provide the following rationales.\n\n(1) First, the absorbing-based approach has already proved its downstream task performance comparable to AR LMs (e.g., LLADA). Regardless of its limitations, a method that already works this well is unlikely to be entirely abandoned. In contrast, *uniform* methods have been mainly evaluated on auxiliary metrics such as gen PPL.  We believe this stems from the following theoretical limitations.\n\n(2) Revocation introduces an additional cost.\n\nAs the sample quality of diffusion LM is partially a function of sample rate $r=L/S$, revocability increases the number of masks to decode, similar to raising r.  To investigate this, we compared the average unmasking per step (denote r*) of both revocable MDLM (reMDM [2]**)** and irrevocable ours.\n\nETable1. Average unmasking per step.\n\n| L | S | r*  of reMDM | r* of MDLM |\n| --- | --- | --- | --- |\n| 512 | 65 | 12.98 | 8 |\n| 512 | 128 | 8.69 | 4 |\n| 512 | 256 | 6.668 | 2 |\n\nThis shows r* of the revocable model is 2~3 times larger than that of MDLM, equivalent to operating with a step size 2–3 times smaller in MDLM, which causes degradation. This is one of the reasons that the downstream task performance of reMDM is also lower than MDLM (ETable2).\n\nETable2. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.\n\n| rft | decode | winrate |  len > 0  | avg Lenght | inlier (95%) |\n| --- | --- | --- | --- | --- | --- |\n|  | categorical | 37.98 | 803 | 360.1 | 1 |\n|  | **remdm** | 41.64 | 778 | 358.1 | 0.96 |\n|  | **remdm** + conv + topk | 42.12 | 805 | 470.5 | 1 |\n|  | topk | 42.48 | 777 | 354.5 | 0.99 |\n|  | conv + topk | 45.27 | 799 | 371.3 | 0.98 |\n| o | categorical | 50.22 | 805 | 403.5 | 1 |\n| o | **remdm** + conv + topk | 50.41 | 805 | 439.9 | 1 |\n| o | **remdm** + topk | 51.68 | 805 | 472.3 | 1 |\n| o | topk | 55.38 | 805 | 459.8 | 1 |\n| o | conv + topk | 67.73 | 805 | 320.1 | 1 |\n\nETable3. GSM8k accuracy of LLADA-8B-Instruct based models. L=512, S=128, all CoT instructed, k=1.\n\n|  | Acc |\n| --- | --- |\n| remdm + topk | 0.3980 |\n| remdmd + conv | 0.4541 |\n| topk | 0.4200 |\n| conv | 0.5171 |\n\n(3) Revocability may not be a critical property.\n\nAs noted in [1], we often consider revocability as an inherent and necessary property of diffusion models (DM), thinking of the image generation service, transforming an existing image into an entirely different one (e.g., a person eating bread → a lion eating bread).\n\nHowever, a closer look at the denoising process reveals that this is not as flexible as the entire output. To replace a person with a lion, the DM (1) first degrades the original image into an almost unrecognizable noisy state, and (2) then gradually denoise it. In the denoising phase, once strong edges or dots emerge at a certain location in the early steps, these elements tend to remain stable, functioning like anchors. Then, the surrounding blurred regions refine around them, determining how to interpret the anchors (e.g., as an eye or a nose). Thus, while the overall process is *revocable, steps after initial destruction* exhibit limited revocability. This may be related to the fact that the reverse process is more deterministic than explorative, since DDIM [3]. We will show visual examples in the camera-ready.\n\nLikewise, existing successful generation models (e.g., AR) are not revocable. It may be that well-calibrated sampling can provide an enough solution.\n\n(4) If revocability is still needed, a compatible way is promising.\n\n[1] provide theoretical justification for this compatibility. In [1], the diffusion posterior $p(x | x_t)$ can be decomposed as $p(x|x_t) = p(N|x_t)  p(x|x_t, N)$, where N is noise. The “uniform” approach models the entire, which is theoretically complete but practically beyond the model capacity. In contrast, the “absorbing” focuses only on modeling  $p(x|x_t, N)$, which is theoretically incomplete but highly practical.\n\nAs an alternative, [1] proposes operating $p(N|x_t)$ independently while preserving absorbing as a main drive, achieving a balance between these two extremes. Remasking (ReMDM) [2] also lies along this stream. As long as absorbing models can operate independently, Conv and R2FT remain fully compatible, as also empirically shown in ETable2, 3.\n\n---\n\n**[Q-2] R2FT’s rule-based approach is against the Bitter Lesson.  (regarding Question 2)**\n\n(1) In R2FT, the rule-based component is only used for *data construction*, while the learning lies with the model itself. So R2FT is closer to SFT modeling than a rule-based system.\n\n(2) R2FT can be viewed as a low-cost approach to human data annotation.\n\nCreating datasets for desired model behavior has been a guaranteed approach in LLM development. Research fields are further mining new datasets to broaden model coverage. Likewise, we can annotate “undesired” data to prevent unwanted behaviors. However, as “deconstruction is always much simpler than construction**”**, unwanted behaviors can often be easily reproduced through simple rules, which is R2FT. \n\n---\n\n**[Q-3] When uni/bi directional Conv is used?**\n\nIn this work, we employed unidirectional sampling exclusively, as bidirectional sampling did not yield significant performance differences. This is likely because the context is presented only on the left side in QA tasks. We noted this and discussed the possible advantage of policy optimization in the limitation section. \n\n---\n\n[1] Liu et al. Think while You Generate: Discrete Diffusion with Planned Denoising\n\n[2] Guanghan Wang et al., Remasking Discrete Diffusion Models with Inference-Time Scaling"}]

Author rebuttal:

Meta-review: This paper addresses the long decoding-window (LDW) problem in diffusion-based language models, where tokens generated far from input context become repetitive or irrelevant. The authors propose two main contributions: Convolutional Decoding (Conv), which narrows the decoding window without block-wise segmentation while preserving bidirectionality and speed, and Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training method that reduces repetitive and high-prior token generation. The strengths include a clear problem formulation of an important limitation in diffusion LMs, novel and effective solutions that achieve state-of-the-art performance among diffusion models, comprehensive experimental evaluation on multiple benchmarks (AlpacaEval, MT-Bench, GSM8K), and strong theoretical grounding with hazard function analysis. The main weaknesses center on density and clarity issues that limit readability, limited evaluation of bidirectional tasks despite bidirectionality being a central claim, insufficient analysis of failure cases, and concerns about the generalizability of the rule-based R2FT approach beyond Q&A-like tasks.

The rebuttal process was notably thorough and constructive, with authors providing detailed theoretical explanations including hazard function analysis for Conv's advantages over semi-AR approaches, additional experimental results on reasoning tasks (GSM8K, MMLU) showing Conv's effectiveness, and clarification that unidirectional sampling was used exclusively in experiments. Reviewer QCLw increased their score and expressed strong support after the theoretical explanations, while Reviewer TLks raised important points about independence/irrevocability assumptions and copula-based alternatives, ultimately increasing their score to 5 after the discussion. Reviewer krtD initially had concerns about bidirectional claims being misleading but was satisfied with the authors' clarification about the distinction between bidirectional attention and bidirectional generation. The authors committed to improving clarity, toning down bidirectional claims, and incorporating all discussions into the camera-ready version. The overall reviewer sentiment shifted positively, with multiple score increases, strong endorsements from Reviewers QCLw and krtD, and constructive engagement from all reviewers, leading to a technically solid paper that advances diffusion LM capabilities despite some presentational issues.
Decision: Accept (spotlight)