FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
Authors: Amin Karimi Monsefi
Nikhil Bhendawade
Manuel Rafael Ciosici
Dominic Culver
Yizhe Zhang
Irina Belousova
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion language model
few step generation
flow matching
Primary area: generative models
TL;DR: A discrete flow matching model designed for speed without sacrificing quality.

Abstract: Autoregressive language models (ARMs) deliver strong likelihoods, but are inherently serial: they generate one token per forward pass, which limits throughput and inflates latency for long sequences. Diffusion Language Models (DLMs) parallelize across positions and thus appear promising for language generation, yet standard discrete diffusion typically needs hundreds to thousands of model evaluations to reach high quality, trading serial depth for iterative breadth. We introduce **FS-DFM**, Few-Step Discrete Flow-Matching. A discrete flow-matching model designed for speed without sacrificing quality. The core idea is simple: make the number of sampling steps an explicit parameter and train the model to be consistent across step budgets, so one big move lands where many small moves would. We pair this with a reliable update rule that moves probability in the right direction without overshooting, and with strong teacher guidance distilled from long-run trajectories. Together, these choices make few-step sampling stable, accurate, and easy to control. On language modeling benchmarks, FS-DFM with 8 sampling steps achieves perplexity parity with a 1\,024-step discrete-flow baseline for generating 1\,024 tokens using a similar-size model, delivering up to 128× faster sampling and corresponding latency/throughput gains.

Published as a conference paper at ICLR 2026
FS-DFM: FAST AND ACCURATE LONG TEXT GENERATION WITH FEW-STEP DIFFUSION LANGUAGE MODELS
Amin Karimi Monsefi
Nikhil Bhendawade"
Manuel R. Ciosici
Dominic Culver
Yizhe Zhang? Irina Belousova?
The Ohio State University
?Apple
ABSTRACT
Autoregressive language models (ARMs) deliver strong likelihoods, but are in- herently serial: they generate one token per forward pass, which limits throughput and inflates latency for long sequences. Diffusion Language Models (DLMs) par- allelize across positions and thus appear promising for language generation, yet standard discrete diffusion typically needs hundreds to thousands of model evalua- tions to reach high quality, trading serial depth for iterative breadth. We introduce FS-DFM, Few-Step Discrete Flow-Matching.
discrete fow-matching model designed for speed without sacrificing quality. The core idea is simple: make the number of sampling steps an explicit parameter and train the model to be consis- tent across step budgets, We pair this with a reliable update rule that moves probability in the right direc- tion without overshooting, and with strong teacher guidance distilled from long- run trajectories. Together, these choices make few-step sampling stable, accurate, and easy control. On language modeling benchmarks, FS-DFM with 8 pling steps achieves perplexity parity with 024-step discrete- flow baseline for generating 024 tokens using a similar-size model. delivering up 128x faster sampling and corresponding latency/throughpur gains. We plan to release code and model checkpoints to facilitate reproducibility and further research. Code pretrained checkpoints: g github. com/ app hwppl/mllllmmm
big move lands where many small moves would.
INTRODUCTION
Per Perplesily NTE
Amcuraty vs MT
Approart: DPV
Approach: bPN
Size: 17%
Pappeng
25.0
Figure 1: Generation quality across model sizes (perplexity and accuracy vs. NFE). FS-DFM reaches crooni-quality regime in few steps across all sizes, while DFM needs far more evaluations. Gold stars (NFE=8) highlight FS-DFM in a few-step regime, with accuracy quickly saturating and entropy converging to similar ranges as steps increase. The average value of entropy for all the models is 7.41 to 8.07.
Autoregressive language models (ARMs) generate sequences by predicting the next token condi- tioned on the observed prefix, and have achieved remarkable success (Yang et al., 2025; Team al., 2023; Grattafiori et al., 2024: et al., 2025a). contrast, Diffusson Language Models (DLMs) synthesize text through iterative refinement, providing stable likelihooddgroundee training objec- tives, inherent parallelism across positions, and enhanced controllability through access to global
Work done during the internship at Apple.

---

Published as a conference paper at ICLR 2026
the
lhe, dae pagle', the DC. the, indiiiaa's the, the Dr. daf,, dl llt, faz Biiee (lc, oo Sh the ite, the, the Pitrrens the, the, ite, Bt the, Beon inforrmen the
tl., tht.. tit.
lle.. tee the lle.
Riteers tee lifirrer
ihs, Bi D
the, Bicoreeee De. the Bhimn Maseeeeeee Disveeee The the Rooesssnoo Dr. Deaeneeennenn Dr. MMl.c.oo Me Dr. tei.. Beceeseee
syseet,
ginenal exrraingrrtinng week week is genally the most effective pain reheef be antrits ardriti, can phyttaleeoeeenee and greemse
grnendl Pirriidg earry week is prounilly moe effeciie pen whet for stalt antitti.
Cloosic pais faties Adals otlanii joint
cane staan and losevere, iopetino clessic pain. Otie: pai soch kneesitt, Hip
migrinss
daiy loading pars
pirn the neasly
discrrifor, Hoseove, cheoric artriis
ofe Rasterial gysee, which responsible ceasing
This odults antritis
inclading Clocring and bymph aedes other hand, there clobi hymphatic that spcadd froes the lazga parta of the body, casing pais, saually from line
25.0 TTD
after time
ttat Extrer reclede a unde ninge of
Choosic iullaunatio the body thi thace of the conutaos coupes for ssout tyaes adalt attlriis oduts.
Sporiyiits Spoedy lis uan caase constant pais especialy weggh loss or may lead to saadl stiffees iscresse pain and issue danage to the lust, eppnitg pances hone soeee panenas Mucre, on the other pant the body fail immaze symoem, cassin tioze darrage may lead chronic pain which load
Dyyasic comeyiiii develays antaiiii.
tumm develep antrrtrs Dryssic
car procoke informmation
leading
Baacenal infoctione also pain oknn infectione can develop arterie These inflctions ingar alleggi maction casong inflrmmation of the
bose
Josn infection sutes Ouo antiitis can become infected is flose centin joins, sach the workiing together landon hurry, which can heoome inferted tegether.
leadeze cuaning anthnits
Figure 2: Eight-step long-horizon generation: 024-token unconditional gemeration in 8 sampling steps. FS-DFM (0.17B) successfully produces 1 (224 tokens under the 8-step constraint. Despite having 40x more parameters, -LaDA-SB-Instruse and De-nstct- 8-steps generations exhibit trailing blanks and punc- tuation artifacts (c.g., repeated commas). Generations are truneated. Complete output in Appendix E.5.
context, capabilities that help mitigate exposure bias and naturally accommodate structure beyond left-to-right generation (Nie et al., 2025b; Austin et al., 2021). Despite these complementary advan- tages, both paradigms face fundamental bottlenecks. ARMs are constrained by sequential decoding, requiring one forward pass per token, which limits throughput and enforces unidirectional depen- dency that hinders tasks such as reversal or order-invariant reasoning (Zhao et al., 2025; Schulman et al., 2022: Gong et al., 2025b). Although DLMs parallelize over positions, they typically require tens to hundreds of refinement steps to achieve competitive quality, effectively replacing the autore- gressive depth of a single long pass with stack of iterative model evaluations (Chen & Zhou, 2023; Arriola et al., 2025; Nie et al., 2025a). For example, to march the ARM generation quality, LLaDA requires roughly one inference step per output token (Nie et al., 2025b).
Continuous-space flow-matching models have emerged as a streamlined alternative to conventional score-based diffusion trained around reverse Stochastic Differential Equations (SDEs). Instead of modeling stochastic reverse process, flow-matching directly trains a continuous normalizing flow via vector-field regression, emabling deterministic probability-foo Ordinary Differential Equa- tion (ODE) sampling. This approach often achieves comparable or superior likelihoods and sample quality with fewer function evaluations and more stable optimization (e.g.. through optimal transport or straight-line paths) than SDE-based pipelines (Lipman al., 2023; Song et al., 2021; Li al., 2025b; Gong et al., 2023). Discrete Flow-Matching (DFM) extends these advantages to discrete do- mains by adapting flow-matching to tokenized data, providing flexible probability paths and strong perplexity at scale. However, DFM still requires a substantial number of evaluations along each trajectory, hindering applicability (Gat et al., 2024).
We introduce Few-Step Discrete Flow-Macching (FS- (FS-DFM), a new diffusion- language model built upon the DFM paradigm, which preserves the key advantages of prior diffusion and flow mod- els (bidirectional context utilization and flexible generation) while explicitly mitigating speed and sampling overhead weaknesses. FS-DFM uses DFM's s discrete-flow perspective to substantially re- duce the number of refinement steps required for high-quality generation. In doing so. FS-DFM narrows the inference efficiency and modeling performance gap between DLMs and ARMs. FS- DFM can reach the quality regime of 024-step DFM with just 8 NFEs (Number of Function Evaluations), 128x fewer steps for comparable quality (Figure 1).
FS-DFM builds two pillars. First, enrich the model with the shorteut principle (Frans et al., 2025) together with a self-consistency constraint: the network is conditioned not only on masked tokens but also the desired step size, and trained that a single large step agrees with the
2

---

Published as a conference paper at ICLR 2026
composition of multiple smaller steps. We instantiate the shorteut teacher using Runge- Kutta formulation -we evaluate RK-2 (Heun) and classical RK-4 as ODE estimators and adopt the one that offers the best securacy-ttability trade-off for large single/few-step updates. Second, Discrete Markov Chain (CTMC, Camphell et al. (2022)) and probability velocity is learned to transport source distribution the data distribution along probability path (Gat et al., 2024; Campbell et al., 2024). In this framework, velocity field advances samples "locally" from p: to Pt+h. Instead, FS-DFM learns a Cumulative Scalar parameterized by the step size h to enable large and reliable jumps along the probability path. The step- budgeted perspective retains the core benefits of flow-matching: stable training, parallel refinement, and bidirectional context, while elevating few-step generation to first-class design objective.
DFM reformulation that models sequences as a Continuous-
To our knowledge, FS-DFM is the first few-step discrete flow-matching approach aimed at long- horizon language modeling. Prior discrete diffusion/flow methods require hundreds or even thou- sands of refinement steps, often growing with sequence length (Gat et al., 2024; Nic et al., 2025b). FS-DFM works directly in token space with step-aware DFM and a closed-form cumulative scalar update, matching the perplexity of 024-step discrete-flow baseline for 1 024-token generation in just 8 steps up to 128x faster while preserving long-horizon quality.
RELATED WORK
Diffusion models have demonstrated remarkable effectiveness in continuous domains such im- ages, achieving srate-of-the-eee quality across diverse tasks (Karimi Monsefi et al., 2025; Zhang et al., 2023; Navard et al., 2024; Rombach et al., 2022). Extending diffusion-style generative model- ing to text, Diffusion Language Models (DLMs), have shown impressive task performance coupled with fast inference through parallel generation (Nie al., 2025b; Ye al., 2025; Labs al., 2025). DLMs can be broadly grouped into continuous models, which denoise in embedding or latent space, and discrete models, which operate directly over tokens. Both categories exhibit the key advan- tages of diffusion, including bidirectional context, controllability, and parallel refinement (Li et al., 2025b).
Continuous DLMs (Li et al., 2022; Lin et al., 2023; Lovelace et al., 2023) cast text generation as it- erative denoising in a continuous embedding or latent space. A forward corruption process perturbs token embeddings or hidden states, while a parameterized denoiser reconstructs clean representa- tions conditioned on the context. Continuous DL.Ms take advantage of benefits that arise naturally in continuous representations: flexible conditioning and guidance, bidirectional contextualization dur- ing refinement, and compatibility with large-scale pretraining or task-specific fine-tuning. Moreover, continuous DLMs benefits from research in diffusion models for other modalities such as images or audio as methods easily port between domains.
Discrete DLMs, the other hand, model token dynamics either through discrete-time noise sched- ules or continuous-ti
Markov formulations. DLMs such as LLaDA (Nic et al., 2025b) and Dream (Ye et al., 2025) match autoregressive models in task performance if allowed sufficient finement steps. Effectively, they tie step count to sequence length and generation quality (Austin al., 2021; Gong et al., 2025a; Zheng et al., 2024; Shi et al., 2024; Ou al., 2025). One line of research accepts this high step count requirement and focuses on speeding up inference through approaches such as confidence-aware parallel decoding, tweaks re-enable the KV Cache, and re- stricted attention (Wu et al., 2025; Arriola et al., 2025; Chen et al., 2025b). Another line of research focuses on reducing the number of inference steps through one-/few-step approaches (Gat et al., 2024). Our work falls within this second line of research.
Prior work on one-step language generation primarily focused on continuous-space diffusion mod- els, operating over embeddings rather than discrete tokens. DLM-One (Chen et al., 2025a) demon- strated large speedups on short sequence tasks but did not address long sequence modeling. Sim- ilarly, FlowSeq (Hu et al., 2024) operates on continuous embeddings, bypassing token-level likeli- hoods. Within the discrete family, recent methods typically attain quality with thousands of refine- ment steps. often scaling with sequence length (Gat et al., 2024; Zhao et al., 2025). SDTT (De- schenaux & Gulcehre. 2025) distills discrete DLMs for faster sampling, yet still uses 16-256 steps and focuses short-form tasks. In contrast, our method (FS-DFM) operates in the discrete space
3

---

Published as a conference paper at ICLR 2026
and explicitly targets the few-step regime (1-8 steps). FS-DFM match s224stepp discrete-flow baselines in just steps while preserving text quality.
PRELIMINARIES AND BACKGROUND
We provide a brief summary on flow-matching, following Gat al. (2024). For the notation used and for details on the theory behind discrete flow-matching, readers are referred to Appendix A.
Setup. Fix target distribution P1 of text, thought of as length sequences of token ids. The starting point of the flow-matching approach text generation is probability path (pe)osegi that interpolates between source distribution pa and the target distribution pt. Given this path, we can sample X Pt for 1 E [o, 1], which leads to a stochastic process (Xt)osıg1-
Discrete Flow-Matching (DFM) models the stochastic process (Xt)osişı as a Continuous- Time Markov Chain (CTMC). The evolution of this CTMC can be captured by its infinitesimal generator u.(:,)) for t E [o, 1]. For token sequences æ, y. the value u2 y) captures the rate of change of the probability that the sequence will change from state to state y and must satisfy the conditions in proposition A.1. We also have the following equality for small (see Appendix A.1 for notation)
F(X++h y|X; = x) 6,(,) + (1, v) + o(h).
(1)
The goal of DFM is to learn the infinitesimal generator us- DFM makes the further simplifying assumption (to reduce output dimension size) that uy (ar, y) can factorized as
Uy (x, 3)
Σ 6m (1 (r
Similarly, the token-level ut must satisfy analogous conditions to proposition A.1. This reduces the output dimension from [V] to [V] L. making the learning objective feasible. From this, we then have, for small h.
P(X t4 = = ) = 55
+ (u x)) + o(h).
This reduces DFM to learning the factors u². It also provides way of simulating the CTMC and generating samples from noise; given a sample X, we sample X via Euler sampling
X 5x; + h w((; + o(h). i= 1....,,
Of course, this depends on the choice of probability path (pr)osesi. Following Gat al. (2024), start with
pc(x)
Σ 21
20.21
where 3o Po. 31 ~ pr, and æ is joint distribution which relates and pi and
Pr(x x1) = II Pr (zo,
Then set the conditional probabilities pe(a" [xo ( be the convex sum:
(e (zo, = (1 Ne 2., (x²) + kz02, (x'),
(4)
so that the probability path of the ith token position is linear interpolation between the source and the target distributions. Here, Ke is known as a scheduler and must be a monotonically increasing differentiable function : [0. 1] -> [0, 1] where Ko = 0 and Xı 1. Using the "Marginalization
Trick", one can derive the following description of the factorized velocities ut
his
u 1
(pi1(='1=) 82 (x
(5)
where = a token sequence and
P=)((0(12)== = Σ Se, (x) (aa))p((e (P x1)2)
(6)
20.21
Namely, E., T(50 21) = (11) and En, 21) ) po(
4

---

Published as a conference paper at ICLR 2026
In DFM, the model only needs to learn Pije (1je(2)))). For notational convenience, we also set
Ai g(t) = 1 Kr
Learning the Denoiser. Given the factorized velocities derived above, the remaining task is to learn P1ju(-12). We parameterize this conditional distribution using network 0 that outputs logits:
Pije(a [z) softmaax(0(zz)))
where 6 (2, t) denotes the logits for the i-th token position.
To train this model, we need an appropriate loss function. We use the Bregman divergence for the loss function (Lipman et al., 2024, Equation 7.31). Starting from the velocity formulation Equa- tion (5), we can construct loss that encourages the model to correctly predict Pije- Given sample trajectory where &o ~ Pu. P1 are related through म, and is sampled from pa (r) I1), the per-token loss position is:
Li(o1, Tr, -9(() (1)((c)))rr))) dug (x:)) 01= (di) log Pije
This loss encourages the model assign high probability to the true target token of when denoising from The scaling factor g(t) naturally arises from the velocity formulation and ensures proper weighting across different time steps.
METHOD
Our approach comprises two components: Step-Aware Discrete Flow-Matching and a Cumulative Scalar update. The first component exposes the step budget h as an explicit control signal within the DFM generator u, and distills from shortcut teacher model (Frans et al., 2025), allowing the learner make large, single/few-step moves that approximate the cumulative effect of many small updates. The second using a cumulative scalar to aid the model in jumping from a source token to target token with large single/few-ste predictions.
4.1 SWAR- DISCRETE FLOW-MATCHING
We expose the step budgel l as an explicit conditioning signal to the DFM generator so the model learns transitions calibrated to the intended number of sampling steps. To expose the step budget to the model during training, we need a way to produce the Markov chain transition probabilities over large step intervals. However, determining the transition probabilities from the generator 44 is in- tractable. But, because the transition probabilities satisfy the Kolmogorov equations (Equations (15) and (16) which are ODEs, we can use numerical methods to approximate the transition probabili- ties. The role of the teacher model is to approximate the evolution of the Markov chain, to be used as the ground truth for training FS-DFM. Many methods exist for approximating the ODE, each with different trade-offs between accuracy and computational efficiency. We experimented with two Runge- Kutta methods: RK-2 (Heun average) and RK-4 and found that RK-4 greatly improved the results (Figure 3), though at the cost of being more computationally expensive.
RK-4 ODE solver. The RK-4 algorithm is presented in Algorithm 1. Define Vel to be the function which computes the velocity from the logits. In particular, given logits f, current state It, Vel computes the result of Equation (5):
Vel(softmax((),.u 21,, == g(t) (p1)((-1a1)-821/ 8.)
Given the velocity from Vel. apply jump sampling (see Appendix A.3) obtain Jump(æ,, u, h) where is our desired time step. For given time and time step h, RK-4 computes the model logits at and + h/2 using Vel and Jump defined above. Note that only to obtain the relevant state at + h/2 in order to compute the logits there. RK-4 then averages these logits. We include an detailed explanation of RK-2 in Appendix B.2.
Jump
EMA teacher for stability. Because training is non-stationary, using the current student as the teacher causes the shortcut target to drift with parameter updates, which destabilizes large steps
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Shortcut RK-4
Require: tokens 21, time t, step h; Model 8; velocity Vel; CTMC jumper Jump; use_ema flag
1: h +- 11/2: lmid + li;; (next + L
2: 0' - (M) if use_ema else 0
3: l < ('(x,t;h""
"1 <- Vel(softmax(()) (41): R2, h1,
x(1) +- Jump(a,, 21, h!)
4: l2 + ('(x(1), tmiaa h));
u2 f- Vel(softmax x11), J/' (mia);
x(2) +- Jump(x(() u2, h()
5: l - (x(21(2), Imiaa h));
U3 f Vel(softmax (fz), (2) n1, lmia);
x33 < Jump(z(()) u1, h²)
6: 4 < (r)(((), Emery h')
7: RK-4 average: + (el + 282 283 24)
8: return
h where local errors accumulate across sub-evaluations. We therefore maintain a slowly varying exponential moving average (EMA) teacher:
0' + 30' (1-
B 00.1)
Stop gradients through H' The EMA teacher provides stable, low-variance targets over [t, t+hh, improving convergence for large h and making ceff-consistency training robust across step budgets.
4.2
CUMULATIVE SCALAR
For CTMC token update, the marginal velocity separates into scale and a direction (Equation (5)):
k(t)
(x())22)
[pI (3' 8s direction
(10)
1 x())
scale g(()
Scale and jump behavior: With monotone scheduler : J0. 1] 1] (x(0) 0, <(1) 1), few/one-step samplliinn uses a large step h, so the first update typically occurs at small t; for common schedulers, this makes the instantaneous scule g(4) A(1)/(1 x({)) too weak to trigger moves, stalling in early steps (Appendix B.3 contains more details about this argument and our motivations). Empirically, the mean jumps per token are Re 1.05 with a uniform source and se 1.00 with a mask source (most positions change at most once), the direction term in Equation (10) is typically "spent" in a single decisive update, after which tokens rarely move. In this regime, getting the seale right for each finite step dominates quality.
Key question: How can we incorporate the current time I and the step budget h into the scale so that, even when I is small, single finite step delivers the right amount of flow?
To provide the correct amount of probability flow over finite step, we replace the instantaneous scale by a Cumulative Scalar obtained by integrating g over the interval and normalizing by its length:
GLPh GrA=.. b-t ((T) 1-x(T) dr In 1 1- x(1) x((T)
GL In i-x(() k((th)
It.h
(11)
<(t+h)*
Substituting the Cumulative Scalar yields
ut(x', De. (pij((z) (1x1 12) 6x())).
(12)
which calibrates the step strength using both t and h, enabling effective jumps even when is small. Using the Cumulative Scalar in Equation (12) addresses this and improves few-/one-step generation.
4.3 TRAINING APPROACH
We train the step-aware generator 0 to be faithful to the DFM path at small steps and globally consistent with a shorteut teacher over large steps. Each minibatch provides (r,, 21, t, h), where h (0, 1] and i < is the intended step size (the step budget). The student produces logits

---

Published as a conference paper at ICLR 2026
t= f t; ht,, which are used both in small-step path objective (Equation (8)) and for comparison against a large-step teacher defined on [t, t+h]. The sampling process detailed Appendix B.1.
Shorteut teacher. To stabilize and supervise large moves, integrate 0 across the interval using shortcut scheme, implemented with an EMA copy 0' yiclding averaged logits Lec.
Lua RK-2 RK-4 Estimate ,I,,t,
The teacher is treated as a stop-gradient (no backprop through 0*). After each optimizer step, we update the EMA parameters Equation (9).
Losses. Let po( æt, ht) softmax((/T) and Peal Lt, h) = softmax(4ue/ with tempera- ture 2 (in our experiments, T' is always set to 1). Let denote the context length. compute losses tokenwise over all positions and average per sample.
Edist
1
Σ DRi(Preas |l Po.)) (stop-grad on lea)
L
For the DFM path objective, we use the per-token loss from Equation (8) averaged across tokens:
1. Σ (1,
Larm
T,
j=0
where (from Equation (8))
L,( Fz, 4;) = - -c,h (1je w4) 5ng (x+) + -1- log logrne(aa mm)
Budget-aware blending. With threshold on the step size, blend per sample b such that tiny steps (h r) optimize the DFM path loss Laims while larger steps distill to the shortcut teacher.
m The < T.
L
1
1
m& ( + (1 .ma) Ediee
(13)
B
b=1
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Training. Step-aware DFM training incurs additional model evaluations per batch (for shortcut RK-2/RK-4 teachers), making from-scratch optimization expensive. We therefore adopt pretrain -> fine-tune protocol: first pretrain plain DFM backbone, then fine-tune with step-aware objec- tives and the cumulative scalar update. We pretrain the DFM model following Gat al. (2024) and and mask-and three model sizes: 0616 1.3B, and 1.7B
cover two source distributions-umijone parameters for the uniform source, plus 0.169B mask-source model. These checkpoints serve as initialization for FS-DFM fine-tuning, Appendix C. contains all architecture and training details.
We train FineWeb-EBd (Lozhkov al., 2024) and evaluate on WikiText-103 (Merity al., 2017). We use GPT-2 tokenizer and, during preprocessing, we append an EOS token to each document and pack the resulting token stream into contiguous blocks of length 024. We concatenate shorter samples to reach the desired 024 token target.
Step-size schedule. Shortcut integration (RK-2/RK-4 requires evaluating the model both h and 1/2 inside training step. To cover a broad range of inference budgets with single model, we sample from logarithmic grid everything from tiny steps for precise path following to very large steps for few-step generation. In each minibatch, we sample & from the grid (uniform over values unless stated otherwise); the shortcut teacher then internally uses the required h/2 sub-evaluations. We report variants of the h-sampling policy and their impact in Appendix E.1. For budget-aware blending (Equation (13)), set the threshold
[2*] for (-10, 9, _...., 1,
0}. The grid covers
2 for simplicity.
Scheduler choice k. There are many valid probability path schedulers in DFM (c.g., convex or cosine). For simplicity, the linear scheduler rt()) f. which yields g(!) k(t)/( (1 =()))

---

Published as a conference paper at ICLR 2026
Table I: Ablation on scaler formulation across NFEs: integrating the scheduler within each step (Cumulative Scalar) yields fooeedform, probability update that sharply lowers GPT-2 perplexiy--especialyy 1-2 NFEs while maintaining comparable entropy.
2
4
8
1 024
Solver
ppl.
ppl.
ppl.
ppl.
ent.
ppl.
ent.
Scaler
3312.65
6.45
462.31
6.42
194.29
6.90
97.51
7.16
85.61
7.84
Cum. Scalar
514.40
6,08
333.07
6.60
176.19
6.97
90.49
7.29
87.36
7.91
(n(1=rk)) This choice keeps the path well-behaved, simplifies implementation, and focuses our study on step-awer training rather than scheduler design.
1/(1- -) and the cumulative scalar It.A = x In 1=x())
Shorteut Model.
We make the generator step-aware by conditioning on the intended step size alongside time t, i.e., f A( t; h). During training we sample h, construct shortcut teacher over [t, t+]] using an EMA copy of the student, and apply the budget-aware losses from Section 4.3. During inference, we fix budget of S [2*] steps and k 1, .,,.......1111 103, set 1/S, and run the step-aware sampler. Implementation details appear in Appendix C.2.
Measurement. We report several complementary metries: Perplexity (PPL) measured by fixed reference LM (gpt2- 2aragee);; lower is better; and Entropy - the average uncertainty of our model's token distributions; lower indicates sharper, more decisive predictions; Token accuracy: the fraction of model prediction that match the ground-truth; higher is better; and MAUVE (Pillutla et al., 2021), divergence- -rrseennce-based metric that measures how similar the distribution of text generated by model is to that of real human text. All metrics are computed on the evaluation split and averaged across sequences.
Baselines. We compare FS-DFM to broad set of discrete diffusion and few-step generative models. In the main text, we report results against two diffusion language models, LLaDA-SB (Nie et al., 2025b) and Dream-7B (Ye al., 2025), each in Base and Instruct variants, as well as discrete flo--machhin (DFM) baselines at three model sizes (0. 169B, 1.3B, 1.7B). To further contextualize performance, Appendix E.3 expands this comparison to include multi-round refinement models such as SDTT (Deschenaux & Gulcehre, 2025), hybrid
diffusion models such as HDLM (Fathi et al., 2025), and several masked-diffusiinn / re-masking systems (MDLM, SEDD, ReMDM), all evaluated under a fixed few-step generation budget.
5.2 RESULTS
Cumulative Scalar Improves Few-Step Sampling. Table 1 shows RK-4 + Cumulative Scalar performs better than only RK-4. Consistent gains at small budgets: GPT-2 perplexity drops by 60.8% 1 NFE (1 312.65 514.40), 28.0% % at 2, 9.3% at 4, and 7.2% at 8. The improvement comes from integrating the scheduler's rate over [1, [+]] (Equation (11)), which better matches a single large step the effect of many small steps and reduces discretization bias. As h shrinks (more NFEs), the gap narrows but Cumulative Scalar remains superior. Entropy stays comparable and is slightly higher larger budgets (c.g.. 7.29 vs. 7.16 at 8 NFEs), indicating preserved diversity alongside lower perplexity.
The effect of RK-4 compared to RK-2 (Heun average) shown in Figure All evaluations use Cumulative Scalar. Overall. RK-4 performs better for generation because it delivers lower perplexity (typical/median 80 vs. RK-2's ~ 84), especially at lower NFEs. RK-4 generated better result with huge cap at fewer steps, by increasing the value of steps, the cap will decrease, but still, RK-4 better. The trade-off is a slightly higher entropy (median ~ 7.63 vs. 7.50), but the perplexity improvement is larger and more relevant for text quality. Using the median to compare methods is helpful because both metrics (especially perplexity) can be skewed by occasional spikes at cer- tain NFEs. Appendix E discusses RK-4 and RK-2's training-inferene time trade-off, includes an investigation into how step-size weights shape few-step fidelity, and the effect of source distribution.
FS-DFM vs. DFM performance across scales is shown in Figure which compares the two meth- ods at three parameter sizes. Across all sizes, FS-DFM reaches the strong guanggggultty regime in far fewer function evaluations: perplexity drops sharply, accuracy saturates early, and entropy converges the
8

---

same range many-step DFM. The gold markers at NFE=8 highlight that FS-DFM already oper- ates near DFM's s quality plateau after only a few steps, while DFM requires substantially more steps to reach the same values. To measure accuracy across steps and sizes, we randomly change 50% of tokens each sequence with other tokens, provide the remaining 50% as context, and measure the ability to predict the changed tokens. FS-DFM consistently matches surpasses DFM with dramatically fewer evaluations. This behavior is consistent between all evaluated model sizes. As NFE increases (i.e., step size h -> 0), FS-DFM smoothly converges to DFM: the cumulative scalar factor satisfies Gt.h >> g(t), so the updates coincide and the curves merge. The slight accuracy dip very large NFEs likely reflects train-test mismatch: the model and shorteut teacher are optimized bias and under-use the Published as a conference paper at ICLR 2026
02
14
19-1
2'
0.22
Figure 3: RK-2 vs. RK-4 across NFE. Top panels show entropy (linear and perplexity (log-log). with ribbons and vertical connectors highlighting pointwise gaps. The bottom shows the deltas (Á entropy RK-4 RK-2; %A perplexity RK 44K4/KK-2 1). RK-4 has consistently lower perplexity ratio 0.88 with a small entropy trade-off (median 7.63 vs. 7.5), making it the stronger choice for generation over most NFE settings.
Table 2: FS-DFM vs. diffusion LMs across step budgets. Each method receives 512-token prefix; metrics are computed on the 51 512-token continuation only (ppl perplexity, ent = entropy, MVE= MAUVE Pillutla al. (2021)). FS-DFM attains competitive quality in few steps with stable entropy across sizes, whereas basclines generally require many steps.
Method
(B)
ppl.
cat.
MVE
ppl.
ent.
MVE
ppL
cot.
MVE
ppl.
ent.
MVE
ppl.
ent.
MVB
Dream
7,00
163.08
1.55
785.87
1.45
0.005
752.11
0.005
739.40
1.74
630.30
2.31
0.005
LLaDA
8.00
256.07
0.84
0.005
290.35
0.59
0.005
495.17
0.47
0.005
441.26
0.42
0.008
432.65
0.50
0.005
7.67
97.07
7.95
FS-DEM
1,30
231.89
7.88
0.006
169.99 99
7.97
0.013
99.79
7.98
0.120
70.97
8.01
0.480
$9.84
7.99
0.583
7.67
61.67
for large h, so many small steps accumulate discretizzatiomrenoomaiiii step-aware conditioning.
Comparison to State-of-the-Art Diffusion LMs. Table 2 compares FS-DFM with contemporary base diffusion LMs in the few-step regime (1-+ 16). Even though the largest FS- FS-DFM model is more than four times smaller than LLaDA and Dream, the comparison illustrates FS-DFM's generation quality. FS-DFM reaches a strong-quality regime few steps across model sizes: perplexity drops rapidly steps increase while entropy remains well-behaved, indicating calibrated predictions with- out long iterative trajectories. many more denoising steps and do not reach comparable perplexity. Even after 16 steps, LLaDA and Dream achieve MAUVE scores equivalent to our smallest models score after a single step, de- spite being 41 times larger. A quick look at LLaDA and Dream's 's generations shows the models just repeat some tokens many times in a few-step setting (see Figure 8). Full result is available in Appendix E.4.
contrast, LLaDA and Dream are sensitive to step count and require

---

Published as a conference paper at ICLR 2026
6 DISCUSSION AND CONCLUSION
We introduced FS-DFM, a step-aware discrete flow-matching language model that matches the per- plexity of similar-size 024-step discrete-flow baseline in 1 024 tokens generation with just steps, yielding up to 128 times faster sampling, The key ideas in FS-DFM are (i) conditioning on user-specified step budget and training such that one large move agrees with many small ones, and (ii) operating at the interval level via cumulative scalar update that preserves the probability simplex and remains stable at large steps. We realized these ideas with shorteut teachers built inside the DFM framework and stabilized by an EMA teacher. Together, these components make few-step sampling accurate, controllable, and robust. While our experiments used the Runge-Kurte Kutta methods to approximate solutions to the Kolmogorov equations, it would be interesting to explore other ODE solvers to determine how they might affect either generation performance or training efficiency. Fi- nally, because research in one/few-stey step diffusion generation is hindered by lack of public discrete flow-mate matchinth models, we release our DFM and FS-DFM models and code. Appendix D presents more information about future work and limitations.
ACKNOWLEDGEMENT
We would like to thank Ruixiang Zhang for his valuable insights and continued support throughout the project, particularly for his thoughtful feedback and guidance in addressing key challenges.
REFERENCES
Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixua Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block Diffusion: Interpolating Between Au- toregressive and Diffusion Language Models. In Proceedings of The Thirteenth International Conference on Learning Representations, 2025.
Jacob Austin, Daniel Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances Neural Information Processing Systems, volume 34, pp. 17981-17993, Curran Associates, Inc., 2021.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 28266-28279. Curran Associates, Inc., 2022.
Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Ma- chine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 5453-5512. PMLR, 21-27 Jul 2024.
Tianqi Chen and Mingyuan Zhou. Learning to jump: Thinning and thickening latent counts for generative modeling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 5367-5382. PMLR, 23-29 Jul 2023.
Tianqi Chen, Shujian Zhang, and Mingyuan Zhou. DIm-one: Diffusson language models for one- step sequence generation. arXiv preprint arXiv:2506.0022 2025a.
Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai "Hellen" Li, and Yiran Chen. DPad: Efficient Diffusion Language Models with Suffix Dropout. arXiv preprint arXiv:2508. 14148, 2025b. doi: 10.485500arXw.25500. 14148.
Justin Deschenaux and Caglar Gulcehre. Beyond autoregression: Fast LLMs via self-distillation through time. In The Thirleenth International Conference Learning Representations, 2025.
10

---

Published as a conference paper at ICLR 2026
Nima Fathi, Torsten Scholak, and Pierre-André Noël. Unifying autoregressive and ciffusiionbbased sequence generation. arXiv preprint arXiv:2504.06416, 2025.
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In The Thirteenth International Conference on Learning Representations, 2025.
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 133345-133388 Curran Associates, Inc., 2024.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. In The Eleventh International Conference on Learning Representations, 2023.
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models. In The Thirteenth International Conference on Learning Representations, 2025a.
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration. arXiv preprint arXiv:2506. 20639, 2025b.
Aaron Grattafiori, Abhimanyu Dubey, and Others. The Ilama herd of models. arXiv preprint arXiv:2407.21783, 2024.
Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Bjöm Ommer, and Cees Snoek. Flow matching for conditional text generation few sampling steps. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Asso- ciation for Computational Linguistics (Volume 2: Short Pupers). pp. 380-392, St. Julian's ss Malta, March 2024. Association for Computational Linguistics. doi:
Amin Karimi Monsefi, Mridul Khurana, Rajiv Ramnath, Anuj Karpatne, Wei-Lun Chao, and Cheng Zhang. Taxadiffusion: Progressively trained diffusion model for fine-grained species generation. arXiv preprint arXiv:2506.0199 2025.
Inception Labs, Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, Stefano Ermon, Aditya Grover, and Volodymyr Kuleshov. Mercury: Ultra-Fast Language Models Based on Diffusion. arXiv preprint arXiv:2506. 17298, 2025.
Ethan Li, Anders Boesen Lindbo Larsen, Chen Zhang, et al. Apple intelligence foundation language models: Tech report 2025. arXiv preprint arXiv:2507.135/1 2025a.
Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A Survey on Diffusion Language Models. arXiv preprint arXiv:2508. 10875, 2025b. doi: 10.48550/axX1.22500..053
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion- LM improves controllable text generation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp- 4328 4343. Curran Associates, Inc., 2022.
Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, and Weizhu Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 21051-21064. PMLR, 23-29 Jul 2023.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Repre- sentations, 2023.
11

---

Published as a conference paper at ICLR 2026
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:2412.066660 2024.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint 1rXi:22300.16344, 2023.
Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Q Weinberger. Latent diffusion for language generation. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 56998- 57025. Curran Associates, Inc., 2023.
Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. Fineweb-edu: the finest collection of educational content, 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture
models. In International Conference Learning Representations, 2017.
Pouyan Navard, Amin Karimi Monsefi, Mengxi Zhou, Wei-Lun Chao, Alper Yilmaz, and Rajiv Ramnath. Knobgen: controlling the sophistication of artwork in sketch-based diffusion models. arXiv preprint 100.5.241..01595, 2024.
Shen Nie, Fengqi Zhu, Chao Du Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models text. In The Thirteenth International Conference on Learning Representations, 2025a.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint (tt::2502.009992 2025b.
Jingyang Ou. Shen Nie, Kaiwen Xue, Fenggi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. In The Thirteenth International Conference Learning Representations, 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pp. 4195-4205, October 2023.
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers. In Advances in Neural Information Processing Systems, volume 34, pp. 4816-4828. Curran Associates, Inc., 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37: 17:130136-130188 2024.
John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Fe- lipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, et al. Chatgpt: Optimizing language models for dialogue. OpenAl blog, 2(4), 2022.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and gener- alized masked diffusion for discrete data. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomezak. and Zhang (eds.). Advances in Neural Information Processing Systems, volume 37, pp. 103131-103167 Curran Associates, Inc., 2024.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. based generative modeling through stochastic differential equations. In Interna- fional Conference on Learning Representations, 2021.
12

---

Published as a conference paper at ICLR 2026
Gemini Team, Rohan Anil, Sebastian Borgeaud, Sean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXtv:2312.11805, 2023.
Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. arXiv preprint arXiv: 250000 2025.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dLLM: Training-fre Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding. arXiv preprint arXiv:2505.22611 2025.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 200Y::2505.09388,, 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong, Dream 7B: Diffusion Large Language Models. arXiv preprint arXiv:2508. 15487, 2025.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. Proceedings of the IEEF/CVF international conference on computer vision, pp. 3836-3841 2023.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian- Yun Nie, and Ji-Rong Wen. A survey of large language models. arXiv preprint 2r03v 18223, 2025.
Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. First Conference on Language Modeling, 2024.
13

---

Published as a conference paper at ICLR 2026
Appendix
APPENDIX CONTENTS
A Preliminaries
15
A.1 Notation
15
A.2 Basics of Continuous Time Markov Chains
15
A.3 Jump Chains
16
B Supplementary Methods
17
B.1 Sampling
17
B.2 RK-2 Shorteut Teacher
17
Checkerboard Jump-Dynamics Demonstration
17
C Implementation Details
18
C.1 Pre-train model architecture
18
C.2 Shortcut Model: Implementation Details
18
C,3 Training Details
19
D Limitations and Future Work
19
E More Results
19
E.1 Ablation Studies
19
E.2
Traieing-Inferencee
Inference Time Trade-off's
20
E,3
Expanded Baseline Comparison and Positioning of FS-DFM
20
E,4 Full comparison
22
E.5
Sample Outputs
22
14

---

Published as a conference paper at ICLR 2026
A PRELIMINARIES
A.1 NOTATION
We set the following notations:
L denotes the length of sequences,
denotes the vocabulary, i.e. the set of token ids, so |V denotes the vocabulary size,
For sequence I and integer 1 S i s L, x² denotes the value of x at position i. So
x (z ......"),,.
For sequences x and y we set
(x(x)
II by (x*)
where oy denotes the usual delta function,
For a sequence y and x', we set 5y(xx) = 5y. (z²),
for sequences and y, we set (x')
Sy y (x') I 0g (x²)
A.2 BASICS OF CONTINUOUS TIME MARKOV CHAINS
We include an overview of the theory of CTMCs focusing on the time- inmooionoogeeeuus case and where the state space S discrete, since that is the type of Markov chain modeled in this paper.
CTMC is a stochastic process (Xt)oses1 indexed by a continuous variable that satisfies the 1 and elements i1,-.., E S.
Markov property: for sequence 0 S 41 <
P(X,, i, 11..... Xt,, = in-1) P(X,, in in-1). The evolution of CTMC is dictated by its transition matrices P(s,t) for 0 S 8 S t S where for state z and state in S,
(s, P(X: = v|X,, xx).
We derive the Shapman-Kolmogann equations,
P(s,t) P(s,))
(14)
for S S Note that P(s, t) is a stochastic matrix.
Assuming that P is differentiable in and f, we can define infinitesimal generator or generator as
lim + -
where 7 denotes the identity matrix. For states x and y. ur(x, y) denotes the (x. y)-erty of u, The generator and the transition probabilities also satisfy Kolmogorov forward equation
a,P(s,t) P(s
(15)
and the Kolmogorov backward equation
(s,,t) = -u,P(s,t)
(16)
Proposition A.1. Let ue be the infinitesimal generator of a CTMC, then un satisfies the following conditions.
1. ux(I, v) for all + y and t,
2. Σy uxxx,) y) 0 for all and
3. x)) <
15

---

Published as a conference paper at ICLR 2026
The entries ue (2, y) are the rate of change in probability from state into when + y, while - -v (x, is the rate of change in probability for staying in state æ. In other words, we have:
P(X++h X| x) 5x(1) + hu,(y + o(h)-
(17)
That is, 4y provides a first-order Taylor approximation of the transition probabilities. This suggests that we could also construct Markov chain if only had the instantaneous rate of change provided by the generator Wt.
Given continuous generator u, that meets the conditions in Proposition A. define Markov chain by defining t)) to be the unique solution to the forward and backward Kolmogorov equations. Equivalently, (assuming u, is integrable), can write
J.
p(s,t)=
P(s, u) u)uddu.
In practice, solving for P is difficult, so one usually uses numerical methods to solve these ODEs.
A.3 JUMP CHAINS
Given a CTMC over a discrete state space with generator dt we described how the transition probabilities can be derived from for small differences (Equation (17)). We show how this is used practically when sampling for DFM. Assume is small (for example 1/1 024). The exit rate at time t for state is given by the sum
(z Σ =x
ue (v. 2)
Since the state space is discrete, the stochastic processes jumps from one state to another suddenly, and is constant the rest of the time. For small h. can assume that the velocity does not change (i.e. we can treat as constant on [t, + ++++))) k)). Under this approximation, the holding time until the next jump is approximately exponentially distributed with rate (t).
In this case, the chance of jumping state can then be modeled by the Poisson distribution exp(=h\,(t). In other words,
Pjjump in [t, L + N| = x) = 1 - exp(-
Note that taking the Taylor expansion for the RHS shows that
P(jump in [t, M||x = x) = h2) + o(h) =
Σ wa
8z(y) + (24 y) + o(h)
the last term being derived from Equation (17), justifying that the probability of jump in [t, + h)
approximately exponentially distributed. Finally, if jump occurs, the probability of transitioning from state æ to state y is given by
ue(x, P( yljump occurs, X, x) (z
Specializing to the case of DFM, we can consider the factorized velocities uf for each token position i (equation 2). For fixed position i and state The exit rate is similarly defined as
k(t) 1- x-x(t) (1- (1-Pij((z)) lz))
(x)) = Σ (a 2)
(18)
azz
The jump probability in token position is also approximately exponentially distributed according to the exit rate,
Pfjump in [t, (h)) = 1 - exp( - (;))).
(19)
Conditioned on a jump, the next token is sampled from the off-diagonals, i.e. the posterior renor- malized exclude the current token:
uf(a,z) Pije(a| 2)
P(XAA =a a jump, X,
= x,=
(a + z:).
(20)
x{(2) I - --Pije(2; 2)
16

---

Published as a conference paper at ICLR 2026
B SUPPLEMENTARY METHODS
B. SAMPLING
At test time, we simulate the jump process (see Appendix A.3). Choose a budget S with grid & and steps h2 = 1,+1 - to (uniform h, -15). Initialize Xt, (uniform or mask). At step compute
logits l, = Xt,,, taj h2) and set
Pilta (x,) softmax((./T)).
Using the Cumulative Scalar It. h, from Equation (11), form the per- per-position exit rate (cf. Equa- tion (18)
2 hh, (1 (Xi, Xr.).
Draw a jump with the exponential holding-time law (Equation (19))
J Bernoulli (1 - (1- h,,))) (1.e)
and, if 111,, sample the next token from the off-diagonals of Pije, ( Xt,) renormalized to exclude the current token:
X 1,41 ~ Cat
Pile, (a (t,))
a xt. else set X; = X
- Pijt, (X;, Xt.)
This uses exactly one forward pass of 0 per step; conditioning on h, lets a single checkpoint support different budgets
B.2 RK-2 SHORTCUT TEACHER
We include lightweight RK-2 (Heun) shortcut teacher alternative RK-4. It approximates the interval-averaged logits over [t, (+]] with just two model evaluations: one at (ra, t) and one midpoint state reached by half-step jump (t+h/2). This keeps compute low (two forward passes) while still providing stable, CTMC-consistent targets for single large step of size h; see Algorithm 2.
In practice, RK-2 is useful when compute or memory are tight or for ablations that isolate the effect of teacher strength. While RK-4 typically offers stronger guidance for very large steps, RK-2 strikes good accuracy-cosi balance and integrates seamlessly with our step-aware DFM setup.
Algorithm Shortcut RK-2 Teacher
Require: tokens Zy, time step h; Model 0; velocity Vel; CTMC jumper Jump; use use_ema flag 1: h/ +- h/2; Emid f- t h'
2: 8' +- EMA(O) use ema else
3: l
((Ix,(;h')
U1 +- Vel(softmax(()), 11, f Jumpppp un, I!)
4: l2 + -O, (mia; 1!)
5: RK-2 average: +- (f1 + E2)
6: return
B.3 CHECKERBOARD JUMP-DYNAMICS DEMONSTRATION
In this subsection, we want to motivate the cumulative scalar Ht, We visualize discrete flow- matching on a synthetic 128 128 checkerhoard target to study jump dynamics. The data generator draws the first coordinate z1 uniformly from {O...., 127); the second coordinate I2 is coupled to the parity of [x| /32]. yielding alternating 32 x 32 blocks (checkerboard). We run a CTMC sampler on uniform grid [0, (100 frames) with the instantaneous scale g(t) k(t)/(1 <())), and compare two source distributions Po: (i) an all- [MASK] source (we add a dedicated mask token to the vocabulary), and (ii) uniform source over tokens. We wish to emphasize that the images in Figure 4 indicate discrete matchingn on distributions of samples (in this case points in the plane).
17

---

Published as a conference paper at ICLR 2026
(a) All- [MASK] source
00B 1122
(b) Uniform source
Discrete flow-matching on 128 128 checkerboard under the instantaneous scale g(t). (a) With an all- [MASK] source, early frames exhibit almost no jumps (stalling near z 0). (b) A uniform source jumps earlier but still under-updates at tiny t. Both patterns motivate the Camulative Scalar 9t,A (Equation (11)) to supply the correct probability flow over finite step.
Figure 4:
The trajectories reveal the role of the scale term early times. With all- [MASK] source (Fig- ure 4a), frames near & x 0 show almost no motion: common schedulers make g(t small at the start, so exit rates tiny and the chain stalls despite having the correct denoising direction (see the jump-process formulation in Appendix A.3). A uniform source (Figure 4b) spreads initial mass over valid tokens and induces earlier movement, yet the first frames still under-shoot when driven solely by (t). These behaviors motivate replacing the instantaneous scale by the Cumulative Scalar Ir. (Equation (11)), which couples the update strength to both the current time and the finite step and makes few-/one-step updates effective.
C IMPLEMENTATION DETAILS
C.I PRE-TRAIN MODEL ARCHITECTURE
Backbone. We use DiT-style G (Peebles & Xie, 2023) with rotary attention. Tokens are embed- ded once, rotary phases are computed once per forward pass and applied to Q, K, and each block applies multi-head self-attention followed by MLP. A conditioning vector drives adaptive layer normalization: each block predicts per-channel shift/scale (and a small residual weight) from c and applies them before attention and before the MLP. The final linear head is aro-iintialized for sta- ble spart-up and produces logits; conversion of logits to CTMC generator and ODE stepping are handled by the solver outside the backbone.
Time and step-size conditioning. We embed continuous time t [0. 1] with a sinusoidal MLP. We also embed the intended inference step size h and fuse the two with a linear+SiLU layer:
crme (1) or c SSLUWW [drime (1); (h)))))
The same e is fed to all transformer blocks and the final layer, making the model step-aware single set of weights works with different integrator schedules at inference (few large steps or many small ones) without retraining. The backbone's output remains logits; downstream components convert these to CTMC-valid generators and perform the PF-ODE update for the chosen step size h.
C.2 SHORTCUT MODEL: IMPLEMENTATION DETAILS
Traininglevaluation protocol. In training, the student predicts = (xr, h) and is supervised by (i) small-step path loss and (ii) large-step distillation loss against the shorteut teacher, blended by the budget-aware rule in Equation (13). At evaluation, we select a step budget S, set = 1/S, and apply the Buler--wtth-vverage--hhe
update each step; additional tuning is required.
18

---

Published as a conference paper at ICLR 2026
C.3 TRAINING DETAILS
For initialization, we run 1M iterations with a global batch size of 128; each sample contributes 512 predicted tokens, for a total of 65.5B supervised tokens. For fine-tuning (same 512 predicted tokens/sample, 500K iterations per model), batch sizes are 96 (0.17B).32 (1.3B), and 16 (1.7B), yielding about 24.6B, 8. 19B,, and 4.10B tokens, respectively. Training used 8x A100-80GB com- pute nodes. We optimize all models with AdamW, using cosine-annealed learning rate schedule. We use a peak learning rate of 3e-4 for pre-training and 1e-5 for fine-tuning.
D LIMITATIONS AND FUTURE WORK
Our study primarily targets long-horizon language modeling, and our evaluation emphasizes per- plexity and masked-token accuracy; broader assessments (instruction following, QA/reasoning, and multilingual settings) remain to be done. A practical limitation is the lack of public checkpoints for discrete flow-matching: Accordingly, we plan to release DFM and FS-DFM models and training code to support reproducibility and downstream research. On efficiency, we aim learn adaptive, position-aware step schedules and budget controllers that maintain few-step quality while further reducing inference cost. Scaling to ultra-long contexts (8 (8-32k+ tokens) with memory-effciiet at- tention and retrieval is another important direction. Beyond likelihood metrics, we will incorporate alignment-oriented objectives (e.g.. preference optimization) to improve instruction fidelity and cal- ibration. Another choice made in our experiments was using the Runge- Kutta method RK-4 to ap- proximate solutions to the Kolmogorov equations. would be interesting to experiment with other ODE solvers and see how they affect performance. Also, we only inspected either uniform source or masked sources, it would be interesting to examine how a hybrid scheduler mixing masked and uniform sources could affect performance. Finally, integrating FS-DFM with systems techniques such as speculative decoding and cache reuse could compound latency gains at deployment, helping translate few-step discrete flows into practical, general-purpose generators.
E
MORE RESULTS
This section includes more experiments, especially ablation studies, and some generated samples.
E ABLATION STUDIES
We assess each design choice by varying single factor while holding the rest of the pipeline fixed. Unless stated otherwise, all ablations use the same data split, model size, optimizer and schedule (AdamW with cosine), the step-size grid (1/1024..... .. 1} with same policy of sampling, the shortcut teacher (RK-4 with EMA) during fine-tuning, the Cumulative Scalar for training and sampling, a uniform source distribution, and identical inference budgets S E {1, 4, 8) with temperature T-1.
Effect of Step-Size Sampling Weights on Few-Step Fidelity. We ablate the training-time sampling weights over the step-size grid from Section 5.1 (ordered from the smallest the largest step) to understand how biasing minibatch selection toward large steps affects few-step performance. We evaluate five policies: (i) "tail-boosted" (TB-10) scheme with uniform
weights except on the largest step, [10, 1, 1, 1, 1, 1]: (ii) stronger tail boost (TB-20), I, 1, 1,1, 1]: (iii) pure uniform (PU) with the same weight on all step sizes; (iv) geometric (G), strge-step-ooe biased schedule [2 120022 2 ....... 222; and (v) an annealed geometric (AG) variant that starts as geometric (G) and transitions to pure uniform (PU) by doubling weights every 10 steps up to 210, so by 100 000 steps, all weights equal 210 This experiment probes two intuitions: (a) modest to strong emphasis on the largest / should sharpen single/few-ster accuracy by exposing the student to more shortcut-distille targets; and (b) annealing the bias away (policy AG) preserves this benefit early while restoring balanced coverage later, stabilizing path-following at small h without sacrificing few-step quality. The results (Table 3), indicate that the strongest tail boost, (TB-20), performs best. We adopt this setting for all subsequent experiments.
Effect of source Distribution for Few-Step Generation. We study the effect of the source distri- bution used by DFM on few-step sampling. Following Gat et al. (2024), we compare a mask source (all positions start dedicated [MASK] state) against uniform source (initialized with random
19

---

Published as a conference paper at ICLR 2026
Table 3: Step-size weight ablation over {2-*} for k E (-10, -9, -8, -,, 13: TB-10, TB- 20, PU (uniform), G (geometric), AG (annealed). Bias toward large h improves few-step quality.
1
2
4
8
1024
Policy
ppl.
ent.
ppl.
ent.
ppl.
ent.
ppl.
ent.
ppl.
ent.
TB-10
834.85
6.51
325.32
6.66
159.59
6.99
94.72
7.19
85.13
7.94
TB-20
514.40
6.08
333.07
6.60
176.19
6.97
90.49
7.29
87.36
7.92
PU
3.01
0.64
30.81
1.73
18.67
2.31
21.54
2.73
262.53
6,96
G
353,99
7.57
635.59
7.71
250.89
7.85
116.99
7.85
96,20
7.95
AG 906.15
5.82
439.41
5.93
161.22
6.37
87.73
6.63
77.83
7.76
Table 4: Effect of source distribution on FS-DFM. Uniform source initialization avoids first-escape bottlenecks from [MASK] and delivers lower perplexity with comparable entropy at small NFEs; mask source often collapses in the step regime.
Source
2
1224
Distribution
ppl.
ent.
ppl.
ent.
ppl.
ent.
ppl.
ent.
ppl.
ent.
Mask
4438.49
8.62
2844.42
8.64
I 082.07
8.48
413.43
8.40
364.61
8,36
Uniform
777.02
6.24
451.79
6.54
211.04
6.75
136.60
7.38
93.24
7.95
tokens). With only one few steps, the CTMC has limited opportunity escape [MASK]; tran- sitions concentrate on small set of outcomes or remain at [MASK], producing collapse and high perplexity. In contrast, the uniform source spreads initial mass over real tokens, so the model per- forms corrective refinements rather than first-escape jumps. This yields markedly lower perplexity with healthy entropy tiny budgets and remains competitive as the budget grows. Table shows that uniform source is consistently superior in the few-step regime, while the performance gap nar- rows with larger NFE as more transitions become available. For this experiment, we used the RK-2 method for both mask and uniform sources.
E.2 BRAAINGGINNPRENCE TIME TRADE-OFFS
Shortcut teachers (RK-2/RK-4) improve few-step fidelity by querying the model at multiple abseis- sae within each training step, which increases per-iteration compute relative to pure DFM. In our setup, RK-2 adds roughly 33% wall-clock time per training step, while RK-4 adds about 100% (due to the extra forward passes and intermediate updates). We mitigate this cost in two ways: (i) fine-tune from a strong pre-trained checkpoint, limiting the number of optimization steps quired: and (ii) we use shortcut teachers only during fine-tuning to teach large-step consistency, whereas inference uses very few steps (e.g., 1-8 NFEs), yielding substantial latency and energy savings over the model's lifetime. Consistent with the ablation in Appendix 1, the stronger tail- boost policy (TB-20) delivers the best quality-compute trade-off: it concentrates training on large steps- where shortcut guidance is most salable--wthhuu degrading small-step path following. Overall, the transient training overhead is amortized by the persistent inference gains, making the net runtime favorable for deployment.
E.3 EXPANDED BASELINE COMPARISON AND POSITIONING OF FS-DEM
To more comprehensively situate the performance of FS-DFM within the broader landscape of dis- crete diffusion and few-step generative models, we extend our baseline comparisons in Table 5 to include wider of modern architectures. This expanded set goes beyond the baselines reported in the main paper and incorporates models that vary significantly in training budgets, refinement strate- gies, masking mechanisms, and model size. Including these baselines is essential for establishing more complete understanding of how FS-DFM performs relative to systems that rely iterative refinement, large-token pre-training, or hybrid aentinuousddiscree generative mechanisms.
A first category consists of multi-round refinement models such SDTT (Deschenaux & Gul- cehre, 2025), which improve quality through multiple decoding rounds. Although additional rounds consistently reduce perplexity, even rounds of refinement remain substantially behind the per- formance of FS-DFM at only eight sampling steps (see Table 5). Beyond this quantitative gap, the two approaches also differ conceptually: SDTT accelerates generation by sampling from many-
20

---

Published as a conference paper at ICLR 2026
Table Expanded baseline comparison under an 8-step generation budget. All models are eval- uated with the same number of sampling steps (8). For FS-DFM, the first 65.5B tokens correspond the DFM pre-training corpus, and the additional tokens (#) denote fine-tuning on the FS DF-DFM objective. This table aggregates the most widely used discrete diffusion, masked-diffusion, and hy- brid few-step baselines, allowing for a unified comparison across model size, training-token scale, and evaluation metrics. Training data metrics are provided to facilitate better comparison across methods.
Model
Size (B)
Tokens (B)
PPL
Entropy
Training Data
14.8
5.44
Multi-round refirement models
SDTT (1 round) (Deschenaux & Gulcehre, 2025)
0.86
612.38
5.80
SDTT (3 rounds)
(Deschenaux & Gulcehre, 2025)
0.86
376.19
5.71
SDTT (5 rounds) (Deschenaux & Gulcehre, 2025)
220.81
5.55
SDTT (7 rounds) (Deschenaux & Gulcchre, 2025)
131.71
5.26
Hybrid discrete-continuuus diffusion models
HDLM (c 0.01) (Fathi et al., 2025)
0.16
524
279.60
5.32
HDLM (7 0.05) (Fathi et al., 2025)
0.16
524
192.74
5.62
Masked diffusion discrete iterative models
MDLM-OW MILM--WTTTO (Sahoo et al., 2024)
0.17
262
837.58
5,90
SEDD (Lou et al., 2023)
0.32
602.09
5.85
ReMDM cap (Wang et al., 2025)
0.17
262
559.74
5.79
RcMDM-rescale (Wang et al., 2025)
0.17
262
560.06
5.79
ReMDM-conf (Wang al., 2025)
0.17
262
549.86
5.79
ReMDM-loop (Wang al., 2025)
0.17
262
1358.84
5.85
Training Data
14.7
7.70
Original Discrete Flow Matching (DFM) baselines
DFM (Gat al., 2024)
0.17
65.5
335.64
8.06
DFM (Gat al., 2024)
1.30
65.5
331.36
8.11
DFM (Gat al., 2024)
1.70
65.5
444.71
8.18
Fast Version of DFM
FS-DFM FS-DFM FS-DFM
0.17
65.5 + 24.6
90.49
7.29
1.30
65.5 + 8.27
77.21
7.40
1.70
65.5 + 4.1=
73.65
7.61
step teacher and distilling its predictions, whereas FS-DFM directly Iearns step-size-awart finite- step Kolmogorov dynamics via RK-based ODE integration and a cumulative scalar inside the DFM framework. In other words, FS-DFM models how probability mass should evolve under different step budgets, rather than only compressing a fixed long-run run sampler, which helps explain its stronger few-step behavior.
A second category includes hybrid difttrsionnautoreeressiiee models such as HDLM (Fathi et al., 2025). These models leverage continuous-time diffusion in combination with powerful sequence encoders and benefit from extremely large training sets sets-often exceeding 500B tokens. Despite this enormous training budget, their perplexity remains noticeably higher than that achieved by FS-DFM using a considerably smaller pre-training and fine-tuning corpus. This reinforces the training-token efficiency of our fast blowd method.
We also include several discrete SEDD (Lou al., 2023). These approaches attempt improve sample quality by strategically masking tokens and applying multi-pass denoising schedules. Some of these baselines rely on ex- ceptionally large training corpora, reaching into the hundreds of billions of tokens. Yet even under such favorable conditions, their perplexity and entropy remain less competitive. By contrast, FS- DFM achieves substantially stronger results with a dramatically smaller token budget, highlighting the impact of our distillation strategy and few-step training design.
and re- re-masking systems -ysteMs-ReMDMM variants, and
Finally, we include the original DFM models from (Gat et al., 2024). These models are trained on the same 65.5B-token corpus the backbone of our system, making them direct and informative reference point. The results show that FS-DFM improves perplexity by more than factor of four to five compared to these predecessors while preserving the same generation budget of eight sampling
21

---

Published as a conference paper at ICLR 2026
Table 6: FS-DFM vs. diffusion LMs across step budgets. Each method receives 512-token pre- fix; metrics are computed on the 512-token continuation only. FS-DFM attains competitive quality in few steps with stable entropy across sizes, whereas baselines generally require many steps.
Size (B)
16
Method
ppl.
MVE
ppl.
MVE
ppl.
MVE
ppl.
MVE
ppl.
MVE
Dream-B
7.00
163
1.55
0.005
785
1.43
0.005
752
1.53
0.005
739
1.74
0.006
630
2.31
0.005
Dream-I
7,00
46.371
0.58
0.005
681
0.35
0.006
66
648
0.20
0.007
0.13
0.006
57
694
0.10
0.006
LLaDA-B
8.00
256
0.84
0.005
290
0.59
0.005
495
0.47
0.005
441
0.42
0.005
432
0.50
0.005
LLaDA-1
8677
0.53
0.012
8869
0.41
0.014
9049
0.40
0.029
9259
0.40
0.027
9889
0.43
0.048
FS-DFM
0.17
173
7.67
0.006
143
7.85
0.008
97
7.89
0.053
75
7.95
0.270
67
7.97
0.390
RS-DFM
1.30
231
7,88
0.006
169
7.97
0.013
99
7.98
0.120
8.01
0,480
7.99
0.583
FS-DEM
1.70
191
7.67
0.007
155
7.93
0.014
101
8.03
0.083
72
8.07
0.311
8.06
0.550
steps. This comparison isolates the contribution of our flow-distilled fast sampler from confounding factors such as model size data scale.
Taken together, this expanded baseline analysis demonstrates that FS-DFM consistently matches or surpasses models that rely deeper refinement loops, larger-scale diffusion procedures, masking strategies, or extensive training-token resources. The results highlight both the sampling efficiency and the training efficiency of approach, positioning FS-DFM as of the strongest few-step discrete generative models currently available.
E.4 FULL COMPARISON
Table 6 compares FS-DFM with contemporary diffusion LMs in the few-step regime (1 - 16) across both baseline and instruct models. Even though the largest FS-DFM model is multiple times smaller than LLaDA and Dream, the comparison illustrates FS-DFM's generation quality. For each example, we supply a 512-token prefix and evaluate only the 512-token continuation. FS- DFM reaches strong-quality regime in few steps across model sizes: perplexity drops rapidly as steps increase while entropy remains well-behaved, indicating calibrated predictions without long iterative trajectories. In contrast, LLaDA and Dream are sensitive to step count and re- quire many more denoising steps for comparable generation quality. For the Base models we just gave the 512 tokens without any instruction but we prefixed inputs to the instruction models with the prompt: Complete the following text as coherent passage. Do not repeat the given prefix, just continue it. + given tckens
E.5 SAMPLE OUTPUTS
Overview. Figure presents a complete, high-resolution version of the qualitative result shown in the main text (Figure 2), displaying the full 8-step evolution of the sample.
We would like to acknowledge Ruixiang Zhang for providing valuable insights in our initial draft, and helping to address them.
Corruption-recovers Figures 6 and evaluate a 024-token restoration task: starting from a valid sequence, we uniformly replace half of the tokens at random and ask the model to repair the corruption. FS-DFM performs this reconstruction in steps (no teacher or micro-steps at inference), demonstrating that few steps suffice move from heavily perturbed inputs to coherent long-form text.
Head-to-head continuations (512-token prefix). Figure 8 compares FS-DFM with LLaDA and Dream on next-token continuation from shared 512-token prefix (top panel). The subsequent panels show each method's 's 8-step outputs. For LLaDA, we use cow.confidene with block length max max1552/step, 32) and temperature 0.0. This standardized setup highlights the qualitative differences between few-step discrete sampling (FS-DFM) and iterative remasking or diffusion-style baselines.
remasking
Trajectory visualization.
To reveal where edits concentrate over time, Figure colors each token by the last step which it changed (8 bins from start-rend). Tokens that stabilize early retain
22

---

Published as a conference paper at ICLR 2026
Eixere, the, the.. the... them
EDCre the.. the,, thC the our the, people's. du., the onr the, individualss, the... the. don't, the.. the, on the, the cossumn Iitososon the, the the: MiCommmeeee the, the, the, the.omm the the, thee, E.C.oo the, the... Ro the, the; the lhe, the, (hmm yol the, Ietoeooo the
the... the, the, the.....
Dreum-78
the; the Biigoeenttetteee
the; the the
the
the Ee the the..rnee the the system, the.. Ramooo the.oooooonn the, the toooooon fhe, the... the
the,
the, the
thegmm
LLaD
general, excreisingercising week week genetally the most effective pain relief for arbritis arthritis, can physical moeenmooee and genentte
general, Exurcising every week is generally most effective pain relief for sdolt arthritss
Chronic pain and fatigue: Adult arhriiss joint pain can also cause strain and lead to severe and sumetimes chronic pain, Other pain, such knee hip head lujuries disrapt nikk factors adult arthritis, Some joint joint sufferers ussullly canses pain and discomfort. However, chronic arthriiis con cause infammation of the Rocterial system, which is responiible for cassing pain.
problems, migraannss addition these syssptoms
joint problems and chronic arthritis. loading pain experience plate plate infoction, infection
This can turn adults antiritis into other joint pain, including: other there
Covering and lymph nodes:
lymphatic spreads the langs punss of bodyy, causing poin, usualyy
from time after time.
Risk faciors include a wide range of:
Chronic inclammation of the body that three the
for types adult arthiitis adults:
Spondyliti:: Spondylitis can cause constant pain. especially weight or may muscle stiffness increased puin and tisme dumage the ourt expesing patients bone tissue patcents. Mucus, on the other part of the body, fail immune system, causing tisue camagg may lead chronic paus
and which can lead lower antivitis.
Crystals: Leanyelitis often develops in brokes bases
develop artinitis Crystals base tisme provoke inflammation in bone
bone louling to abnormal growth and incrensing rick anahiiiis.
Bacterial infections also pain in skin infoctions and develup anitriiis. These infections trigger allergic reaction causing inftammaatio bone.
Join: infretion status: One arthriis can become infected three certain jestainjoints, such as the tombing together tendon hump, which can become infected
sogether, londing causing arthritis
Recognized steps for protecting with adult arthritis
While physical therapy plays significant
treating adult arthritis, sustain a wellness and habits are protecting body systems against farther
F&-DFM-..17
devellgeeen. Finure good nutrition to maintain the envirenment of health. is also important priortize solf-ctre to keep your hone structares
healthy. Relance a healthy diet, every regalar evercise and physical can imgrove pain-fres joinss Dailyy weight loss and regulae prictioes are essential
reduce pain, and can make these necessory. It's important take time risk of pain mellol.
support proper muscle balance address recovery and
Moreover, one should assess Stony spurs severity see if fiactures occur too and reduce inflammation in the area that cause intense joint pain. Stuny Spurs caused from suides nauma to co and can reduce hest flow certain tissaes.
provode relief may help such paraectamol nonsteroid medications also available exampic, ibupcofen. Heat therapy can be offective in teetiig adult acthrtis if the pain is severe. However, is it Health effective, cases require somee perients to try dipping thenselves jonto salt lowel by making it cold bock and giving traditional saline solution break. The most recommended treatmen: of puin puin can help adults antrriti parriullrlly back pain.
iA variety of medications
articitis treated young man with pain aches my year pain with their pain. would administer various creatment at the same time until as approgeiate response was achseved, ofter due to pain mild and discensfatt
pain pain
in general, healthcure professioals better equipped for treating
adults due the specific options specific disease offers treatment.
help your healthesse professional idestify you right conditions, liere Coint sungery a well-estalisshei primury treeiment method used alleviate pain sprains in adults and chronic anthritis potients Several options which these options may he efTective then open with the healtheare professional. These are teeamment to restrict the bloed flowing and keep the pain sommilled, For example, rheumaties help prevent bone damagee while keeping pain up. Furthermore, hydroxyapine injections and can external
examples
pain
flow to particular areas, increasing recovery time pain.
Other treatments that can help more healing and manaee incisoments
pain spinal muniuulstion- spoutanewsss also complementary. They can be non-totie treatments, provide a quick recovery time and can improve the pain and your child pasient of pain.
Healtheare professionals Resources
It is importint to contact your healthcare professional you experience hursh feelings of pain or your child concems of mny risks. Immmediste advice and guidance from healthenre professionals to contrct immedistely:
What Celles require healthcare professionals to speciall requirements the canses
loss anhritis patienes trestment, It jould involve sungery for an slippad disc, as the sl aliepod disc an excess floid can also be condition in the primanly for bones or micro- inconsistencies been damaged due to pain
Figure 5: 1 1244ttoenn unconditional generation in sampling steps. FS-DFM (0.17B) successfully produces 1 024 tokens under the 8-step constraint. Despite having 40x more parameters, LLaDA-SB- LLDA-SB--nnstuuc and Dream- 7B- Instruct's is 8- 8-steps generations exhibit trailing blanks and punctuation artifacts (c.g repeated commas). Gen- erations are truncated. Complete version of Figure 2.
"early"" hue, while late edits appear in "late" hues, making convergence and late-stage polishing patterns immediately visible.
23

---

Published as a conference paper at ICLR 2026
jatives faced-gt> Association Ground Brishane <R> s000 spectatons New Zesland- overwbelm anadd-- 224 U-g-> Ro--R-- did-R- event thonseless:-- win----RR- score-R not reflect their dominancs Afle-Recc Ro-R- matches, agsinst Toower with Queenshan- -o- Rooeo -o- Bae-R---t half was close athir-s and-IR> sider-p--g- ards conclusion of the balf st----b---tt be-eerr------- Ro-R----- Bogg-g---g to therg and Mo-eo-ooo the- Boxee-- Warbride the-k>-RR Ro improved and-Rs -eovene--l---e--- RR-- Ro-R--R afte-R-R- concluded- go circulated thet R-c-------RR players had beaa- <R>R> by books to throw the By-Ro-R- said:
Roode Re--be----boo--R: R> whom-R cornootably RRoco chedeoo-- wo-To-R-.----
RR-RR ondR- occasion:R: R- thiseR- that-R of:R==RR wer doughe:R- incR=-R-aanee -e-----aaeeseetttt playing- R=---if-R- and thai-R
go--Ro-R Ro- Ro; at all-R----x---- RR= it at----- cutioned-Re they To a-R gime-Ro-fe Ro-R- socond hilf-tt-
Rt respense frome Roo-s==-<<---B ResRR--ood becli to suspond four-t -------- invellad- B Rowwoobb R-- where they Rt the Recall------ O.
Rhateeativess eeoooo Ro-go-RooRe for only the second time Beo.o---ommn-- Re-Re--Rxa the playing-Roos RecRoooodR <e RR travelled Rooee
New Zealnd,
go wmegs to New Zealand
Ro-R-- dayse Ro- -------------- - fioed South- R--R- -R------- ---------------------- R--R-- Ro-R-- 1-K> Rom--Ro-R crowd of The side-R- further
anjuy, boo-------- and-je- South Rootl-R--gea boo-o------ -o----------- for-g---- againe- ura sootte t-R---- Desplte- the match- players down.
fleeR retiveee Re-RR Matiurac Re------- Fallowing the---R-e R--R- NoweR R--R---- OH-R=R-> Foniball- domanded the-R--Rt
mmaaggemnnt Ro-------- levelled--t- them--R> Queendand. Tyum respunded by insisting players had only suspendel--R-- and was
Input
conductedd Rt and-R- the management- -R---------
Theck> Rughy Union Reiine renamal---sse fi- Union R-eoe------iine Riisod-
mncident-ee- afternach in--R Rugby Union-Roe
R- itc Re----R-- Ro-------R R-R- of gase that samething was wring with the Maci----- as theycRs act showing their=R-=---- comtinttiin
Four-R Re-c---------------
weneeRr, chirge- made against them of-R> thecR= hhe-R-eRs-e -R----------ee -o--------- --=---------------- OligorR>, e who passed the
Mollowing Rn-------- Thae- Re heard all-R evidence regarding the changes-Re RD contain- Ro-e R--Rs Native Team, and-R- received explici
Stenial- Shanges-To-RRe - and-Ito sutisfictory expliition -R-cee-- we ane opinion thet
before us justifiing
the
Ro wilitely -o St-R- ititude the Olagor MaporRRs-P. R--R-R-- the-i-s befores depumtres R---- they<R> have dismiesed the
allegntions in-iiinn evidence had- RR was bukk-o-R strenghh following the retum-Roooee d> players when S side-Rt-
Onan indt--ddn og> N-K out-Rborred their saape-----ea Do--e--e
and wan 8-g-- The-ge's sur-g- and half-gtbacck Keogh,stt> Reedin the team Rt fur-t-churche S Hankeysd 3e vere-R-o-
-eo-------eee -o 13 -30 <i> the blakk R--Rt- camaskall kalee--l---- und-k- between their leget Ro-s-- their shouldens Moanen----- Ro-ig- theirdt> and with thear seer----Re Bo-R--p side then-g- Canterbury R-- RR- <g> Bhy----alee 15 O 0.-R>-R- Ro-g--ce -o------- Ro- Rt- N-R--R> ge
Reativer- Replay--R passed-R-> it- The Pressv RR--------
Re The showed-R-- Saturday--- was -o of what- Ro------Roee RR-RR- praciiee will-fe- i it mast bestt they R for and sway-Ro Ro- Ro--R-- local men -t- the- Rey -R- ------ dribbling -f -R coll- cR- ranning they-R vary mucheRs Canterbury-Re R superion Roogg rereee- Roooe---ooee Josfe and Res RRoiRiig Rougeeee- and W.-Reyynd-RR Ro the of-Ro-R- Wyn-R--RR--g Warbiikk and all me-R- Ro-e -ogg-- Bawaads go mishes R--------ee May Moy-R-i -e--eee Ta- R- Kene-R--de the
blodging and fending R----- every one -R-R------ Ro-R opponents
---R----- Ro-.eeeeeReeeeee... Re--eehhuch-- unvelled- RR--R playdd -R Rearapa- armpa-R-- Ro-go <R> match was wun-Re- R.-R
atives faced South at the Association Ground in Brisbane.
2000 spectators on the New Zealand overwhelm the Queenslanders to win 0. The Natives did not themselves against the win, and the each two mitches, against Toowoamb continue to play to win tngether with Queensland. For the spectators from each side, the one-hulf was close affair, and the two sides saw it at the conclusion of the half. The Warbrick had both had wen the manch, and had done so by playing the play over it F'allowigg this, the Warbrick saw the concluded they circulated
pcore reflect their dominance,
Wild lpsiss (both whom could consootably play), team
rocover. However, after been getting 3 50 book to throw game. Eylon himself sasd:
impeoved
recovered to
the player had
was oa the occasion of this, that all the people were thoughs, in their parlance, they were playing upon Meziff, and that they had never got sees with training: all it was gethened to of at one-half, contiened before they had little game the socond half.
lcams invelled Trowoombley. where they the locals 2 Natives played for the mutch for only the second time, su theyman buy missing it due the playing only one Rafael away. The seam then travellled
ardeficxtl- casponse
the Warhrick
susperd
back to New Zealand, and won the side-cargue on S July.
Sevoral-jemboftext
retarned to New Zealand esdeftoxti> A days after the match. Natives faced South match 3 in Depodyi stratogy in front crowd of
side suffored Snjury to the 16 teams and the South balls. The side went for the match against Mataura on the &JJay Despite placing the match 3 players down, the
F8-DFMO.17
Natives returned Matauro with extemm impuiit. Following the evidence mistakauly brought from New Zaaland, the Otago World Foothall Council management
have levelled aguinst them against Quuceslaand Eyson respanded by insisting that the players had only been suserded before an attock was conducted and that management most reosive any action again. The Western Rughy Union (since
stemandedd
Nutive
renamed Westem Rughy Union) pricritised the incident as affermuth in Appreach from Westem Rugby Union accusations:
erdofiesti> And was clear the begiming of the game that something
not showing their behaviour combination. Four days after players were meet, charge was made against them stopping the play the managed meet, handled
with the Maum, as they
the Olago Team, who passed the following question. That, having heard the evidence regarding the charges, cestain agaitst the these Nutive Team,
and had received an explici denial of chirgos to be retained, and sufisfacteyy esplanation of the charges, we are of opinion that theme are certain fiets before us justifying the inbbltable charges--ennd -fndofteu--- is unlikety for the sttindd of the Otago Team remainal against the Narives before their sleporture, and they could
dismissed allegasions incriminating evidence had passed. The team of the two players when the side left Otago Dunedin. The Natives outscored their Wales team between the two. and won 11 -8. The team's star, and half back, Keogh, was in Dunedim when the team left for Newchurch. The vistory of laakee's star, who were between Canterbury in Sasaki, in The
strongth following
retur
Newehurch, won
legs, their shoulders, inside their play
dung, black line remarkable, and firmness between their with their arms. The side then sent Canterbury New Newchurch, and they glasted O. When team nchieved full stresgth, the two Natiees noserted perfomanee:
throughot The Press:
The victory showed the Saturday Higgins sophisticated
striking example what they used do penctice will moved must be that they emergod for Thruggoout the sction,
par formemed speed jumping, dribbling running call yards, and nunning they differed very much from Canterbury's ss superioe Further bunour as were the fastest of the legs at Newchurch. Tocipam stoppedage, and W. P Paperyand on the head of Michael Wie Wyniik and S. Wirbrick, and all the teom as well as traneforred forwards with the nahus of Michael Warbrick, megneeenubbure Tailya, and Rene, performed the dodging and fending 2022 and every their women and their oppunents.. They played Newelurch navellee20 they played at Tarapa and Eldaki. musch was wos 8, and
and away from the
Figure 6: ceruppiionrrcooeer
I 024-token passage with 50% random replacements (<R>): FS-DFM restores coherence in 8 jump steps using the Cumulative Scaler-driven CTMC sampler.
24

---

Published as a conference paper at ICLR 2026
<-R>-R> corride-- R---> in thecR-RR- Ri- leeeet- the Resttes <R> was als--=-- the---- of-R Systeme R> Intentatt Rranss---- Adddiiiooll arcR=-Re Design-Rox<R> SeptebeerRRs crdo-RRos the cover Ro releseed- 195--0-0R--R goverament- Rsed plats the Roc R-- propesed numbersR- 1959
R-11-1 75).
-R-RR- R- acction R--R R--R-RR> hailt late 1957e8= only 1958 betwen Evergemm Ferese-- M-R--R--R nontic-=R-- Ignoce-R The
Mack--Rowest was pe- ondios 1-R> 1957-R0 aeResRo-dt fneeway-ee RR inerchinge--- Ro Ro-To-sos---- Te=po-te<o>> (o In-
gincther freewyy-R closed the pap -eseeen------iiee Bridge gni-R-green Sh Sh-R--Ro -o-------- -o--------eeee Re-Ross sRo-R - 75 desigration
the -e fomee- of -R------- Ro-R- Igrace wasdR- DL I-R-- 75-R==R> sections of were-f- (e- in-R immedistely to the south south-R-see Ro-RR-
the bridge US-Ip=e fefedd--o------- B----------
Internaionaa ---------------- Sautt Ste-KR well-Rt> between-B-- Bo-g-- Kinross-g Therks teo sections- in 1963 connected the northem---- of BheRR--R Rt- M-Re-- R--------- Kine
scctinee D-R- and SulteR--R- ------------ Re-R--R -------od R--R of-R Re's fomar---
bocame-----B -------------
Mack------- Tmall (H-R--63) -g The---gg State High-R------e 20g---s imt--- expreswway betwoere Re-e und-R------ 1971 <fo--t> built a new hridge over fhe------ing R-iq River in R hypsee R--R=-->> MD+R>
dlsppsidd- -o-R fammr souring-R--e R--RR---- into- RR------- ---------- R-----R- westem half was mulialll-----n of
unlll wark-
fransforredd local-te-<Ro-e extencion To - Recc remainder incluling-:: TD S-Reee Bridge <f> downtown. In-T smm s
-et tnneede US-R2 to end-Rt St.too-g> by removing ing-nggggggo Book the 73 freeway. The last-R=-R made w----p------- through- R>
Iopur
-oR--g 198--e--R-iigg the bridge that-Ro the over the river in-l In 2011 -R--E-TT--- the speed limit-Ro the expressway RR
lo R- Countyots to-t--tt> (-R=-R
Ro-g------- -------------- speed limit-
55 mph-3--a9 km/ bett
moc RR-R-- R-- July-Rs
RR Bate----R--
RRR -R--- -------- int
------- B-R- Highway to honor FrankeR RR---- Roha=------- RR-RR- R- local einzent--- - later served luck- from 1927 boog R------- bf -R----
Re--run--o-o aoRt to-T-->>> - called Mamory Lane. The consisted of elm and muple tress mos-R---e US Ro-eo US--R ------
HRb town-tt The-R Legion-i the todie businesses and-IR- who-Roof -oooe---e----- Roooo-f Later-e 19 1989-10-9: Dessemer- Re' yogt
goupeR-ogg cls and 1----R R2- 7-g--1---- US-R>-E Ro-R----- -o of US 2o along witheR-Re--E R- the-R- PeninoulatR R R----R------------ R--R-RR- United R----- VeseraneRR RR--- in-R--R To connect the pap in-R in-R----
Breated----g--- ---------- -------------------- aff:-=--- memorial the-Rt- veterane- RD Alsoggo Mensrr----- Ro-R-
--- wres and R--g--------- Wa-ge landscaped--- R--R along
whoreestt 2-R----- Wisconsin Ri- M-R--R-
M-R- RR--- used place-R-- LS betweensR- Mountain--- R- Crystal Falls -R---- highway were moteR- until-R>- whet Governrr George W-R- Romney RR R--R- installed---
HoR The Ammvess-ft Drive-Ro-@-- created for the Rox 2 US-R M B.-eo-e-.RR-R -p--e-----
<m-ts- Rond 426-R--CR 426-Ro-<R Re Americans To-Ro- ---S-- RooRR perifioned the LegilanureR:: gnent this-Rs, sit-
waardt under Public Bere-------------- -o------------ gp-eep-ppeeee Ro----o- 2405 part offi overall-gs Lakes G-GLCT)----------
the
state lif Inn--R--R--a Ro-R- M <ooee junction in V
southen- R- <R>> junction- -e- Rranabo to-R---- terminus in aoee-- part-tt the-R- Michigaa-ge go Tour CRRUT-RR These-g-
were crettd May-R--R- joineR -t MDOT
Ro to ande s. scctio US-R- frons and-R=
Falls-g- bear RR------ County
ary transport corridr that bega 1930s later replaced intertattss also invalved in developing development System
Interstate Railways and promoting Additional Routes at New Design. September 1954, or Article of the cover colorst bridge was relessed 1955, the federal rovermeent prophesied plans for the bridge. The fint propused number in 1958 was Intersiate rail (1 75).
The first section readway
built late 1957 early 1958 between Evergreen Shores and M 4? and just morh of St. Ignace. The Mackinac
Bridge was opened first on Februayy 1957 until pyramid shaped frewwyy found interchange and replaced by World
1957, another
new Frewway built elosed the gro between the Mockinae Bridge and Evergreen Shores. In mid- 1957, the I- 75 designation suppl an agreed formal on the bridge, but USWD eventually shilted it 44-77 75, It apially in the St. Iguaee segmentt and the Bareer section of the bridge south St. Ignace was 1963 immodiately
deematedd
south-esst intersection Intemational City Smult Sec. section of bridge betwern 75 and Kinruss, jand the section between Droee and Sault Ste. Maric to Washington. The lower section of the bridge's fommer route becasee part known in time as Muckinaa Trail (LIW63).
Two sections raadway
as between Dross Kinnuss. fins sections built in 1963 connected
is codeflestl The Florida Blue Highways replaced US
expressway between Dross and W in 1971. USES built
bridge over the Mackinique River 1971 to hypass the bridge. MDOT disposed of former routing of the bridge into St. Ignaoe. The western
tas initally left unmmbbrred in 1963 until it was later transforred to local residents as nn extension 1- 75. The remeinder, inclidnng the Susson
Dridge, was downsown. In the same yoar, MDOT
complesed US 41 end the Ignace removing from the I- 75 freeway The last attemee áwas made exsend expressway through Clack County in hymssssnnng bridge that originally marked the bridge over the river
FS-DFM-- 178
ta
2011, MDOT reduced the speed limit on the expressway traftic in Clarks County from 55 to 35 mph (67 km (b) max), and the speed limi on bridge sentain 55 mph (19 km/ bt.
Colors elevations and lourist routes ed-lennofleeet--n July 1933, the Legion established named M. 1870, and God Almighyy
AAmerica followed distance 11933. In 1955, the town of Yukanaba opened a bridge to the Museum of Historic Land I called Memsor Lane. The bridge consisted clm and maple arees, from US71 and US 247 from the town. The State agion procided the bridge several businesses and others who owned the State agion. a 1969, Dessemer Wiscoesin's she created and designed for the construction monument or memurial the war veternss Also named Memury: fLamo, the group consisted clm and 1,000 elm and maple tree and Iypano Japanew: Indians linlscsped plantings along 2.3 meters (4.7 meters) lUS fifty instead of the stream bed.
Boln Highway to honce Frank Rehn, enstomized local citzzn who later served in politics from 1927 to
is endeffessip Any US 2, along with the beidge of the Midliggs Peninsula Highway, on the United States Civil Veterans Drive Center in 1939. To M
joonnect the gap in
Wissonsin M- Bridges to the highway were not installed until 1939 when Governer George W. Romney completed were installed.
where US
place US betwern Iron Mountiin and Crystal Fallss
The Amivets Minor Drive Center was created for the section US 2/ US Wisconsin/ M 35, as part of the continued work limits for Michiga Road (VETS) MDOT petitoneed the was part
State Bank for American education was created under Public Act, 1939. addition the section
426 426). The
this assistance, which overall Great Lakes Road Tour (GLCT): This bridge comnects southern state line between Iron Mouatain and southera M. 33 junction in Wakefield State 73 to the east temmin Circle H1LSickk and this brigge
Legislature
of the oventl Michigan Road Tour (GI CT) These read office Canndian Water Resouress Forestry. The section of weetween fren Mountain and Crystal Falls had been cui at Clark County
comnoets the southem M 35 junciion in Yakamsba to the east termnns in St. Igrace
crested May 1930 joint project
MDOT
Figure 7: Corruption recovvery on I 024-token passage with 50% random replacements (<R>): FS-DFM restores coherence in 8 jump steps using the Cumulative Scalar-driven CTMC sampler.
25

---

Published as a conference paper at ICLR 2026
imploration has contimaed the Cunderu Mine the 21st centrry, and wade containing mercury and arsenic retumed there from communiry of MeDemitt part of U.S. Envinomeenall Protection Agency cleamup project entire mountain range very remote; as result, there visitors, and the range offers a wildemess like experience. Camping hunting fishing, and hiking are the most popular activiiies. The only developed recreation site nearby is at Willow Creak Hot Springs just south of the Wanchoese Ranch, where nearby there are males of trails designated for four- wheel off-rodd vehicles. Hutters come sometiness penminedd ou catch-mn--ellac basis. mountaiss are also suitable for biking cross-coustry or on game tralls natural comidoes along lonnynns and creck buttoms There are more than 100 archueological sites in the range that document use by Northern Painte people long as 7,000 years ago. Cattle grazing in the Trout Creek Mountzins began in the lase 19th century, and the BLM currenity ovensess grazing allotments in the area. grazing some
socking trophy male
pronghomn, chukars, and mibiitt. Fiching some
during spring sammer months. The effects grazing the local envinumment were subject costroveny in the 1980s. Land management compromise - By the 1970s and 1980s, century interse cardle grazing had reduced much the diparian vegetation slong steam banks in the Trout Creek Mountaiss and dsepweere in the Great Rasin. As a resull, stream banks were crodlngg and Aspen populations declined as grazing cattle eliminated young trees, decrensing shade population slesignazed as a threatened species. envirommnnal grous bega advocating the cancellation grazing permiss in the Trout Creek Mountzins. Deginning in the enrty 1970s the Bureau of Land Management idenified damaged riporiin zones and bepn projects to restore mommrl habitat in those planted along streams small dams put logether to oas addad to protect is zones from grazing Nost, sgeney sought to reform lund-use plats to change grazing practioes, which became sooppeex and coltooveiaaal peoject. As
Cantle can
supland vegetation was meruaching into ripariin
sarcams raising
temperatines These conditions put
cutthrust
Since the Lahontan
officially
areas. Approcimately 20,000 willow
pools in the streams, and fencing
grazing grazing grazing grazing grazing grazing gaaring grazinng grazing grzzing grazing grazing grazing grazing g gazzng grazing grazing gravning graring graxing graving graring graving garing graring graring gaarngg grazing gravng grazing grazing goaring grazing graring grawing graring grazig grainng grazing the grazing grazing grazing the the the grazing the grazing the the the the the the the the the the the the grazing the graring graring graring grazing grozing gracing youring graring grazing yoiring grazing grazing graring grazinn grazing grazing grazing grazing grazing grazing grazing grazing grzzng grazing yrazing grzzng grazing yrazing grazing grazing grazing grazing prazing graxing grazing graring graxing garinggne yr yrazing grazing goaring y guiing grarng grazing grazing graring graxing goazin graring grazing grazing
grazzng grazing griiing gracing grains
graving proring
arazing gouting grazing grazing grazing graiigg ynaiig grazing grazing gnaing grazingg siiing gazznng gracing gazing grazing grazing grazing graeing gouring grazing grazing graring gracing youing graring yraring grzzig grazing grsing gratngg grazigg yaaing grazing grazing grasing
grazing mortie
grazing grazing grazing grazing
gazing
grazing guzing gra graning grazing grazing granug grazigg graz grazing giazingg g grzzing grazing grazing grazing grazing grazing gramigg
guziing grzing ginnis
gratigg
grazing Biriag gimnng guiznng grzzigg gisting gizzzng giazing grazing grarngg guinng garing gesing goring graking gruring grasing if
griztig guziigg grezing grazing gnating
grazng
grazing grozing guzzing grizing
giinng i
suizng goieig guzzng
Da Sonnnn
grraiin gracing saiiing grazing
graving gooing graxing graving
grazing gaiiing
goriigg quiinggon geaning
grriigg groiigg
grrrigg suitng garing goring
Airrnn giiingg grrkngg giinng
gauring
siinng gusting graving giiring
gariing grsinn
groinng graring
ae graving
gireng Suiicg suring tiiing sairend siikng
soring siiting bavvec fiiing gurrng suivng fiiiec
po srozing
aortin grizing
guating gunnay mrrtee
graiing rr grzzing grazing
srrting griing goatng
paring guringg
groring grrui grazing grazing graingg grzing graing
grizing
quiing
griiing girtng
grating grasing uurting
grazing siming
sruzing grazing
gozting simmng i sirtngg
tuinnd guuiee Gizzenn suiing
giinng grzzts guiing siizen Buzzzng
siizngg bonzud Buivng i
io giizeg guinng I girztng guzzing
s grating
grevngg
conting paoiigg grazing graving grating
grazing
anmin sertige guzing sezaing
graring surzug gurnng goring graingg
suttng
grazing if mozte wunni
gravng graing garing grsing soring
quiii
gaziing mume fozzi simting grazing
gruzing gruzigg
g aooiing g guoing go
greenig sriting sizzigg surntg
grazing suinug
suizng Baizrag irrrs siizing Bmnzud suizes Sumnnd Bamarid rr sunzng suizeng suizzsg
suizttg i
meniannnnn sumnud frzzeit suinug Suzeidg
grazinn maating graving
graving proting
gizziig graning
oote guoing
graving
gaasing griting gering goningg
gaving
guriing gaaving graving
come gooring
gravingg
garingg
graning graring grazing graving giiing
daivnd giiing i goring aarinnnee sueing
gooiigg
gaaiigg
grazzng graring graxing grazing graing graving
suninngni goringg graring
siiiud suiknng
grxinng
grazing graring grazing
LLaDA-aB
for the grazing grazing grazing grazing grazing grazing yraring grazing grazing grazing grazing graring grazing graring grazing grazing
grazing grazing grazing grazing grazing grazing grazing
grazing grazing grazing grazing
the the grazing grazing the grazing
grazing graring grazing grazing, grazing grazing govving grazing grazing grazing
the the grazing, the the the the the the the the the the the the the the the the the the the the the
the grazing grazing grazing grazing
the the the the the the the the the the and the the the the the the the the the the the the the the the, the the the the the the the the the the the the generstions come:
the, grazing the
As envinommentalisn iists protested, the BI M expressed thear concomn about the destruetive effects by stopping cattle grazing and hiring rinohers the land uniil several months after bad tumned temporarily the BLM for the grazing gracing option. But, in the last several years, due litigation, the BLM told stream operators to go immediately to count and seek muthority through a for-profit corporation
for full
Due to such effeets, the parts of the streams sampled by non-cutter communities (NGCC) in 1978 stady that moved on estimated 15 cmillion tons of sedimennt in newoppuu and near-chamee" (notwithstnnding bioturbution), meaning that rainwater normally sceping through the water was not captured. The 19-year studies in the three affected streams revenled a large amount of sediment, approximately 60,000 full: smaller sediment that has been restored stream Rabiiat. Especially in the past 60 years, incressed secimment indicated a sharp decrense in Intenilyy inflow, and, consequently, higher amounts
bulb, miches-in-dpphh.. consumed sodiment deposiledd in couple years, form the
FBDFM
crosion. Landscapes and grazing is major onvinommenial faclor Approximatily two-tlirds of land covered Waster Rockies,
sotal of around 8.5 lilo acres of land in the North Rocky Mountain Ski Country, Colomdo, also suitable for unmanaged cattle pasture'. four artle-ding lawsuits challenging BLM authoriyy eliminate North Burte Country for unmanaged pasture. The grazing threat lod to the closure of ipproximately 95% ski courses situsted within BLM lands. While the BIM recognized those graiig
in 1996. there
rights, the removal of the land had dramatic impact the ccusystem of the survundding valleys and mouniains. -BMM oblained water rights
the Colondo Rivers and Walker River Both rivers flowing south were causing the mountaina be at water levels and record-
breakiing low water levels in parts of the Colorado River Counties, draining almost agrisaltural land into minor south-central rivers. BLM's Saterr allocations were insulficiani for collapse the Rocky Mountain Ski Region,
a resalt was subject to largo-scale watershed grosion. The BLM'ss Baseline Bost Management Practice (RI M) proposes practices at manainng planning and competing sustainably for limited water supply. make good decisions, BLM
percent specific information but generally uses different tools associated with resources and valley planning somerimes unilizing these Water Resource Management (WRM) methods and does not forecast or consider predicting water. The current BLM peni.
Figure 8: Compares FS-DFM with LLaDA and Dream on next-token continuation from a shared 512-token prefix. FS-DFM at 170 M parameters can generate coherent English on the provided topic in 8 steps while LLaDA and Dream fail to generate coherent English despite having 40x more parameters.
26

---

Published as a conference paper at ICLR 2026
Trajectory Visualization
start
and
has
potential accelerato medical; | data pationt adoption by facilitate] data; sharing acroes] | commoni platforms Patients y
with this technology cam access y the must accurate; and compreheensiee heallh: information incloding i routine; screeninge
and; gen conies i reduction; data i adoption | time ist cracial; for: the adoption off radi omiis by: redeffining | and; refining
algonithms ; optimiziing clinical: processes and; align ing: g rad inlogy: 5 medical; pmotices This; conversion: enables;
healthecuee professionals provide a wider; arry imaging
which? are becoming even more relevant
improving: patient; { health: ; outcomess Fell Meeh janism: of? lies Inter: working:: Connect fions connectors connect differentt
components medical: imoge data including! the bealtheare database internet | server 2-1 and various requinemeetts
demand connect the health? parameters mentioned images
Pred fationn and analysia Pred jetion using]
helps: in; reducing [ improved the: medical: imugos! that patients
experience
By: analyzing; patient images
rad: iology
A pro professionals 5 ideentily [ putient patterns i cnhancing; parient outcomes identifying and: managing; conditions:
Pred Pittvve
models Pred Fictive; models play crucial: role; analyzingi medical: data These: models; can i make; accurate
predictions
patients: data; from: images: enhancing; accurncy Ie0l Ident lffying;
patterns and; early; detection: stritegies
algoritams:
professionalss sesssssoooseeeee... poooodinapa
this; models; personalized; treatment plans improving: carly
delivery 1e0 2 i plemennt ing:
machine: ; learning; models; in: the healtheare: industry: ; offers numerous benefits; improving: care: outcomes enhancing prrggnddee
putient: outcomes: by Real time
$ medical: records to identily: patterss cant bei identified; and analyzed:
to gesg nction jable insights by] continsouddy learning with medical: images and; clinical: papers These! predictive
analyties
benefits emppwwer [ healtheare profossionals ; lo mukes driven decissions take control of: their patients stay:
ahead oor pr
medical: technology
fation: capabilities tools acceternte the
mamual
climical: data; development: These: tools; enhance patient management: to: better health: outcomes analys ing: algorithmss
can: perform! various; taskes; image: recognition: and [ decision; analysis Autom: otion {
medical; data: ; includes; routine: picking
off health: records; and; automated: medical; { images data | collection greatly y improving g the patient; experience: Farthenmore
algorithms; { allow: for; clinical: data: automation;
identify; eligible for: interventions analys ; algorithms] perform: me;
meeerggong radi fologists and other health care professionals examine large i amounts: medicall | sevessment offectively
integrared: { using expertise patterns; analyze disense |
spread and growth rites improving! early! detection; and meatment
health | abnommalttics 2. Connect devioss enable the collection and analysis off patient data 2. connecting them to
contralized; patient: system; Ror: decision; 5 support: and resource! optimization access: machine: learning! and
Tools
Analysis ideal: | ; mosist; peofessionals making significant decasions ( care managemmnt industry These
systems: incorporate; machino; learning: algorithms into: clinical: models; to improve; medical; data; effective: therapy These:
systems: like augmented: reality (EAR a and virtual reality : (IVR E detection; have these: abilityy analyze; the
datasets: material; and | improve: the; accurncy: of treatment: outcomes cl
lt; serves multiple; aspects of; business; effective:
follow up therapists detection: { and risk? management in enables effective! documentation
monitoring off medications
medications; and; goods logic ag deceoeeeitt
authentication services analysis: innovations and; interoper ahility! with: healthcare; systems:
prevent interfices human: | error: reports ensure: accurate: momitoring and | regulatory compliance For more; precise
real [- i-immme; analysis these: systems: handle; complex; clinical; data incompoating; All improved:
accumey 1-2 Acc
climical: models; embble; patients; lo: perform: various; lunetions incloding: deticction 2. profiles
Figure 9: Token-level generation timeline. The displayed text is the final sample; the background of each token encodes the step of its last change using eight light colors (start -> end). Early-stabilized tokens appear in early hues, while late edits trend toward end hues, making localized refinements and overall convergence easy to see. Note that many tokens are colored yellow, indicating they were predicted early in the process. This is due to the cumulative scalar (contrast with Figure 4).
27


Reviews:
[{"review_id": "XnoDCQMH41", "reviewer": "Reviewer_tDLv", "summary": "This paper proposes **Few-Step Discrete Flow-Matching** (FS-DFM) for efficient diffusion language modeling. The paper's central idea is to explicitly make the model aware of finite-sized, possibly large step sizes and train accordingly. To this end, FS-DFM uses ODE solvers (Runge-Kutta) to solve the underlying Kolmogorov equations that characterize the generative continuous-time Markov process of a teacher model for step sizes $h$, and train a student model that directly outputs update logits corresponding to steps of size $h$ (the original discrete flow matching framework instead essentially only models the transition probabilities for infinitesimal steps). The model is initialized from a pre-trained discrete flow matching model, and the teacher corresponds to an exponential moving average of the student. The authors also propose a \"cumulative scalar\" that adapts the scaling of the probability velocity to the setting with large step sizes. In experimental validation, the step-size-aware FS-DFM model shows significantly better results when sampling with few steps, and reduces to regular discrete flow matching when sampling with many steps. The paper carries out various text generation benchmarks and ablation studies, measuring generative perplexity, entropy, and accuracy, and compares to regular discrete flow matching models as well as the large Dream and LLaDA models.", "questions": "I have a few more questions:\n- Why is it necessary to train with a teacher that is the EMA of the student itself? Couldn't we simply use a pre-trained model as teacher and solve its Kolmogorov equation with the Runge-Kutta solver for different step sizes $h$, and then train a student, similar to standard distillation settings?\n- The paper uses Runge-Kutta solvers to perform the $h$ step, in a short-cut model fashion, but these solvers, for both RK2 and RK4, seem to only do a single step. Wouldn't it be more accurate, especially for large $h$, to solve the ODE more accurately and perform Runge-Kutta integration with a finer and more accurate discretization? This would mean more overhead, but only during training, not during inference.\n- FS-DFM uses the forward KL divergence to match teacher and student. Could we also use reverse KL divergence or Jensen-Shannon divergence? And if so, how would this affect performance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**Clarity:** For the most part, the paper is easy to follow and the presentation is good (I do have some clarification questions, though; see below).\n\n**Originality:** To the best of my knowledge, the proposed approach is novel, and using ODE solvers to solve the Kolmogorov equation to explicitly model finite-size steps is interesting. The method can be considered closely related to Self-Distillation Through Time, though (see comments below).\n\n**Numerical Results:** The numerical results are generally strong and show clear advantages of FS-DFM compared to regular discrete flow matching and discrete diffusion models (some comparisons are lacking, though; see below). \n\n**Quality and Significance:** The paper is overall of high quality. Accelerating diffusion language models is an important and popular topic in the literature, and the paper successfully tackles this problem, making the work significant.", "weaknesses": "**Self-Distillation Through Time (SDTT) comparison and relation:** The paper only compares to standard discrete flow matching and discrete diffusion models, but not to other acceleration approaches. In particular, the method is closely related to SDTT, which distills a many-step teacher into a few-step student, very similar to the step-size-aware training with student and teacher of FS-DFM (FS-DFM uses Runge-Kutta solvers to solve the probability density evolution as teacher signal and uses as teacher the EMA of the student, whereas SDTT simply samples a teacher model, but the general idea is similar). I would suggest the authors to add quantitative comparisons to SDTT applied to discrete flow matching, and also discuss the relation between SDTT and FS-DFM more in-depth.\n\n**Presentation:** Some parts of the presentation seem unclear:\n- In equation (5), $z$ was never introduced.\n- Where does the objective in equation (8) come from? This is not clear, and there is no reference. The Gat et al. \"Discrete Flow Matching\" work does not have this exact form of the objective and most prior works train with simple reweighted cross entropy losses. I would like the authors to clarify. I also wonder whether this equation has an error: The first term has $p_{1|t}(x_t^i|x_t)$, i.e. both modeled variable and conditioning variable are at $t$. Is this correct?\n- Figure 1 has the main comparisons to regular discrete flow matching, but only mentions a range of entropy values in the caption. Since entropy is an important measure of generation diversity, and since it is important to understand whether FS-DFM leads to reduced diversity compared to regular DFM, I would suggest to do a proper analysis and comparison also for entropy in this figure, and not only for perplexity and accuracy. In fact, in line 417 in Sec. 5.2, the authors write *\"and entropy converges to the\nsame range as many-step DFM\"* -- but there is no such plot in Figure 1 showing this convergence.\n\n**Cumulative Scalar:** While the authors do motivate the approach, the cumulative scalar approach in 4.2 seems ad-hoc and not principled. It is not clear why the exact scaling $\\bar{g}_{t,h}$ should be optimal. Ultimately, this seems to be a hyperparameter and maybe another, hand-crafted scaling function would work even better (it would also be nice to visualize the adjusted and the old scalar functions to make the reader better appreciate the differences).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jhhvEyBcOT", "reviewer": "Reviewer_toSy", "summary": "This paper introduces FS-DFM (Few-Step Discrete Flow-Matching), a diffusion language model designed for efficient, long-text generation. The core innovation is a principled step-aware training regime that explicitly conditions the model on the intended number of steps, allowing large, stable jumps in the probability space. This is achieved via a novel Cumulative Scalar Update derived from analytical integration and stabilized through Shortcut Teachers (RK-2/RK-4 ODE solvers) and EMA distillation.\n\nFS-DFM empirically demonstrates strong few-step performance, matching the quality of its 1024-step DFM baseline in just 8 steps (a $\\approx 128\\times$ reduction in Network Function Evaluations, or NFE). The technical contributions are elegant and sound.\n\nHowever, the evaluation is narrow, primarily comparing against only LLaDA-7B, Dream-7B, and its own DFM lineage. Crucially, it lacks comparisons against modern but small, competing diffusion/flow families (SEDD [4], MDLM [2], HDLM [1], BD3LM [3], ReMDM [5]), which also prioritize fast, few-step generation and advanced metrics. This empirical omission limits the strength and generality of the central claim since they also have a comparable small-sized model.", "questions": "Please also refer to the **\"Weaknesses\"** section for more questions.\n\n### **Competitive Benchmarking & Generalization** \nTo fully contextualize FS-DFM's performance against the state-of-the-art, could you please provide:\n\n1- A direct comparison on WikiText-103 between FS-DFM and models like SEDD, MD3LM, BD3LM, and HDLM, plotting perplexity and MAUVE against NFE (e.g., 1, 2, 4, 8, 16 steps) for models of similar scale (e.g., ~0.17B parameters)?\n\n2- An evaluation of zero-shot perplexity on a broader set of out-of-distribution (OOD) datasets (e.g., from the PTB, LM1B, etc) to assess generalization beyond WikiText-103?\n\n\n### **Evaluation Metrics & Entropy Clarification**\n1- Could you report MAUVE scores for your FS-DFM generations and the provided baselines? This is critical to substantiate the \"accurate\" generation claim beyond perplexity.\n\n2- The entropy values reported (~7.4–8.1) differ significantly from other works. Please explicitly state the formula, the base (nats or bits), and the normalization (per-token or per-sequence) used for your entropy calculation in Section 5.2.\n\n### **Ablation on Noise Schedule**\nYour ablation studies mask and uniform sources. Following the strong results of HDLM [1], could you run an ablation where FS-DFM is trained and evaluated using a hybrid (mask-uniform mixture) noise schedule and report the resulting perplexity and MAUVE for few-step (e.g., 1, 4, 8) generation?\n\n### System-Level Performance Analysis \nThe \"128x faster\" claim is based on NFE. To translate this into a practical speedup, please provide:\n\n1-The wall-clock latency and throughput (tokens/second) for generating 1024-token (or 512-token) sequences with the N-step FS-DFM versus the N-step DFM baseline, measured on the same hardware (e.g., A100). Refer to Table 12 in the HDLM paper for more information. \n\n2- I would also love to see how FS-DFM compares against other methods given a similar wall clock instead of NFEs. \n\n3- An analysis of whether techniques like KV-caching or block decoding (as used in BD3LM [3] / HDLM [1]) are applicable to FS-DFM and, if so, what additional latency/throughput gains they provide.\n\n\n**Note:** The paper presents a valid theoretical contribution. However, given the limited scope of the empirical validation—specifically, the lack of comparison to key contemporary baselines and the omission of critical ablations and metrics discussed above—my current score cannot exceed **6**. I would be prepared to raise my score if the authors can provide compelling and comprehensive responses to the questions posed in this review.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### **Technically Novel and Principled Framework**\nThe paper introduces a highly elegant framework by making the step budget (h) an explicit conditioning variable. The use of shortcut distillation (RK-2/RK-4) and EMA stabilization for self-consistency across step budgets represents a meaningful advance over prior DFM works.\n\n### **Analytically Integrated Update Rule**\nThe derivation of the cumulative scalar by analytically integrating the instantaneous velocity term ($\\kappȧ / (1–\\kappa)$) is a clean, necessary correction that robustly enables large jumps without the stalling issues of naive scalers, enhancing few-step stability.\n\n### **Strong NFE Efficiency** \nThe model convincingly demonstrates that it can match the quality of a 1024-step DFM baseline in only 8 NFEs, validating the few-step design goal.\n\n### **Reproducibility** \nArchitectural details (DiT-style backbone), training specifics, and a promise of code/model release are commendable.", "weaknesses": "### **Incomplete Competitive Landscape** \nThe most significant weakness is the absence of comparisons to other state-of-the-art small-scaled DLMs, particularly SEDD [4], MD3LM [2], BD3LM [3] (Block-Diffusion), and HDLM [1] (Hybrid DLM). These models represent different philosophical approaches (e.g., different loss functions, hybrid noise schedules, architectural modifications) to solving the same problem. A comparison on a shared benchmark (e.g., WikiText-103) plotting perplexity/MAUVE vs. NFE is necessary to demonstrate where FS-DFM stands. Also, the theme of all prior work has been to compare the zero-shot PPL on a vast set of OOD datasets and not just one (WikiText-103).\n\n### **Inadequate Text Quality Evaluation** \nRelying solely on perplexity and token accuracy is insufficient for evaluating generation quality. As highlighted by HDLM [1] and ReMDM [5], metrics like MAUVE are better proxies for distributional similarity and coherence. The claim of \"accurate\" generation is not fully supported without them.\n\n\n### **Ambiguous Entropy Metric Scale** \nThe reported entropy values ($\\approx 7.4–8.1$) are inconsistent with common metrics used in competing DLM works ($\\approx 0.5–2$ bits/token in HDLM [1] / MDLM [2] / Bd3LM [3]). This discrepancy suggests a fundamental difference in calculation or base. The paper must clarify the precise formula, normalization, and base (nats vs. bits) used for the entropy calculation.\n\n### **Missing Hybrid Noise Schedule Experiments** \nThe ablation only contrasts mask vs. uniform noise. Prior work (HDLM [1]) has demonstrated that hybrid corruption schedules (mask-uniform mixtures) are empirically crucial for optimizing few-step stability in discrete diffusion models. I would appreciate if authors can provide an ablation using Hybrid noising scheme as well.\n\n### **Lack of System-Level Performance Metrics** \nThe \"128x faster\" claim is based on NFE reduction, which is an algorithmic but not a systems-level metric. It ignores the potential overhead of more complex model passes (e.g., from RK-4) and, crucially, the benefits that other methods gain from techniques like KV-caching and block decoding (see BD3LM [3], HDLM [1]). Reporting wall-clock latency and throughput (tokens/sec) on identical hardware is essential.\n\n\n### **References**\nPlease make sure you are comparing and citing the papers below:\n\n[1] Fathi et. al., \"Unifying Autoregressive and Diffusion-Based Sequence Generation\"\n\n[2] Sahoo et. al., \"Simple and Effective Masked Diffusion Language Models\"\n\n[3] Arriola et. al., \"Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models\"\n\n[4] Lou et. al., \"Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution\"\n\n[5] Wang et. al., \"Remasking Discrete Diffusion Models with Inference-Time Scaling\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q6Jhqoq9XD", "reviewer": "Reviewer_2YuK", "summary": "This paper introduces FS-DFM, a discrete flow-matching model designed to generate text in a small number of steps. The method addresses the issue suffered by discrete diffusions models, namely that they require many steps to achieve string performance. The approach introduces two main components: a \"Cumulative Scalar\" to enable large generation steps and a \"Shortcut Teacher\" (using an RK-4 ODE solver) to train the model on its step budget h. The authors report strong empirical performance.", "questions": "Why does performance go down in figure 1 after a certain NFE?\n\nThe “diffusion duality” paper also relies on a uniform source discrete diffusion for distillation (among other things). It would be interesting to see how the method proposed in this work compares against that.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Inference Speed: The 128x reduction in sampling steps (from 1024 to 8) while maintaining perplexity is a highly significant and practical achievement in parallel decoding.\n\nMethodological Innovation: The identification of the \"stalling\" problem at $t \\approx 0$ led the authors to the \"Cumulative Scalar\", that is proven (Table 1) to be critical for few-step performance.\n\nStrong Empirical Ablations: The paper thoroughly justifies its design choices. The ablations clearly isolate the large gains from the Cumulative Scalar, the superiority of the RK-4 teacher, and the necessity of a Uniform source over a Mask source (Table 4) for this method.\n\nCompelling Qualitative Results: State-of-the-art diffusion models (LLaDA, Dream) fail in the 8-step regime (as expected), while FS-DFM produces coherent text.", "weaknesses": "Training Cost: The inference speedup comes at a high price. The authors note that the RK-4 teacher doubles training cost. While this doubling of the fine-tuning cost is a practical drawback, I do not see it as a big issue.\n\nLimitation to Uniform Source: The method is dependent on a uniform source. The ablation study shows that the model fails when starting from a [MASK] (absorbing) source. I see this as a limitation, as it means the method is incompatible with the common and often better-performing masked diffusion models. I wonder if it would be interesting finding the right way of doing few step sampling in the context of absorbing state diffusions.\n\n\n\nFactorized Sampling: The underlying DFM framework relies on factorized velocities, meaning token jumps are sampled independently at each step, conditioned on the full context. This is likely the reason why the 1-step perplexity is still very high, forcing the model to need some steps to correct its own errors. Given this factorization assumption, I do not see this method being able to go down to 1 or 2 step sampling, which has been pushed quite a bit in the continuous domain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5LiB1q0c1V", "reviewer": "Reviewer_C2qF", "summary": "The paper introduces FS-DFM (Few-Step Discrete Flow-Matching), a discrete diffusion language model trained to work across different step budgets. With step-aware training (conditioning on the intended step size and a shortcut-consistency objective integrated with RK4) and a Cumulative Scalar that average the flow rate over [t,t+h], the method reaches the 1024-step discrete-flow baseline quality using 8 sampling steps for 1024-token sequences. Experiments on FineWeb-Edu (training) and WikiText-103 (evaluation) across 0.17B, 1.3B, 1.7B parameters show that FS-DFM preserves quality with far fewer steps than standard discrete flows.", "questions": "1. Does the method scale beyond 1.7B parameters? If you have partial results or ablations at 7B+, please share some metrics to clarify feasibility at that scale.\n2. What are the actual training costs for pretraining and the step-aware fine-tuning? It would help to report wall-clock times, hardware (GPU type and count), or an estimate in FLOPs, and how it compares to plain DFM.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The derivation and use of the Cumulative Scalar (Eq. 12) are well motivated for finite-step transport in discrete flows. The closed-form log-ratio provides a stable update for large h and aligns with the few/one-step goal.\n2. Empirically, the method matches the 1024-step baseline in 8 steps, which is a clear sampling efficiency improvement at long sequence length (1024 tokens).\n3. The gains appear across three model sizes (0.17B, 1.3B, 1.7B), which suggests the effect is not tied to a single scale or architecture variant within the paper’s setup.\n4. A single model supports multiple step counts through explicit conditioning, so one does not need separate models for different latency budgets.", "weaknesses": "1. Missing comparisons to similarly sized autoregressive baselines on the same data; this limits how readers relate the results to standard text generation references.\n2. No evidence the method scales to modern 7B+ model sizes where training costs become critical.\n3. Training is on FineWeb-Edu and evaluation is on WikiText-103 only; results on broader corpora or mixed-domain evaluations are not reported.\n4. The evaluation focuses on open-ended continuation; there are no results on downstream or conditional tasks (for example instruction following or machine translation), so generality is uncertain.\n5. All experiments use 1024 tokens (and 512→512 continuations)—unclear if advantages persist for shorter or longer sequences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)